Fixing Significant Time Drift (7 Hours Behind) on CentOS 6: NTPD Troubleshooting and Alternative Solutions


2 views

When your CentOS 6 server shows a 7-hour time difference compared to reference servers, this typically indicates one of three scenarios:

  • Incorrect timezone configuration
  • Significant NTP synchronization failure
  • Hardware clock (RTC) misconfiguration

First, check your current time configuration with these commands:

# Check system time
date

# Check hardware clock
hwclock --show

# Verify timezone
ls -l /etc/localtime
cat /etc/sysconfig/clock

# Check NTP status
service ntpd status
ntpq -pn

For large time differences (over 1000 seconds), NTPD will refuse to sync. You'll need manual adjustment:

# Stop NTPD first
service ntpd stop

# Set time manually (example for UTC+7)
date -s "2023-11-15 14:30:00"

# Write to hardware clock
hwclock --systohc

# Alternative: Use ntpdate for one-time sync
ntpdate -u pool.ntp.org

Modify /etc/ntp.conf with these key settings:

server 0.pool.ntp.org iburst
server 1.pool.ntp.org iburst
server 2.pool.ntp.org iburst
server 3.pool.ntp.org iburst

# Enable panic threshold (in seconds)
tinker panic 0

# Increase polling time
minpoll 4
maxpoll 7

Then restart services:

service ntpd start
chkconfig ntpd on

For problematic NTPD cases, chrony often performs better:

yum install chrony
service chronyd start
chkconfig chronyd on

# Configuration (/etc/chrony.conf)
server pool.ntp.org iburst
makestep 1.0 3

After implementation, monitor synchronization:

# For NTPD:
ntpq -pn

# For chrony:
chronyc tracking
chronyc sources
  • Regularly check system logs (/var/log/messages)
  • Consider adding cron job for periodic sync verification
  • Monitor hardware clock battery on physical servers

When running time(NULL) or checking /proc/driver/rtc, you'll notice the system clock shows approximately 7 hours difference from the actual time. This isn't normal drift (which would be seconds/minutes), suggesting either incorrect timezone configuration or NTP synchronization failure.

First verify both hardware and system clocks:

# Check hardware clock
hwclock --show

# Check system clock
date

# Check timezone
ls -l /etc/localtime
cat /etc/sysconfig/clock

The 7-hour offset typically indicates:

  • UTC vs localtime mismatch in BIOS
  • Incorrect timezone files
  • NTP daemon not properly syncing
  • Large initial offset exceeding NTP threshold

For large offsets (>1000s), NTP won't sync automatically. Manually set time first:

# Stop ntpd temporarily
service ntpd stop

# Set time manually (example for US Pacific Time)
date -s "2023-11-15 14:30:00"

# Write to hardware clock (use --utc if BIOS uses UTC)
hwclock --systohc

Edit /etc/ntp.conf to add:

# Emergency large offset handling
tinker panic 0

# Force sync even with large offset
server 0.pool.ntp.org iburst
server 1.pool.ntp.org iburst
server 2.pool.ntp.org iburst

Create a monitoring script /usr/local/bin/check_time_sync.sh:

#!/bin/bash
MAX_OFFSET=60  # seconds
current_offset=$(ntpdc -c kerninfo | awk '/offset:/ {print $2}' | tr -d '-')

if [ $(echo "$current_offset > $MAX_OFFSET" | bc) -eq 1 ]; then
    logger "NTP offset too large: ${current_offset}s"
    service ntpd restart
fi

Add to /etc/sysconfig/ntpd:

OPTIONS="-g -x -p /var/run/ntpd.pid"

The -g allows large corrections, while -x prevents clock stepping that might disrupt applications.

For better handling of unstable clocks:

yum install chrony
service ntpd stop
chkconfig ntpd off

# Edit /etc/chrony.conf
server 0.centos.pool.ntp.org iburst
server 1.centos.pool.ntp.org iburst
makestep 1.0 3

For CentOS 6 systems:

rm -f /etc/localtime
ln -s /usr/share/zoneinfo/America/Los_Angeles /etc/localtime  # Example for PDT