When working with database clusters in air-gapped environments, maintaining time synchronization becomes critical yet challenging. Your current setup shows precisely this pain point: despite configuring a local NTP server (192.168.178.20) and clients, you're observing significant time drift ranging from seconds to minutes within just 10 minutes of synchronization.

Your server configuration contains some common pitfalls for isolated NTP setups:

# Problematic elements in current server config
server 127.127.1.0 prefer
fudge 127.127.1.0
restrict default

The main issues are:

1. Using the local clock as preferred source without proper tuning
2. Overly permissive default restrictions
3. Missing key parameters for stable timekeeping

Here's a proven configuration for air-gapped Ubuntu servers:

# /etc/ntp.conf on your NTP server (192.168.178.20)
driftfile /var/lib/ntp/ntp.drift
statsdir /var/log/ntpstats/

# Local clock as stratum 8 (not 10)
server 127.127.1.0 iburst
fudge 127.127.1.0 stratum 8

# Network restrictions
restrict default kod limited nomodify notrap nopeer noquery
restrict 127.0.0.1
restrict 192.168.178.0 mask 255.255.255.0 nomodify notrap

# Tuning parameters
tinker panic 0
tos maxdist 30
tos mindist 0.001
tos maxclock 30
tos minclock 3

# Logging
logconfig =syncall +clockall

For your database servers that need tight synchronization:

# /etc/ntp.conf on client machines
driftfile /var/lib/ntp/ntp.drift

server 192.168.178.20 iburst minpoll 3 maxpoll 4

restrict default ignore
restrict 127.0.0.1
restrict 192.168.178.20 nomodify notrap noquery

# Force more frequent synchronization
tos mindist 0.001
tos maxclock 30

Follow this exact sequence after updating configurations:

# On all machines:
sudo service ntp stop
sudo ntpd -gq
sudo service ntp start

# Verify synchronization:
ntpq -pn
watch -n 1 ntpq -c as

# Check for errors:
tail -f /var/log/syslog | grep ntp

If you still see rejections after the above steps:

1. Check hardware clocks:

   sudo hwclock --debug --show

2. Force immediate sync:

   sudo ntpdate -u -b 192.168.178.20

3. Adjust kernel timekeeping:

   echo 1 | sudo tee /proc/sys/kernel/ntp_adj

For database environments, consider these additions:

# Add to /etc/sysctl.conf
kernel.ntp_tick=1
kernel.ntp_maxerror=500
kernel.ntp_est_error=500

Monitor drift weekly with:

ntpdc -c loopinfo
ntptime

When dealing with database clusters in air-gapped environments, time synchronization becomes critical yet challenging. The scenario involves 8 Ubuntu 14.04 servers without internet access, where time drift causes database consistency issues. We'll explore a robust local NTP solution.

The primary NTP server (192.168.178.20) needs this optimized configuration:

# /etc/ntp.conf on the master server
server 127.127.1.0 iburst prefer
fudge 127.127.1.0 stratum 8
driftfile /var/lib/ntp/ntp.drift
logfile /var/log/ntp.log

# Access control
restrict default kod nomodify notrap nopeer noquery
restrict 127.0.0.1
restrict 192.168.178.0 mask 255.255.255.0 nomodify notrap

# Tuning parameters
tinker panic 0
tos maxclock 25
tos minclock 12

For clients showing rejection states, implement this aggressive sync configuration:

# /etc/ntp.conf on clients
server 192.168.178.20 iburst minpoll 3 maxpoll 4 prefer
restrict 192.168.178.20 nomodify notrap noquery

driftfile /var/lib/ntp/ntp.drift
logfile /var/log/ntp.log

# Force synchronization parameters
tos orphan 12
tinker allan 1500
tinker dispersion 500

When clients show rejection (status 905a), follow this recovery sequence:

sudo service ntp stop
sudo ntpd -gq -x -c /etc/ntp.conf
sudo service ntp start
sudo ntpdate -u 192.168.178.20

Check synchronization status with:

ntpq -pn
watch -n 1 "ntpdc -c sysinfo"

Implement these cron jobs for persistent synchronization:

# /etc/cron.d/ntp-sync
*/5 * * * * root /usr/sbin/ntpdate -u 192.168.178.20 && /sbin/hwclock --systohc

For logging and analysis:

tail -f /var/log/ntp.log | grep -E 'sync|adjust|step'