The default 1000-second panic threshold in NTPd serves as a critical safeguard against:
- Catastrophic time jumps that could disrupt time-sensitive applications
- Protection against misconfigured or malicious time sources
- Prevention of clock drift corruption in distributed systems
In virtualized environments and during initial server provisioning, we often encounter:
# Common VMware time drift after suspend/resume
ntpq -p
remote refid st t when poll reach delay offset jitter
==============================================================================
*ntp.example.com .GPS. 1 u 125 256 377 1.234 +543.2 8.15
For Puppet-managed configurations, consider this approach:
# Puppet manifest snippet for conditional panic threshold
class { 'ntp':
servers => ['pool.ntp.org'],
tinker => $::is_virtual ? {
true => 'panic 0',
default => undef,
},
restrict => [
'default kod nomodify notrap nopeer noquery',
'-6 default kod nomodify notrap nopeer noquery'
],
}
Consider these approaches first:
- VMware Tools time synchronization
- BIOS/NTP pre-sync in provisioning automation
- Stratum-aware NTP pool configuration
When disabling panic threshold, implement robust monitoring:
# Nagios check for excessive time drift
define command {
command_name check_ntp_offset
command_line /usr/lib/nagios/plugins/check_ntp_offset -H $HOSTADDRESS$ -w $ARG1$ -c $ARG2$
}
For large-scale deployments:
# Recommended ntp.conf for VMware hosts
tinker panic 0
server ntp1.internal.corp iburst minpoll 4 maxpoll 4
server ntp2.internal.corp iburst minpoll 4 maxpoll 4
disable monitor
tos orphan 5
In modern infrastructure management, we frequently encounter scenarios where time synchronization becomes problematic:
- New physical servers with incorrect BIOS time (sometimes off by months)
- VMware VMs experiencing significant clock drift after suspend/resume cycles
- Cloud instances that boot with stale time information
The default NTP behavior of rejecting synchronization when the offset exceeds 1000 seconds (the "panic threshold") creates operational challenges in these cases.
The panic threshold exists for good reason - it prevents NTP from making massive time adjustments that could:
# Default panic threshold in ntpd (from ntp.conf):
tinker panic 1000 # 1000 second threshold
When this threshold is exceeded, ntpd will:
- Log a "clock stepped" message
- Stop adjusting the clock
- Mark itself as unsynchronized
In controlled environments where large corrections are acceptable, disabling the threshold can make sense:
# Example Puppet manifest to disable panic threshold:
class profile::ntp {
file { '/etc/ntp.conf':
content => template('profile/ntp.conf.erb'),
# ... other parameters
}
}
# ERB template snippet:
<% if @disable_panic_threshold -%>
tinker panic 0
<% end -%>
Disabling this safeguard introduces several potential issues:
Risk | Impact | Mitigation |
---|---|---|
Time jumps affecting applications | Transaction errors, log inconsistencies | Monitor for large corrections |
Hidden hardware clock issues | Persistent time problems | BIOS battery checks |
Security implications | Potential certificate validation issues | Strict NTP source validation |
Instead of completely disabling the panic threshold, consider these alternatives:
# More flexible configuration options:
tinker step 1.0 # Allow stepping for offsets > 1 second
tinker stepout 5 # Try stepping up to 5 times before giving up
# Combined with larger (but not infinite) thresholds:
tinker panic 3600 # 1 hour threshold instead of 1000s
For VMware environments specifically, ensure you're using:
- VMware Tools time synchronization as a fallback
- Proper
disable_time_sync="FALSE"
settings in VMX files - Regular checks of host time synchronization
When modifying the panic threshold:
- Document the change and its rationale
- Implement monitoring for large time corrections
- Consider gradual correction rather than immediate stepping
- Test thoroughly in non-production first
# Example monitoring check for large NTP corrections:
#!/bin/bash
THRESHOLD=60
CORRECTION=$(grep 'time reset' /var/log/ntp.log | awk '{print $NF}')
if (( $(echo "$CORRECTION > $THRESHOLD" | bc -l) )); then
alert "Large NTP correction: ${CORRECTION}s"
fi