Fixing Time Drift in Resumed KVM Guests: NTP and Clock Source Configuration


1 views

When working with KVM guests managed by libvirt, a common issue arises during guest suspension and resumption. The guest's system clock fails to account for the elapsed time during suspension, leading to significant time discrepancies. This occurs because:

  • The guest's internal clock stops during suspension
  • Standard time synchronization mechanisms may not activate immediately after resume
  • Clock source behavior differs between physical and virtual environments

Before making changes, verify your current clock source configuration:

# Check available and current clock sources
cat /sys/devices/system/clocksource/clocksource0/available_clocksource
cat /sys/devices/system/clocksource/clocksource0/current_clocksource

# Typical output:
# kvm-clock tsc hpet acpi_pm
# kvm-clock

To maintain accurate time in resumed KVM guests, implement this multi-layered solution:

1. Configure KVM Clock Properly

Add these parameters to your guest's kernel command line (in /etc/default/grub):

GRUB_CMDLINE_LINUX="... clocksource=kvm-clock no-kvmclock-vsyscall ..."

Then update grub and reboot:

update-grub
reboot

2. Implement NTP Synchronization

Install and configure chrony (preferred for virtual environments):

apt install chrony  # Debian/Ubuntu
yum install chrony  # RHEL/CentOS

# Configure /etc/chrony/chrony.conf or /etc/chrony.conf
server ntp.ubuntu.com iburst
makestep 1.0 3

3. Add libvirt Hook Script

Create a resume hook to force time synchronization:

# /etc/libvirt/hooks/qemu
#!/bin/bash

if [ "$2" = "resume" ]; then
    virsh list --name | while read vm; do
        virsh qemu-agent-command "$vm" '{"execute":"guest-set-time"}'
    done
fi

chmod +x /etc/libvirt/hooks/qemu

4. Alternative: Use guest agent commands

For more precise control, use the QEMU guest agent:

# Manual time sync command
virsh qemu-agent-command VM_NAME '{"execute":"guest-set-time"}'

# Alternatively, trigger NTP sync
virsh qemu-agent-command VM_NAME '{"execute":"guest-exec", "arguments":{"path":"/usr/bin/chronyc", "arg":["-a", "makestep"]}}'

After implementing these changes, test the solution:

# Suspend and resume the guest
virsh suspend VM_NAME
sleep 60
virsh resume VM_NAME

# Check time difference
date; ssh VM_NAME date
  • Ensure the QEMU guest agent is installed and running in the guest
  • Verify NTP service status with systemctl status chronyd
  • Check kernel messages for clock-related errors: dmesg | grep -i clock
  • For Windows guests, use virsh qemu-agent-command with appropriate Windows time sync commands

When using KVM virtualization with libvirt, suspended guests often experience significant time drift upon resumption. This occurs because:

  • The guest's internal clock freezes during suspension
  • No automatic synchronization occurs during resume
  • kvm-clock alone isn't sufficient for post-resume correction

First verify your current clock source setup:

# Check available and active clock sources
cat /sys/devices/system/clocksource/clocksource0/available_clocksource
cat /sys/devices/system/clocksource/clocksource0/current_clocksource

# Sample output should show kvm-clock as active:
# kvm-clock tsc hpet acpi_pm 
# kvm-clock

We need multiple layers of time synchronization:

1. XML Configuration for libvirt Domain

Edit your VM's configuration:

<domain type='kvm'>
  ...
  <clock offset='utc'>
    <timer name='rtc' tickpolicy='catchup' track='guest'/>
    <timer name='pit' tickpolicy='delay'/>
    <timer name='hpet' present='yes'/>
    <timer name='kvmclock' present='yes'/>
  </clock>
  ...
</domain>

2. Guest OS Configuration

Inside the guest, implement these measures:

For Linux Guests

# Install NTP tools
sudo apt install chrony  # or ntp for older systems

# Configure chrony to sync with host
echo "server _gateway iburst" | sudo tee -a /etc/chrony/chrony.conf

# Enable qemu-guest-agent for time sync
sudo systemctl enable --now qemu-guest-agent

For Windows Guests

# In PowerShell, enable time synchronization integration
Set-VMIntegrationService -VMName "YourVMName" -Name "Time Synchronization" -Enabled $true

3. Host-side Automation

Create a libvirt hook to force time sync on resume:

#!/bin/bash
# /etc/libvirt/hooks/qemu

GUEST_NAME="$1"
ACTION="$2"

if [ "$ACTION" = "start" ] || [ "$ACTION" = "reconnect" ]; then
    # Wait for guest to be fully booted
    sleep 20
    
    # Force time sync via guest agent
    virsh qemu-agent-command "$GUEST_NAME" '{"execute":"guest-set-time"}'
fi

Validate your setup with these commands:

# Check current time difference between host and guest
host_time=$(date +%s)
guest_time=$(virsh qemu-agent-command $VM_NAME '{"execute":"guest-get-time"}' | jq -r '.return')
echo "Time difference: $((host_time - guest_time)) seconds"

# Monitor chrony synchronization
chronyc tracking
chronyc sources
  • If using NTP instead of chrony, ensure ntpd service is active
  • For Windows guests, verify Hyper-V Time Sync Service is running
  • Check libvirt logs if guest agent commands fail: journalctl -u libvirtd
  • Consider adding <feature policy='require' name='kvmclock'/> in CPU features

For environments requiring microsecond precision:

# Add these parameters to your QEMU command line (or via libvirt XML):
-rtc base=utc,driftfix=slew
-global kvm-pit.lost_tick_policy=delay