Debugging Systemd Timer Failures: Why Your Service Unit Isn’t Starting and How to Fix It


2 views

When manually executing systemctl start letsencrypt-example_com.service, everything works perfectly. However, the same service fails to trigger when activated via its companion timer unit, despite the timer showing as active in systemctl list-timers.

Your current timer configuration has several potential issues:

[Timer]
OnUnitActiveSec=1min     # This triggers 1min after service activation
Persistent=true          # Good for catching missed runs
Unit=letsencrypt-example_com.service

The key problem is using OnUnitActiveSec instead of OnCalendar for periodic execution. Here's how we should modify it:

[Timer]
OnCalendar=*-*-1,15 02:00:00  # Runs at 2AM on 1st and 15th of each month
RandomizedDelaySec=1h         # Spreads load across the hour
Persistent=true
Unit=letsencrypt-example_com.service

The service unit has dependencies that might affect timer execution:

[Unit]
Requires=nginx_reload.service
Before=nginx_reload.service

Add explicit ordering to prevent race conditions:

[Unit]
Wants=nginx_reload.service
After=nginx_reload.service

Use these commands to investigate timer behavior:

# Show timer activation details
journalctl -u letsencrypt-example_com.timer -n 50

# Check service execution attempts
journalctl -u letsencrypt-example_com.service --since "1 hour ago"

# Verify timer definition
systemd-analyze verify /etc/systemd/system/letsencrypt-example_com.timer

Here's a proven timer configuration for Let's Encrypt renewal:

[Unit]
Description=Let's Encrypt renewal timer for example.com

[Timer]
OnCalendar=*-*-01,15 03:00:00
AccuracySec=1h
Persistent=true
Unit=letsencrypt-example_com.service

[Install]
WantedBy=timers.target

Since the service runs as user 'letsencrypt', ensure:

# Set proper permissions for webroot
chown -R letsencrypt:www-data /srv/files/letsencrypt/www
chmod -R 750 /srv/files/letsencrypt/www

# Validate script execution
sudo -u letsencrypt /bin/sh /usr/local/bin/letsencrypt-renew.sh example.com

# Reload systemd after changes
systemctl daemon-reload

# Manually test the timer
systemctl start letsencrypt-example_com.timer
systemctl status letsencrypt-example_com.timer
journalctl -xe

When examining the timer unit status, we notice it shows as active but the service remains inactive:

# systemctl list-timers
NEXT  LEFT  LAST  PASSED  UNIT  ACTIVATES
n/a  n/a  Fri 2016-05-06 13:10:13 CEST  1h 51min ago letsencrypt-example_com.timer letsencrypt-example_com.service

The current timer setup has several potential issues:

[Timer]
OnUnitActiveSec=1min
Persistent=true
Unit=letsencrypt-example_com.service

The primary problem stems from using OnUnitActiveSec which triggers relative to the service's last activation. For a certificate renewal scenario, we should use absolute timing with OnCalendar or fixed intervals with OnUnitInactiveSec.

Here's a more appropriate timer configuration for certificate renewal:

[Unit]
Description=Run letsencrypt-example_com every 60 days

[Timer]
OnCalendar=*-*-1,15 03:00:00
RandomizedDelaySec=3600
Persistent=true
Unit=letsencrypt-example_com.service

[Install]
WantedBy=timers.target

The service unit could benefit from additional robustness measures:

[Service]
Type=oneshot
ExecStart=/bin/sh /usr/local/bin/letsencrypt-renew.sh example.com www.example.com
User=letsencrypt
Group=www-data
Restart=no
SuccessExitStatus=0 1
MemoryLimit=512M
CPUQuota=50%

To diagnose timer issues, use these commands:

# Check timer activation
journalctl -u letsencrypt-example_com.timer

# Verify service attempts
journalctl -u letsencrypt-example_com.service

# Test timer without waiting
systemd-analyze calendar "*-*-1,15 03:00:00"

# Check next elapse time
systemctl list-timers --all

Here's a verified configuration that works for certificate renewal:

# /etc/systemd/system/letsencrypt-renewal.timer
[Unit]
Description=Weekly LetsEncrypt renewal check

[Timer]
OnCalendar=Mon *-*-* 02:00:00
RandomizedDelaySec=1h
Persistent=true

[Install]
WantedBy=timers.target

# /etc/systemd/system/letsencrypt-renewal.service
[Unit]
Description=LetsEncrypt Certificate Renewal
After=network-online.target

[Service]
Type=oneshot
ExecStart=/usr/bin/certbot renew --quiet --agree-tos
ExecStartPost=/bin/systemctl reload nginx
User=root
Group=root

Ensure proper permissions for the letsencrypt user:

# Set directory permissions
chown -R letsencrypt:www-data /etc/letsencrypt
chmod -R 750 /etc/letsencrypt

# Verify webroot access
chown letsencrypt:www-data /srv/files/letsencrypt/www
chmod 755 /srv/files/letsencrypt/www