How to Force Kill and Restart a Systemd Service on Configuration Reload


3 views

When dealing with legacy daemons that don't support graceful configuration reloads (via SIGHUP), we often need a complete process restart. The standard systemctl reload behavior doesn't accommodate this workflow out of the box.

The naive approach of combining kill and restart in ExecReload fails because:

[Service]
ExecReload=/bin/kill -9 $MAINPID && /usr/bin/MYSERVICE

This creates multiple issues:

  • Race conditions between kill and restart
  • Systemd loses process tracking
  • No proper service state transitions

We need to leverage systemd's restart mechanisms properly:

[Service]
ExecStart=/usr/bin/MYSERVICE
ExecReload=/bin/systemctl kill --signal=KILL MYSERVICE.service
Restart=on-failure
RestartSec=5s

Here's a full service file that handles this properly:

[Unit]
Description=Legacy Daemon with Kill-Restart Reload

[Service]
Type=simple
ExecStart=/usr/local/bin/legacy-daemon --config /etc/legacy.conf
ExecReload=/bin/systemctl kill --signal=KILL legacy-daemon.service
Restart=always
RestartSec=5s
TimeoutStopSec=30
KillMode=process

[Install]
WantedBy=multi-user.target

After implementing:

# systemctl daemon-reload
# systemctl start legacy-daemon
# systemctl reload legacy-daemon  # Should kill and restart
# journalctl -u legacy-daemon -f  # Verify restart occurred

For services needing pre-restart cleanup:

[Service]
ExecStart=/usr/bin/complex-daemon
ExecReload=/usr/local/bin/daemon-cleanup ; /bin/systemctl restart complex-daemon.service
Restart=no  # Since we handle it explicitly
  • Ensure your application can handle sudden termination
  • Consider file descriptor leaks during hard kills
  • Monitor for restart loops in production
  • For stateful services, implement proper shutdown handlers

When managing traditional UNIX daemons with systemd, we often encounter scenarios where a simple configuration reload isn't sufficient - sometimes the service needs complete termination and restart. The standard ExecReload directive falls short here as it typically just sends SIGHUP.

With a basic service definition:

[Service]
ExecStart=/usr/bin/MYSERVICE
Type=simple

The systemctl reload command fails because:

  • No reload mechanism is defined
  • systemd doesn't assume kill/restart behavior by default

The proper approach involves creating a custom reload handler:

[Service]
ExecStart=/usr/bin/MYSERVICE
ExecStop=/bin/kill -TERM $MAINPID
ExecReload=/bin/sh -c "/bin/kill -TERM $MAINPID && /usr/bin/MYSERVICE"
Restart=on-failure
Type=simple

Key components:

  • Explicit termination command (ExecStop)
  • Compound reload command using shell
  • Restart policy for robustness

For complex services, consider using a reload wrapper script:

#!/bin/bash
# /usr/libexec/MYSERVICE-reload

systemctl stop MYSERVICE.service
systemctl start MYSERVICE.service

Then reference it in your unit file:

[Service]
ExecReload=/usr/libexec/MYSERVICE-reload

After implementation:

# Validate the unit file
systemd-analyze verify /etc/systemd/system/MYSERVICE.service

# Test the reload behavior
systemctl daemon-reload
systemctl start MYSERVICE
systemctl reload MYSERVICE

# Check status
journalctl -u MYSERVICE -f

For configuration file changes specifically, consider using path triggers:

[Unit]
Description=My Service
After=network.target

[Path]
PathModified=/etc/MYSERVICE.conf

[Service]
ExecStart=/usr/bin/MYSERVICE
Type=simple

This automatically restarts when the config file changes.