Every Linux sysadmin has faced this - you're rebooting a remote server through SSH, and your session hangs indefinitely until TCP timeout kicks in. This happens because sshd
gets terminated by systemd during shutdown sequence before it can properly close active connections.
The root cause lies in the service shutdown ordering. During system shutdown:
- Systemd kills services in parallel by default
- Network-related services often get terminated before
sshd
- Active SSH sessions get orphaned without proper closure
We need to ensure sshd
stops before network services during shutdown. Here's how:
# Create custom systemd unit override sudo mkdir -p /etc/systemd/system/ssh.service.d sudo nano /etc/systemd/system/ssh.service.d/shutdown.conf
Add these contents:
[Unit] After=network.target network-online.target Conflicts=shutdown.target reboot.target halt.target Before=shutdown.target reboot.target halt.target
For more control, create a pre-shutdown script:
# /usr/lib/systemd/system-shutdown/graceful-ssh.sh #!/bin/bash [ "$1" = "reboot" ] || [ "$1" = "halt" ] && { systemctl stop ssh sleep 3 # Give SSH time to close connections }
Make it executable:
sudo chmod +x /usr/lib/systemd/system-shutdown/graceful-ssh.sh
Verify the shutdown sequence without actually rebooting:
systemd-analyze critical-chain ssh.service
If you're using older Debian versions with SysVinit, modify the init script:
# Add to /etc/init.d/ssh stop() { # Send TERM signal to all sshd processes killall -TERM sshd sleep 5 # Force kill if needed killall -KILL sshd || true }
After implementing the fix, you should see clean disconnects in your SSH client:
client$ ssh server server$ sudo reboot # Should immediately show "Connection closed by remote host"
When rebooting a Debian server with active SSH connections, administrators often encounter frozen client sessions that persist until TCP timeout. This occurs because the default sshd
termination process doesn't properly close established connections before shutting down.
The most effective solution involves modifying the SSH service unit file to implement proper shutdown sequencing:
# Create an override directory if it doesn't exist
sudo mkdir -p /etc/systemd/system/ssh.service.d/
# Create an override config file
sudo nano /etc/systemd/system/ssh.service.d/shutdown.conf
Add these directives to the override file:
[Service]
KillMode=process
ExecStop=-/usr/bin/killall -w -s TERM sshd
TimeoutStopSec=30
For environments where modifying systemd isn't preferable, consider these options:
# Method 1: Custom shutdown script
sudo nano /usr/local/bin/graceful_ssh_shutdown.sh
#!/bin/bash
/usr/bin/pkill -TERM sshd
sleep 5
/usr/bin/pkill -KILL sshd
exit 0
Then make it executable and add to reboot sequence:
sudo chmod +x /usr/local/bin/graceful_ssh_shutdown.sh
sudo ln -s /usr/local/bin/graceful_ssh_shutdown.sh /etc/rc6.d/K01ssh-shutdown
Verify the changes work without a full reboot:
# Reload systemd configuration
sudo systemctl daemon-reload
# Test the shutdown behavior
sudo systemctl restart ssh
Monitor active SSH sessions during the test:
watch -n 1 "netstat -tnpa | grep sshd"
If sessions still hang:
- Check system logs:
journalctl -u ssh.service
- Verify kill signals are being sent properly
- Adjust
TimeoutStopSec
if needed for your environment