We've all been there - you're running a long compile job or database migration through SSH when suddenly your connection drops. The process keeps running server-side, but you've lost visibility. Here's how to regain control without restarting your work.
The most robust approach is using terminal multiplexers. Here's how to set up tmux:
# First install tmux if needed
sudo apt-get install tmux # Debian/Ubuntu
sudo yum install tmux # CentOS/RHEL
# Start a named session
tmux new -s mysession
# Detach from session (Ctrl+b then d)
# Later, reattach from any connection:
tmux attach -t mysession
For older systems where tmux isn't available, GNU screen works similarly:
screen -S sessionname
# Detach with Ctrl+a then d
# Reattach with:
screen -r sessionname
For quick reconnection attempts without multiplexers:
# During initial SSH connection:
ssh -o ServerAliveInterval=60 user@host
# If dropped, check for hanging processes with:
ps aux | grep your_process
Create a resilient SSH wrapper script:
#!/bin/bash
while true; do
ssh -o TCPKeepAlive=yes -o ServerAliveInterval=15 user@host
sleep 5
echo "Reconnecting..."
done
If you need to check on processes without reattaching:
# Find your process ID
ps aux | grep process_name
# Check process status
cat /proc/PID/status
# View output if writing to file
tail -f /path/to/output.log
When working with long-running processes over SSH, sudden disconnections can be frustrating. Linux systems provide several ways to maintain and restore sessions:
# Check if your SSH server supports TCPKeepAlive
ssh -o TCPKeepAlive=yes user@host
The most robust solution is using tmux before starting your process:
# Install tmux if not present
sudo apt-get install tmux # Debian/Ubuntu
sudo yum install tmux # CentOS/RHEL
# Basic workflow:
tmux new -s mysession # Start new session
./long_running_script.sh # Run your process
# Detach with Ctrl-b d
# After reconnection:
tmux attach -t mysession
For older systems where tmux isn't available:
screen -S mysession
./my_process
# Detach with Ctrl-a d
# Reattach later:
screen -r mysession
If you didn't use session management tools:
# Find your process
ps aux | grep process_name
# Check for running shells
who -a
w
Add these to ~/.ssh/config:
Host *
ServerAliveInterval 60
ServerAliveCountMax 5
TCPKeepAlive yes
ControlMaster auto
ControlPath ~/.ssh/control:%h:%p:%r
ControlPersist 1h
For processes running in nohup:
# Check output if redirected
tail -f nohup.out
# Reattach to existing process if possible
reptyr $(pidof your_process)
Always consider:
- Running important processes with nohup
- Redirecting output to log files
- Using systemd services for critical processes
nohup ./script.sh > output.log 2>&1 &