How to Run Long-Running SSH Commands Without Disruption: tmux, nohup, and screen Solutions


3 views

When you execute a command via SSH, it runs in an interactive shell session. This creates two critical vulnerabilities:

  • The process terminates if your local connection drops
  • Output becomes inaccessible after disconnection

For a 24-hour Python script (or any lengthy operation), this behavior is unacceptable. Here are three professional solutions:

The most basic method uses nohup to detach from terminal:

nohup python long_script.py > output.log 2>&1 &

Key components:

  • > output.log redirects stdout to file
  • 2>&1 combines stderr with stdout
  • & runs in background

For interactive monitoring and reattachment:

# First session
tmux new -s longjob
python script.py
# Detach with Ctrl+B then D

# Later reattach
tmux attach -t longjob

Advanced tmux features:

  • Multiple windows/panes
  • Session persistence
  • Scrollback buffer

Similar to tmux but with different keybindings:

screen -S longprocess
python analyze_data.py
# Detach with Ctrl+A then D

# Resume with
screen -r longprocess
Method Best For Pros Cons
nohup Fire-and-forget jobs Simple, no dependencies No reattachment
tmux Interactive monitoring Modern features, splitting Learning curve
screen Legacy systems Ubiquitous Older codebase

For production environments, consider creating a service:

[Unit]
Description=Long Running Python Job

[Service]
User=deploy
ExecStart=/usr/bin/python3 /path/to/script.py
Restart=always

[Install]
WantedBy=multi-user.target

When executing a Python script or any long-running command via SSH, your process becomes vulnerable to network interruptions. If your local connection drops, the remote process typically terminates with:

Write failed: Broken pipe
Connection to host closed.

This happens because SSH ties the remote process to your local terminal session by default.

Here are three reliable methods to keep processes running after SSH disconnects:

1. Using nohup

The classic approach that works on all Unix-like systems:

nohup python long_script.py > output.log 2>&1 &

This:

  • Detaches from terminal with nohup
  • Redirects output to a file
  • Runs in background with &

2. GNU Screen Session

More flexible than nohup, allowing reattachment:

screen -S long_task
python long_script.py
# Detach with Ctrl+A then D
# Reattach later with:
screen -r long_task

3. tmux (Modern Alternative)

Similar to screen but with more features:

tmux new -s long_process
python script.py
# Detach: Ctrl+B then D
# Reattach: tmux attach -t long_process

For critical production workloads, create a systemd service:

[Unit]
Description=Long Running Python Script

[Service]
User=youruser
ExecStart=/usr/bin/python3 /path/to/script.py
Restart=always

[Install]
WantedBy=multi-user.target

Then enable with:

sudo systemctl enable long_script.service
sudo systemctl start long_script.service

After detaching, you can check progress with:

tail -f output.log  # For nohup method
screen -ls         # For screen sessions
tmux list-sessions # For tmux
journalctl -u long_script.service -f  # For systemd

Choose the method that best fits your workflow and infrastructure requirements.