How to Run Persistent Scripts Over SSH Without Termination on Disconnect


2 views

When executing scripts via SSH on Ubuntu or other Linux systems, the process typically terminates when you close the SSH connection. This happens because:

  • The script becomes a child process of the SSH session
  • SIGHUP (hang-up signal) is sent to all processes when the terminal closes
  • Processes are tied to the shell's process group

The simplest method is using nohup (no hang-up):

nohup ./long_script.sh &

This will:

  • Ignore the SIGHUP signal
  • Redirect output to nohup.out
  • Run in background with &

For interactive scripts or when you need to reattach later:

sudo apt-get install tmux
tmux new -s mysession
./long_script.sh
# Detach with Ctrl+B then D
# Reattach later with: tmux attach -t mysession

For critical services, create a systemd unit file:

sudo nano /etc/systemd/system/longscript.service

File contents:

[Unit]
Description=My Persistent Script

[Service]
ExecStart=/path/to/script.sh
Restart=always
User=username

[Install]
WantedBy=multi-user.target

Then enable and start:

sudo systemctl daemon-reload
sudo systemctl enable longscript
sudo systemctl start longscript

If you already started the script:

./long_script.sh & # Start in background
jobs -l # Note the job ID
disown -h %1 # Replace 1 with your job ID

Always verify your processes are running:

ps aux | grep long_script
journalctl -u longscript -f # For systemd services
tmux list-sessions # For tmux
  • Ensure proper logging for background processes
  • Handle cleanup properly (use trap for signals)
  • Consider resource limits and monitoring
  • For critical tasks, implement proper error handling

When executing scripts via SSH on Ubuntu (or any Linux system), processes typically terminate when the SSH session ends. This occurs because:

  • Processes receive SIGHUP (hangup signal) upon SSH disconnection
  • The shell's process tree gets cleaned up
  • Default behavior ties process lifetime to the terminal session

1. Using nohup

The classic approach for detaching processes:

nohup ./long_running_script.sh > output.log 2>&1 &

Key components:

  • nohup prevents SIGHUP from terminating the process
  • > output.log redirects stdout
  • 2>&1 combines stderr with stdout
  • & runs the process in background

2. Screen/Tmux Session Management

For interactive scripts or monitoring needs:

# Install tmux (modern alternative to screen)
sudo apt-get install tmux

# Basic usage:
tmux new -s mysession
./interactive_script.sh
# Detach with Ctrl+B then D
# Reattach later: tmux attach -t mysession

3. Systemd Service Units

For production-grade process management (Ubuntu 9.04+):

# Create service file
sudo nano /etc/systemd/system/myscript.service

[Unit]
Description=My Persistent Script

[Service]
ExecStart=/path/to/script.sh
Restart=always
User=myuser
WorkingDirectory=/path/to/
StandardOutput=syslog
StandardError=syslog

[Install]
WantedBy=multi-user.target

Then enable and start:

sudo systemctl daemon-reload
sudo systemctl enable myscript
sudo systemctl start myscript

Process Disowning

For already-running processes:

./script.sh &
jobs -l # Note PID
disown -h %1

Daemonization with setsid

setsid ./daemon_script.sh > /dev/null 2>&1 </dev/null &
Method Best For Persistence Monitoring
nohup Quick tasks Good Limited
tmux Interactive scripts Excellent Full
systemd Production services Best Advanced
  • Check process status with ps aux | grep scriptname
  • Verify nohup output file permissions
  • For systemd: journalctl -u myscript -f
  • Test with simple scripts before production use

Example test script (test_script.sh):

#!/bin/bash
while true; do
    echo "$(date) - Still running" >> /tmp/persistent_test.log
    sleep 60
done