When you launch a process in background (either via &
or bg
) and then logout from an SSH session, here's what actually happens under the hood:
# Example 1: Background process creation
$ scp largefile user@remote:/path/ &
[1] 12345 # Process goes to background immediately
# Example 2: Suspended process backgrounding
$ scp largefile user@remote:/path/
^Z # CTRL-Z suspends process
[1]+ Stopped scp largefile user@remote:/path/
$ bg 1 # Resume in background
[1]+ scp largefile user@remote:/path/ &
Modern Linux systems (using systemd or traditional init) implement process re-parenting when the parent process dies. Your shell (bash/zsh) sends SIGHUP to its child processes during logout, but the kernel intervenes when:
- The process is in background state
- The parent process (your shell) terminates
In such cases, the kernel reparents the process to init (PID 1), which becomes its new parent. This explains why pstree
shows init as the parent after relogin.
The observed behavior differs based on several factors:
# Test case 1: Foreground process
$ scp largefile user@remote:/path/
# Logout kills it (SIGHUP received)
# Test case 2: Background process with shell built-in
$ sleep 3600 &
# Logout may kill it depending on shell settings
# Test case 3: Disowned process
$ scp largefile user@remote:/path/ &
$ disown
# Logout won't affect it (no SIGHUP sent)
nohup
serves three distinct purposes:
- Ignores SIGHUP (signal disposition change)
- Redirects stdout/stderr to nohup.out
- Prevents terminal I/O attempts from failing
Contrast this with simple backgrounding:
# Without nohup
$ python long_running.py > output.log 2>&1 &
# May still terminate if shell sends SIGHUP
# With nohup
$ nohup python long_running.py > output.log 2>&1 &
# Guaranteed to survive logout
SSH daemons can influence this behavior through these mechanisms:
ClientAliveInterval
settings- SIGHUP handling during controlled logout vs connection drop
- Terminal vs non-terminal sessions (
-t
flag)
For bulletproof background execution in SSH sessions, consider:
# Option 1: nohup + disown
$ nohup ./script.sh & disown
# Option 2: tmux/screen
$ tmux new -d -s mysession './script.sh'
# Option 3: systemd service
$ sudo systemd-run --user --scope --unit=myjob ./script.sh
For different scenarios:
Requirement | Solution |
---|---|
Temporary background task | command & |
Survive accidental disconnect | nohup command & disown |
Long-running critical process | tmux or systemd service |
Need output redirection | nohup command > log 2>&1 & |
When dealing with background processes in Linux, it's crucial to understand how the system manages process relationships and signals. The observed behavior where your scp
process survived logout and was adopted by init
is indeed expected in most modern Linux systems.
# Launch process in background
$ scp largefile user@remote:/path/ &
[1] 12345
# View process hierarchy
$ pstree -p
sshd(1234)───bash(5678)───scp(12345)
Contrary to common belief, background processes don't automatically receive SIGHUP on logout in many cases. The key factors determining this behavior are:
- Terminal handling configuration
- Shell implementation (bash vs zsh etc.)
- Process group leadership
- System initialization system (systemd, sysvinit, etc.)
When you properly exit a shell (via exit
command rather than connection drop), modern systems handle orphaned processes more gracefully:
# Before logout:
$ ps -o pid,ppid,pgid,sid,comm
PID PPID PGID SID COMMAND
12345 5678 12345 1234 scp
# After logout (new session):
$ ps -o pid,ppid,pgid,sid,comm
PID PPID PGID SID COMMAND
12345 1 12345 1234 scp
The nohup
command remains valuable in specific scenarios:
# Classic usage for long-running processes
$ nohup ./long_script.sh > output.log 2>&1 &
# Equivalent modern approaches
$ setsid ./long_script.sh > output.log 2>&1 &
$ (./long_script.sh > output.log 2>&1 &)
For reliable background process management:
- Use
disown
for existing jobs:disown %1
- Consider
tmux
orscreen
for critical processes - For systemd systems:
systemd-run --user --scope ./script.sh
To investigate unexpected process behavior:
# Check signal handlers
$ grep Sig /proc/12345/status
# Trace signal delivery
$ strace -p 12345 -e trace=signal