How to Automatically Reattach or Recreate Named GNU Screen Sessions for Persistent Remote Work


12 views

When working with remote servers, GNU Screen sessions provide invaluable persistence. However, managing named sessions with automatic reattachment/recreation presents several nuanced challenges:

# Current working but limited approach
ssh host -t screen -x thesessionname

# Almost-there solution with -R flag
ssh host -t screen -R -S thesessionname

The key distinction between these approaches lies in how they handle session states:

  • -x: Attaches to existing session (attached or detached) but fails if session doesn't exist
  • -R: Reattaches to first detached session or creates new one, ignoring named sessions

Here's a script that implements the exact behavior we want:

#!/bin/bash
SESSION="thesessionname"
HOST="your-remote-host"

ssh -t $HOST <<'ENDSSH'
  # Check if session exists (attached or detached)
  if screen -list | grep -q "$SESSION"; then
    # Session exists - attach to it
    exec screen -x "$SESSION"
  else
    # Create new session with custom options
    exec screen -S "$SESSION" -c ~/.screenrc.work
  fi
ENDSSH

For production environments, consider this more robust version with logging and auto-recovery:

#!/bin/bash
SESSION="prod-session"
HOST="prod-server"
LOG="$HOME/.screen_recovery.log"

ssh -t $HOST >> $LOG 2>&1 <<'ENDSSH'
  if ! screen -list | grep -q "$SESSION"; then
    echo "$(date) - Session $SESSION not found. Creating new one." >> $LOG
    screen -dmS "$SESSION" bash -c 'while true; do /path/to/main_script; sleep 5; done'
  fi
  exec screen -x "$SESSION"
ENDSSH

Depending on your workflow, these alternatives might be preferable:

# Using tmux instead of screen (modern alternative)
ssh host -t tmux attach -t thesessionname || tmux new -s thesessionname

# Using systemd to keep session alive
[Unit]
Description=Persistent Screen Session
After=network.target

[Service]
Type=simple
User=youruser
ExecStart=/usr/bin/screen -S persistent -D -m /path/to/command
Restart=always
RestartSec=5s

[Install]
WantedBy=multi-user.target
  • Use screen -wipe in cron to clean up dead sessions
  • Implement session locking for sensitive operations
  • Consider using byobu for enhanced session management
  • Set up SSH config aliases for frequently accessed hosts

When working with long-running processes on remote servers, GNU Screen's session persistence is incredibly valuable - until it breaks. The fundamental issue arises when we want:

  • Automatic reattachment to existing named sessions
  • Seamless recreation if the session terminates
  • Prevention of duplicate sessions when already attached

The standard approach using screen -x thesessionname works for reattachment but fails when:

# Current working but non-resilient method
ssh host -t screen -x thesessionname

# Almost-there solution with -R flag
ssh host -t screen -R -S thesessionname

The -R flag's behavior of only checking for detached sessions creates duplicate sessions when one is already attached - which is exactly what we're trying to avoid.

We can implement a shell script that intelligently handles both scenarios. Here's a comprehensive solution:

#!/bin/bash
SESSION_NAME="thesessionname"
REMOTE_HOST="host"

ssh -t $REMOTE_HOST <<'ENDSSH'
  # Check if session exists (attached or detached)
  if screen -list | grep -q "\.${SESSION_NAME}\s"; then
    # Session exists - reattach
    screen -x $SESSION_NAME
  else
    # Create new session with logging
    screen -L -Logfile ~/screenlog.$SESSION_NAME -S $SESSION_NAME
  fi
ENDSSH

For production environments, consider adding session validation and auto-recovery:

#!/bin/bash
SESSION_NAME="prod_session"
REMOTE_HOST="production.example.com"
LOG_DIR="/var/log/screen_sessions"

ssh -t $REMOTE_HOST <<'ENDSSH'
  # Create log directory if missing
  mkdir -p $LOG_DIR
  
  # Check session status
  if ! screen -list | grep -q "\.${SESSION_NAME}\s"; then
    echo "Session not found - creating new"
    screen -dmS $SESSION_NAME -Logfile $LOG_DIR/$SESSION_NAME.log bash
    sleep 1 # Allow session creation
  fi
  
  # Verify session responsiveness
  if ! screen -S $SESSION_NAME -X stuff "echo healthcheck\n"; then
    echo "Session unresponsive - recreating"
    screen -S $SESSION_NAME -X quit
    screen -dmS $SESSION_NAME -Logfile $LOG_DIR/$SESSION_NAME.log bash
  fi
  
  # Finally attach
  screen -x $SESSION_NAME
ENDSSH

For maximum reliability, implement a monitoring cron job on the remote server:

# Add to remote server's crontab (crontab -e)
*/5 * * * * /usr/bin/screen -list | grep -q "\.important_session\s" || \
/usr/bin/screen -dmS important_session /path/to/your/script.sh

While Screen is venerable, Tmux offers more robust session management:

ssh host -t tmux new -A -s thesessionname

The -A flag automatically attaches to existing sessions or creates new ones, solving our core problem elegantly.