How to Automatically Reattach or Recreate Named GNU Screen Sessions for Persistent Remote Work


1 views

When working with remote servers, GNU Screen sessions provide invaluable persistence. However, managing named sessions with automatic reattachment/recreation presents several nuanced challenges:

# Current working but limited approach
ssh host -t screen -x thesessionname

# Almost-there solution with -R flag
ssh host -t screen -R -S thesessionname

The key distinction between these approaches lies in how they handle session states:

  • -x: Attaches to existing session (attached or detached) but fails if session doesn't exist
  • -R: Reattaches to first detached session or creates new one, ignoring named sessions

Here's a script that implements the exact behavior we want:

#!/bin/bash
SESSION="thesessionname"
HOST="your-remote-host"

ssh -t $HOST <<'ENDSSH'
  # Check if session exists (attached or detached)
  if screen -list | grep -q "$SESSION"; then
    # Session exists - attach to it
    exec screen -x "$SESSION"
  else
    # Create new session with custom options
    exec screen -S "$SESSION" -c ~/.screenrc.work
  fi
ENDSSH

For production environments, consider this more robust version with logging and auto-recovery:

#!/bin/bash
SESSION="prod-session"
HOST="prod-server"
LOG="$HOME/.screen_recovery.log"

ssh -t $HOST >> $LOG 2>&1 <<'ENDSSH'
  if ! screen -list | grep -q "$SESSION"; then
    echo "$(date) - Session $SESSION not found. Creating new one." >> $LOG
    screen -dmS "$SESSION" bash -c 'while true; do /path/to/main_script; sleep 5; done'
  fi
  exec screen -x "$SESSION"
ENDSSH

Depending on your workflow, these alternatives might be preferable:

# Using tmux instead of screen (modern alternative)
ssh host -t tmux attach -t thesessionname || tmux new -s thesessionname

# Using systemd to keep session alive
[Unit]
Description=Persistent Screen Session
After=network.target

[Service]
Type=simple
User=youruser
ExecStart=/usr/bin/screen -S persistent -D -m /path/to/command
Restart=always
RestartSec=5s

[Install]
WantedBy=multi-user.target
  • Use screen -wipe in cron to clean up dead sessions
  • Implement session locking for sensitive operations
  • Consider using byobu for enhanced session management
  • Set up SSH config aliases for frequently accessed hosts

When working with long-running processes on remote servers, GNU Screen's session persistence is incredibly valuable - until it breaks. The fundamental issue arises when we want:

  • Automatic reattachment to existing named sessions
  • Seamless recreation if the session terminates
  • Prevention of duplicate sessions when already attached

The standard approach using screen -x thesessionname works for reattachment but fails when:

# Current working but non-resilient method
ssh host -t screen -x thesessionname

# Almost-there solution with -R flag
ssh host -t screen -R -S thesessionname

The -R flag's behavior of only checking for detached sessions creates duplicate sessions when one is already attached - which is exactly what we're trying to avoid.

We can implement a shell script that intelligently handles both scenarios. Here's a comprehensive solution:

#!/bin/bash
SESSION_NAME="thesessionname"
REMOTE_HOST="host"

ssh -t $REMOTE_HOST <<'ENDSSH'
  # Check if session exists (attached or detached)
  if screen -list | grep -q "\.${SESSION_NAME}\s"; then
    # Session exists - reattach
    screen -x $SESSION_NAME
  else
    # Create new session with logging
    screen -L -Logfile ~/screenlog.$SESSION_NAME -S $SESSION_NAME
  fi
ENDSSH

For production environments, consider adding session validation and auto-recovery:

#!/bin/bash
SESSION_NAME="prod_session"
REMOTE_HOST="production.example.com"
LOG_DIR="/var/log/screen_sessions"

ssh -t $REMOTE_HOST <<'ENDSSH'
  # Create log directory if missing
  mkdir -p $LOG_DIR
  
  # Check session status
  if ! screen -list | grep -q "\.${SESSION_NAME}\s"; then
    echo "Session not found - creating new"
    screen -dmS $SESSION_NAME -Logfile $LOG_DIR/$SESSION_NAME.log bash
    sleep 1 # Allow session creation
  fi
  
  # Verify session responsiveness
  if ! screen -S $SESSION_NAME -X stuff "echo healthcheck\n"; then
    echo "Session unresponsive - recreating"
    screen -S $SESSION_NAME -X quit
    screen -dmS $SESSION_NAME -Logfile $LOG_DIR/$SESSION_NAME.log bash
  fi
  
  # Finally attach
  screen -x $SESSION_NAME
ENDSSH

For maximum reliability, implement a monitoring cron job on the remote server:

# Add to remote server's crontab (crontab -e)
*/5 * * * * /usr/bin/screen -list | grep -q "\.important_session\s" || \
/usr/bin/screen -dmS important_session /path/to/your/script.sh

While Screen is venerable, Tmux offers more robust session management:

ssh host -t tmux new -A -s thesessionname

The -A flag automatically attaches to existing sessions or creates new ones, solving our core problem elegantly.