We've all been there - running a critical server upgrade via SSH when suddenly the connection drops. The upgrade process gets orphaned, leaving system files locked and the server in limbo. Here's how to properly recover without restarting the entire upgrade.

First, check for active upgrade processes:

ps aux | grep -E 'apt|dpkg|upgrade'

Look for processes like /usr/bin/dpkg --status-fd or /usr/bin/perl /usr/bin/aptitude. Note the PID (process ID) of any running upgrade-related process.

The ideal solution if you had foresight to use a terminal multiplexer:

# If using screen:
screen -r

# If using tmux:
tmux attach

If you didn't use these initially, we'll need alternative methods.

For processes that expect terminal input, try reattaching STDIN:

sudo reptyr PID

Note: You may need to install reptyr first and configure ptrace permissions:

sudo apt-get install reptyr
echo 0 | sudo tee /proc/sys/kernel/yama/ptrace_scope

If you encounter locked files when trying to resume:

sudo lsof /var/lib/dpkg/lock
sudo kill -9 PID  # only if absolutely necessary
sudo rm /var/lib/apt/lists/lock
sudo rm /var/cache/apt/archives/lock

Ubuntu's package manager has built-in recovery mechanisms:

sudo dpkg --configure -a
sudo apt-get -f install

For future upgrades:

• Always use screen or tmux before starting upgrades
• Configure SSH keepalives in /etc/ssh/sshd_config:

ClientAliveInterval 60
ClientAliveCountMax 5

Or in your local SSH config (~/.ssh/config):

Host *
    ServerAliveInterval 60
    TCPKeepAlive yes

We've all been there - you're in the middle of a critical server upgrade via SSH when suddenly your connection drops. The process was waiting for your input (like whether to keep old config files), but now you're locked out with aptitude files showing as locked. Here's how to recover gracefully.

When your SSH session terminates unexpectedly, the upgrade process continues running but becomes detached from your terminal. Linux provides several tools to manage such situations:

# Check for existing screen sessions
screen -ls

# List all running processes
ps aux | grep -i upgrade

# Check for locked package files
sudo lsof /var/lib/dpkg/lock
sudo lsof /var/lib/apt/lists/lock

If the upgrade process is still running, you have several options to reconnect:

# Method 1: Using screen (if you started with it)
screen -r

# Method 2: Using tmux (alternative to screen)
tmux attach

# Method 3: Reconnecting to the process directly
sudo tail -f /var/log/apt/term.log
sudo dpkg --configure -a

If you can't reattach, follow this recovery sequence:

# First, check for and remove any lock files
sudo rm /var/lib/apt/lists/lock
sudo rm /var/cache/apt/archives/lock
sudo rm /var/lib/dpkg/lock

# Then force package configuration
sudo dpkg --configure -a

# Finally, complete any interrupted upgrades
sudo apt-get -f install
sudo apt-get update
sudo apt-get upgrade

Consider these best practices for long-running upgrades:

# Always use screen or tmux for important operations
sudo apt-get install screen tmux

# Example tmux session
tmux new -s upgrade_session
sudo do-release-upgrade

# Or with screen
screen -S upgrade_session
sudo do-release-upgrade

When things go wrong, these commands can provide insights:

# View package manager logs
cat /var/log/dpkg.log

# Check partial upgrades
sudo apt-get check

# Examine broken packages
sudo apt-get -f install

# View held packages
sudo apt-mark showhold