When dealing with Ubuntu server reboots, many developers encounter the frustrating "Waiting for Redis to shutdown..." loop. This typically occurs when Redis processes refuse to terminate gracefully, often requiring a forced reboot. The issue frequently appears in environments using monitoring tools like Godrb to keep Resque processes running.

Several factors can contribute to this behavior:

1. Resque monitoring interference: Tools like Godrb may actively maintain Redis connections
2. Improper shutdown sequence: Background jobs not completing before shutdown attempt
3. PID file issues: Stale or incorrect process identification files

Here are proven methods to resolve the shutdown hang:

1. Proper Service Stop Sequence

Create a custom shutdown script (/usr/local/bin/stop_redis_properly.sh):

#!/bin/bash
# Stop monitoring services first
sudo systemctl stop godrb
sudo systemctl stop resque

# Then stop Redis
sudo systemctl stop redis-server

# Force kill if necessary (15 seconds timeout)
sleep 15
if pgrep redis-server; then
    sudo pkill -9 redis-server
    sudo rm /var/run/redis/redis-server.pid
fi

2. Redis Configuration Tuning

Modify your redis.conf:

# Set shutdown timeout (milliseconds)
timeout 15000

# Disable protected mode when running with monitoring
protected-mode no

# Adjust save settings to prevent shutdown delays
save ""
appendonly no

3. Systemd Unit File Modification

Edit /etc/systemd/system/redis.service:

[Service]
ExecStop=/bin/redis-cli shutdown
TimeoutStopSec=15
KillMode=process

When troubleshooting, these commands help identify the issue:

# Check active connections
redis-cli client list

# Monitor shutdown process
sudo strace -p $(pgrep redis-server)

# Verify PID file consistency
sudo ls -l /var/run/redis/

• Always stop dependent services before Redis during shutdown
• Implement proper connection cleanup in monitoring tools
• Regularly test shutdown procedures in staging environments

When working with Redis on Ubuntu servers, many developers encounter the frustrating scenario where the system gets stuck on "Waiting for Redis to shutdown..." during reboots. This typically forces administrators to perform hard reboots, potentially leading to data corruption.

From my experience managing Redis clusters, these are the most frequent causes:

# Check active Redis connections (run before shutdown)
redis-cli info clients | grep connected_clients
# Or more detailed:
redis-cli client list

The Godrb monitoring tool mentioned in the original post can indeed interfere with clean shutdowns. Here's how to properly configure it:

# Example God configuration for Redis-aware monitoring
God.watch do |w|
  w.name = "resque"
  w.start = "rake resque:work"
  w.stop_signal = 'TERM'
  w.stop_timeout = 30.seconds
  w.before_stop = Proc.new {
    # Clean Redis connection handling
    Redis.current.quit rescue nil
  }
end

Redis needs to complete several operations before shutting down. Implement this in your init scripts:

#!/bin/bash
# Sample shutdown script for Redis
REDIS_CLI="/usr/bin/redis-cli"
$REDIS_CLI SHUTDOWN SAVE
sleep 5
if pgrep redis-server; then
    $REDIS_CLI SHUTDOWN NOSAVE
fi

For modern Ubuntu systems using systemd, these adjustments often help:

# /etc/systemd/system/redis.service.d/override.conf
[Service]
TimeoutStopSec=30
ExecStop=/usr/bin/redis-cli shutdown
KillSignal=SIGTERM