When working with remote servers over unstable networks, rsync transfers often fail midway with errors like:

rsync: connection unexpectedly closed

While the --partial flag helps preserve partially transferred files, rsync lacks built-in retry functionality.

Here's a bash script that implements exponential backoff for robust transfers:

#!/bin/bash

MAX_RETRIES=10
INITIAL_WAIT=1
MAX_WAIT=60
TARGET_DIR="/path/to/destination"
SOURCE="user@remote:/path/to/source"

retry_count=0
wait_time=$INITIAL_WAIT

while [ $retry_count -lt $MAX_RETRIES ]; do
    rsync -azP --partial --progress "$SOURCE" "$TARGET_DIR"
    exit_code=$?
    
    if [ $exit_code -eq 0 ]; then
        echo "Transfer completed successfully"
        exit 0
    fi
    
    echo "Transfer failed (attempt $((retry_count+1))/$MAX_RETRIES), retrying in $wait_time seconds..."
    sleep $wait_time
    
    # Exponential backoff with max cap
    wait_time=$((wait_time * 2))
    if [ $wait_time -gt $MAX_WAIT ]; then
        wait_time=$MAX_WAIT
    fi
    
    retry_count=$((retry_count+1))
done

echo "Max retries reached. Transfer failed."
exit 1

• Exponential backoff to prevent overwhelming the server
• Configurable maximum retry attempts
• Progress reporting with -P flag
• Archive mode (-a) for proper file attribute preservation
• Compression (-z) for better performance on slow connections

For critical transfers, add checksum verification after completion:

#!/bin/bash

# ... previous script parts ...

rsync -azP --partial --progress "$SOURCE" "$TARGET_DIR"
if [ $? -eq 0 ]; then
    echo "Verifying checksums..."
    rsync -avzc --checksum --dry-run "$SOURCE" "$TARGET_DIR"
    if [ $? -eq 0 ]; then
        echo "Verification successful"
        exit 0
    else
        echo "Verification failed - restarting transfer"
    fi
fi

For ongoing synchronization needs, consider combining with inotify-tools:

#!/bin/bash

while true; do
    inotifywait -r -e modify,create,delete "$TARGET_DIR"
    ./retry_rsync.sh  # The script we created earlier
done

We've all been there - trying to sync files from a remote server that keeps dropping connections. The dreaded rsync: connection unexpectedly closed error appears randomly, leaving partially transferred files and frustration.

While the --partial flag helps resume interrupted transfers, it doesn't solve the fundamental issue of automatic retries. Here's what happens without proper handling:

rsync -avz --partial user@remote:/path/to/files /local/path
# Connection fails mid-transfer
# Manual intervention required to restart

Here's a robust bash script that will keep retrying until the transfer completes successfully:

#!/bin/bash

REMOTE="user@remote:/path/to/files"
LOCAL="/local/path"
MAX_RETRIES=10
RETRY_DELAY=30

for ((i=1; i<=MAX_RETRIES; i++)); do
    rsync -avz --partial --progress "$REMOTE" "$LOCAL"
    if [ $? -eq 0 ]; then
        echo "Transfer completed successfully"
        exit 0
    fi
    echo "Attempt $i failed. Retrying in $RETRY_DELAY seconds..."
    sleep $RETRY_DELAY
done

echo "Maximum retries reached. Transfer failed."
exit 1

For even better reliability, implement exponential backoff between retries:

#!/bin/bash

REMOTE="user@remote:/path/to/files"
LOCAL="/local/path"
MAX_RETRIES=5
INITIAL_DELAY=5

delay=$INITIAL_DELAY
for ((i=1; i<=MAX_RETRIES; i++)); do
    rsync -avz --partial --progress "$REMOTE" "$LOCAL"
    if [ $? -eq 0 ]; then
        echo "Transfer completed successfully"
        exit 0
    fi
    echo "Attempt $i failed. Next retry in $delay seconds..."
    sleep $delay
    delay=$((delay * 2))
done

echo "Maximum retries reached. Transfer failed."
exit 1

For critical systems, consider these enhancements:

• Add logging to track transfer attempts
• Implement email notifications for failures
• Combine with inotifywait for continuous syncing
• Add checksum verification after transfer

Here's an example with logging:

#!/bin/bash

LOG_FILE="/var/log/rsync_retry.log"
{
    echo "Starting transfer at $(date)"
    # Previous script contents here
    echo "Transfer completed at $(date)"
} >> "$LOG_FILE" 2>&1