When using rsync in Python scripts via subprocess
, unexpected password prompts can completely halt automation workflows. This typically occurs when:
- SSH public key authentication fails
- Invalid remote username is provided
- Permissions on the remote machine are misconfigured
These critical options will make rsync behave predictably in scripts:
rsync \
--archive \
--compress \
--stats \
--no-motd \
--rsh="ssh -o BatchMode=yes" \
source/ user@host:destination/
Here's how to properly call rsync from Python with error handling:
import subprocess
from subprocess import CalledProcessError
def run_rsync():
try:
result = subprocess.run(
["rsync", "-az", "--rsh=ssh -o BatchMode=yes", "/local/path/", "remoteuser@host:/remote/path/"],
check=True,
capture_output=True,
text=True
)
print("Transfer successful")
return True
except CalledProcessError as e:
print(f"Rsync failed with error: {e.stderr}")
return False
Add these to your SSH config (~/.ssh/config
) for bulletproof automation:
Host backup-server
HostName server.example.com
User backupuser
IdentityFile ~/.ssh/backup_key
IdentitiesOnly yes
PasswordAuthentication no
BatchMode yes
For production systems, consider these additional safeguards:
# Timeout after 30 seconds if connection hangs
rsync --timeout=30 ...
# Limit bandwidth to prevent network saturation
rsync --bwlimit=5000 ...
# Set connection attempt limits
rsync --max-tries=2 ...
Test your configuration with this diagnostic command before scripting:
ssh -T -o BatchMode=yes -o StrictHostKeyChecking=yes user@host echo "Test successful"
When using rsync in Python scripts with subprocess
, unexpected password prompts can break automation workflows. This typically happens when:
- Using incorrect remote usernames
- SSH key authentication fails
- Permission issues occur on the remote machine
Add these critical parameters to your rsync command:
rsync -e "ssh -o BatchMode=yes" --rsync-path="sudo -u correct_user rsync" \
--password-file=/dev/null -q --no-motd
Here's a robust Python implementation using subprocess:
import subprocess
def safe_rsync(source, dest):
try:
result = subprocess.run(
["rsync", "-azP",
"-e", "ssh -o BatchMode=yes -o ConnectTimeout=30",
"--rsync-path='sudo -u valid_user rsync'",
source, dest],
check=True,
timeout=300,
stdout=subprocess.PIPE,
stderr=subprocess.PIPE
)
return True
except subprocess.CalledProcessError as e:
print(f"Rsync failed with error: {e.stderr.decode()}")
return False
except subprocess.TimeoutExpired:
print("Rsync timed out")
return False
For production environments, consider these additional measures:
# 1. Strict host key checking (avoids MITM attacks)
ssh_options = "-o StrictHostKeyChecking=yes -o UserKnownHostsFile=/path/to/known_hosts"
# 2. Connection hardening
ssh_options += " -o ConnectTimeout=15 -o ServerAliveInterval=10"
# 3. Full non-interactive command
rsync_cmd = f"rsync -azP -e 'ssh {ssh_options}' --delete /local/path user@host:/remote/path"
When troubleshooting, test SSH connectivity first:
ssh -v -o BatchMode=yes user@host
Common exit codes to handle:
- 5: Error in starting rsync on remote machine
- 12: Connection timeout
- 255: SSH failure