Many developers encounter corrupt packet
errors when using rsync to transfer large files, especially in Windows-to-Linux environments. The error typically manifests as:
rsync: writefd_unbuffered failed to write 4092 bytes to socket [sender]: Broken pipe (32)
rsync: connection unexpectedly closed (11337 bytes received so far) [sender]
rsync error: unexplained error (code 255) at io.c(605) [sender=3.0.9]
This issue usually stems from one or more of these factors:
- SSH session timeouts during long transfers
- Network instability or packet loss
- Buffer size mismatches between client and server
- Windows-specific path handling issues in Cygwin
- Server-side resource limitations
1. Adjust SSH Keepalive Settings
Add these parameters to your SSH command:
rsync -ravq -e "ssh -o TCPKeepAlive=yes \
-o ServerAliveInterval=15 \
-o ServerAliveCountMax=3 \
-o ConnectTimeout=60" \
--delete ./local_dir user@$SERVER:/dest_dir
2. Use Rsync's Built-in Retry Mechanism
Add these rsync-specific options:
rsync --partial --progress --timeout=30 \
--rsh="ssh -o TCPKeepAlive=yes" \
--bwlimit=10000 \
./local_dir user@$SERVER:/dest_dir
3. Increase TCP Window Size
For very large files, adjust the TCP window size:
rsync -avz --bwlimit=0 --block-size=16384 \
--rsh="ssh -o TCPKeepAlive=yes" \
./local_dir user@$SERVER:/dest_dir
Network Quality Verification
Run a network test before transferring:
ping -t $SERVER
# And/or:
mtr --report $SERVER
Alternative Transfer Methods
If rsync continues to fail, consider these alternatives:
# Using tar over ssh:
tar czf - ./local_dir | ssh user@$SERVER "tar xzf - -C /dest_dir"
# Using scp with resume capability:
scp -o ServerAliveInterval=15 -C ./largefile user@$SERVER:/dest_dir
- On the server:
sudo sysctl -w net.ipv4.tcp_window_scaling=1
- On Windows: Disable power saving features for network adapters
- For Cygwin: Ensure you're using the latest version with updated SSH packages
The "corrupt packet" error in rsync typically occurs during large file transfers when the SSH connection times out or becomes unstable. From your error message:
rsync: writefd_unbuffered failed to write 4092 bytes to socket [sender]: Broken pipe (32)
rsync: connection unexpectedly closed (11337 bytes received so far) [sender]
This suggests network-level transmission problems, specifically with the SSH layer underneath rsync.
The immediate solution is to enhance SSH's connection resilience. Your current command already includes some timeout parameters, but we can improve it:
rsync -rav --progress --partial -e "ssh -o TCPKeepAlive=yes -o ServerAliveInterval=15 -o ServerAliveCountMax=3 -o ConnectTimeout=60" --delete ./local_dir user@$SERVER:/dest_dir
Key improvements:
- Added
TCPKeepAlive=yes
to maintain TCP connections - Increased
ServerAliveInterval
to 15 seconds - Set
ConnectTimeout
to 60 seconds - Added
--partial
to resume interrupted transfers
For extremely large files, consider using rsync's native protocol instead of SSH tunneling:
rsync -rav --progress --partial --rsync-path="sudo rsync" --port=873 --delete ./local_dir rsync://user@$SERVER/dest_dir
This requires setting up rsyncd on the server side but eliminates SSH overhead.
Since you're using Cygwin on Windows, try these additional optimizations:
export CYGWIN="tty notitle glob"
rsync -8 -rav --bwlimit=5000 --outbuf=N -e "ssh -o IPQoS=throughput" --delete ./local_dir user@$SERVER:/dest_dir
The -8
flag forces 8-bit clean transmission, while --outbuf=N
disables output buffering.
To pinpoint the exact failure point, run rsync with verbose debugging:
rsync -vvvv --progress -e "ssh -v" --delete ./local_dir user@$SERVER:/dest_dir
This will generate detailed output showing exactly where the transfer fails.
If rsync continues to fail, consider these alternatives:
# Using tar over ssh
tar czf - ./local_dir | ssh user@$SERVER "tar xzf - -C /dest_dir"
# Using lftp mirror
lftp -e "mirror -R --parallel=4 ./local_dir /dest_dir; quit" sftp://user:$PASSWORD@$SERVER