How to Monitor Individual File Transfer Progress in Rsync Like Wget


2 views

When transferring massive files (like 20GB datasets) over unstable connections, rsync's default --progress behavior can be frustrating. Unlike wget which shows continuous progress per file, rsync often only displays final transfer statistics - leaving you guessing whether the transfer is still active or has stalled.

For detailed per-file monitoring, combine these flags:

rsync -avh --progress --stats source_file destination_path

Key parameters explanation:

  • -v: Verbose output
  • -h: Human-readable file sizes
  • --progress: Show transfer progress
  • --stats: Display comprehensive transfer statistics

For real-time visualization similar to wget, try these approaches:

# Method 1: Detailed per-file progress
rsync -avh --info=progress2 --info=name0 source dest

# Method 2: With transfer speed and ETA
rsync -avh --progress --human-readable --partial source dest

When dealing with spotty networks, these additional parameters help:

rsync -avh --progress --partial --append-verify \
--timeout=30 --contimeout=40 source dest

This combination:

  • Resumes interrupted transfers (--partial)
  • Verifies file integrity (--append-verify)
  • Sets reasonable timeouts

For even better progress visualization, pipe through pv:

tar cf - bigfile | pv -s 20G | ssh user@host "tar xf - -C /target"

Transferring a database backup with full visibility:

rsync -avh --progress --stats \
--info=progress2 --info=name0 \
--partial --append-verify \
database_backup.sql user@remote:/backups/

This gives you:

  • Continuous progress bar
  • Transfer speed
  • Percentage complete
  • Time remaining estimate

When dealing with large file transfers (like our 20GB example) over potentially unstable connections, the default --progress flag in rsync can leave you guessing about transfer status. The fundamental issue isn't just about seeing progress - it's about getting granular, real-time feedback per file during the transfer process.

Combine these flags for best results:

rsync -avh --progress --stats source destination

The magic happens through multiple complementary flags:

  • --progress: Shows per-file transfer progress (not just completion)
  • -v: Increases verbosity
  • -h: Human-readable output
  • --stats: Provides transfer summary

Here's what you'll see during transfer:

largefile.iso
          25%   5.01GB   50.23MB/s    0:03:12

This shows:

  • Current progress percentage
  • Amount transferred
  • Current transfer speed
  • Estimated time remaining

For even better monitoring:

rsync -avh --progress --stats --partial --inplace source destination

Key additions:

  • --partial: Keeps partially transferred files
  • --inplace: Updates files directly (better for large files)

For single-file transfers, combine rsync with pv:

pv largefile.iso | rsync --archive --inplace --progress --partial --partial-dir=.rsync-partial - destination/

Add these to handle unstable connections:

rsync -avh --progress --stats --partial --partial-dir=.rsync-partial \
    --timeout=30 --contimeout=60 --bwlimit=10000 source destination

This configuration:

  • Sets connection timeouts
  • Limits bandwidth (10MB/s in this case)
  • Creates a dedicated directory for partial transfers

Install this Perl script for graphical progress:

rsync -avz --progress source dest | rsyncprogress