When debugging rsync operations or maintaining audit trails, we often need two critical pieces of information simultaneously: precise timestamps and detailed file transfer lists. The default rsync behavior forces a choice between these logging formats, but with proper configuration we can get both.

Rsync offers three primary logging approaches:

• Standard output redirection (>> file.log) captures the progress output
• --log-file parameter enables structured logging
• --log-file-format allows customizing the output format

We can leverage both standard output and custom log formats simultaneously:

rsync -avz --progress --delete \
--log-file=/var/log/rsync_combined.log \
--log-file-format="%t [%i] %f %b %l" \
/web/path/public_html/ $newhost:/web/path/public_html \
>> /var/log/rsync_transfers.log

The key to effective logging lies in understanding format placeholders:

• %t: Current timestamp
• %i: Transfer item information
• %f: Filename
• %b: Number of bytes transferred
• %l: Length of file (for non-regular files)

For more comprehensive logging, consider this enhanced version:

rsync -avz --progress --stats --human-readable --delete \
--log-file=/var/log/rsync_full_%Y-%m-%d.log \
--log-file-format="%t [%n] [%o] %f (%b bytes)" \
--out-format="%t %f %''b %l" \
/src/ user@host:/dest/ \
2>&1 | tee -a /var/log/rsync_console.log

Here's sample output from both logs:

# Custom log format output
2023/11/15 14:30:22 [sender] [to-check] file1.txt (4096 bytes)
2023/11/15 14:30:22 [receiver] [created] dir/subfile.jpg (102400 bytes)

# Progress output
2023/11/15 14:30:22 file1.txt 4,096 4,096 100%
2023/11/15 14:30:22 dir/subfile.jpg 102,400 102,400 100%

For production systems, implement log rotation with logrotate:

/var/log/rsync_*.log {
    daily
    missingok
    rotate 30
    compress
    delaycompress
    notifempty
    create 640 root adm
    sharedscripts
    postrotate
        /usr/lib/rsyslog/rsyslog-rotate
    endscript
}

If logs aren't appearing as expected:

• Verify write permissions on log directories
• Check for SELinux/AppArmor restrictions
• Ensure sufficient disk space for logs
• Test with simpler --log-file-format strings first

For systems using systemd, consider journal logging:

rsync ... | systemd-cat -t rsync-transfer
journalctl -t rsync-transfer -f

---

Many sysadmins face this common rsync logging challenge - you either get detailed file transfer information or proper timestamps, but not both simultaneously. This becomes particularly frustrating when troubleshooting sync operations or auditing changes.

The standard rsync output with -v flag shows transferred files but lacks timestamps:

rsync -avz --progress source/ destination/ >> sync.log
# Output shows files but no timestamps
error_log  5740980 100%   36.98MB/s    0:00:00 (xfer#1, to-check=1405/1524)

While --log-file-format provides timestamps, it omits file details:

rsync -avz --log-file=sync.log --log-file-format="%t %o %f [%b]" source/ dest/
2012/01/03 17:30:05 [10505] Total bytes sent: 2345435

Combine both approaches using tee and custom logging:

rsync -avz --progress --delete \
    --log-file=/var/log/rsync_details.log \
    --log-file-format="%t %f (%l bytes)" \
    /source/path/ dest:/target/path/ \
    | tee -a /var/log/rsync_transfers.log

This gives you two complementary logs:

• rsync_details.log: Timestamped file-level records
• rsync_transfers.log: Full transfer progress output

Customize the log format using these placeholders:

%t - Timestamp
%f - Filename
%l - File length
%b - Bytes transferred
%o - Operation (send/receive)
%P - Module path

Example format string for comprehensive logging:

--log-file-format="%t [%o] %P%f (%l bytes) transferred: %b bytes"

For production systems, consider this wrapper script:

#!/bin/bash
LOG_DIR="/var/log/rsync"
TIMESTAMP=$(date +%Y%m%d_%H%M%S)
LOG_FILE="$LOG_DIR/rsync_$TIMESTAMP.log"

rsync -avz --progress --delete \
    --log-file="$LOG_FILE.details" \
    --log-file-format="[%t] %o %f (%l) %b" \
    /source/path/ user@host:/target/path/ \
    | tee -a "$LOG_FILE.transfer"
    
gzip "$LOG_FILE".*  # Compress logs after completion

Add this to your /etc/logrotate.d/rsync:

/var/log/rsync/*.log {
    daily
    missingok
    rotate 30
    compress
    delaycompress
    notifempty
    sharedscripts
    postrotate
        /usr/bin/systemctl reload rsync > /dev/null
    endscript
}