How to Generate Patch Files with Rsync Instead of Direct File Transfers: A Diff-Based Approach


10 views

When synchronizing files between servers, developers often need more granular control than simple file copying. The ability to preview changes as diff patches before actual transfer helps in:

  • Auditing security-sensitive modifications
  • Verifying deployment changes in CI/CD pipelines
  • Maintaining change documentation

While rsync doesn't natively output unified diffs, we can combine it with standard Unix tools:

rsync -avn --out-format="%f" source/ destination/ | \
while read file; do
    diff -u "source/$file" "destination/$file" >> changes.patch
done

For a more robust solution handling edge cases:

#!/bin/bash

# Configurable parameters
SOURCE_DIR="/path/to/source"
DEST_DIR="/path/to/dest"
PATCH_FILE="$(date +%Y%m%d)-changes.patch"

# Generate differential patch
rsync -avn --delete --out-format="%n" "$SOURCE_DIR/" "$DEST_DIR/" | \
grep -v '/$' | \
while read -r file; do
    if [ -f "$SOURCE_DIR/$file" ] && [ -f "$DEST_DIR/$file" ]; then
        diff -u "$DEST_DIR/$file" "$SOURCE_DIR/$file" >> "$PATCH_FILE"
    elif [ ! -f "$DEST_DIR/$file" ]; then
        echo "+++ NEW FILE: $file" >> "$PATCH_FILE"
        cat "$SOURCE_DIR/$file" | sed 's/^/+/' >> "$PATCH_FILE"
    fi
done

echo "Patch generated: $PATCH_FILE"

This technique proves particularly valuable when:

  • Config file management: Review exactly which lines differ in nginx/Apache configurations
  • Code deployments: Verify changes between development and staging environments
  • Compliance tracking: Maintain audit trails of file modifications

For large directory trees, consider these optimizations:

# Use checksum comparison for accuracy
rsync -acn --out-format="%f" source/ dest/

# Parallel processing for speed
find source/ -type f -print0 | parallel -0 -j8 diff -u {} dest/{/}

When synchronizing files between servers, many developers want to preview changes before executing the actual transfer. While rsync's -n (dry run) flag shows which files will be modified, it doesn't reveal the content differences. Here's how to generate patch files that document these changes.

Rsync wasn't primarily designed as a diff tool, but we can combine it with other Unix utilities to achieve patch generation:


rsync -avn --out-format="%f" source/ destination/ | \
while read file; do 
    diff -u "source/$file" "destination/$file" >> changes.patch
done

For production use, consider this enhanced version that handles edge cases:


#!/bin/bash

SOURCE_DIR="/path/to/source/"
DEST_DIR="/path/to/dest/"
PATCH_FILE="rsync_changes_$(date +%Y%m%d).patch"

# First get the list of changed files
changed_files=$(rsync -aic --out-format="%f" "$SOURCE_DIR" "$DEST_DIR")

# Generate unified diff for each changed file
echo "" > "$PATCH_FILE"  # Clear existing patch file
while IFS= read -r file; do
    if [ -f "$SOURCE_DIR$file" ] && [ -f "$DEST_DIR$file" ]; then
        diff -u "$SOURCE_DIR$file" "$DEST_DIR$file" >> "$PATCH_FILE"
    fi
done <<< "$changed_files"

To apply the changes later:

patch -p0 < changes.patch

For binary files or more complex comparisons:


rsync -avc --dry-run --itemize-changes source/ dest/ | \
grep '^[><]' | awk '{print $2}' | \
xargs -I {} diff -u source/{} dest/{} > binary_changes.patch
  • This method works best for text files
  • Binary files require special handling
  • Permission changes won't be captured in the patch
  • New files will show as full diffs

For Git repositories, you can combine rsync with git diff:


rsync -avn --delete --exclude='.git' source/ dest/ | \
while read file; do
    (cd source && git diff --no-index -- "$file" "../dest/$file")
done > git_changes.patch