When dealing with multiple source and destination directory pairs in rsync, we often face a common dilemma: should we run separate rsync commands for each pair or find a way to consolidate them? Running individual commands becomes impractical when dealing with numerous directory pairs (N to N mapping) where each source has a unique destination.

Here's how to efficiently sync multiple source directories to their respective destinations in a single rsync command:

rsync -av \
  /home/me/source1/ /home/someoneelse/dest1/ \
  /mnt/somedisk/source2/ /home/someoneelse/dest2/

The critical points to remember:

• Source directories must end with trailing slashes
• Destination directories must not have trailing slashes (except the last one)
• Order matters: sources and destinations must be paired sequentially

For N directories where N is large or variable, we can use shell scripting:

#!/bin/bash
declare -A dir_pairs=(
    ["/home/me/source1"]="/home/someoneelse/dest1"
    ["/mnt/somedisk/source2"]="/home/someoneelse/dest2"
    # Add more pairs as needed
)

for src in "${!dir_pairs[@]}"; do
    rsync -av "${src}/" "${dir_pairs[$src]}/"
done

When dealing with complex synchronization patterns, consider these additional techniques:

# Using find with xargs for dynamic directory lists
find /data/sources -maxdepth 1 -type d -name "project_*" | \
xargs -I {} rsync -av {}/ /backup/destination/{##}/

# Parallel execution for better performance
parallel -j 4 rsync -av {1}/ {2}/ ::: /src1 /src2 ::: /dest1 /dest2

1. Mismatched directory pairs (always verify your paths)
2. Forgetting trailing slashes on source directories
3. Running without --dry-run first when testing
4. Not preserving permissions (add -p flag when needed)

For large-scale operations, these optimizations can help:

# Using compression for remote transfers
rsync -avz source/ user@remote:destination/

# Limiting bandwidth when needed
rsync -av --bwlimit=1000 source/ destination/

# Using checksum only when necessary
rsync -avc source/ destination/

When dealing with complex file synchronization tasks, developers often need to map multiple source directories to corresponding destination paths efficiently. The standard rsync command handles single source-to-destination pairs well, but requires special handling for multiple directory mappings.

For the specific case mentioned with two directory pairs:

rsync -av /home/me/source1/ /home/someoneelse/dest1/ && \
rsync -av /mnt/somedisk/source2/ /home/someoneelse/dest2/

For more flexible N-to-N directory synchronization, we can use arrays in bash:

#!/bin/bash

# Define source and destination arrays
sources=(
    "/home/me/source1"
    "/mnt/somedisk/source2"
    "/var/log/source3"
)

dests=(
    "/home/someoneelse/dest1"
    "/home/someoneelse/dest2"
    "/backup/logs"
)

# Verify array lengths match
if [ ${#sources[@]} -ne ${#dests[@]} ]; then
    echo "Error: Source and destination counts don't match"
    exit 1
fi

# Perform sync
for i in "${!sources[@]}"; do
    rsync -av --delete "${sources[$i]}/" "${dests[$i]}/"
done

For systems with multiple cores, we can significantly improve performance using GNU parallel:

parallel -j 4 rsync -av --delete {1}/ {2}/ ::: "${sources[@]}" ::: "${dests[@]}"

Add robust error handling and logging to your sync script:

LOG_FILE="/var/log/rsync_multi.log"

for i in "${!sources[@]}"; do
    echo "Syncing ${sources[$i]} to ${dests[$i]}" >> "$LOG_FILE"
    if ! rsync -av --delete "${sources[$i]}/" "${dests[$i]}/" >> "$LOG_FILE" 2>&1; then
        echo "[ERROR] Failed to sync ${sources[$i]}" >> "$LOG_FILE"
        # Add your error recovery logic here
    fi
done

For maintainability with many directory pairs, consider a config file approach:

# sync_config.cfg
/home/me/source1 /home/someoneelse/dest1
/mnt/somedisk/source2 /home/someoneelse/dest2
/var/log/source3 /backup/logs

Then process it with:

while read -r src dest; do
    [[ -z "$src" || "$src" == \#* ]] && continue
    rsync -av --delete "$src/" "$dest/"
done < sync_config.cfg

• Always test with --dry-run first
• Consider using --checksum for critical data
• For remote sync, setup SSH keys for password-less operation
• Monitor --progress for large transfers