When duplicating a Magento installation (approximately 15,000 files totaling 50MB) on a server with modest specs (2.4GHz Xeon, 2GB RAM), the standard cp -a command can become painfully slow. This is primarily due to:

• Overhead of individual file operations
• Inode creation for each file
• Lack of parallel processing
• SSH session limitations

1. rsync with Compression and Delta Transfer

rsync -avz --progress source/ destination/

Key advantages:

• -z: Enables compression during transfer
• --progress: Shows real-time transfer status
• Only transfers changed portions of files (delta algorithm)

2. tar Pipe for Single Archive Transfer

# Local to remote
tar cf - source | ssh user@remote "cd /path/to/destination && tar xf -"

# Remote to remote (via local)
ssh user@source-server "tar cf - /path/to/source" | ssh user@dest-server "cd /path/to/dest && tar xf -"

3. Parallel rsync for Multi-Core Utilization

# Install parallel if needed
sudo apt-get install parallel

# Run parallel rsync
cd source && find . -type f | parallel -j 8 rsync -a {} destination/{}

Using mbuffer for Network Optimization

# On source server
tar cf - source | mbuffer -m 1G | ssh user@remote "mbuffer -m 1G | tar xf - -C destination"

# Installation:
sudo apt-get install mbuffer   # Debian/Ubuntu
sudo yum install mbuffer       # RHEL/CentOS

Filesystem-Specific Solutions

For ZFS/BTRFS filesystems:

# ZFS snapshot clone
zfs snapshot pool/source@backup
zfs clone pool/source@backup pool/destination

# BTRFS send/receive
btrfs subvolume snapshot -r source source-snap
btrfs send source-snap | ssh user@remote "btrfs receive /path/to/destination"

Test results for 15,000 files (50MB total) on 2.4GHz Xeon:

When migrating Magento installations (typically containing 15,000+ small files totaling ~50MB), standard cp -a operations become inefficient due to:

• Metadata preservation overhead
• Sequential file processing
• Lack of compression during transfer

Tested on Xeon 2.4GHz/2GB RAM server:

rsync -az --progress --delete \
    -e "ssh -T -c aes128-gcm@openssh.com -o Compression=no" \
    /path/to/source/ user@remote:/path/to/dest/

Key parameters:

• -a: Archive mode (preserves permissions)
• -z: Compression during transfer
• --delete: Remove extraneous files
• SSH cipher optimization for small files

For environments without rsync:

(cd /path/to/source && tar -cf - .) | \
    ssh user@remote "cd /path/to/dest && tar -xpf -"

After copy operations:

# Update base URLs
mysql -e "UPDATE core_config_data SET value='http://new.url/' \
    WHERE path LIKE 'web/%/base_url';"

# Clear cache
rm -rf var/cache/* var/page_cache/*