When managing files on remote servers, duplicating directories is a fundamental operation. Whether you're creating backups, setting up staging environments, or migrating data, SSH provides several robust methods to accomplish this task efficiently.
The most efficient approach for large directories is rsync, which supports delta transfers and preserves permissions:
rsync -avz -e ssh /path/to/source/ username@remote_host:/path/to/destination/
Key flags explanation:
-a: Archive mode (recursive, preserves permissions)
-v: Verbose output
-z: Compression during transfer
-e: Specify remote shell (SSH)
For systems without rsync, this method works well:
tar czf - /path/to/source | ssh username@remote_host "tar xzf - -C /path/to/destination"
This creates a compressed tar archive and pipes it directly through SSH to be extracted on the remote server.
While scp typically copies files, you can use it recursively:
scp -r -p /path/to/source username@remote_host:/path/to/destination
Note: scp is being deprecated in favor of sftp or rsync in modern systems.
To properly handle symlinks during duplication, add these rsync options:
rsync -avzL -e ssh /path/to/source/ username@remote_host:/path/to/destination/
The -L flag resolves symbolic links and copies the actual files.
For large directories:
- Use --partial with rsync to resume interrupted transfers
- Consider --bwlimit to control bandwidth usage
- Exclude unnecessary files with --exclude patterns
For regular duplication tasks, set up SSH key authentication to avoid password prompts:
ssh-keygen -t ed25519
ssh-copy-id username@remote_host
When working with remote servers, directory duplication often becomes necessary during deployment, backups, or testing scenarios. SSH provides several powerful methods to accomplish this task efficiently while maintaining file permissions and structure.
For most use cases, rsync over SSH offers the best combination of speed and reliability:
rsync -avz -e ssh user@remote_host:/path/to/source/directory /path/to/destination
Key flags explanation:
- -a: Archive mode (preserves permissions, ownership, timestamps)
- -v: Verbose output
- -z: Compression during transfer
- -e: Specifies the remote shell to use (SSH in this case)
When dealing with particularly large directories, combining tar with SSH can be more efficient:
ssh user@remote_host "tar cf - /path/to/source" | tar xf - -C /path/to/destination
This creates a tar archive on the remote server and pipes it directly to the local extraction.
For simpler cases where you just need a quick copy:
scp -r user@remote_host:/path/to/source /path/to/destination
Note that scp doesn't preserve all metadata as reliably as rsync.
To maintain hard links, extended attributes, and other special file properties:
rsync -avzXA -e ssh --hard-links user@remote_host:/source/ /destination/
For scripting purposes, set up passwordless SSH authentication:
ssh-keygen -t rsa ssh-copy-id user@remote_host
Then you can run duplication commands without manual password entry.
After duplication, verify integrity with:
rsync -avzcn -e ssh /destination/ user@remote_host:/source/
The -n flag performs a dry run, -c adds checksum verification.
For large transfers over high-latency connections:
- Use rsync's --partial flag for resumable transfers
- Adjust SSH compression with -C (may help or hurt depending on content)
- Consider --bwlimit to avoid saturating network bandwidth
Permission denied errors often occur due to:
rsync -avz --rsync-path="sudo rsync" -e ssh user@host:/source/ /dest/
For space constraints during transfer:
rsync -avz -e ssh --max-size=1G user@host:/source/ /dest/