When using SCP for remote directory transfers between servers, a common frustration occurs when the target directory already exists. The default behavior creates an undesired nested directory structure instead of overwriting contents.

Consider this standard SCP command:

scp -prq source_server:source_dir/ target_server:target_dir/

Two scenarios emerge:

• Target doesn't exist: Creates target_dir with source_dir contents
• Target exists: Creates source_dir inside target_dir (undesired nesting)

Method 1: Wildcard Approach (Preferred)

The most reliable single-command solution:

scp -prq source_server:source_dir/* target_server:target_dir/

Key advantages:

• Works whether target exists or not
• Properly handles hidden files when using -p flag
• Maintains original permissions and timestamps

Method 2: Fallback Script

For maximum compatibility:

#!/bin/bash
# Attempt wildcard method first
if ! scp -prq source_server:source_dir/* target_server:target_dir/ 2>/dev/null; then
    # Fallback to standard method
    scp -prq source_server:source_dir target_server:target_dir
fi

Handling Large Transfers

For improved performance with massive directories:

rsync -azP --delete source_server:source_dir/ target_server:target_dir/

Automating Across Multiple Servers

Example for bulk operations:

for server in server{1..10}; do
    scp -prq source_server:source_dir/* ${server}:target_dir/
done

The behavior stems from how UNIX path resolution works. When specifying a target directory:

• SCP must preserve the source directory name if the exact target path exists
• This prevents accidental overwrites in the filesystem

• Permission errors: Ensure write access to target parent directory
• Hidden files missing: Always include -p flag
• Partial transfers: Consider using rsync with --partial flag

---

When using SCP to transfer directories between remote servers, you'll encounter a specific behavior that often causes frustration:

scp -prq server1:source_dir server2:target_dir

The command works perfectly when target_dir doesn't exist - it creates it and copies all contents. However, when target_dir exists, SCP creates source_dir inside target_dir instead of overwriting it.

SCP's behavior stems from its design philosophy of being non-destructive. When the target exists:

• For files: SCP will overwrite by default
• For directories: SCP creates a subdirectory to preserve existing content

Method 1: Copy Contents Instead of Directory

The most reliable solution is to copy the contents rather than the directory itself:

scp -prq server1:source_dir/* server2:target_dir/

This ensures all files are copied directly into target_dir, overwriting existing files with the same names.

Method 2: Fallback Approach

For a more robust solution that handles both cases, use this script:

#!/bin/bash

# First try copying contents
scp -prq server1:source_dir/* server2:target_dir/ 2>/dev/null

# If failed (target directory might not exist), copy the whole directory
if [ $? -ne 0 ]; then
    scp -prq server1:source_dir server2:target_dir
fi

Using rsync Instead

For more control, consider rsync which handles this case natively:

rsync -avz --delete server1:source_dir/ server2:target_dir/

The trailing slashes are crucial - they ensure contents are copied into the target directory rather than creating a subdirectory.

SSH Pipeline Approach

For large directories, this method can be more efficient:

ssh server1 "tar -czf - source_dir" | ssh server2 "tar -xzf - -C /path/to/parent && mv /path/to/parent/source_dir target_dir"

When implementing any solution:

• Permission requirements on both servers
• Network reliability for large transfers
• Error handling for partial transfers
• Verification of copied data integrity