When transferring web files between servers, hidden configuration files like .htaccess, .env, and .gitignore are often crucial yet frustratingly omitted by default SCP behavior. The standard recursive copy command:

scp -rp src/ user@server:dest/

fails to capture these dotfiles, leaving applications broken after deployment.

The behavior stems from Unix shell expansion rules - wildcard patterns like * explicitly exclude dotfiles by design for safety. SCP inherits this behavior when processing directory contents.

Here are three production-tested approaches:

1. Rsync Alternative

The most robust method uses rsync which handles hidden files properly:

rsync -avz --include='.*' --exclude='*' src/ user@server:dest/
rsync -avz src/ user@server:dest/

First command copies only hidden files, second copies everything else.

2. Tar Pipe Technique

For systems without rsync, create a tar archive including hidden files:

tar czf - -C src/ . | ssh user@server "tar xzf - -C dest/"

3. SCP Workarounds

If you must use SCP, these patterns work:

scp -rp src/{*,.*} user@server:dest/

Or explicitly list hidden files:

scp -rp src/.htaccess src/.env user@server:dest/

Watch for these special cases:

• The . and .. entries when using .* patterns
• Permission preservation with -p flag
• Symbolic link handling with -L if needed

For CI/CD pipelines, use this bash function:

transfer_all_files() {
  local src=$1
  local dest=$2
  rsync -avz --no-perms --chmod=Du=rwx,Dg=rx,Do=rx,Fu=rw,Fg=r,Fo=r \
    --include='.*' --exclude='lost+found' \
    "$src" "$dest"
}

---

When transferring web files like .htaccess, .env, or Git's .gitignore, you'll notice SCP's default behavior ignores dotfiles. This isn't a bug - it's by design in many Unix tools to prevent accidental operations on system directories (. and ..).

The common pattern:

scp -rp src/ user@server:dest/

misses hidden files because:

• Wildcard expansion happens before SCP sees the files
• The -r flag doesn't override dotfile exclusion

Method 1: Explicit File Listing

Best for known files:

scp -rp src/{.htaccess,.env,config.php} user@server:dest/

Method 2: Rsync Alternative

For complete directory mirroring:

rsync -avz --include='.*' src/ user@server:dest/

Method 3: Tar Pipe (Most Reliable)

Works with complex permissions:

cd src && tar czf - . | ssh user@server "tar xzf - -C /path/to/dest"

Watch for:

• Symlinks (add -L to rsync or -h to tar)
• Special permissions (preserve with -p)
• Maximum command length (tar avoids this)

Method	Speed	Reliability
SCP with wildcards	Fast	Medium
Rsync	Variable	High
Tar pipe	Slowest	Highest

For deployment scripts:

#!/bin/bash
SRC="/var/www/html/"
DEST="user@production:/var/www/html/"
EXCLUDE="--exclude=*.tmp --exclude=*.bak"

rsync -azP --delete $EXCLUDE --include='.*' $SRC $DEST