“Rsync Skipping Main Directory: Debugging Remote File Transfer Issues Between Windows and macOS”


11 views

When running rsync between different operating systems, you might encounter this puzzling behavior where rsync appears to skip the main directory entirely. Here's what's happening under the hood:

$ rsync -avz -e ssh user@remote:/source/path/ ./destination/
receiving file list ... done
./
sent 26 bytes  received 68 bytes  17.09 bytes/sec
total size is 0  speedup is 0.00

The issue stems from how rsync handles directory structures across different platforms. The key factors are:

  • Windows and macOS handle directory permissions differently
  • The trailing slash in the source path matters significantly
  • Default rsync behavior varies between OS combinations

Here are several approaches that work for Windows-to-macOS transfers:

1. Explicit Directory Creation

Force create the destination directory structure first:

mkdir -p ./export
rsync -avz --ignore-existing user@remote:/source/path/ ./export/

2. Using --recursive Flag

Explicitly tell rsync to handle directories:

rsync -rvz -e ssh user@remote:/source/path/ ./export/

3. Alternative Syntax

Try this pattern which often works better across platforms:

rsync -avz user@remote:/source/path/. ./export/

Add these flags to understand what's happening:

rsync -avz --progress --stats --dry-run -e ssh user@remote:/source/path/ ./export/

For permission-related skips, consider:

rsync -avz --no-perms --no-owner --no-group user@remote:/source/path/ ./export/

This command consistently works across my Windows-to-macOS setup:

rsync -rtvz --progress --delete \
    --exclude='.DS_Store' \
    -e ssh user@remote:/Library/WebServer/sites/staging/app1/ \
    ./export/

When running rsync between Windows and macOS systems for remote deployment testing, you might encounter directory skipping behavior even when using standard flags like -a or -v. Here's what's happening in your specific case:

$ rsync -avz -e ssh user@remote:/path/ ./local
receiving file list ... done
./
sent 26 bytes  received 68 bytes  17.09 bytes/sec
total size is 0  speedup is 0.00

Rsync skips directories primarily due to three reasons:

  • Permissions mismatch between source and destination
  • Missing trailing slash in source path
  • Filesystem differences between Windows and Unix-like systems

Try these command variations to resolve the issue:

# Solution 1: Force directory creation with -d
rsync -avzd -e ssh user@remote:/path/to/source/ ./destination/

# Solution 2: Include --delete and --prune-empty-dirs
rsync -avz --delete --prune-empty-dirs -e ssh user@remote:/source/ ./dest/

# Solution 3: Explicitly include all files
rsync -avz --include='*/' --include='*' --exclude='*' -e ssh user@remote:/source/ ./dest/

When syncing between Windows and Unix systems:

# Handle line endings and permissions:
rsync -avz --no-perms --no-owner --no-group --chmod=ugo=rwX -e ssh user@remote:/src/ ./dst/

# For Cygwin/Windows builds, add:
rsync -rtvz --rsync-path="mkdir -p /remote/path && rsync" user@remote:/src/ /cygdrive/c/local/path/

Add these flags for better diagnostics:

rsync -avv --progress --stats -e ssh user@remote:/src/ ./dst/

# Check what would happen without actual transfer:
rsync -avzn -e ssh user@remote:/src/ ./dst/

For complex directory structures with empty directories:

rsync -avz --relative --delete -e ssh user@remote:/base/path/./subdir ./local/

# Alternative using find and rsync:
ssh user@remote 'cd /src && find . -type d -print0' | xargs -0 mkdir -p
rsync -avz -e ssh user@remote:/src/ ./dst/