When using rsync for backups or file transfers, we often need to handle hidden files (dotfiles) differently based on their location in the directory structure. The challenge arises when we want to:

• Exclude all dotfiles and dot-directories only in the root of our transfer
• Preserve hidden files in subdirectories that we're actively syncing

The obvious approach rsync -a --exclude=".*" source/ dest/ fails because:

# This excludes ALL dotfiles everywhere
rsync -a --exclude=".*" source/ dest/

Similarly, attempts like --exclude="./.*" don't work because rsync's pattern matching interprets paths differently than shell globbing.

The proper solution involves using rsync's filter rules with the -f option:

rsync -av \
    -f "- .*" \        # Exclude all top-level dotfiles
    -f "+ */" \        # Include all directories
    -f "+ */.??*" \    # Include dotfiles in subdirectories
    -f "+ *" \         # Include all non-dotfiles
    source/ dest/

Let's break down what each filter does:

1. - .* - Exclude all patterns matching .* (dotfiles) at the root
2. + */ - Include all directories (so we recurse into them)
3. + */.??* - Include dotfiles in subdirectories (. followed by at least 2 chars)
4. + * - Include all remaining files (non-dotfiles)

For those who prefer a simpler approach without filter rules:

rsync -av --exclude="/.*" source/ dest/

The leading slash in /.* tells rsync to only match dotfiles in the transfer root.

Always verify your exclude patterns with --dry-run first:

rsync -avn --exclude="/.*" source/ dest/

Here's how I structure my backup script to preserve Git repos while excluding top-level config:

#!/bin/bash
BACKUP_DIR="/mnt/backup"

rsync -av \
    --delete \
    -f "- .*" \
    -f "+ */" \
    -f "+ */.??*" \
    -f "+ *" \
    ~/projects/ $BACKUP_DIR/projects/

When using rsync for backups or file synchronization, many developers encounter a common frustration: the --exclude=".*" pattern affects all directories in the transfer, not just the top-level directory. This behavior becomes problematic when you want to:

• Exclude /.git but preserve /project/.git
• Skip /.env but keep /config/.env
• Ignore /.DS_Store but transfer /photos/.DS_Store

Rsync's filtering system provides the precise control we need. The key is to combine exclusion and inclusion patterns with proper anchoring:

rsync -a \
    --exclude="/.*" \
    --include="*/" \
    --include="**/.*" \
    --exclude="*" \
    source/ destination/

Let's break down what this does:

• --exclude="/.*" - Excludes dotfiles/directories at root level
• --include="*/" - Allows directory traversal
• --include="**/.*" - Includes dotfiles in subdirectories
• --exclude="*" - Excludes everything else (but previous includes take precedence)

Here are some real-world scenarios with specific solutions:

# Case 1: Exclude top-level node_modules but keep others
rsync -a \
    --exclude="/node_modules" \
    --include="*/" \
    --include="**/node_modules/**" \
    --exclude="*" \
    app/ backup/

# Case 2: Skip top-level .git but preserve project .git folders
rsync -a \
    --exclude="/.git" \
    --include="repos/*/.git/**" \
    --exclude="*" \
    code/ archive/

Always verify your patterns using rsync's --dry-run and -v (verbose) flags:

rsync -anv --exclude="/.*" --include="*/" --include="**/.*" source/ dest/ | grep -E '(^deleting|^sending)'

This will show you exactly what would be transferred or skipped without making any changes.

Complex filter rules can impact performance. For large directory trees:

• Place more specific patterns first
• Use --filter with rule files for complex scenarios
• Consider --prune-empty-dirs to clean up unnecessary directories

# Using a filter file for better maintainability
echo "- /.*" > rsync-filter
echo "+ */" >> rsync-filter
echo "+ **/.*" >> rsync-filter
echo "- *" >> rsync-filter

rsync -a --filter="merge rsync-filter" source/ dest/