How to Exclude All Directories Matching a Pattern in Rsync: A Complete Guide for Developers


2 views

When using rsync for backups or file synchronization, excluding directories that match specific patterns is a common requirement. The original poster encountered this exact issue when trying to exclude all directories under /root that start with "branch".

/root
    /branch1
    /branch2
    /branch3
    /other
    /stillAnother
    /etc

Many developers first try simple pattern matching like --exclude 'branch*', but discover it doesn't work as expected. The issue stems from how rsync processes patterns:

  • Simple patterns only match filenames, not paths
  • Without proper path specification, rsync won't match directory names
  • The pattern needs to account for the full relative path

To properly exclude directories matching a pattern at any level, you need to use:

rsync -avh --exclude='/branch*/' --stats --delete --link-dest=$LNK source/ destination/

Key elements of this solution:

  • The leading slash (/) matches from the root of the transfer
  • The trailing slash (/) ensures only directories are matched
  • Quotes around the pattern prevent shell expansion

For more complex scenarios, consider these approaches:

# Exclude multiple patterns
--exclude={'/branch*/','/temp*/','/cache*/'}

# Use exclusion file for complex rules
--exclude-from='exclude_patterns.txt'

# Example exclude_patterns.txt content:
/branch*/
/*/temp_*/
backup_*/*

Always verify your exclude patterns with --dry-run before executing:

rsync -avhn --exclude='/branch*/' source/ destination/

The -n flag performs a dry run, showing what would be transferred without actually copying files.

Here's how this technique applies to common scenarios:

# Exclude all node_modules directories in a project
rsync -avh --exclude='/node_modules/' project/ backup/

# Skip temporary directories in a web server backup
rsync -avh --exclude={'/tmp/','/cache/','/sessions/'} /var/www/ backup/

# Ignore version control directories
rsync -avh --exclude={'/.git/','/.svn/','/.hg/'} project/ remote:/backup/

If exclusions still aren't working:

  • Ensure you're using absolute paths if transferring from root
  • Check for trailing slashes in your source specification
  • Verify pattern matching isn't being affected by shell expansion
  • Consider using --exclude vs --exclude-from for complex rules

When dealing with rsync exclusions, many users encounter unexpected behavior with pattern matching - especially when trying to exclude entire directory trees. The frustration compounds when the same exclusion pattern works differently between command line and configuration files.

The fundamental problem lies in how rsync processes path patterns. For directory exclusion to work properly, you need to understand that:

--exclude='branch*'    # Only matches at the current level
--exclude='*/branch*'  # Matches at any subdirectory level
--exclude='/branch*'   # Absolute path exclusion (from transfer root)

Based on your directory structure, here are verified working solutions:

# Option 1: Exclude from transfer root
rsync -avh --exclude='/branch*' source/ destination/

# Option 2: Double wildcard pattern
rsync -avh --exclude='**/branch*' source/ destination/

# Option 3: Filter file approach
echo '/branch*' > exclude.txt
rsync -avh --exclude-from='exclude.txt' source/ destination/

For more complex scenarios, consider these patterns:

# Exclude all branch directories but keep their empty structure
rsync -avh --include='*/' --exclude='*/branch*/***' source/ dest/

# Exclude branch directories but preserve specific files
rsync -avh --exclude='*/branch*' --include='*/branch*/important.txt' source/ dest/

When patterns don't work as expected, use these techniques:

# Dry run with verbose output
rsync -avh --dry-run --exclude='/branch*' -vv source/ destination/

# Print the exclude list being processed
rsync --exclude='/branch*' --exclude='*.tmp' --verbose --dry-run . /tmp/dest

For large directory trees, the exclusion method impacts performance:

  • Single patterns in command line: Fastest
  • Multiple --exclude options: Moderate overhead
  • External filter files: Additional I/O cost