Solving rsync Wildcard Inclusion Issues: How to Properly Use –files-from and –include-from for Pattern Matching


15 views



When working with rsync's pattern matching capabilities, many developers encounter the frustrating "No such file or directory" error when attempting to use wildcards with either --files-from or --include-from. Let's examine why this happens and how to solve it.

The fundamental issue lies in how rsync processes include patterns versus file lists. The --include-from option expects pattern matching rules, while --files-from expects exact file paths.

For directory patterns with wildcards, we need to use --include-from with proper pattern syntax:

rsync -av --include-from=list.lst --include='*/' --exclude='*' /home/user/ /path/to/backup/

The key additions here are:

  • --include='*/' to allow directory traversal
  • --exclude='*' as the base exclusion rule

Here's a fully functional implementation that handles all the patterns from the original question:

# list.lst contents:
.gnupg/
.pki/
.gnome2/keyrings/
.mozilla/firefox/*.default/bookmarkbackups/
.mozilla/firefox/*.default/bookmarks.html
.mozilla/firefox/*.default/*.db
.mozilla/firefox/*.default/*.sqlite

# rsync command:
rsync -av \
  --include-from=list.lst \
  --include='*/' \
  --exclude='*' \
  /home/user/ \
  /path/to/backup/

For complex wildcard patterns that rsync can't handle directly, combining find with rsync often works better:

find /home/user -path '*/.gnupg' -o \
             -path '*/.pki' -o \
             -path '*/.gnome2/keyrings' -o \
             -path '*/.mozilla/firefox/*.default/bookmarkbackups' -o \
             -path '*/.mozilla/firefox/*.default/bookmarks.html' -o \
             -path '*/.mozilla/firefox/*.default/*.db' -o \
             -path '*/.mozilla/firefox/*.default/*.sqlite' \
             -print0 | rsync -av --files-from=- --from0 / /path/to/backup/

When troubleshooting rsync patterns:

  1. Add -vvv for maximum verbosity
  2. Test with --dry-run first
  3. Simplify patterns to identify which ones fail
  4. Remember that rsync processes patterns in order


When working with rsync's filtering options, many developers encounter the "No such file or directory" error when attempting to use wildcards in include patterns. The root cause lies in how rsync processes these patterns during the transfer operation.

Before diving into solutions, it's crucial to understand these two similar but distinct options:

--include-from=FILE  # Reads include patterns from FILE
--files-from=FILE    # Reads exact file paths to transfer from FILE

For your specific case with wildcards in paths like .mozilla/firefox/*.default/, you need to use the --include pattern syntax correctly:

# Correct format for list.lst:
+ .gnupg/
+ .pki/
+ .gnome2/keyrings/
+ .mozilla/firefox/*.default/bookmarkbackups/
+ .mozilla/firefox/*.default/bookmarks.html
+ .mozilla/firefox/*.default/*.db
+ .mozilla/firefox/*.default/*.sqlite
+ */
- *

Here's the complete working command:

rsync -av \
    --include-from=/path/to/list.lst \
    --exclude='*' \
    /home/user/ \
    /path/to/backup
  • Always include + */ to allow directory traversal
  • Explicitly exclude everything else with - *
  • Use trailing slashes for directories in your patterns
  • The -av flags ensure proper recursive operation

If you prefer using --files-from, you'll need to first generate the actual file list:

find ~ -path '*.mozilla/firefox/*.default/*.db' -o \
       -path '*.mozilla/firefox/*.default/*.sqlite' > filelist.txt
rsync -av --files-from=filelist.txt / /path/to/backup

Add these flags to troubleshoot filtering issues:

--debug=FILTER2  # Shows pattern matching details
--dry-run       # Performs trial run without changes