When working with rsync's pattern matching capabilities, many developers encounter the frustrating "No such file or directory" error when attempting to use wildcards with either
--files-from
or--include-from
. Let's examine why this happens and how to solve it.The fundamental issue lies in how rsync processes include patterns versus file lists. The
--include-from
option expects pattern matching rules, while--files-from
expects exact file paths.For directory patterns with wildcards, we need to use
--include-from
with proper pattern syntax:rsync -av --include-from=list.lst --include='*/' --exclude='*' /home/user/ /path/to/backup/
The key additions here are:
--include='*/'
to allow directory traversal--exclude='*'
as the base exclusion ruleHere's a fully functional implementation that handles all the patterns from the original question:
# list.lst contents: .gnupg/ .pki/ .gnome2/keyrings/ .mozilla/firefox/*.default/bookmarkbackups/ .mozilla/firefox/*.default/bookmarks.html .mozilla/firefox/*.default/*.db .mozilla/firefox/*.default/*.sqlite # rsync command: rsync -av \ --include-from=list.lst \ --include='*/' \ --exclude='*' \ /home/user/ \ /path/to/backup/
For complex wildcard patterns that rsync can't handle directly, combining
find
with rsync often works better:find /home/user -path '*/.gnupg' -o \ -path '*/.pki' -o \ -path '*/.gnome2/keyrings' -o \ -path '*/.mozilla/firefox/*.default/bookmarkbackups' -o \ -path '*/.mozilla/firefox/*.default/bookmarks.html' -o \ -path '*/.mozilla/firefox/*.default/*.db' -o \ -path '*/.mozilla/firefox/*.default/*.sqlite' \ -print0 | rsync -av --files-from=- --from0 / /path/to/backup/
When troubleshooting rsync patterns:
- Add
-vvv
for maximum verbosity- Test with
--dry-run
first- Simplify patterns to identify which ones fail
- Remember that rsync processes patterns in order
When working with rsync's filtering options, many developers encounter the "No such file or directory" error when attempting to use wildcards in include patterns. The root cause lies in how rsync processes these patterns during the transfer operation.
Before diving into solutions, it's crucial to understand these two similar but distinct options:
--include-from=FILE # Reads include patterns from FILE --files-from=FILE # Reads exact file paths to transfer from FILE
For your specific case with wildcards in paths like
.mozilla/firefox/*.default/
, you need to use the--include
pattern syntax correctly:# Correct format for list.lst: + .gnupg/ + .pki/ + .gnome2/keyrings/ + .mozilla/firefox/*.default/bookmarkbackups/ + .mozilla/firefox/*.default/bookmarks.html + .mozilla/firefox/*.default/*.db + .mozilla/firefox/*.default/*.sqlite + */ - *
Here's the complete working command:
rsync -av \ --include-from=/path/to/list.lst \ --exclude='*' \ /home/user/ \ /path/to/backup
- Always include
+ */
to allow directory traversal - Explicitly exclude everything else with
- *
- Use trailing slashes for directories in your patterns
- The
-av
flags ensure proper recursive operation
If you prefer using --files-from
, you'll need to first generate the actual file list:
find ~ -path '*.mozilla/firefox/*.default/*.db' -o \
-path '*.mozilla/firefox/*.default/*.sqlite' > filelist.txt
rsync -av --files-from=filelist.txt / /path/to/backup
Add these flags to troubleshoot filtering issues:
--debug=FILTER2 # Shows pattern matching details
--dry-run # Performs trial run without changes