How to Exclude Directories in LFTP Mirror Command Using -x Option: A Regex Guide for Developers


2 views

The -x option in lftp's mirror command uses regular expressions to exclude files and directories during transfer operations. Many developers struggle with its exact syntax, especially when dealing with directory exclusions.

For excluding the .svn directory, you need to match both the directory name and its path components. Here's the proper syntax:

mirror -R -e -x '/\.svn(/|$)' /documents/ /test

This pattern works because:

  1. \/ matches the forward slash (escaped)
  2. \.svn matches the literal ".svn"
  3. (/|$) matches either a trailing slash or end-of-string

Here are some useful patterns for different exclusion needs:

# Exclude multiple directories
mirror -x '/\.(svn|git|idea)(/|$)' -x '/node_modules(/|$)' ...

# Exclude files with specific extensions
mirror -x '\.(tmp|bak|swp)$' ...

# Case-insensitive exclusion
mirror -x -i '/\.svn(/|$)' ...

If your patterns aren't working as expected:

  1. Use set xfer:log 1 to enable transfer logging
  2. Check with cls -x 'pattern' to test matching before mirroring
  3. Remember patterns are applied to full remote paths

When excluding many directories:

  • Combine patterns with | (OR) operator when possible
  • Avoid overly complex regex patterns
  • Consider using --parallel=number to speed up large transfers

When using lftp's mirror command for directory synchronization, the -x (exclude) option follows standard shell globbing patterns by default, not regular expressions. This is a common point of confusion among developers transitioning from tools like rsync.

To exclude .svn directories during mirroring, you should use:

mirror -R -e -x .svn/ /documents/ /test

Key observations about this syntax:

  • The trailing slash indicates directory exclusion
  • No regex anchors (^$) are needed
  • Pattern matching is case-sensitive

For more complex exclusion scenarios, lftp supports multiple patterns:

mirror -R -e -x ".svn/" -x "*.tmp" /src/ /backup/

Some useful pattern examples:

  • -x "*.log": Exclude all log files
  • -x "/temp/": Exclude top-level temp directory
  • -x "build/": Exclude build directories at any level

When dealing with large directory structures:

mirror -R -e --parallel=4 -x "node_modules/" -x ".git/" /projects/ /backup/

The --parallel option can significantly improve sync speed when combined with exclusions.

For users who prefer regex patterns, you can enable Perl-compatible regex with:

set ftp:use-pcre on
mirror -R -e -x '\.svn$' /documents/ /test

Remember to test your patterns with cls -x pattern before running the actual mirror operation.