How to Use SCP to Transfer Only Files with Specific Extensions in Linux


2 views

When working with remote servers, we often need to transfer only specific file types rather than entire directories. The standard SCP command doesn't natively support filtering by multiple extensions like *.tar.gz and *.war simultaneously.

The naive approach of trying:

scp 192.168.1.2:/srv/myfiles/'*.tar.gz *.war' .

fails because:

  • SCP treats the entire quoted string as a single pattern
  • The space between extensions breaks the pattern matching
  • SCP's globbing doesn't support multiple patterns in one command

Here are three reliable methods to achieve selective file transfer:

Method 1: Using SSH with find and tar

This is the most robust solution:

ssh 192.168.1.2 'find /srv/myfiles $-name "*.tar.gz" -o -name "*.war"$ -print0 | tar -czvf - --null -T -' | tar -xzvf -

Breaking it down:

  • find locates files with either extension (-o for OR condition)
  • -print0 handles filenames with spaces correctly
  • tar creates an archive stream (- for stdout) of matching files
  • Local tar extracts the stream

Method 2: Using rsync Instead

rsync offers more flexible pattern matching:

rsync -avz --include="*.tar.gz" --include="*.war" --exclude="*" 192.168.1.2:/srv/myfiles/ .

Key points:

  • --include patterns specify what to transfer
  • --exclude="*" prevents other files from being included
  • -a preserves attributes, -v verbose, -z compression

Method 3: Multiple SCP Commands with Shell Globbing

For simpler cases with a few patterns:

scp 192.168.1.2:/srv/myfiles/*.{tar.gz,war} .

Or separately:

scp "192.168.1.2:/srv/myfiles/*.tar.gz" .
scp "192.168.1.2:/srv/myfiles/*.war" .

The ssh+tar method is most efficient for:

  • Large numbers of files
  • Files with spaces or special characters in names
  • When bandwidth is limited (compression helps)

While rsync is better for:

  • Partial transfers or resume capability
  • When you need to verify file integrity
  • Recursive directory transfers with patterns

Common pitfalls and solutions:

# Handle cases where no files match:
shopt -s nullglob
scp 192.168.1.2:/srv/myfiles/*.tar.gz . || echo "No tar.gz files found"

For more complex patterns, consider creating a temporary file list:

ssh 192.168.1.2 'find /srv/myfiles -name "*.tar.gz" -o -name "*.war" > /tmp/filelist'
scp 192.168.1.2:/tmp/filelist .
scp -T 192.168.1.2:"$(

When working with remote servers, we often need to copy only specific file types while excluding others. The standard SCP command doesn't directly support filtering by multiple extensions, which can be frustrating when dealing with directories containing hundreds of files.

The attempt:

scp 192.168.1.2:/srv/myfiles/'*.tar.gz *.war' .

fails because:

  • SCP's glob pattern handling differs from local shell expansion
  • The single quotes prevent proper pattern interpretation
  • Space-separated patterns aren't processed as expected

Method 1: Using find + scp (Most Reliable)

Execute this on your local machine:

ssh user@192.168.1.2 "find /srv/myfiles -type f $-name '*.tar.gz' -o -name '*.war'$" | \
while read file; do
    scp "user@192.168.1.2:$file" .
done

Method 2: tar + ssh Pipeline (For Large Transfers)

ssh user@192.168.1.2 "tar czf - /srv/myfiles/*.tar.gz /srv/myfiles/*.war" | tar xzvf - -C .

Method 3: rsync Alternative (For Repeated Transfers)

rsync -avz --include='*.tar.gz' --include='*.war' --exclude='*' \
user@192.168.1.2:/srv/myfiles/ .
  • All methods require SSH key authentication for automation
  • The find method preserves original file permissions
  • For many files (1000+), Method 2 (tar) is most efficient
  • Method 3 (rsync) is best for incremental transfers

For complex criteria (e.g., files modified within last 7 days):

ssh user@192.168.1.2 "find /srv/myfiles -type f -mtime -7 $-name '*.tar.gz' -o -name '*.war'$" | \
while read file; do
    scp "user@192.168.1.2:$file" ./recent_backups/
done