How to Bulk Move Files Between Directories on the Same FTP Server Using LFTP


6 views

When attempting to move multiple files between directories on an FTP server using lftp, you'll quickly discover that standard shell wildcards don't work as expected. The command:

lftp> mv remote.dir1/* remote.dir2/

Fails with the error:

550 remote.dir1/*: The filename, directory name, or volume label syntax is incorrect

This occurs because FTP servers typically don't support shell-style wildcard expansion in the same way local shells do.

Option 1: Using LFTP's Built-in Mirror Command

The most efficient method is to use lftp's mirror command with the --Remove-source-files flag:

lftp> mirror --Remove-source-files remote.dir1 remote.dir2

This will:
1. Copy all files from remote.dir1 to remote.dir2
2. Delete the original files in remote.dir1
3. Preserve directory structure if present

Option 2: Scripted Approach with Find and MV

For more control, you can generate and execute move commands:

lftp> find remote.dir1 -type f -exec "mv {} remote.dir2/"

Note: Some FTP servers may not support the find command.

Option 3: Shell Script with File List

When dealing with very large directories:

#!/bin/bash
lftp -u user,pass ftp.example.com < /tmp/filelist.txt
!sed 's|^|mv remote.dir1/|;s|$| remote.dir2/|' /tmp/filelist.txt > /tmp/mv_commands.txt
source /tmp/mv_commands.txt
quit
EOF

Moving Specific File Types

To move only .txt files:

lftp> find remote.dir1 -name "*.txt" -exec "mv {} remote.dir2/"

Preserving Timestamps

Add the -a flag to maintain original file attributes:

lftp> mirror -a --Remove-source-files remote.dir1 remote.dir2

If lftp limitations are problematic, consider:

  • ncftp: ncftpbatch -f commands.txt
  • curlftpfs: Mount FTP as local filesystem
  • Python's ftplib for custom solutions

Remember that all operations must occur server-side when restricted to pure FTP environments.


When working with FTP servers through command-line tools like LFTP, many users encounter limitations when trying to perform batch operations. The standard mv command in LFTP works perfectly for single files:

lftp> mv remote.dir1/file1.txt remote.dir2/

But fails spectacularly when attempting wildcard operations:

lftp> mv remote.dir1/* remote.dir2/
550 remote.dir1/*: The filename, directory name, or volume label syntax is incorrect

This behavior occurs because most FTP servers don't support wildcard operations natively. Unlike local file systems or more modern protocols like SFTP, traditional FTP requires explicit paths for each operation.

Here are several effective approaches to solve this problem:

1. Using LFTP's Mirror Command

The mirror command can be repurposed for this task:

lftp> mirror --no-empty-dirs --include="*" --delete-source-files remote.dir1/ remote.dir2/

2. Scripted Solution with LFTP

Create a script file (move_files.lftp) containing:

set ftp:list-options -a
cd remote.dir1
ls | while read filename; do mv "$filename" ../remote.dir2/; done

Then execute it with:

lftp -f move_files.lftp

3. Using Find and Xargs

For more complex scenarios, combine LFTP with standard Unix tools:

lftp -c "open ftp://user:pass@server; find remote.dir1 -type f" | \
xargs -I {} lftp -c "open ftp://user:pass@server; mv {} remote.dir2/"

If LFTP isn't mandatory, consider these alternatives:

1. ncftp

ncftp> mput remote.dir1/* remote.dir2/

2. curlftpfs

Mount the FTP as a local filesystem:

curlftpfs ftp://user:pass@server /mnt/ftp
mv /mnt/ftp/remote.dir1/* /mnt/ftp/remote.dir2/
fusermount -u /mnt/ftp
  • Always verify file counts before and after operations
  • Consider using --dry-run first when available
  • For very large directories, process files in batches
  • Check for filename encoding issues (LFTP's set ftp:charset)