When dealing with large remote files where only specific portions contain relevant data, downloading entire files becomes inefficient. Traditional FTP clients typically lack native support for partial downloads, forcing developers to implement workarounds.
The FTP protocol actually supports partial file transfers through two mechanisms:
- REST (Restart): Sets the point at which file transfer should begin
- RANGE: Specifies byte ranges for partial transfers (though not universally supported)
Method 1: Using cURL with Range Headers
curl -u username:password \
--range 0-1000 \
ftp://example.com/largefile.dat \
-o partial.dat
Method 2: Python ftplib Implementation
from ftplib import FTP
def download_range(host, user, passwd, remote, local, start, end):
ftp = FTP(host)
ftp.login(user, passwd)
# Set restart position
ftp.voidcmd('TYPE I') # Binary mode
ftp.voidcmd('REST ' + str(start))
# Calculate range length
length = end - start + 1
# Retrieve partial content
with open(local, 'wb') as f:
def callback(data):
f.write(data)
return len(data) # Return actual length
ftp.retrbinary(f'RETR {remote}', callback, blocksize=8192, rest=start)
ftp.quit()
# Example usage
download_range('ftp.example.com', 'user', 'pass',
'largefile.dat', 'partial.dat', 0, 1000)
Method 3: Using lftp Client
lftp -u username,password ftp.example.com
get -o local_file.dat -c /remote/file.dat -O 0 -e 1000
quit
Note that partial download support depends on:
- FTP server implementation (vsftpd, ProFTPD, etc.)
- Protocol extensions (some support custom RANGE commands)
- Firewall configurations that might interfere with passive mode transfers
When native FTP partial downloads aren't available:
- Use HTTP instead if the server supports it (better range request support)
- Create server-side scripts to extract needed portions
- Consider SFTP/SCP with streaming processing
If encountering "550 File not found" errors:
- Verify the FTP server supports REST commands
- Check that binary transfer mode is set (TYPE I)
- Confirm the file exists and permissions are correct
- Test with different clients to isolate the issue
When working with large remote files, downloading the entire file just to access a small portion is inefficient and wastes bandwidth. Many developers need to extract only specific segments (like the first 1000 bytes) from massive files stored on FTP servers. While manual interruption (Ctrl+C) works for ad-hoc downloads, automation requires a more reliable solution.
The FTP protocol does support partial file retrieval through the REST
(restart) command, but implementation varies across clients and servers. The syntax mentioned in some documentation (get filename:start-end
) is not universally supported.
1. Using cURL for Precise Byte Ranges
For modern systems, cURL provides the most reliable way to fetch partial files:
curl -u username:password \
-r 0-999 \
ftp://example.com/path/to/largefile.dat \
-o partial.dat
This downloads only the first 1000 bytes (0-999) of the file.
2. Alternative Approach with wget
wget can also handle partial downloads when combined with FTP:
wget --user=username --password=password \
--ftp-user=username --ftp-password=password \
-O - ftp://example.com/file.dat | head -c 1000 > partial.dat
3. Python Implementation
For programmatic control, use Python's ftplib
with custom retrieval logic:
from ftplib import FTP
def download_partial(host, user, passwd, filename, start, end, output):
with FTP(host) as ftp:
ftp.login(user, passwd)
with open(output, 'wb') as f:
def callback(data):
if f.tell() + len(data) > end:
data = data[:end - f.tell()]
f.write(data)
raise Exception("Download complete")
f.write(data)
ftp.retrbinary(f'RETR {filename}', callback, rest=start)
# Download first 1000 bytes
download_partial('ftp.example.com', 'user', 'pass',
'largefile.dat', 0, 999, 'partial.dat')
Some FTP servers (especially legacy systems like z/OS) may not support range requests properly. In such cases:
- Check if the server supports
REST
command withFEAT
- Consider using SFTP/SCP instead, which often has better partial download support
- For z/OS specifically, you might need JCL scripts to extract portions before transfer
If you encounter "550 File not found" errors:
- Verify the server supports range requests
- Ensure the filename doesn't contain special characters
- Try different clients (lftp, ncftp often have better support)