Troubleshooting Rsync “File too large” Error When Copying to VFAT Filesystem


3 views

When working with external drives formatted as VFAT (commonly known as FAT32), you'll encounter a fundamental limitation: individual files cannot exceed 4GB in size. This explains why your 4.4GB video file fails while smaller files transfer successfully.

The rsync error clearly indicates the issue:

rsync: write failed on "/media/video/Truth.In.24.ESPNHD.720p.HDTV.x264.mkv": File too large (27)
rsync error: error in file IO (code 11) at receiver.c(389) [receiver=3.1.0]

Option 1: Reformat the Drive

The most permanent solution is to reformat your USB drive to a filesystem that supports large files:

sudo umount /dev/sdX1
sudo mkfs.ext4 /dev/sdX1

Replace /dev/sdX1 with your actual device identifier.

Option 2: Split Large Files

If reformatting isn't possible, you can split the file before transfer:

split -b 2000m Truth.In.24.ESPNHD.720p.HDTV.x264.mkv Truth.In.24.part

Then reassemble after transfer:

cat Truth.In.24.part* > Truth.In.24.ESPNHD.720p.HDTV.x264.mkv

To avoid future issues, implement a pre-transfer check in your backup script:

#!/bin/bash
MAX_SIZE=4294967295 # 4GB - 1 byte

for file in /home/Media/Movies/*; do
    size=$(stat -c%s "$file")
    if [ $size -gt $MAX_SIZE ]; then
        echo "Warning: $file exceeds VFAT size limit ($size bytes)"
    fi
done

Consider using tools that automatically handle large files:

  • tar pipe: tar cf - bigfile | (cd /media/video && tar xvf -)
  • rsync with --partial: May help with interrupted transfers

I recently encountered the same frustrating error while attempting to back up media files to an external USB drive using rsync. The error message clearly indicates a file size limitation, but what's puzzling is that it occurred with a 4.4GB file while successfully transferring other 4GB files.

The root cause stems from using a VFAT filesystem (commonly known as FAT32) on your USB drive. While many believe FAT32 has a 4GB file size limit, the actual threshold is 4,294,967,295 bytes (2^32 - 1), which explains why some files just under 4GB work while others fail.

# Check your filesystem type:
df -Th /media/video

Here are three reliable approaches to solve this problem:

1. Reformat the Drive to a Modern Filesystem

For Linux systems, ext4 is ideal:

# Unmount the drive first
sudo umount /dev/sdX1

# Format as ext4 (replace X with your actual device)
sudo mkfs.ext4 /dev/sdX1

2. Use NTFS for Cross-Platform Compatibility

If you need Windows compatibility:

sudo mkfs.ntfs -Q /dev/sdX1

3. Split Large Files During Transfer

If reformatting isn't an option, use split and cat:

# Split the file
split -b 2G Truth.In.24.ESPNHD.720p.HDTV.x264.mkv split_file_

# Transfer parts
rsync -zv split_file_* /media/video/

# Reassemble on destination
cat split_file_* > Truth.In.24.ESPNHD.720p.HDTV.x264.mkv

Consider these rsync alternatives when dealing with filesystem limitations:

# Using tar with pipe (works within filesystem limits)
tar -cf - bigfile.mkv | (cd /media/video/ && tar -xf -)

# Using pv for progress monitoring
tar -cf - bigfile.mkv | pv | (cd /media/video/ && tar -xf -)

If you must keep VFAT, verify mount options that might affect large file handling:

# Current mount options
mount | grep /media/video

# Remount with different options (temporary)
sudo mount -o remount,rw,exec,uid=1000 /dev/sdX1 /media/video