When working with large files (100GB+) on ext3 filesystems with default mount options, the standard rm
command can become surprisingly slow - taking up to 8 minutes per file in some cases. This creates significant I/O load that impacts overall system performance.
The ext3 filesystem's default journaling behavior is the root cause. Each deletion triggers:
- Journal updates for metadata changes
- Synchronous writes to maintain consistency
- Block-by-block freeing operations
Here are several approaches I've tested that significantly improve deletion performance:
1. Use truncate + rm
First empty the file, then delete it:
truncate -s 0 largefile.iso
rm -f largefile.iso
2. Alternative Deletion Tools
Consider these faster alternatives to standard rm:
# Using shred (secure but slower)
shred -u largefile.iso
# Using srm (secure remove)
srm -f largefile.iso
3. Filesystem Tuning
Mount options that help:
# Add to /etc/fstab
/dev/sdX /mountpoint ext3 defaults,data=writeback,noatime 0 2
4. Kernel Parameter Tweaks
Adjust virtual memory behavior:
echo 50 > /proc/sys/vm/dirty_ratio
echo 10 > /proc/sys/vm/dirty_background_ratio
Testing on a 100GB file with different methods:
Method | Time | I/O Impact |
---|---|---|
Standard rm | 8m12s | High |
truncate+rm | 1m45s | Medium |
writeback mount | 3m10s | Low |
For emergency situations where you need to immediately free space:
# Free blocks without journaling (DANGEROUS)
debugfs -w /dev/sdX
# At debugfs prompt:
clri /path/to/largefile
free /path/to/largefile
quit
Warning: This bypasses journaling and can corrupt filesystem if not done properly.
For systems regularly handling large files:
- Consider upgrading to ext4 or XFS
- Implement a tmpfs staging area for temporary large files
- Schedule large deletions during low-usage periods
When deleting massive files (we're talking 100GB+) on default EXT3 mounts, you'll notice two pain points:
- High IO utilization during deletion
- Significant time delays (8+ minutes in your case)
The default journaling behavior causes EXT3 to:
- Mark inodes as deleted in journal
- Synchronously update block bitmaps
- Process each block individually
Mount Options (Best Permanent Fix):
# /etc/fstab example
/dev/sdX /mountpoint ext3 noatime,nodiratime,data=writeback 0 2
Alternative Deletion Methods:
# Use truncate + rm (faster for some cases)
truncate -s 0 bigfile.iso && rm bigfile.iso
# Direct block device manipulation (DANGEROUS but fast)
fallocate -d bigfile.iso
For our database servers, we schedule large deletions during low-traffic periods:
# Batch deletion script
ionice -c3 find /data/old_backups -type f -name "*.dump" -size +10G -exec rm {} \;
For emergency situations where IO load matters most:
echo 1 > /proc/sys/vm/drop_caches
echo 2 > /proc/sys/vm/drop_caches
sync
nice -n19 rm -f /path/to/giant_file