MongoDB Disk Space Not Freed After Dropping Large Collection: Diagnosis and Solutions


2 views

When you drop a collection in MongoDB, the storage space isn't immediately returned to the filesystem. This is because MongoDB uses preallocated data files and maintains a local database for replication purposes. The storage engine (typically WiredTiger or MMAPv1) handles data deletion differently.

Several factors contribute to this behavior:

  • Preallocation: MongoDB preallocates data files in advance
  • Journal Files: The journal may still contain references
  • Storage Engine: WiredTiger compacts data asynchronously

Here are effective methods to free up disk space:

1. Compact the Database

For WiredTiger storage engine:

db.runCommand({ compact: 'collectionName' })

2. Repair Database (Requires Disk Space)

First ensure you have at least 110GB free space elsewhere:

mongod --repair --dbpath /var/lib/mongodb

3. Dump and Restore

A more reliable but time-consuming method:

mongodump --db yourDB --out /backup/path
mongo yourDB --eval "db.dropDatabase()"
mongorestore /backup/path

4. Enable Compression

For WiredTiger, configure compression during startup:

storage:
  wiredTiger:
    collectionConfig:
      blockCompressor: zlib

Check actual storage metrics with these commands:

// View database stats
db.stats(1024*1024) // Shows size in MB

// View collection stats
db.collection.stats()
  • Schedule regular maintenance compaction
  • Use TTL indexes for time-series data
  • Consider sharding for very large collections
  • Monitor storage with tools like MongoDB Ops Manager

Here's how we fixed a similar 200GB collection issue:

// Step 1: Create enough temporary space
$ mkdir /mnt/tmp_mongo
$ ln -s /mnt/tmp_mongo /var/lib/mongodb/tmp

// Step 2: Run repair with alternative directory
$ mongod --repair --dbpath /var/lib/mongodb --repairpath /var/lib/mongodb/tmp

// Step 3: Verify space recovery
$ du -sh /var/lib/mongodb

When dealing with MongoDB's storage engine, many developers encounter a puzzling situation: after dropping large collections, the disk space isn't immediately reclaimed. This occurs because MongoDB uses a pre-allocation strategy for storage files and maintains data files even after collection deletion.

MongoDB's default storage engine (WiredTiger since 3.2) handles file storage differently from traditional databases:

  • Data files grow in size incrementally (up to 2GB per file)
  • Deleted space is marked as available but not returned to OS
  • Namespace files maintain metadata even after collection drops

Here are effective methods to recover disk space, with examples:

1. Compact the Database

This defragments the data files without requiring additional space:

use yourDatabase
db.runCommand({ compact: 'yourCollection' })

2. Repair Database with --repair Option

For more thorough space recovery (requires free space equal to current DB size):

mongod --dbpath /var/lib/mongodb --repair

3. Dump and Restore Approach

When other methods fail, this nuclear option guarantees space recovery:

# Dump remaining data
mongodump --db yourDatabase --out /backup/

# Drop the entire database
mongo yourDatabase --eval "db.dropDatabase()"

# Restore the data
mongorestore --db yourDatabase /backup/yourDatabase

For production systems, consider implementing these maintenance routines:

// JavaScript function to compact all collections
function compactAll() {
    db.getCollectionNames().forEach(function(coll) {
        if(!coll.startsWith('system.')) {
            print('Compacting: ' + coll);
            db.runCommand({ compact: coll });
        }
    });
}
  • Configure WiredTiger's cacheSizeGB appropriately
  • Set storage.journal.enabled: true for consistent writes
  • Monitor storage with db.stats() and db.collection.stats()

Remember that MongoDB intentionally keeps the storage allocated for performance reasons. Regular maintenance is key to managing disk space effectively in production environments.