Troubleshooting a Hanging “yum clean all” Command on CentOS: Database Recovery Steps


2 views

When yum clean all hangs without CPU usage or output (even with -v --noplugins), we're typically dealing with RPM database corruption. The strace output showing a freeze at futex during /var/lib/rpm/Packages access confirms this.

Before nuclear options, try these sequenced steps:

# First attempt - kill any locked processes
ps aux | grep -E 'yum|rpm'
kill -9 [pid_of_stuck_process]

# Second attempt - manual DB rebuild
systemctl stop packagekitd  # Prevent interference
mv /var/lib/rpm/__db* /tmp/
rpm --rebuilddb

For severe corruption where rpm --rebuilddb also hangs:

# Create emergency backup
mkdir /root/rpm_db_backup
cp -a /var/lib/rpm /root/rpm_db_backup

# Generate new DB from installed packages
rpm -qa > /root/packages.list
rm -f /var/lib/rpm/__db* /var/lib/rpm/Packages
rpm --initdb

# Reimport packages (takes time on large systems)
while read pkg; do
    rpm --reinstall $pkg 2>/dev/null || rpm -ivh --justdb $pkg
done < /root/packages.list

If all else fails, this will rebuild everything from metadata (may miss local modifications):

yum install -y yum-utils
package-cleanup --cleandupes
rm -rf /var/cache/yum/*
yum clean all
yum makecache
  • Regularly backup /var/lib/rpm
  • Use yum-complete-transaction after interrupted updates
  • Consider dnf on CentOS 8+ for better transaction handling

If you've ever encountered a situation where yum clean all just hangs indefinitely without any CPU usage or output, you're not alone. This particularly frustrating issue tends to occur on CentOS servers during routine maintenance.

The classic signs of this issue include:

  • Complete freeze at yum clean all command
  • No CPU usage despite the hanging process
  • Strace output showing a hang at futex operation
  • Normal server operation otherwise

Before diving into solutions, let's verify the exact state of the system:

# Check for existing yum processes
ps aux | grep yum

# Verify disk space availability
df -h

# Check for file system errors
ls -la /var/lib/rpm/

From the strace output, we can see the process hangs when accessing the RPM database files. This suggests corruption or locking issues in the Berkeley DB files that RPM uses.

When standard recovery fails, try these more aggressive approaches:

# First, kill any stuck yum processes
pkill yum

# Then attempt manual database recovery
rm -f /var/lib/rpm/__db*
rpm --rebuilddb

# Alternative method using db_recover
cd /var/lib/rpm
db_recover -v -h .

For particularly stubborn cases, you might need to rebuild the entire RPM database:

# Backup existing database
mkdir /root/rpmdb-backup
cp -a /var/lib/rpm /root/rpmdb-backup/

# Reinitialize database
rm -rf /var/lib/rpm
rpm --initdb

# Reinstall all packages to rebuild database
rpm -qa | xargs rpm -e --justdb --nodeps
yum reinstall $(rpm -qa)

To minimize chances of this happening again:

  • Regularly clean yum cache: yum clean all
  • Maintain sufficient disk space in /var
  • Avoid interrupting yum operations
  • Consider using dnf on newer CentOS versions