When managing a CentOS server that has evolved into critical infrastructure (web development, Subversion, and internal wiki hosting), the yum update decision becomes a calculated risk assessment. Having personally maintained such systems for years, I've developed a methodology that balances security and stability.
For your specific case with 250 pending updates:
- Criticality Analysis: First identify packages handling:
# List security-related packages rpm -qa | grep -E 'httpd|php|openssl|kernel|subversion|mod_ssl|postgresql|mysql'
- Change Impact: Core services like Apache or database systems deserve special attention
# Check running services systemctl list-units --type=service --state=running
Instead of a blanket yum update
, implement phased updates:
# First update only security packages
yum update --security
# Then update non-critical packages in batches
yum update $(rpm -qa | grep -vE 'kernel|httpd|mysql|postgresql')
Before any update, establish version snapshots:
# Create package version manifest
rpm -qa --qf "%{NAME}-%{VERSION}-%{RELEASE}.%{ARCH}\n" | sort > /root/pkglist-pre-update.txt
# Download all current packages (for emergency downgrade)
mkdir /root/rpm_backup
yumdownloader $(rpm -qa --qf "%{NAME}\n") --destdir=/root/rpm_backup
After updates, implement these checks:
# Verify service status
for service in httpd svnserve wiki-service; do
systemctl is-active $service || echo "$service failed!"
done
# Test critical functionality
curl -I localhost > /dev/null || echo "Web server check failed"
svn co file:///var/svn/repo/testcheckout || echo "SVN test failed"
For emergency rollbacks of problematic updates:
# Single package downgrade example
rpm -Uvh --oldpackage /root/rpm_backup/package-name-version.rpm
# Full system rollback (if using BTRFS)
snapper list
snapper undochange 42..47
Remember that CentOS's conservative update policy means most updates are vetted, but complex dependency chains can still cause issues. Your approach of ghosting the drive before updating is exactly what I'd recommend for high-stakes systems.
When managing a CentOS server that's evolved into mission-critical infrastructure (web development, SVN, and internal wiki), the yum update decision becomes non-trivial. Here's my battle-tested approach after maintaining similar systems for a decade.
# Always verify these before major updates:
sudo yum check-update # View pending updates
sudo yum history # Review past transactions
df -h # Check disk space
For high-availability systems, I always recommend creating a staging clone. Here's how to quickly mirror your production setup:
# Create LVM snapshot for testing (requires LVM setup)
lvcreate -L10G -s -n centos_test /dev/centos/root
mount /dev/centos/centos_test /mnt/test
chroot /mnt/test /bin/bash
Instead of blanket updates, consider these targeted approaches:
# Security-only updates
sudo yum update --security
# Kernel update isolation
sudo yum update kernel
sudo yum update --exclude=kernel*
For web-facing services, implement these safety nets before updating:
# Create emergency rollback points
sudo yum install yum-utils
sudo yum-complete-transaction --cleanup-only
sudo yum history addon-info last saved_tx
# Database protection
mysqldump -u root -p --all-databases > pre_update_backup.sql
Post-update validation is crucial. This script checks common failure points:
#!/bin/bash
# Service status check
systemctl list-units --type=service --state=failed
# Web stack verification
curl -I localhost | grep "HTTP"
svnadmin verify /path/to/repos
Configure these Nagios checks to detect update-related issues:
define service {
service_description Package Version Consistency
check_command check_nrpe!check_yum_consistency
max_check_attempts 3
normal_check_interval 15
}
Here's my 5-step recovery playbook:
- Identify broken packages:
rpm -Va
- Check logs:
journalctl -xe --no-pager
- Rollback:
sudo yum history undo last
- Isolate:
yum deplist problematic-package
- Manual repair:
rpm -e --nodeps broken-package