Migrating Munin Server with Historical Data Preservation: A Step-by-Step Guide


2 views

Munin stores its historical data in RRD (Round Robin Database) files located in /var/lib/munin/ by default. These binary files contain all the metrics collected over time. The directory structure typically follows:

/var/lib/munin/
├── domain1.example.com
│   ├── cpu-user.rrd
│   ├── memory-used.rrd
│   └── ...
├── domain2.example.com
│   ├── disk-usage.rrd
│   └── ...
└── ...

Before moving anything, ensure both old and new servers run the same Munin version to avoid compatibility issues:

# Check Munin version
munin-node -V

Create a complete backup of your existing data:

# Create backup
sudo tar -czvf munin_backup.tar.gz /var/lib/munin /etc/munin/munin.conf

Choose one of these reliable transfer methods:

Method 1: Direct Rsync

# On new server
sudo rsync -avz -e ssh oldserver:/var/lib/munin/ /var/lib/munin/

Method 2: Tarball Transfer

# On old server
sudo tar -czvf munin_data.tar.gz -C /var/lib/munin .

# On new server
sudo mkdir -p /var/lib/munin
sudo tar -xzvf munin_data.tar.gz -C /var/lib/munin

Transfer your configuration files while maintaining proper permissions:

# Copy main config
sudo scp oldserver:/etc/munin/munin.conf /etc/munin/

# Copy plugin configs if any
sudo scp oldserver:/etc/munin/plugin-conf.d/* /etc/munin/plugin-conf.d/

After transferring files, perform these essential tasks:

# Set proper ownership
sudo chown -R munin:munin /var/lib/munin

# Restart services
sudo systemctl restart munin-node
sudo systemctl restart munin-httpd

Use these commands to verify your data transferred correctly:

# Check file count comparison
find /var/lib/munin -type f | wc -l

# Sample a specific RRD file
rrdtool info /var/lib/munin/example.com/cpu-user.rrd

For nodes that will continue reporting to the new server, update their configuration:

# In /etc/munin/munin-node.conf on each node
allow ^127\.0\.0\.1$
allow ^192\.168\.1\.100$  # New server IP

For a seamless transition, consider running both servers temporarily while switching DNS or load balancer configurations.

If graphs appear broken after migration:

# Rebuild HTML files
sudo su - munin --shell /bin/bash -c /usr/share/munin/munin-update

# Check for permission issues
sudo munin-check

Before migrating your Munin server, it's crucial to understand where and how data is stored. Munin typically stores its data in:

/var/lib/munin/   # Main data directory
├── munin-update  # Update status files
├── munin-data    # RRD database files
├── cgi-tmp       # CGI generated images
└── html          # Static HTML output

The most critical components are the RRD (Round Robin Database) files in munin-data which contain all your historical metrics.

First, ensure both servers run the same Munin version to avoid compatibility issues:

# On both old and new servers:
munin-node --version

Create a backup of your current configuration:

# On old server:
tar -czvf munin_backup.tar.gz /etc/munin /var/lib/munin

Method 1: Direct File Transfer

The simplest approach is to copy the entire data directory:

# On new server:
rsync -avz old-server:/var/lib/munin/ /var/lib/munin/

For configuration files:

rsync -avz old-server:/etc/munin/ /etc/munin/

Method 2: Database-Level Migration

If using MySQL backend (less common), dump and restore the database:

# On old server:
mysqldump -u munin -p munin_db > munin_db.sql

# On new server:
mysql -u munin -p munin_db < munin_db.sql

After transferring files, adjust file permissions:

chown -R munin:munin /var/lib/munin
chown -R munin:munin /etc/munin

Update the munin-node configuration if IP addresses changed:

# In /etc/munin/munin.conf on new server:
[old-node.example.com]
    address 192.168.1.100
    use_node_name yes

Check if the RRD files were properly transferred:

rrdtool info /var/lib/munin/munin-data/server1/cpu-user.rrd

Force an immediate update to test functionality:

su - munin --shell=/bin/bash -c "/usr/share/munin/munin-update"

Permission problems: The munin user must have proper access to all files.

# Typical solution:
chmod -R 755 /var/lib/munin

Time zone mismatches: Ensure both servers use the same time zone to prevent graph anomalies.

timedatectl list-timezones
sudo timedatectl set-timezone Region/City

Consider setting up a cron job for regular Munin data backups:

0 3 * * * root tar -czf /backups/munin-$(date +\%Y\%m\%d).tar.gz /var/lib/munin /etc/munin