Configuring Monit for Disk Space Monitoring: Best Practices for / and /var Partitions


6 views

Monit provides robust filesystem monitoring capabilities that go beyond simple disk space checks. When setting up monitoring for critical partitions like / and /var, we can implement comprehensive checks that help prevent system issues before they occur.

For your current setup with /dev/md0 mounted on / and /dev/md2 on /var, here's the proper configuration:

check filesystem rootfs with path /
    if space usage > 80% for 5 cycles then alert

check filesystem varfs with path /var
    if space usage > 85% for 5 cycles then alert
    if inode usage > 85% then alert

Consider adding these additional checks to your configuration:

check filesystem rootfs with path /
    if space usage > 80% then alert
    if inode usage > 80% then alert
    if changed fsflags then alert
    if service time > 100 milliseconds for 5 cycles then alert

For ext4 filesystems, you might want to add SMART monitoring:

check device root_disk with path /dev/sda
    if SMART status != "OK" then alert

For LVM setups:

check filesystem lvm_root with path /
    if space usage > 90% then exec "/usr/local/bin/cleanup_old_logs.sh"

Enhance your monitoring with proactive measures:

check filesystem varfs with path /var
    if space usage > 90% then exec "/usr/bin/logger -t monit 'CRITICAL: /var space low'"
    if space usage > 95% then exec "/sbin/service apache2 graceful"

After updating your monitrc, always validate the configuration:

monit -t
monit reload
monit status

To test alerts without filling your disk, you can temporarily lower thresholds:

check filesystem testfs with path /tmp
    if space usage > 1% then alert

Watch out for these issues:

  • Using device paths instead of mount points in checks
  • Not monitoring inode usage (critical for mail servers)
  • Setting thresholds too low causing false alarms

For comprehensive monitoring, combine with process checks:

check process apache with pidfile /var/run/apache2.pid
    start program = "/etc/init.d/apache2 start"
    stop program = "/etc/init.d/apache2 stop"
    if cpu > 60% for 2 cycles then alert
    if totalmemory > 500 MB for 5 cycles then restart

When setting up Monit to monitor disk space, it's crucial to understand both the device paths and mount points. From your configuration, we can see:

mount
/dev/md0 on / type ext3 (rw)
/dev/md2 on /var type ext3 (rw)

Here's the proper way to configure Monit for your root filesystem:

check filesystem rootfs with path /
    if space usage > 90% for 5 cycles then alert
    if inode usage > 85% for 5 cycles then alert

For your /var partition, the configuration would be:

check filesystem var_partition with path /var
    if space usage > 80% then alert
    if inode usage > 75% then alert

Beyond basic space monitoring, consider these additional checks:

check filesystem rootfs with path /
    if space usage > 90% then alert
    if inode usage > 85% then alert
    if changed fsflags then alert
    if permission != 755 then alert
    if changed uid then alert
    if changed gid then alert

For a more production-ready setup with multiple thresholds and actions:

check filesystem var_partition with path /var
    if space usage > 80% then alert
    if space usage > 90% then exec "/usr/local/bin/cleanup_old_logs.sh"
    if space usage > 95% then exec "/usr/local/bin/emergency_cleanup.sh"
    if inode usage > 80% then alert
    if inode usage > 90% then exec "/usr/local/bin/report_inode_crisis.sh" timeout 60 seconds

After making changes, always validate your configuration:

monit -t
service monit restart
monit status

If you prefer to monitor the raw devices instead of mount points:

check device md0 with path /dev/md0
    if space usage > 90% then alert

check device md2 with path /dev/md2
    if space usage > 80% then alert