How to Monitor 404/403 HTTP Status Codes with Monit Without False Alerts


2 views

When configuring Monit to monitor web services, many developers encounter a frustrating behavior: Monit treats HTTP 4xx status codes (like 404 Not Found or 403 Forbidden) as complete failures, triggering unnecessary alerts and restarts. This becomes problematic when you actually want to verify that these error pages are being served correctly as part of your application's behavior.

Monit's HTTP check is designed to verify service availability, not specific HTTP status codes. The default configuration considers any response that isn't a successful 2xx code as a failure. Here's what happens with a typical configuration:

if failed 
   host example.com 
   port 80
   protocol http
   request "/non-existent-page"
then alert

This will trigger an alert even if your server correctly returns a 404 page with proper headers.

Monit 5.2+ introduced the status parameter which allows custom status code checking. Here's how to modify your configuration:

check process httpd with pidfile /var/run/httpd.pid
  start program = "/etc/init.d/httpd start"
  stop program = "/etc/init.d/httpd stop"
  
  if failed
     host hostname
     port 80
     protocol HTTP
     request "/special-404-page"
     status = 404
  then exec "/bin/bash -c '/bin/echo -e \"hostname\\thttpd\\t3\\t404_VERIFIED\" | ...'"

For scenarios where multiple status codes are valid responses, you can use the with status syntax:

if failed
   host api.example.com
   port 443
   protocol https
   request "/auth-check"
   with status = 200 or status = 403
then alert

Here's a complete working example for monitoring both 404 and 403 pages:

check process nginx with pidfile /var/run/nginx.pid
  start program = "/etc/init.d/nginx start"
  stop program = "/etc/init.d/nginx stop"
  
  # Check 404 page
  if failed
     host example.com
     port 80
     protocol http
     request "/should-404"
     status = 404
     timeout 5 seconds
  then exec "/path/to/notify.sh 404_OK"
  
  # Check 403 page
  if failed
     host example.com
     port 80
     protocol http
     request "/admin"
     status = 403
     timeout 5 seconds
  then exec "/path/to/notify.sh 403_OK"

If your custom status checks aren't working as expected:

  1. Verify Monit version (monit -V) supports status checks
  2. Check for syntax errors in your configuration
  3. Test the HTTP request manually with curl first
  4. Ensure Monit has proper network access to your service

Many developers using Monit for service monitoring encounter a common frustration: the tool treats HTTP 4xx status codes (like 404 Not Found or 403 Forbidden) as connection failures. This behavior can trigger unnecessary alerts and restarts when you actually want to verify that these error pages are being served correctly.

By default, Monit considers any HTTP response status code outside the 2xx range as a failure. This makes sense for most monitoring scenarios, but becomes problematic when you specifically need to verify error page delivery.

# Standard HTTP check that fails on 4xx responses
if failed 
   host example.com 
   port 80
   protocol http
   request "/nonexistent-page"
then alert

Monit actually provides a way to customize the acceptable status codes through the status parameter. Here's how to modify your check:

check process httpd with pidfile /var/run/httpd.pid
  start program = "/etc/init.d/httpd start"
  stop program = "/etc/init.d/httpd stop"
  
  if failed 
     host hostname 
     port 80
     protocol HTTP 
     request "/special-404-page"
     status = 404
  then exec "/bin/bash -c '/bin/echo -e \"hostname\\thttpd\\t3\\tFAILED\" | /usr/sbin/send_nsca -H nagiosserver -c /etc/send_nsca.cfg'"

For scenarios where you need to accept multiple status codes (like both 403 and 404), you can use this approach:

if failed 
   host hostname 
   port 80
   protocol HTTP 
   request "/admin" 
   status > 399 and < 405
then alert

When combining with other monitoring tools like Nagios, ensure your status checks align:

if failed 
   host hostname 
   port 80
   protocol HTTP 
   request "/wp-admin" 
   status = 403
then exec "/usr/local/bin/nagios_notify.sh 'WordPress admin accessible'"

This technique is particularly useful for:

  • Testing custom error page configurations
  • Verifying authentication/authorization systems
  • Monitoring maintenance mode pages
  • Checking URL whitelisting/blacklisting

If your custom status checks aren't working as expected:

  1. Verify Monit version (5.8+ recommended)
  2. Check for syntax errors in the status condition
  3. Test the endpoint manually with curl first
  4. Review Monit logs for connection details