How to Configure Nagios Service Notifications: Email for WARNINGS and Pager for CRITICAL Alerts


3 views

The key to customizing Nagios notifications lies in the service_notification_commands parameter and proper contact/contactgroup configuration. The original setup simultaneously triggers both email and pager for all alerts, which isn't ideal for tiered response scenarios.

We need to implement a two-layer notification system:

  1. Email notifications for WARNING state changes
  2. Pager notifications exclusively for CRITICAL state changes

First, create separate notification commands in your Nagios configuration:


# Email-only notification command
define command {
    command_name    notify-service-by-email-only
    command_line    /usr/bin/printf "%b" "***** Nagios *****\n\nNotification Type: $NOTIFICATIONTYPE$\n\nService: $SERVICEDESC$\nHost: $HOSTALIAS$\nAddress: $HOSTADDRESS$\nState: $SERVICESTATE$\n\nDate/Time: $LONGDATETIME$\n\nAdditional Info:\n\n$SERVICEOUTPUT$\n" | /bin/mail -s "** $NOTIFICATIONTYPE$ alert - $HOSTALIAS$/$SERVICEDESC$ is $SERVICESTATE$ **" $CONTACTEMAIL$
}

# Pager-only notification command
define command {
    command_name    notify-service-by-pager-only
    command_line    /usr/bin/printf "%b" "Service: $SERVICEDESC$\nHost: $HOSTALIAS$\nState: $SERVICESTATE$\nTime: $SHORTDATETIME$\nInfo: $SERVICEOUTPUT$" | /usr/bin/pager_command $CONTACTPAGER$
}

Modify your contact definition to implement tiered notifications:


define contact {
    name                            tiered-contact
    service_notification_options    w,u,c,r,f,s
    host_notification_options       d,u,r,f,s
    service_notification_commands   notify-service-by-email-only,notify-service-by-pager-only
    host_notification_commands      notify-host-by-email,notify-host-by-pager
    register                        0
    service_notification_period     24x7
    host_notification_period        24x7
}

To achieve the conditional behavior, we'll use Nagios' notification escalation features:


define serviceescalation {
    host_name               *
    service_description     *
    first_notification      1
    last_notification       0
    notification_interval   0
    contact_groups          pager-admins
    escalation_options      c
    escalation_period       24x7
}

After implementing these changes:

  1. Restart Nagios to apply configuration changes
  2. Test with service nagios configtest first
  3. Force a WARNING state alert to verify email-only delivery
  4. Force a CRITICAL state alert to verify both email and pager delivery

For more granular control, consider these additional parameters:


define contact {
    # ... existing parameters ...
    service_notification_options    w
    host_notification_options       d,u
}

define contactgroup {
    contactgroup_name       email-admins
    alias                   Email Administrators
    members                 admin1,admin2
    service_notification_commands   notify-service-by-email-only
    host_notification_commands      notify-host-by-email
}


In enterprise monitoring systems, differentiating notification channels based on alert severity is crucial. The requirement is to modify Nagios' default behavior where both email and pager notifications fire for all service states (WARNING, CRITICAL, etc.). Instead, we want:

  1. Email notifications for WARNING and CRITICAL states
  2. Pager notifications ONLY for CRITICAL states

This requires modifying both contact definitions and notification commands. Here's the technical approach:

define command {
    command_name    notify-service-critical-pager
    command_line    /usr/bin/printf "%b" "$NOTIFICATIONTYPE$\n\nService: $SERVICEDESC$\nHost: $HOSTALIAS$\nAddress: $HOSTADDRESS$\nState: $SERVICESTATE$\n\nDate/Time: $LONGDATETIME$\n\nAdditional Info:\n\n$SERVICEOUTPUT$\n" | /bin/mail -s "** $NOTIFICATIONTYPE$ Alert - $HOSTALIAS$/$SERVICEDESC$ is $SERVICESTATE$ **" $CONTACTPAGER$
}

define command {
    command_name    notify-service-warning-email
    command_line    /usr/bin/printf "%b" "$NOTIFICATIONTYPE$\n\nService: $SERVICEDESC$\nHost: $HOSTALIAS$\nAddress: $HOSTADDRESS$\nState: $SERVICESTATE$\n\nDate/Time: $LONGDATETIME$\n\nAdditional Info:\n\n$SERVICEOUTPUT$\n" | /bin/mail -s "** $NOTIFICATIONTYPE$ Alert - $HOSTALIAS$/$SERVICEDESC$ is $SERVICESTATE$ **" $CONTACTEMAIL$
}

The contact definition needs to reference these custom commands:

define contact {
    name                            priority-contact
    service_notification_options    w,u,c,r,f,s
    host_notification_options       d,u,r,f,s
    service_notification_commands   notify-service-warning-email,notify-service-critical-pager
    host_notification_commands      notify-host-by-email
    register                        0
    service_notification_period     24x7
    host_notification_period        24x7
}

For more complex scenarios, consider using Nagios event handlers or modifying notification scripts:

#!/bin/bash
# Custom notification script: notify_custom.sh
case "$SERVICESTATE" in
    "CRITICAL")
        # Send both email and pager
        echo "$NOTIFICATIONTYPE$ Alert for $HOSTALIAS$/$SERVICEDESC$" | mail -s "CRITICAL Alert" $CONTACTEMAIL$
        echo "$NOTIFICATIONTYPE$ Alert for $HOSTALIAS$/$SERVICEDESC$" | mail -s "CRITICAL Alert" $CONTACTPAGER$
        ;;
    "WARNING")
        # Send only email
        echo "$NOTIFICATIONTYPE$ Alert for $HOSTALIAS$/$SERVICEDESC$" | mail -s "WARNING Alert" $CONTACTEMAIL$
        ;;
esac

After implementation:

  1. Force a WARNING state (e.g., set disk threshold temporarily)
  2. Verify only email notification is received
  3. Force a CRITICAL state
  4. Verify both email and pager notifications are received