Automating HP DL380 G5 Drive Failure Alerts via Email: A No-Reboot Scripting Solution


4 views

Managing aging HP ProLiant DL380 G5 servers presents unique challenges, especially when critical components like hard drives fail. The built-in HP Management Agents often lack modern notification capabilities, leaving administrators unaware of failures until manual checks occur.

Before implementing our solution, ensure:

  • HP ProLiant Support Pack (PSP) installed
  • Working SMTP server accessible from your network
  • Basic familiarity with command-line tools
  • Administrative access to the servers

We'll leverage HP's hpacucli utility combined with a simple shell script. Here's the complete implementation:


#!/bin/bash
# HP Drive Health Monitor Script
MAILTO="admin@yourdomain.com"
SMTP_SERVER="smtp.yourdomain.com"
TEMP_FILE="/tmp/hp_drive_status.txt"

# Check controller status
/usr/sbin/hpacucli controller all show status > $TEMP_FILE

# Parse for failures
if grep -q "Failed" $TEMP_FILE || grep -q "Predictive Failure" $TEMP_FILE; then
    mailx -s "ALERT: HP DL380 G5 Drive Failure Detected" -S smtp=$SMTP_SERVER $MAILTO < $TEMP_FILE
fi

# Clean up
rm $TEMP_FILE

1. Save the script as /usr/local/bin/hp_drive_monitor.sh

2. Make it executable:

chmod +x /usr/local/bin/hp_drive_monitor.sh

3. Add to cron (run every 30 minutes):

*/30 * * * * /usr/local/bin/hp_drive_monitor.sh

For environments preferring SNMP:


# Configure HP System Management Homepage
1. Access https://[server-ip]:2381
2. Navigate to Administration → Alert Mail Settings
3. Configure SMTP server and recipient
4. Set severity threshold to "Warning"

Missing hpacucli: Install via:

yum install hpacucli # For RHEL/CentOS
apt-get install hpacucli # For Debian/Ubuntu

Mail delivery failures: Test SMTP connectivity first:

echo "Test message" | mailx -s "SMTP Test" -S smtp=$SMTP_SERVER $MAILTO

html

For HP ProLiant DL380 G5 servers, drive failure notifications typically rely on HP's Management Agents and the Integrated Lights-Out (iLO) system. The key components involved are:

  • HP System Management Homepage (SMH)
  • HP Insight Management Agents
  • SMTP configuration capabilities
  • SNMP trap forwarding (alternative method)

Here's the most straightforward approach without server reboots:

# First, verify HP agents are installed
rpm -qa | grep hp-snmp-agents
rpm -qa | grep hp-health

# If missing, install them without reboot
wget http://downloads.hp.com/pub/softlib2/software1/pubsw-linux/p1234567890/v12345/hp-health-10.xx.x.x.x.x.rpm
rpm -Uvh --nodeps hp-health-*.rpm

# Configure SMTP in hpasmcli
hpasmcli -s "SHOW SMTP"
hpasmcli -s "SET SMTP SERVER smtp.yourdomain.com"
hpasmcli -s "SET SMTP SENDER serveralert@yourdomain.com"
hpasmcli -s "SET SMTP ALERT-EMAIL recipient@yourdomain.com"

For environments where direct SMTP isn't available:

# Configure SNMP in /etc/snmp/snmpd.conf
traphandle .1.3.6.1.4.1.232.1 /usr/local/bin/handle_drive_failure.sh

# Sample handler script (handle_drive_failure.sh)
#!/bin/bash
read oid
read host
read desc
if [[ "$desc" == *"physicaldrive"*"failed"* ]]; then
  echo "$desc" | mail -s "Drive Failure Alert" admin@yourdomain.com
fi

If alerts aren't working:

  • Verify the HP agents are running: service hpasm restart
  • Check SMTP connectivity: telnet smtp.yourdomain.com 25
  • Test alert generation: hpasmcli -s "TEST SMTP ALERT"
  • Review logs: tail -f /var/log/hp-snmp-agents/hp-health.log

For more granular control:

# Configure specific severity levels
hpasmcli -s "SET SMTP ALERT-LEVEL critical,warning"

# Multiple recipients (comma separated)
hpasmcli -s "SET SMTP ALERT-EMAIL admin1@domain.com,admin2@domain.com"

# Customize email subject
echo "EVENT-SUBJECT: [CRITICAL] Server Alert from %hostname%" > /etc/hp-snmp-agents/event.conf