How to Configure HP Smart Array P400 for ZFS Direct Disk Access (Disabling Hardware RAID)


2 views

When implementing ZFS on enterprise HP hardware, many administrators encounter this frustrating scenario: The Smart Array P400 controller automatically groups disks into hardware RAID arrays (typically RAID-5 by default), preventing ZFS from accessing physical disks directly. This fundamentally breaks ZFS's advanced features like:

  • End-to-end checksumming
  • Adaptive replacement algorithms
  • Per-disk failure detection

HP's documentation suggests this controller doesn't support true JBOD mode, but we have two working approaches:

Method 1: Individual RAID-0 Volumes

# Using HPACUCLI (example for disk 1:1:1):
hpacucli ctrl slot=1 create type=ld drives=1:1:1 raid=0
# Repeat for each disk (1:1:2, 1:1:3, etc)

This creates single-disk RAID-0 arrays, effectively passing through disks while satisfying the controller's requirement for RAID configuration.

Method 2: HBA Mode Flash (Advanced)

Some technicians have successfully cross-flashed the P400 with LSI firmware to enable true IT mode. Sample steps:

# Backup original firmware first!
megarec -writesbr 0 sbrempty.bin
megarec -cleanflash 0
sas2flsh -o -f 2118it.bin -b mptsas2.rom

Warning: This voids warranty and carries bricking risk. Proceed only if you have recovery hardware.

After either method, verify disk visibility in Linux:

ls -l /dev/disk/by-id/
# Should show individual disks like:
# scsi-3600508b1001c0ad0 -> ../../sda
# scsi-3600508b1001c0ad1 -> ../../sdb

# ZFS pool creation example:
zpool create tank \
  mirror scsi-3600508b1001c0ad0 scsi-3600508b1001c0ad1 \
  mirror scsi-3600508b1001c0ad2 scsi-3600508b1001c0ad3

When using the RAID-0 method, you might want to disable controller caching:

hpacucli ctrl slot=1 modify nobatterywritecache=enable
hpacucli ctrl slot=1 modify driveswritecache=disable

This prevents potential data consistency issues while allowing ZFS's superior caching algorithms to work.

For critical ZFS deployments, consider replacing the P400 with:

  • LSI 9211-8i (flashed to IT mode)
  • Adaptec 71605 (true HBA mode support)
  • HP H240 (official HBA variant)

The investment often pays off in better performance and simpler maintenance.

When implementing ZFS on enterprise hardware like HP ProLiant servers, we often encounter this fundamental conflict: hardware RAID controllers want to manage disks, while ZFS needs direct disk access. The HP Smart Array P400 controller typically presents disks as RAID members by default, forcing them into hardware RAID configurations.

ZFS requires raw disk access to implement its advanced features:

  • End-to-end checksumming for data integrity
  • Adaptive replacement caching
  • Efficient software RAID (mirroring, RAID-Z)
  • Predictive failure analysis

When disks are hidden behind hardware RAID, ZFS loses visibility into the actual physical devices.

For the HP ML370 G5 with P400 controller, follow these steps:

# Access the controller BIOS during boot (Ctrl+M)
1. Select "Array Configuration Utility"
2. Choose "Create Array"
3. For each physical disk:
   - Press Enter on the disk
   - Select "Make Unassigned Drive"
4. Save configuration and exit

After configuration, check disk visibility in Linux:

ls -l /dev/disk/by-id/
# Should show individual SAS disks like:
# scsi-3600508b1001c1c3e2d4a5b6c7d8e9f0 -> ../../sda
# scsi-3600508b1001c1c3e2d4a5b6c7d8e9f1 -> ../../sdb

With individual disks visible, create your ZFS pool:

zpool create tank \
  mirror scsi-3600508b1001c1c3e2d4a5b6c7d8e9f0 \
         scsi-3600508b1001c1c3e2d4a5b6c7d8e9f1 \
  mirror scsi-3600508b1001c1c3e2d4a5b6c7d8e9f2 \
         scsi-3600508b1001c1c3e2d4a5b6c7d8e9f3 \
  raidz1 scsi-3600508b1001c1c3e2d4a5b6c7d8e9f4 \
         scsi-3600508b1001c1c3e2d4a5b6c7d8e9f5 \
         scsi-3600508b1001c1c3e2d4a5b6c7d8e9f6

When using the P400 in JBOD mode:

  • Disable controller cache (battery-backed cache interferes with ZFS)
  • Set disk timeout to 30 seconds in /etc/sysctl.conf: 
    vm.blockdev.timeout_seconds = 30
  • Monitor disk temperatures - the controller may no longer report them

If JBOD mode isn't available:

  1. Configure each disk as single-disk RAID0
  2. Use the controller's HBA mode if supported (requires firmware update)
  3. Consider replacing with a true HBA like LSI 9200-8e