After six months of hands-on experience with OpenSolaris and ZFS across Sun Fire x4540 and commodity hardware, I've identified a critical pain point: most RAID controllers (like PERC and Smart Array) fail to provide proper JBOD disk passthrough. While workarounds exist - such as creating multiple RAID 0 virtual disks - these compromise ZFS's native hotswap capabilities and require reboots during disk replacements.

For optimal ZFS operation, these are my top controller recommendations:

# lspci -nn | grep -i "SAS\|SATA"
01:00.0 Serial Attached SCSI controller [0107]: Broadcom / LSI SAS2008 PCI-Express Fusion-MPT SAS-2 [1000:0072] (rev 03)

BBWC presents an interesting paradox in ZFS environments:

• Pro: Dramatically improves synchronous write performance (critical for NFS/iSCSI)
• Con: Creates potential data consistency risks during power failures

Here's how to properly configure a common HBA for ZFS:

# Check controller mode
sas2flash -listall

# Verify disk visibility
format <<EOF
EOF

# ZFS pool creation with direct disks
zpool create tank \
  mirror /dev/disk/by-id/ata-WDC_WD40EFZX-68AWUN0_WD-WX32D80HKD7E \
  mirror /dev/disk/by-id/ata-WDC_WD40EFZX-68AWUN0_WD-WX32D80HKD7F

After half a year of hands-on experience with OpenSolaris and ZFS across Sun Fire x4540 and commodity hardware platforms, I've identified a critical hardware limitation: most mainstream RAID controllers (like Dell PERC or HP SmartArray) lack true JBOD passthrough functionality. While workarounds exist - such as creating multiple RAID 0 virtual disks - these solutions compromise essential ZFS features like hot-swap capability, forcing system reboots during disk replacements.

For optimal ZFS performance and management, these SAS/SATA HBAs are field-tested solutions:

# LSI/Broadcom/Avago based controllers (gold standard)
- LSI 9211-8i (PCIe 3.0, SAS2008 chipset)
- LSI 9300-8i (PCIe 3.0, SAS3008 chipset)
- Dell H200/H310 (crossflashed to IT mode)

# Marvell based alternatives
- Adaptec ASR-71605H (true HBA mode available)
- HighPoint RocketRAID 2720SGL

These controllers support proper disk passthrough when configured in "IT mode" (Initiator Target), presenting raw disks to ZFS without RAID abstraction layers.

Battery-Backed Write Cache presents both opportunities and challenges:

• Pros: Dramatically improves synchronous write performance (critical for NFS/SMB)
• Cons: Potential data consistency risks if not properly handled

ZFS best practice configuration for BBWC controllers:

# For LSI controllers with cache module
sas2flash -list # verify controller details
megacli -LDSetProp WB -LAll -aAll # force write-back mode
zpool create tank mirror sda sdb cache /dev/sdc # example pool with cache device

Here's my current production setup on a Supermicro chassis:

# Controller detection
$ pciconf -lv | grep -A4 sas
mpt0@pci0:3:0:0: class=0x010700 card=0x30e0103c chip=0x00721000 rev=0x02 hdr=0x00

# ZFS pool creation with proper disk visibility
$ zpool create -f tank raidz2 /dev/disk/by-id/scsi-35000c500a1b3e27b \
    /dev/disk/by-id/scsi-35000c500a1b3e27c \
    /dev/disk/by-id/scsi-35000c500a1b3e27d \
    /dev/disk/by-id/scsi-35000c500a1b3e27e

Proper HBA selection enables true hot-swap capability. With an LSI 9300-8i controller in IT mode:

1. Physically replace failed disk
2. Rescan SCSI bus without reboot: devfsadm -Cv
3. Replace in ZFS: zpool replace tank 1234567890 /dev/disk/by-id/newdisk

The entire process completes without system downtime - impossible with RAID controller workarounds.