Understanding SAS Expander Transparency with RAID Controllers: A Deep Dive into Multi-Expander Configurations


3 views

When working with SAS (Serial Attached SCSI) storage architectures, expanders function similarly to network switches by enabling multiple devices to share a single controller connection. The key advantage is transparent operation - the controller sees all downstream devices as if they were directly connected, maintaining full compatibility with RAID operations.

The LSI MegaRAID 9260-16i you mentioned perfectly illustrates this concept. With its 4 SFF-8087 ports (16 lanes total), it can:

  • Directly connect 16 drives (4 per port)
  • Connect to expanders that each support dozens of additional drives

Yes, you can absolutely create RAID arrays spanning multiple expanders. The controller's firmware handles this transparently. For example:


# Sample MegaCLI command showing RAID creation across expanders
./MegaCli64 -CfgLdAdd -r10[252:0,252:1,252:2,252:3] WB RA Cached -a0
# Where 252:0 might be on Expander 1 and 252:3 on Expander 3

While the architecture supports this flexibility, consider these factors:

  1. Bandwidth Allocation: Each expander connection shares the bandwidth of its upstream port
  2. Expander Management: Most modern SAS expanders support SMP (Serial Management Protocol) for monitoring
  3. Zoning Capabilities: Advanced expanders allow partitioning of the SAS domain

When spanning RAID arrays across expanders, performance characteristics change:

Configuration IOPS Impact Latency Impact
Single Expander Minimal +5-10μs
Multiple Expanders 10-15% reduction +15-25μs

When working with multi-expander setups, these CLI commands prove invaluable:


# List all physical disks regardless of expander
./MegaCli64 -PDList -aALL

# Check expander health status
./MegaCli64 -AdpGetSASPhy -a0

# Verify end-to-end connectivity
./sas2ircu 0 display

In SAS (Serial Attached SCSI) architectures, expanders function similarly to network switches, enabling connection scalability while maintaining protocol transparency. When using an LSI MegaRAID 9260-16i controller with 4 SFF-8087 ports (16-drive capacity), expanders multiply connectivity without requiring special controller configuration.

The SAS controller enumerates all devices through expanders exactly as if directly connected, maintaining complete endpoint visibility. The SAS protocol's hierarchical addressing (similar to SCSI LUNs) handles device identification:

# Sample Linux CLI showing expander transparency
$ lsscsi
[0:2:0:0]   disk    LSI      MR9260-16i      4.13  /dev/sda
[0:2:1:0]   disk    LSI      Expander1       1.23  /dev/sdb
[0:2:2:0]   disk    LSI      Expander2       1.23  /dev/sdc

For the 9260-16i controller specifically:

  • RAID groups can combine disks from any connected expander
  • Performance characteristics match direct-attached disks
  • Zoning limitations only apply in SAS-2.1+ topologies

Example MegaCLI command creating a RAID10 across expanders:

MegaCli -CfgSpanAdd -r10 -Array0[32:2,32:5] -Array1[64:1,64:6] -a0
# Where:
# 32: = expander 1 SAS address
# 64: = expander 2 SAS address

When using multiple expanders with the 9260-16i:

  1. Ensure expander firmware compatibility (minimum v1.23 for LSI units)
  2. Monitor link bandwidth saturation (4x6Gbps per SFF-8087)
  3. For optimal performance, balance drives across expander ports

Use these SAS topology inspection tools:

# View physical topology
$ sas2ircu 0 display

# Check link speeds
$ cat /sys/class/sas_device/expander-0\:0\:0/device/phy-0\:0\:0/negotiated_linkrate