Linux SCSI Address Mapping: Why Virtual Disks Show Different Host/Target IDs in VM Environments


2 views

When working with virtualized SCSI disks in Linux environments, you'll often encounter discrepancies between:

  • The SCSI addresses configured in your hypervisor (e.g., VMware ESX)
  • The addresses reported by Linux kernel SCSI subsystem
  • The addresses shown in BIOS or other operating systems

The Linux kernel handles SCSI addressing through a multi-layered approach:

struct scsi_device {
    unsigned int id;      // Target ID
    unsigned int lun;     // LUN
    unsigned int channel; // Virtual channel (often 0)
    struct Scsi_Host *host; // Pointer to host adapter
};

The key components that affect address translation:

  1. SCSI host controller driver registration order
  2. Bus rescan and device enumeration sequence
  3. Virtualization layer abstraction

Based on your test cases, we can identify these mapping patterns:

Driver vHW Pattern Linux Pattern
LSI SAS 0:X 0:(X/3)
LSI SCSI 1:X 2:X
pvSCSI 2:X 1:X

To programmatically match virtual hardware addresses to Linux addresses:

#!/bin/bash
# Map VMware SCSI addresses to Linux device names
map_scsi_address() {
    local driver=$1
    local hw_host=$2
    local hw_target=$3
    
    case $driver in
        "lsi_sas")
            echo "0:$((hw_target/3))"
            ;;
        "lsi_scsi")
            echo "2:$hw_target"
            ;;
        "pvscsi")
            echo "1:$hw_target"
            ;;
        *)
            echo "$hw_host:$hw_target"
            ;;
    esac
}

# Example usage:
map_scsi_address "lsi_sas" 0 6  # Returns 0:2

To debug SCSI host assignments:

# View SCSI host controller assignments
cat /proc/scsi/scsi

# Check driver loading order
dmesg | grep -i scsi

# Examine udev device rules
udevadm info -q all -n /dev/sdX

Create udev rules for consistent device naming:

# /etc/udev/rules.d/99-vmware-scsi.rules
ACTION=="add", SUBSYSTEM=="scsi", ATTR{vendor}=="VMware*", \
ATTR{model}=="Virtual disk*", ATTR{wwid}=="*", \
RUN+="/bin/sh -c 'echo $kernel > /etc/vmware-disks/%s{host}-%s{target}-%s{lun}'"

Linux determines SCSI device addressing through a multi-layered process involving both kernel-level drivers and hardware interactions. The addressing scheme consists of three components:

host:channel:target:lun

In virtualized environments like VMware, this gets particularly interesting because:

  1. The hypervisor presents virtual SCSI controllers
  2. Each controller gets assigned a host number
  3. Virtual disks receive target IDs within their controller

From your observations, we can identify clear patterns in how different VMware SCSI drivers handle addressing:

Driver Virtual HW Addr Linux Detected Addr Pattern
LSI SAS 0:0 0:0 Target ID preserved
LSI SAS 0:3 0:1 Target ID remapped
LSI SCSI 1:1 2:1 Host number incremented
pvSCSI 2:2 1:2 Host numbers reversed

To understand what's happening, we can examine the SCSI host assignments:

# cat /proc/scsi/scsi
Attached devices:
Host: scsi0 Channel: 00 Id: 00 Lun: 00
  Vendor: VMware   Model: Virtual disk     Rev: 1.0
Host: scsi0 Channel: 00 Id: 01 Lun: 00
  Vendor: VMware   Model: Virtual disk     Rev: 1.0

The discrepancy occurs because:

  • Different SCSI drivers may register their host controllers in different orders
  • Some drivers perform target ID remapping for compatibility reasons
  • The Linux SCSI midlayer may reassign host numbers during initialization

For consistent device identification, consider these approaches:

Method 1: Using by-path naming

# ls -l /dev/disk/by-path/
total 0
lrwxrwxrwx 1 root root  9 Apr 30 10:00 pci-0000:02:00.0-scsi-0:0:0:0 -> ../../sda
lrwxrwxrwx 1 root root 10 Apr 30 10:00 pci-0000:02:00.0-scsi-0:0:1:0 -> ../../sdb

Method 2: Custom udev rules

# /etc/udev/rules.d/99-vmware-scsi.rules
SUBSYSTEM=="block", ENV{ID_PATH}=="*-scsi-0:0:1:*", SYMLINK+="disk/vmware/%k"

Method 3: Querying through sysfs

# find /sys/class/scsi_device/*/device -name block -exec ls {}/sd* \;

For each driver type, we can implement specific solutions:

LSI Logic SAS

# Force consistent host numbering
options mptspi tm2Enable=0

LSI Logic Parallel SCSI

# Set explicit host numbers
options mptscsih mpt_msi_disable=1 hba_count=2

VMware Paravirtual SCSI

# Ensure proper host assignment
options vmw_pvscsi ring_pages=32 cmd_per_lun=254

When troubleshooting SCSI addressing issues:

# View SCSI host assignments
$ cat /sys/class/scsi_host/host*/proc_name

# Check driver parameters
$ systool -v -m mptspi

# Monitor SCSI events
$ scsi_logging_level -s -S -E -T -M 7
$ dmesg -w

Several boot parameters can influence SCSI host numbering:

scsi_mod.scan=sync
scsi_mod.default_dev_flags=0
scsi_mod.max_report_luns=255

These issues aren't unique to VMware environments. Similar behavior can occur with:

  • Physical HBAs in multi-controller systems
  • iSCSI implementations
  • Fibre Channel environments