Grub2 Multi-Disk Installation: Solving BIOS/Linux Device Order Mismatch on RAID Systems


2 views

When dealing with complex storage configurations, particularly systems with both internal and external disks (like your 7-disk setup), Linux device enumeration (/dev/sdX) often doesn't match the BIOS boot order. This creates a dangerous situation where GRUB might install to /dev/sda while the BIOS actually boots from /dev/sdd or another disk.

The most reliable solution is to install GRUB to all potential boot devices. Here's how to implement this on Debian stable with GRUB2:

# First install to the default location (usually sda)
grub-install /dev/sda

# Then install to all other disks in your system
for disk in sdb sdc sdd sde sdf sdg; do
    if [ -e /dev/$disk ]; then
        grub-install /dev/$disk
    fi
done

To ensure future update-grub operations maintain bootability across all disks, create a custom hook:

cat > /etc/grub.d/42_multidisk <<'EOF'
#!/bin/sh
exec tail -n +3 $0

for disk in /dev/sd[a-g]; do
    [ -e "$disk" ] && grub-install "$disk"
done
EOF

chmod +x /etc/grub.d/42_multidisk
update-grub

Check GRUB is properly installed to all devices:

for disk in /dev/sd[a-g]; do
    if [ -e "$disk" ]; then
        echo -n "Checking $disk: "
        dd if=$disk bs=512 count=1 2>/dev/null | strings | grep -q GRUB \
            && echo "GRUB found" || echo "No GRUB signature"
    fi
done

Since you mentioned RAID, be aware that some configurations (especially hardware RAID) might present virtual devices to the BIOS. In such cases, you might need to:

# Install to RAID array devices if present
if [ -e /dev/md0 ]; then
    grub-install /dev/md0
fi

If you want to identify which disk the BIOS is actually booting from, try this diagnostic method:

# Create unique identifiers on each disk
for disk in /dev/sd[a-g]; do
    if [ -e "$disk" ]; then
        echo "BIOS_TEST_$(basename $disk)" > /boot/$(basename $disk)_marker
    fi
done

# After reboot, check which marker exists
find /boot -name '*_marker' -exec cat {} \;

When dealing with multiple disks (especially in RAID configurations), we often encounter situations where BIOS reported disk order differs from Linux's device assignment. This creates boot problems because GRUB might install to /dev/sda while BIOS attempts to boot from /dev/sdd.

GRUB2 installation consists of two main components:

  1. Core bootloader code written to the disk's boot sector
  2. Configuration files and modules stored in /boot/grub

Here's how to properly install GRUB on all potential boot disks in Debian:

# First, identify all disks in the system
lsblk -d -o NAME,ROTA,SIZE,MODEL

# Install GRUB to each disk (repeat for all disks from sda to sdg)
for disk in /dev/sd{a..g}; do
    echo "Installing GRUB to $disk"
    grub-install --target=i386-pc --recheck --no-floppy --boot-directory=/boot $disk
done

# Update GRUB configuration
update-grub

For more reliable identification, consider using disk-by-id:

ls -l /dev/disk/by-id/

# Example installation using disk ID
grub-install --target=i386-pc /dev/disk/by-id/ata-ST3000DM001-1CH166_W502YZ99

For software RAID (mdadm) configurations, you need to:

  1. Install GRUB on each physical disk
  2. Ensure your RAID array includes a bootable partition
  3. Check that mdadm can assemble the array early enough in boot

Example for RAID1:

mdadm --detail --scan >> /etc/mdadm/mdadm.conf
update-initramfs -u

After installation, verify GRUB is properly installed on all disks:

# Check boot sector for GRUB signature
dd if=/dev/sda bs=512 count=1 2>/dev/null | strings | grep GRUB

# Check boot files existence
ls -l /boot/grub/

Create a script to handle GRUB updates across all disks:

#!/bin/bash
# /usr/local/bin/update-all-grub

DISKS=$(lsblk -d -o NAME | grep -E '^sd[a-z]$' | sed 's/^/\/dev\//')

for disk in $DISKS; do
    echo "Updating GRUB on $disk"
    grub-install --target=i386-pc $disk
done

update-grub