When managing storage in Linux environments, the relationship between LVM (Logical Volume Manager) and software RAID (mdadm) is critical. There are two primary approaches:
# LVM-on-RAID (Traditional approach)
Physical Disks -> RAID -> LVM -> Filesystem
# RAID-on-LVM (Alternative approach)
Physical Disks -> LVM -> RAID -> Filesystem
LVM on Software RAID Advantages:
- Better performance as RAID operates directly on physical devices
- Proven stability with mdadm handling disk failures
- Easier disk replacement in degraded arrays
- Supports all RAID levels (0,1,5,6,10)
RAID on LVM Advantages:
- More flexible volume management before RAID configuration
- Easier to resize underlying physical volumes
- Potential for more complex storage topologies
Benchmark tests consistently show better throughput with LVM-on-RAID:
# Sample benchmark command for LVM-on-RAID
hdparm -tT /dev/mapper/vg0-lv0
# Expected results on SSD RAID10:
Timing cached reads: 3000 MB in 2.00 seconds
Timing buffered reads: 1800 MB in 3.00 seconds
Here's how to properly set up LVM on software RAID:
# Create RAID array
mdadm --create /dev/md0 --level=1 --raid-devices=2 /dev/sda /dev/sdb
# Prepare for LVM
pvcreate /dev/md0
vgcreate vg0 /dev/md0
lvcreate -L 100G -n lv0 vg0
# Filesystem creation
mkfs.ext4 /dev/vg0/lv0
This architecture might make sense when:
- Using advanced LVM features like thin provisioning
- Implementing complex storage tiering
- Needing frequent volume resizing
For most production environments, I recommend:
- Use LVM-on-RAID for better performance and reliability
- Monitor RAID health with
mdadm --detail /dev/mdX
- Implement proper LVM snapshots for backups
- Consider write-intent bitmaps for RAID rebuilds
When setting up storage systems in Linux environments, the relationship between LVM (Logical Volume Manager) and software RAID creates fundamental architectural decisions. The two primary approaches are:
- LVM on top of software RAID (mdadm → LVM)
- RAID implemented within LVM (LVM → RAID)
This remains the most battle-tested configuration in production environments:
# Creating RAID first
mdadm --create /dev/md0 --level=1 --raid-devices=2 /dev/sda1 /dev/sdb1
# Then setting up LVM
pvcreate /dev/md0
vgcreate vg_raid /dev/md0
lvcreate -L 100G -n lv_data vg_raid
Key Advantages
- Stable performance characteristics: The RAID layer handles striping/mirroring before LVM sees the storage
- Easier recovery: MDADM tools are mature for array reconstruction
- Clear separation of concerns: RAID handles redundancy, LVM handles volume management
Modern LVM versions support RAID types natively through device mapper:
# Direct LVM RAID1 creation
pvcreate /dev/sda1 /dev/sdb1
vgcreate vg_raid /dev/sda1 /dev/sdb1
lvcreate --type raid1 -L 100G -n lv_data vg_raid
Potential Benefits
- Simplified stack: Fewer layers in the storage subsystem
- Flexible reconfiguration: Easier to convert between RAID levels
- Thin provisioning integration: Works seamlessly with LVM thin pools
Benchmark results from a 4-disk system (WD Red 4TB, XFS filesystem):
Configuration | Sequential Read | Random 4K Write |
---|---|---|
LVM on RAID5 | 420 MB/s | 7800 IOPS |
RAID5 via LVM | 390 MB/s | 7200 IOPS |
LVM on RAID10 | 580 MB/s | 14200 IOPS |
RAID10 via LVM | 550 MB/s | 13800 IOPS |
System Maintenance
With LVM-on-RAID, you'll typically:
# Check array status
mdadm --detail /dev/md0
# Then check LVM status
vgdisplay vg_raid
Whereas LVM-RAID consolidation provides:
# Single command for status
lvdisplay --maps vg_raid/lv_data
Recovery Scenarios
Disk failures behave differently:
- Traditional MDADM: Clear degraded array warnings in syslog
- LVM RAID: Requires checking both LVM and kernel raid events
For database workloads: Stick with mdadm RAID10 + LVM for best predictable performance
For flexible development environments: Consider LVM RAID for easier reconfiguration between RAID levels
For large storage arrays: The mdadm layer still provides more mature monitoring tools