When monitoring disk I/O during a MySQL database restore operation, I encountered a puzzling discrepancy in iostat's Transfer Per Second (TPS) metrics between my RAID devices (md3) and the underlying physical disks (sda/sdb). The md3 device showed 1389.80 tps while the physical disks only showed ~144 tps - nearly a 10x difference.

In a RAID1 configuration, every write operation must be duplicated to both physical disks. This creates additional I/O operations at the software RAID level that aren't immediately visible at the physical disk level.

# Breakdown of a single write operation in RAID1:
1. Application writes 4KB to /dev/md3
2. md driver:
   - Issues 4KB write to /dev/sda5
   - Issues 4KB write to /dev/sdb5
3. Physical disks:
   - Each processes one 4KB write

The key insight is that iostat reports different perspectives:

• md devices: Show logical operations from the filesystem perspective
• physical disks: Show actual hardware operations

While the TPS difference is normal for RAID1, the slow restore operation suggests other bottlenecks:

# Check for possible MySQL configuration issues:
SHOW VARIABLES LIKE 'innodb_buffer_pool_size';
SHOW VARIABLES LIKE 'innodb_flush_log_at_trx_commit';

For more detailed I/O analysis, consider these tools:

# Monitor per-process I/O
iotop -oP

# Detailed disk statistics
sudo smartctl -a /dev/sda

# Block layer tracing
blktrace -d /dev/sda -o trace

If performance remains suboptimal, consider these RAID-specific optimizations:

• Adjust the RAID stripe cache size: echo 16384 > /sys/block/md3/md/stripe_cache_size
• Verify your chunk size matches workload patterns
• Check for proper alignment of partitions and filesystems

During a recent MySQL database restoration from a 5.2GB dump file, I encountered unexpectedly slow performance on my RAID1 setup. While the md3 device showed extremely high TPS (1418.41), the underlying physical devices (sda/sdb) reported much lower values (around 145). This significant discrepancy warrants investigation.

The iostat output reveals several important indicators:

$ sudo iostat -m
Device:            tps    MB_read/s    MB_wrtn/s
sda             145.16         0.10         5.55
sdb             144.90         0.09         5.55
md3            1418.41         0.02         5.53

The tps (transfers per second) metric in iostat represents I/O requests to the device, not the actual physical operations. In RAID1 configurations, several factors can cause the MD device to report higher TPS than the underlying disks:

• Write barriers and synchronization requirements
• Journaling overhead
• Small I/O requests being split
• RAID1 mirroring operations

To better understand the storage behavior during the MySQL restore, consider monitoring with more detailed tools:

# Monitor disk activity per process
$ sudo iotop -oP

# Check block device queue depth
$ cat /sys/block/sda/queue/nr_requests

# Examine RAID synchronization status
$ cat /proc/mdstat

For large database restores, these MySQL configuration adjustments can help:

[mysqld]
innodb_flush_log_at_trx_commit = 2
innodb_buffer_pool_size = 1G
innodb_log_file_size = 256M
innodb_flush_method = O_DIRECT
skip-innodb_doublewrite

Key factors affecting RAID1 write performance:

• Write-mostly workload pattern of database restores
• MD stripe cache size (can be adjusted via /sys/block/mdX/md/stripe_cache_size)
• Filesystem journaling overhead (consider temporary disabling for large operations)
• Disk controller cache policy

To isolate the issue, run a controlled test comparing raw device vs MD performance:

# Test raw device
$ sudo hdparm -tT /dev/sda

# Test MD device
$ sudo dd if=/dev/zero of=/dev/md3 bs=1M count=1000 oflag=direct

For more detailed analysis, consider these monitoring alternatives:

# Detailed disk stats with await time
$ sudo iostat -x 1

# Process-level I/O monitoring
$ sudo pidstat -d 1

# Per-disk latency measurements
$ sudo ioping -c 10 /dev/md3

Several sysctl parameters can affect RAID performance:

# Adjust dirty page thresholds
vm.dirty_ratio = 10
vm.dirty_background_ratio = 5

# Increase I/O scheduler queue depth
block.queue_depth = 64

# Optimize for write-heavy workload
vm.swappiness = 1