When SMART reports unreadable sectors (like LBA 0x00d39eee in your logs), we're dealing with physical media degradation. Unlike full disk failures, single bad blocks can often be salvaged through sector reallocation - if handled properly.
WD's diagnostic tools indeed blindly overwrite bad blocks. What we need instead is controlled reallocation that:
- Preserves filesystem metadata visibility
- Minimizes data destruction
- Works on mounted systems when possible
First verify the bad block location:
smartctl -l xerror /dev/sda
For ext3 filesystems, we can force reallocation while mounted using this sequence:
# Convert LBA to physical sector (usually 512-byte sectors)
SECTOR=$((13868782 * 512 / $(blockdev --getss /dev/sda)))
# Force read to trigger reallocation
dd if=/dev/sda of=/dev/null bs=512 count=1 skip=$SECTOR iflag=direct
Since you're using LVM, we need to account for the additional mapping layer. First identify the physical extent:
pvdisplay --maps /dev/sda | grep -B1 $(bc <<< "ibase=16; D39EEE")
Then verify if it's metadata or actual file data using debugfs:
debugfs -R "icheck $SECTOR" /dev/mapper/vg-root
For recurring cases, I've used this wrapper script:
#!/bin/bash
DEV=$1
LBA=$2
SECTOR=$(($LBA * 512 / $(blockdev --getss $DEV)))
echo "Attempting reallocation of sector $SECTOR on $DEV"
# First try normal read
dd if=$DEV of=/dev/null bs=512 count=1 skip=$SECTOR iflag=direct 2>/dev/null
# Check if still pending
if smartctl -l xerror $DEV | grep -q $LBA; then
echo "Forcing reallocation with write"
dd if=/dev/zero of=$DEV bs=512 count=1 seek=$SECTOR oflag=direct
fi
If the bad block contained file data, use ext3grep to identify affected files:
ext3grep /dev/mapper/vg-root --ls --inode $(debugfs -R "icheck $SECTOR" /dev/mapper/vg-root | awk '/Block/ {print $2}')
When your Linux system reports SMART errors like "Currently unreadable (pending) sectors" through smartd
, it indicates the disk has identified problematic blocks but hasn't yet reallocated them. The key information we extract from your logs:
Error: UNC at LBA = 0x00d39eee = 13868782
This gives us the exact logical block address (LBA) we need to target. Before proceeding though, let's confirm the disk's reallocation status:
smartctl -a /dev/sdX | grep -E "Reallocated|Pending|Uncorrectable"
While traditional methods involve full disk scans with badblocks
, we can be surgical with these steps:
# 1. Identify the affected block in filesystem context
filefrag -v -b4096 /mount/point | grep 13868782
# 2. Force read attempt to trigger reallocation (requires root)
dd if=/dev/sdX of=/dev/null bs=4096 count=1 skip=13868782 iflag=direct
# 3. Verify reallocation occurred
smartctl -A /dev/sdX | grep Reallocated_Sector_Ct
For your LVM/ext3 setup, we need extra precautions:
# Check if block contains filesystem metadata
debugfs -R "icheck 13868782" /dev/mapper/vg-root
# If it returns inode numbers, check their importance:
debugfs -R "stat <inode>" /dev/mapper/vg-root
Here's a bash script that automates the verification and recovery process:
#!/bin/bash
DEVICE="/dev/sdX"
LBA="13868782"
BLOCK_SIZE=4096
# Convert LBA to filesystem block
FSBLOCK=$(($LBA * 512 / $BLOCK_SIZE))
echo "[+] Forcing read of problematic block $LBA (fs block $FSBLOCK)"
dd if=$DEVICE of=/dev/null bs=$BLOCK_SIZE count=1 skip=$FSBLOCK iflag=direct 2>&1 | grep -v records
echo "[+] Checking SMART status"
RECOUNT=$(smartctl -A $DEVICE | grep Reallocated_Sector_Ct | awk '{print $10}')
if [ "$RECOUNT" -gt 0 ]; then
echo "[✓] Success: $RECOUNT sectors reallocated"
else
echo "[!] Warning: No reallocation occurred - run extended SMART test"
smartctl -t long $DEVICE
fi
After forcing reallocation, verify filesystem integrity:
# For ext3 filesystems
e2fsck -v -f /dev/mapper/vg-root
# Check for any damaged files
grep "15868782" /var/log/messages* | grep -i error
Remember to monitor the disk closely going forward - a single reallocated sector often indicates future failures. Consider adding this to your regular maintenance cron jobs:
# Weekly extended SMART test
0 3 * * 0 /usr/sbin/smartctl -t long /dev/sdX