When you encounter a failed disk in your ZFS pool showing only a GPTID like gptid/5fe33556-3ff2-11e2-9437-f46d049aaeca but it's not listed in glabel status, here's how to track it down systematically:

# First get the physical device mapping
zdb -C | grep -A10 "gptid/5fe33556"
# This might reveal the underlying device like ada2

The camcontrol devlist output shows a gap in device numbering - we see ada0, ada1, ada3 but no ada2. This strongly suggests ada2 is the failed device. To confirm:

# Check dmesg for disk errors
dmesg | grep -i ada2
# Or check smartctl if the disk is partially responsive
smartctl -a /dev/ada2

Since you have ada0 available as a spare, here's how to perform a remote replacement:

# First offline the failed disk
zpool offline tank gptid/5fe33556-3ff2-11e2-9437-f46d049aaeca

# Then replace using the spare
zpool replace tank gptid/5fe33556-3ff2-11e2-9437-f46d049aaeca /dev/ada0

# Monitor resilvering progress
zpool status -v

For more precise disk identification (especially in large arrays):

# Map GPTIDs to physical devices
gpart list | grep -B5 "gptid/5fe33556"

# Get serial numbers directly
for disk in /dev/ada*; do
  echo -n "$disk: "; smartctl -i $disk | grep -i serial
done

To avoid this situation in the future:

# Create a disk mapping file for reference
zpool status -v > /root/zpool_disk_mapping_$(date +%Y%m%d).txt
camcontrol devlist >> /root/zpool_disk_mapping_$(date +%Y%m%d).txt

When working with ZFS pools on FreeNAS, disk failures can present unique identification challenges due to the abstraction layers between physical disks and their representation in the storage pool.

The initial zpool status output shows:

gptid/5fe33556-3ff2-11e2-9437-f46d049aaeca  UNAVAIL      0     0     0  cannot open

This indicates a disk has failed, but the GPTID alone doesn't directly map to physical hardware.

The glabel status output provides crucial mapping information:

gptid/5f3c0517-3ff2-11e2-9437-f46d049aaeca     N/A  ada1p2
gptid/60570005-3ff2-11e2-9437-f46d049aaeca     N/A  ada3p2
gptid/60ebeaa5-3ff2-11e2-9437-f46d049aaeca     N/A  ada4p2

Notice the missing ada2 in your camcontrol devlist output while seeing consecutive ada devices from 0-10 confirms ada2 is the failed disk.

To get comprehensive disk information, run:

smartctl -i /dev/ada2

If the disk is truly failed, this command will likely timeout or return errors.

For remote replacement using ada0 (your spare), execute:

zpool replace tank gptid/5fe33556-3ff2-11e2-9437-f46d049aaeca /dev/ada0

Then verify resilvering status:

zpool status -v

For more detailed analysis consider:

# List all GEOM providers
geom disk list

# Check disk health
smartctl -a /dev/ada2

# View kernel messages for disk errors
dmesg | grep ada2

1. Always document drive bay to device name mappings
2. Consider using disk labels instead of relying on device names
3. Set up email alerts for ZFS pool status changes
4. Regularly check SMART status on all drives