Many developers encounter this exact situation when launching EC2 instances with instance store volumes. While AWS clearly shows the storage allocation during instance configuration (like the 850GB on m5d.medium instances), the storage isn't automatically mounted or formatted when the instance launches.
First, let's verify the available block devices. SSH into your instance and run:
lsblk
NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT
xvda 202:0 0 8G 0 disk
└─xvda1 202:1 0 8G 0 part /
xvdb 202:16 0 850G 0 disk
Here we can see the 850GB volume (/dev/xvdb
) that isn't yet mounted.
For Amazon Linux, follow these steps:
# Check if the volume has a filesystem
sudo file -s /dev/xvdb
# If output shows "/dev/xvdb: data", we need to create a filesystem
sudo mkfs -t ext4 /dev/xvdb
# Create a mount point
sudo mkdir /data
# Mount the volume
sudo mount /dev/xvdb /data
# Verify the mount
df -h
To ensure the volume mounts after reboots, add this to /etc/fstab
:
/dev/xvdb /data ext4 defaults,nofail 0 2
While instance stores offer high performance (better I/O than EBS), they're ephemeral. The data disappears when:
- The instance stops (not reboots)
- The instance terminates
- The underlying host fails
Perfect use cases include:
- Temporary cache files
- Processing large datasets where you can recreate the data
- High-performance temporary storage
- Batch processing jobs
For production environments, automate the mounting process using instance user data:
#!/bin/bash
mkfs -t ext4 /dev/xvdb
mkdir /data
mount /dev/xvdb /data
echo "/dev/xvdb /data ext4 defaults,nofail 0 2" >> /etc/fstab
Instance stores provide:
- Higher throughput than EBS volumes
- Lower latency
- No additional cost (included in instance price)
For database workloads, consider combining instance store for temporary tables with EBS for persistent data.
For persistent storage needs, EBS volumes are better despite their performance trade-offs because they:
- Persist independently of instance state
- Support snapshots
- Can be encrypted
When launching an EC2 instance with instance store volumes, you're dealing with physically attached storage that offers high performance but is ephemeral in nature. Unlike EBS volumes that persist independently of the instance lifecycle, instance store volumes are tied to the host machine.
Many developers encounter this issue where the secondary storage doesn't automatically appear. Here's what's happening:
# Check available block devices
lsblk
# You might see only:
# NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT
# xvda 202:0 0 8G 0 disk
# └─xvda1 202:1 0 8G 0 part /
The device might appear under a different name like /dev/xvdb or /dev/nvme1n1 depending on the instance type.
Here's a step-by-step process to make your 850GB available:
# 1. Identify the raw device
sudo fdisk -l
# 2. Format the volume (only needed first time)
sudo mkfs -t ext4 /dev/xvdb
# 3. Create mount point
sudo mkdir /data
# 4. Mount the volume
sudo mount /dev/xvdb /data
# 5. Verify
df -h
To ensure the volume mounts automatically after reboots, add this to /etc/fstab:
/dev/xvdb /data ext4 defaults,nofail 0 2
While AWS recommends EBS for persistence, instance store shines for:
- Temporary cache files
- High-performance scratch space
- Processing jobs with built-in redundancy
- CI/CD build environments
Here's how you might use instance storage in a data processing application:
import os
import shutil
# Set working directory on instance store
WORK_DIR = "/data/processing"
def setup_environment():
if not os.path.exists(WORK_DIR):
os.makedirs(WORK_DIR)
def process_large_file(input_path):
# Copy to high-speed instance store
temp_path = os.path.join(WORK_DIR, "temp_file.dat")
shutil.copy2(input_path, temp_path)
# Process file (example: simple analysis)
with open(temp_path, 'r') as f:
data = f.read()
# Processing logic here
# Clean up
os.remove(temp_path)
Instance store typically offers:
- Higher IOPS than standard EBS
- Lower latency as it's physically attached
- No network overhead like EBS
Since instance store is ephemeral, implement regular sync to S3:
# Simple sync script example
aws s3 sync /data s3://your-bucket/instance-store-backup/