When working with NVMe SSDs in Linux, you'll encounter two distinct device file types:
crw------- 1 root root 243, 0 Dec 12 16:09 /dev/nvme0 # Character device
brw-rw---- 1 root disk 259, 0 Jan 14 01:30 /dev/nvme0n1 # Block device
The character device (/dev/nvme0
) provides raw access to the NVMe controller. It's used for:
- Low-level controller management
- Sending admin commands (format, firmware update, etc.)
- Retrieving SMART data and logs
Example using nvme-cli
to get SMART data:
sudo nvme smart-log /dev/nvme0
The block devices (/dev/nvme0n1
, /dev/nvme0n2
, etc.) represent:
- Actual namespaces (storage partitions)
- Regular block storage operations
- Filesystem creation and mounting
Example creating a filesystem:
sudo mkfs.ext4 /dev/nvme0n1
sudo mount /dev/nvme0n1 /mnt/nvme
The character device enables direct controller access while bypassing Linux block layer:
// Example C code for raw NVMe access
int fd = open("/dev/nvme0", O_RDWR);
ioctl(fd, NVME_IOCTL_ADMIN_CMD, &cmd);
Block devices provide buffered, filesystem-friendly access:
// Regular file operations work normally
FILE *f = fopen("/mnt/nvme/file.txt", "w");
fprintf(f, "Hello NVMe!");
Use Case | Device Type |
---|---|
Storage operations | Block device |
Performance testing | Both (compare results) |
Controller management | Character device |
Developing NVMe tools | Character device |
Raw character device access can yield lower latency but requires more development effort. The Linux NVMe driver automatically creates both interfaces to support different access patterns.
Example comparing performance:
# Test block device
fio --filename=/dev/nvme0n1 --direct=1 --rw=randread --bs=4k --numjobs=1 --runtime=60
# Test raw device (requires custom tool)
nvme perf /dev/nvme0 -s 4096 -q 32 -t 60
When working with NVMe drives in Linux, you'll notice two distinct device files appear in /dev
:
crw------- 1 root root 243, 0 Dec 12 16:09 /dev/nvme0
brw-rw---- 1 root disk 259, 0 Jan 14 01:30 /dev/nvme0n1
The first (/dev/nvme0
) is a character device (denoted by 'c'), while the second (/dev/nvme0n1
) is a block device (denoted by 'b'). This distinction exists because they serve fundamentally different purposes in the system.
The character device (/dev/nvme0
) provides raw access to the NVMe controller. This is primarily used for:
- Administrative commands (formatting, firmware updates)
- Controller-level operations
- Low-level debugging
Here's an example of using the character device to get controller information:
#include <linux/nvme_ioctl.h>
#include <sys/ioctl.h>
#include <fcntl.h>
int main() {
int fd = open("/dev/nvme0", O_RDWR);
struct nvme_admin_cmd cmd = {
.opcode = nvme_admin_identify,
.nsid = 0, // 0 for controller
.addr = (__u64)buffer,
.data_len = 4096
};
ioctl(fd, NVME_IOCTL_ADMIN_CMD, &cmd);
close(fd);
return 0;
}
The block device (/dev/nvme0n1
) represents the actual namespace (storage area) and is used for:
- Regular filesystem operations
- Partition management
- Standard I/O operations
Example of reading from the block device:
#include <fcntl.h>
#include <unistd.h>
int main() {
int fd = open("/dev/nvme0n1", O_RDONLY);
char buffer[512];
read(fd, buffer, sizeof(buffer));
close(fd);
return 0;
}
Use the character device when:
- You need to send NVMe admin commands
- Performing controller management tasks
- Developing NVMe driver-level features
Use the block device when:
- Working with filesystems
- Performing regular read/write operations
- Using standard storage utilities (dd, fdisk, etc.)
Here's how you might use both device types together:
// First use character device to format
int ctrl_fd = open("/dev/nvme0", O_RDWR);
struct nvme_format_cmd fmt_cmd = {
.opcode = nvme_admin_format_nvm,
.nsid = 1, // namespace ID
.cdw10 = (1 << 4) // set format bit
};
ioctl(ctrl_fd, NVME_IOCTL_ADMIN_CMD, &fmt_cmd);
close(ctrl_fd);
// Then use block device to create filesystem
system("mkfs.ext4 /dev/nvme0n1");
While the block device offers convenient filesystem access, the character device provides lower latency for certain operations. For high-performance applications, you might want to bypass the block layer entirely:
// Bypass filesystem for direct I/O
int fd = open("/dev/nvme0n1", O_RDWR | O_DIRECT);
posix_memalign(&buffer, 4096, 4096);
read(fd, buffer, 4096);