Understanding NVMe Device Types: When to Use Character (/dev/nvme0) vs Block (/dev/nvme0n1) Devices in Linux


1 views

When working with NVMe SSDs in Linux, you'll encounter two distinct device file types:

crw------- 1 root root 243, 0 Dec 12 16:09 /dev/nvme0  # Character device
brw-rw---- 1 root disk 259, 0 Jan 14 01:30 /dev/nvme0n1 # Block device

The character device (/dev/nvme0) provides raw access to the NVMe controller. It's used for:

  • Low-level controller management
  • Sending admin commands (format, firmware update, etc.)
  • Retrieving SMART data and logs

Example using nvme-cli to get SMART data:

sudo nvme smart-log /dev/nvme0

The block devices (/dev/nvme0n1, /dev/nvme0n2, etc.) represent:

  • Actual namespaces (storage partitions)
  • Regular block storage operations
  • Filesystem creation and mounting

Example creating a filesystem:

sudo mkfs.ext4 /dev/nvme0n1
sudo mount /dev/nvme0n1 /mnt/nvme

The character device enables direct controller access while bypassing Linux block layer:

// Example C code for raw NVMe access
int fd = open("/dev/nvme0", O_RDWR);
ioctl(fd, NVME_IOCTL_ADMIN_CMD, &cmd);

Block devices provide buffered, filesystem-friendly access:

// Regular file operations work normally
FILE *f = fopen("/mnt/nvme/file.txt", "w");
fprintf(f, "Hello NVMe!");
Use Case Device Type
Storage operations Block device
Performance testing Both (compare results)
Controller management Character device
Developing NVMe tools Character device

Raw character device access can yield lower latency but requires more development effort. The Linux NVMe driver automatically creates both interfaces to support different access patterns.

Example comparing performance:

# Test block device
fio --filename=/dev/nvme0n1 --direct=1 --rw=randread --bs=4k --numjobs=1 --runtime=60

# Test raw device (requires custom tool)
nvme perf /dev/nvme0 -s 4096 -q 32 -t 60

When working with NVMe drives in Linux, you'll notice two distinct device files appear in /dev:

crw------- 1 root root 243, 0 Dec 12 16:09 /dev/nvme0
brw-rw---- 1 root disk 259, 0 Jan 14 01:30 /dev/nvme0n1

The first (/dev/nvme0) is a character device (denoted by 'c'), while the second (/dev/nvme0n1) is a block device (denoted by 'b'). This distinction exists because they serve fundamentally different purposes in the system.

The character device (/dev/nvme0) provides raw access to the NVMe controller. This is primarily used for:

  • Administrative commands (formatting, firmware updates)
  • Controller-level operations
  • Low-level debugging

Here's an example of using the character device to get controller information:

#include <linux/nvme_ioctl.h>
#include <sys/ioctl.h>
#include <fcntl.h>

int main() {
    int fd = open("/dev/nvme0", O_RDWR);
    struct nvme_admin_cmd cmd = {
        .opcode = nvme_admin_identify,
        .nsid = 0, // 0 for controller
        .addr = (__u64)buffer,
        .data_len = 4096
    };
    ioctl(fd, NVME_IOCTL_ADMIN_CMD, &cmd);
    close(fd);
    return 0;
}

The block device (/dev/nvme0n1) represents the actual namespace (storage area) and is used for:

  • Regular filesystem operations
  • Partition management
  • Standard I/O operations

Example of reading from the block device:

#include <fcntl.h>
#include <unistd.h>

int main() {
    int fd = open("/dev/nvme0n1", O_RDONLY);
    char buffer[512];
    read(fd, buffer, sizeof(buffer));
    close(fd);
    return 0;
}

Use the character device when:

  • You need to send NVMe admin commands
  • Performing controller management tasks
  • Developing NVMe driver-level features

Use the block device when:

  • Working with filesystems
  • Performing regular read/write operations
  • Using standard storage utilities (dd, fdisk, etc.)

Here's how you might use both device types together:

// First use character device to format
int ctrl_fd = open("/dev/nvme0", O_RDWR);
struct nvme_format_cmd fmt_cmd = {
    .opcode = nvme_admin_format_nvm,
    .nsid = 1, // namespace ID
    .cdw10 = (1 << 4) // set format bit
};
ioctl(ctrl_fd, NVME_IOCTL_ADMIN_CMD, &fmt_cmd);
close(ctrl_fd);

// Then use block device to create filesystem
system("mkfs.ext4 /dev/nvme0n1");

While the block device offers convenient filesystem access, the character device provides lower latency for certain operations. For high-performance applications, you might want to bypass the block layer entirely:

// Bypass filesystem for direct I/O
int fd = open("/dev/nvme0n1", O_RDWR | O_DIRECT);
posix_memalign(&buffer, 4096, 4096);
read(fd, buffer, 4096);