It’s no surprise that NVMe speed is impressive: A blue-ribbon consortium of storage and server vendors developed NVMe as a high-performance interface specification that accelerates NAND SSDs using the PCIe bus. NVMe’s logical device interface takes advantage of SSD’s low latency and parallelism to improve IOPs and throughput and reduce latency.
The NVMe protocol provides high bandwidth and low latency with flash-specific improvements. It supports current NAND flash and will scale to support future high-performance devices that depend on persistent memory technologies. NVMe speeds outstrip the performance of the SATA and SAS drives that came beforehand, but speed alone isn’t the only reason to consider implementing an SSD in your servers or data centers. This guide covers both features and limitations of NVMe storage devices.
- How fast are NVMe SSDs?
- NVMe architecture
- 6 features that boost NVMe speed
- 3 NVMe limitations
- NVMe-oF (NVMe over Fabrics)
- Comparing NVMe speed vs. other types of SSDs
- Bottom line: NVMe speed
How fast are NVMe SSDs?
NVMe (Non-Volatile Memory Express) is designed for exponentially more efficient memory processing and higher read/write speeds. It easily surpasses speeds of older storage protocols like SATA and SAS. Major NVMe SSD vendors include Samsung, Kingston Technology, Crucial, Kioxia, and Western Digital.
Up to 64K queues
NVMe is purpose-built for speed with an architecture that uses PCI Express. PCIe standards map operations through shared memory, simplify internal software, and optimize I/O with up to 64,000 queues.
Speed increase over legacy protocols
NVMe’s features make it significantly faster than legacy SAS and SATA SSDs, not to mention SAS/SATA HDDs. These speeds are often five to ten times faster than SATA hard drives and SSDs.
Some of the fastest NVMe drives can read 7 GB/s and write at 5-6 GB/s. The same drives deliver 500,000+ random read IOPs and 500,000 write IOPs.
Learn more about SSD performance and pricing.
NVMe uses the PCI Express bus standard on a computer or server to connect the NVMe SSD to the central processing unit. This more direct access to the CPU allows NVMe drives to bypass another communication step. For example, SATA SSDs must use the SATA controller to communicate with the CPU. This additional step requires more time, so read/write speeds aren’t as fast. NVMe drives’ closer communication with the CPU improves speed.
The multiple threads on an NVMe SSD also improve performance. Multiple threads on a drive allow parallel processing; the storage disk can perform multiple operations at a time. This is beneficial for computers and servers running major applications at once.
The NVMe spec is delivered in standard-sized PCIe expansion cards, or a 2.5” form factor with a four-lane PCIe interface going through a U.2 connector. A popular choice because of its easy deployment, U.2 connects SSDs to a host. NVMe U.2 drives often feature four PCIe lanes, two SAS lanes and one SATA lane for broad interface support in the 2.5” form factor.
M.2 mini-board specification for PCIe, SATA or USB form factors are a popular standard for consumer-level NVMe usage. M.2 boards come in several sizes, including the smallest available PCIe footprint.
Storage capacity on NVMe disks starts at consumer-sized 450GB. Top storage capacities currently hit around 30 TB for the data center, though that’s the very high end.
6 features that boost NVMe speed
The NVMe interface improves on its predecessors because it’s designed to accommodate faster speeds, improve server performance, and map commands more efficiently. These features of NVMe SSDs make them overall faster drives than legacy disks and SATA SSDs.
The architecture exploits SSD parallelism to reduce IO overhead. HDDs, and tape for that matter, are sensitive to access patterns. Sequential data enables faster performance while random data slows down data access. SSDs run in parallel so random or sequential data have little effect on SSD performance.
Hybrid flash arrays often bottlenecked since the SSD tiers were capable of more speed than the HDD storage interfaces could support. IT compensated to a point with multicore processors and a lot of RAM. But it’s more efficient and less expensive to deploy NVMe that is engineered to take advantage of SSD speeds. NVMe takes advantage of the PCIe bus and standard, which can use multiple lanes for data transmission.
Additional NVMe enhancements include the ability to support a single queue capacity of 64K, and to process about 64K of these long queues at the same time. Along with latency reduction, this accelerates performance for busy servers processing simultaneous requests.
NVMe employs remote direct memory access (RDMA) using the PCIe bus. This enables the interface to map IO commands and responses to host shared memory, which frees CPU resources. NVMe also streamlines its command set, issuing less than half of the CPU instructions as SATA or SAS.
Controller Memory Buffer
The NVMe buffer enables the host to formulate commands in controller memory instead of depending on fetch commands through the PCIe. NVMe passes memory blocks instead of SCSI commands, which results in lower latency. NVMe also arbitrates priority commands by observing service level agreement parameters.
NVMe supports multi-host reservations in Windows Clusters that coordinate host access by managing shared namespaces.
3 NVMe limitations
NVMe-based drives are incredibly beneficial for speed-intensive operations, but it’s not a walk in the park to purchase and deploy them, especially for large enterprises with heavy processing needs. NVMe limitations include cost, lack of system support, and design.
NVMe-based storage is more expensive than other storage devices. M.2 and U.2 SSDs, broadly speaking, cost more than hard drives. Over the past few years, the cost difference between NVMe drives and other drives has decreased, and NVMe SSDs have become more affordable. But for massive volumes of data, the high-capacity drives are still extremely expensive. You pay for the high speeds, and for a lot of businesses this isn’t reasonable, especially for large amounts of archive data.
Lack of system support
Many older computer systems and servers don’t support NVMe protocols. If your business’s data center hardware isn’t compatible with M.2, U.2, or E.1 flash drives, NVMe doesn’t make sense. Developing systems that cooperate with all SSDs takes time, and NVMe is a relatively new protocol.
Design and capacity
Many popular NVMe drives just aren’t designed for large data centers. The M.2 SSDs you hear so much about are really better designed for gaming systems and other consumer needs, or for the IT team at a small startup. But for the most part, the popular NVMe SSDs aren’t feasible for storing large amounts of data. Storage capacity on M.2 drives typically ranges from 400 GB to 8 TB, but 8 terabytes is the very high end and drives with that capacity are costly.
Some vendors have begun making data-center-specific drives. The Enterprise and Data Center Standard Form Factor (EDSFF) is specifically designed for enterprise use cases. But the higher capacity drives are still expensive. Using many enterprise-grade NVMe drives ranges from pricey to downright infeasible for the majority of businesses.
NVMe-oF (NVMe over Fabrics)
NVMe over fabrics is a relatively recent protocol that makes NVMe devices available over a network. This is beneficial for data center environments where data needs to quickly move between environments. The standard for NVMe-oF was first published in 2016, and it isn’t yet a widely used data center technology.
For all of NVMe’s performance benefits, its speed is dependent on direct attachment to an individual host. A storage controller is a necessity for shared storage, and it manages capacity provisioning, some data protection, physical addressing, and protocol translations. But add a storage controller and NVMe slows down, defeating the justification for its high cost.
NVMe-oF is a way of solving the problem. The specification enables NVMe message-based commands to transfer data over Ethernet, Fibre Channel (FC) or InfiniBand fabrics without going through slow storage controllers. Storage admins can take NVMe SSD out of the server and connect it over the fabric. Remote SSD storage services operate at the speed of memory to memory transfer with extremely low latency.
The spec uses RDMA for InfiniBand, Converged Ethernet, and Internet Wide Area. It uses a different method to transport across Fibre Channel.
Learn more about the origins of NVMe over fabrics.
Comparing NVMe speed vs. other types of SSDs
This table compares the sequential read and write speeds of NVMe-based drives and SATA drives. NVMe-based SSDs consistently perform faster, but speeds vary significantly even amongst these drives.
|Form Factor/Specification||Capacity||Top Sequential Read Rate||Top Sequential Write Rate|
|Samsung PM9A3||U.2||3.84 TB||6,900 MB/s||4,100 MB/s|
|Kingston DC15000M||U.2||3.84 TB||3,300 MB/s||2,700 MB/s|
|Kioxia XD6 Series||E1.S||3.84 TB||6,500 MB/s||2,350 MB/s|
|Kingston Technology DC450R||2.5″||3.84 TB||560 MB/s||530 MB/s|
|Seagate Nytro 1361||2.5″||3.84 TB||530 MB/s||500 MB/s|
Read and write speeds listed are provided by the hardware vendor and represent the best possible rates, under ideal conditions. Note that they may not accurately reflect everyday performance under normal data center conditions.
Learn more about the differences between NVMe and SATA.
Bottom line: NVMe speed
NVMe-based SSDs are optimized for fast data processing and improve on their predecessors’ performance. But they aren’t ideal for every storage use case. Depending on your business needs, an NVMe SSD may not be the right choice if:
- You’re storing big data volumes that need to be quickly accessed.
- You don’t have the budget for multiple NVMe drives.
- You need to deploy archive storage.
Additionally, before you implement NVMe storage, make sure that your current storage hardware supports it. You don’t want a situation where storage solutions don’t work together; this could stall scaling operations down the road.
For smaller storage systems, supporting one or two critical applications, a high-performance NVMe drive might work well. In data storage, it isn’t always best to immediately chase the latest hype, but with careful consideration and the right use case, NVMe storage can be worthwhile.
Is your business considering a new SSD? Read about the best solid state drives next.
This article was updated March 2023 by Jenna Phipps.