It’s a little known fact that flash-based storage can be too much for most systems. Designed back in the days when slow hard disk drives (HDDs) carried out the reading and writing of data, today’s channels for information transport often can’t cut it when loaded up with flash. The result is bottlenecking applications: the combined might of multicore processors, abundant RAM and flash pack far more processing punch than can be relayed by the associated storage protocols and bus architectures.
Enter Non-Volatile Memory Express (NVMe). It's a PCIe-based approach to resolving those bottlenecks. And it’s about to capture the imagination of the storage world.
“I’ve been at this for more than 20 years and NVMe is one of the most revolutionary, most anticipated and most exciting developments I’ve seen,” said Doug Rollins, Senior Technical Marketing Engineer, Enterprise Solid State Drives for the Storage Business Unit at < ahref="http://www.micron.com">Micron Technology.
So what is it? According to non-profit NVM Express, Inc., NVMe is an “optimized, high-performance, scalable host controller interface with a streamlined register interface and command set designed for non-volatile memory based storage.” It was developed as a standard to reduce latency and provide faster performance. Already, early NVMe products have demonstrated up to six times greater 4KB random and sequential read/write performance compared to SATA disk as well as SATA-based solid state drives (SSDs).https://o1.qnsr.com/log/p.gif?;n=203;c=204660765;s=10655;x=7936;f=201812281308090;u=j;z=TIMESTAMP;a=20400368;e=i
“The full value of flash has been restricted by legacy protocols,” said Rollins. “NVMe is designed from the ground up to take advantage of the capabilities of flash, enabling far greater performance and unlocking the value flash can provide.”
To understand just how restricted, let’s take a look at PCIe: It evolved as a means of providing more bandwidth and lower latency. PCIe Gen 3 supports about 7.87 Gbps per lane which relieved some of the storage bottleneck. But even with PCIe and SSDs, chokepoints remain in the form of the SATA and SAS protocols which offered enough bandwidth for HDDs and early generation SSDs, but not enough for the latest applications and flash architectures.
A simple comparison with SATA on queuing highlights how important this new technology might be. SATA allowed for only one command queue which was capable of holding 32 commands per queue. NVMe enables 65,536 (64K) queues each with 64K commands per queue.
“The storage I/O capabilities of flash can now be fed across PCIe faster to enable modern multi-core processors to complete more useful work in less time,” said Greg Schulz, an analyst with StorageIO Group.
Some NVMe products are already on the market and a lot more are coming. With more than 80 companies belonging to NVM Express Inc., expect all the usual suspects to be offering NVMe-enabled storage, appliances and servers in the very near future. Further, a raft of startups is bound to emerge touting NVMe-only strategies and the abandonment of any kind of HDD and perhaps even advocating that users cast aside non-NVMe-based SSDs. Smart users, therefore, are advised to inform themselves on this technology well before deployment.
“Preparation is everything,” said Rollins. “Start from an appreciation of your most critical applications and where the most significant bottlenecks exist.”
Schulz advised users to tie NVMe plans into refresh cycles. Anyone looking to buy new servers should gravitate towards models with motherboards, BIOS and PCIe G3 slots that are NVMe ready. Similarly, storage array and appliance vendors will begin to introduce products that can incorporate this new protocol. Even if NVMe is not initially used on these boxes, it is probably wise to pay a mark-up to have it included. You will also begin to see vendors offering equipment with slots that utilize an 8637 connector, which can be used either for NVMe, SAS or SATA.
But NVMe is not just about the hardware. Schulz cautioned users to pay attention to software.
“Check if your operating system or hypervisor has drivers for NVMe devices, or if there are updates that can be applied,” said Schulz. “Drivers are available for the major Linux versions including RHEL, Centos and Ubuntu as well as for Windows 8/2012, VMware and other hypervisors.”
NVMe, then, requires both hardware and software parts. It is likely to be a good fit for many servers, particularly those being used for content provisioning, software defined storage platforms and database servers. But it isn’t about to eliminate SAS or SATA — at least not any time soon.
“NVMe will co-exist with your existing SAS and SATA devices and will operate as part of a tiered storage strategy,” said Schulz. “Start planning for NVMe in your future by shopping for NVMe-capable servers today.”
Training is another important part of future NVMe success. After all, NVMe is initially going to be deployed in business-critical applications where its higher cost can be justified by far greater performance that can be translated directly to the bottom line. You don’t want to be fumbling around with it in those kinds of environments.
“Ensure the support teams charged with maintenance and management of the platforms in which you plan to deploy NVMe are well trained,” said Rollins. “You need a staff that is already prepared, educated and able to handle any challenge that comes their way.”
No Golden Ticket
Those familiar with the Willie Wonka movies and books know all about golden tickets. And it’s understandable that some storage managers, on hearing about the marvels of NVMe will imagine that they are about to obtain one. But like any new breakthrough, expectations can often exceed reality.
Jeff Sisilli, Senior Director Product Marketing, Flash Solutions, DDN Storage, agreed that NVMe can be a powerful tool to satisfy performance hungry applications, reduce hardware requirements and shorten time to results. But he advised users to not look upon it as a golden ticket that will resolve each and every performance woe or application bottleneck.
“Despite the marketing buzz, it’s important to keep in mind that flash alone won't always accelerate your applications or make your workload faster,” he said. “Where actual speed-ups can be achieved is dictated by knowing your specific data patterns and identifying where the I/O bottlenecks are in your environment.”
In other words, homework and prior testing will pay dividends in terms of delivering the biggest bang for your NVMe buck. It's like choosing between adding racing tires to a Camaro or a Toyota Yaris. In the former, the tires may buy you sufficient performance to warrant the investment. In the latter, you are wasting your money.
Sisilli mentioned that DDN is unveiling its next generation storage, server and I/O acceleration platforms this fall. This will include a PCIe Fabric in which you can plug in an abundance of NVMe as a way to drive much more performance at lower latency.
“We’re talking lots and lots of performance,” said Sisilli. “48 NVMe slots compared to the typical 2 to 6 slots you see on today’s commodity servers.”
Photo courtesy of Shutterstock.