Enterprise Storage Forum content and product recommendations are editorially independent. We may make money when you click on links to our partners. Learn More.
NVMe over Fabrics (NVMe-oF) is an approach to storage networking that uses the non-volatile memory express standard to provide high-speed data transfers across entire local area networks. It’s particularly well-suited for data center environments, because the messaging model of network communication it uses allows it to transmit data quickly over longer distances. This guide explains NVMe-oF and the networking protocols it uses as well as its major benefits and weaknesses.
What is NVMe over Fabrics?
NVMe is a storage and transfer specification designed for solid state drives that dramatically speeds up data access and response times. It works by accessing flash storage through a Peripheral Component Interconnect express (PCIe) bus, which supports up to 64,000 command queues instead of the single command queue supported by traditional mechanical disks and all flash SSDs.
NVME-oF uses network fabrics like Fibre Channel and ethernet to extend those benefits to a network, improving data transfer speeds between servers and devices.and Ethernet. Businesses can transfer data between servers and computers on a local area network with NVMe-oF, even if the machines are in different locations. NVMe-oF is ideal for environments like data centers that have the networking infrastructure to support it.
How does NVMe-oF work?
Rather than using a PCIe bus on a single device, NVMe-oF uses messaging technology to communicate over a network, and can be configured alongside multiple networking standards. Here are a few of the most popular:
- Fibre Channel uses fiber optic technology to transmit data between servers, computers, and remote locations like data centers. When NVMe is deployed alongside Fibre Channel networking, it’s known as NVMe/FC.
- Remote Direct Memory Access (RDMA) performs transmissions between two machines without going through the operating system. RDMA protocols include both Infiniband and RDMA over Converged Ethernet (RoCE). When NVMe is deployed with Infiniband, it’s known as NVMe/IB. When NVMe uses converged Ethernet, it’s known as NVMe/RoCE.
- Transmission Control Protocol (TCP) is a standard for transmitting data packets over a network, typically the internet. When configured alongside NVMe, it’s known as NVMe/TCP.
Which of these standards a business chooses to use will depend upon its existing networking protocols as well as data center architecture and needs.
Benefits of NVMe-oF
NVMe over Fabrics presents multiple advantages for enterprises that are able to implement it successfully, including reduced vendor lock-in, networking flexibility, and high data transfer rates.
Vendor and network flexibility
Because organizations can choose different network protocols and storage arrays for an NVMe configuration, vendor lock-in is less likely. NVMe over Fabrics provides flexibility for businesses that want to support different networks and storage solutions when correctly configured.
Data transfer speeds
Due to its design, NVMe performs faster than its predecessors, including SAS and SATA. NVMe-oF is able to process stored data at rates that support mission-critical applications and can support data center processing requirements such as rapid retrieval of object-stored data.
Parallelism
Because of NVMe’s significant queue depth, it can process many requests at the same time. This parallelism is beneficial for businesses with multiple memory-intensive applications. For businesses with widely distributed local area networks, NVMe over Fabrics extends that parallelism to the whole network rather than a single server supporting a single application.
NVMe-oF alternatives
Because NVMe over Fabrics is a new technology, its alternatives are primarily older storage networking protocols like iSCSI and ISER that don’t support NVMe. Businesses might choose these over NVMe because their existing technology supports them or because their IT staff is more familiar with them.
iSCSI is a networking protocol that allows SCSI to work with an Ethernet connection. Its main benefit is its widespread use. Storage and networking teams unfamiliar with NVMe over Fabrics are likely to be more experienced using iSCSI, and unlike NVMe-oF, organizations might also already have the technology for iSCSI in place. The major disadvantage of iSCSI is its speed — it’s slower than NVMe-oF.
Similarly, the iSCSI Extensions for RDMA standard, or ISER, allows iSCSI to use RDMA. It provides greater bandwidth and avoids the overhead of standard TCP/IP processes, but, like iSCSI, it doesn’t support NVMe.
Bottom line: NVMe over Fabrics
NVMe-oF is a storage and transfer specification that brings high speeds to entire business networks rather than individual machines. It’s particularly beneficial for enterprises with multiple mission-critical applications and hot storage needs. While NVMe-oF is a promising technology, it needs to be consistently and reliably implemented. Before it can truly benefit businesses and add value to their existing storage infrastructures, storage teams have to know how it works and if it suits their needs.
Learn more about storage networking solutions for your business.
