What is NVMe?
NVMe stands for Non-Volatile Memory Express, or Non-Volatile Memory Host Controller Interface Specification (NVMHCIS) is an open logical-device interface specification for accessing non-volatile storage media attached via PCIe bus. It is new storage access and transport protocol for accessing high-speed storage media that brings many advantages.
NVMe was designed for SSD. It communicates between the storage interface and the System CPU using high-speed PCIe sockets, independent of the storage form factor. Input or Output tasks performed using NVMe drivers begin faster, transfer more data and finish faster than older storage models using older drivers, such as AHCI (Advanced Host Controller Interface), a feature of SATA SSDs.
NVMe is becoming the new industry standard for both servers in the data center and client devices such as laptops, desktop PCs and even next-gen gaming consoles.
The NVMe protocol accesses flash storage via a PCI Express (PCIe) bus to deliver a high-bandwidth, low-latency user experience, which supports tens of thousands of parallel command queues. It much faster than hard disks and traditional all-flash architectures, which are limited to a single command queue.
NVMe is often NAND flash memory that comes in several physical form factors, including solid-state drivers (SSDs), PCI express (PCIe) add-in cards, M.2 cards, U.2, and other forms.
NVMe allows host hardware and software to exploit the levels of parallelism possible in modern SSDs. As a result, NVMe reduces input or output overhead and brings various performance improvements relative to previous logical-device interfaces, including multiple long command queues and reduced latency.
NVMe devices are chiefly available in standard-sized PCI Express expansion cards and as 2.5-inch form-factor devices that provide a four-lane PCI Express interface through the U.2 connector (formerly known as SFF-8639). Storage devices using SATA express and the M.2 specification, which support NVM Express as the logical-device interface, are a popular use-case for NVMe and have become the dominant form of solid-state storage for servers, desktops and laptops alike.
History of NVMe
- The Non-Volatile Memory Host Controller Interface (NVMHCI) Work Group started to develop the NVMe specification in 2009 and published the 1.0 version on 1st March 2011.
- The 1.0 specification included the NVM command set, administration command set, queuing interface, and security features.
- The NVMHCI Work Group, commonly known as the NVM Express Work Group, released an update to the NVMe specification on Oct. 11, 2012.
- NVMe 1.1 added support for SSDs with multiple PCIe ports to enable multipath I/O and namespace sharing.
- The NVM Express Work Group held its first plugfest in May 2013 to enable companies to test its products’ compliance with the NVMe specification and check interoperability with other NVMe products.
- The NVMe 1.2 specification emerged on Nov. 3, 2014, with enhancements such as support for live firmware updates, improved power management and the option for end-to-end data protection.
- On Nov. 17, 2015, the NVM Express organization ratified the 1.0 version of the NVM Express Management Interface (NVMe-MI) to provide an architecture and command set to manage a non-volatile memory subsystem out of the band.
- NVMe-MI enables a management controller to perform SSD device and capability discovery, health and temperature monitoring, and non-disruptive firmware updates.
- Without NVMe-MI, IT managers generally relied on proprietary, vendor-specific management interfaces to enable the administration of PCIe SSDs.
- In September 2016, the CompactFlash Association announced that it would be releasing a new memory card specification, CFexpress, which uses NVMe.
Specification
Specifications for NVMe released to date include:
- 1.0e (January 2013)
- 1.1b (July 2014)
- 1.2 (November 2014)
- 2a (October 2015)
- 2b (June 2016)
- 2.1 (June 2016)
- 1.3 (May 2017)
- 3a (October 2017)
- 3b (May 2018)
- 3c (May 2018)
- 3d (March 2019)
- 1.4 (June 2019)
- 4a (March 2020)
How NVMe Works
NVMe maps input, output commands, and shared memory responses in a host computer over the PCIe interface. The NVMe interface supports parallel I/O with multicore processors to facilitate high throughput and mitigate CPU bottlenecks.
NVMe offers a more streamlined command set to process an I/O request than the SCSI and ATA command sets do. NVMe requires fewer than half the number of CPU instructions than the SCSI command set does with SAS devices, and the ATA command set uses with SATA drives.
NVMe supports 64,000 commands in a single message queue and a maximum of 65,535 I/O queues. By contrast, a SAS device typically supports up to 256 commands, and a SATA drive supports up to 32 commands in one queue.
However, NVMe-based PCIe SSDs are currently more expensive than SAS- and SATA-based SSDs of equivalent capacity and high-end enterprise NVMe SSDs may consume more power than SAS or SAS SSDs. The SCSI Trade Association claims the more mature SAS SSDs offer additional advantages over NVMe PCIe SSDs, such as greater scalability and time-tested capabilities. NVMe PCIe SSDs also may provide a level of performance that many applications do not require.
AIC (add-in card)
Almost all early NVMe solid-state drives are HHHL (half-height, half-length) or FHHL (full-height, half-length) AIC, with a PCIe 2.0 or 3.0 interface. An HHHL NVMe solid-state drive card is easy to insert into a PCIe slot of a server.
U.2 (SFF-8639)
U.2 is also known as SFF-8639. It is a computer interface for connecting solid-state drives to a computer. It uses up to four PCI Express lanes. Available servers can combine up to 4 U.2 NVMe solid-state drives.
U.3 (SFF-8639)
U.3 is built on the U.2 spec and uses the same SFF-8639 connector. It is a tri-mode standard, combining SAS, SATA and NVMe support into a single controller.
U.3 can also support hot-swap between the different drives where firmware support is available. U.3 drives are still backward compatible with U.2, but U.2 drives are not compatible with U.3 hosts.
M.2 (NGFF)
M.2 is also known as the Next Generation Form Factor (NGFF), uses an M.2 NVMe solid-state drive computer bus. Interfaces provided through the M.2 connector are PCI express 3.0 or PCI express 4.0.
NVMe over Fabrics (NVMe-oF)
NVMe is more than faster flash storage. It is also an end-to-end standard that enables vastly more efficient transport of data between storage systems and servers.
NVM Express over Fabrics (NVMe-oF) is the concept of using a transport protocol over a network to connect remote devices, contrary to NVMe, where devices are connected directly to the PCIe bus.
NVMe-oF is a communication protocol that allows one computer to access block-level storage devices attached to another computer via remote direct memory access over several transport protocols.
- NVMe over Fabrics extends NVMe’s performance and latency benefits across network fabrics such as Ethernet, Fibre Channel, and InfiniBand.
- It provides higher IOPS and reduced latency from the host software stack through the Data Fabric to the storage array.
NVMe over Fibre Channel (NVMe/FC)
With the recent release of NetApp’s AI data management platform provides NVMe over Fibre Channel support today.
In September 2014, a standard for using NVMe over Fiber Channel (FC) was proposed, and this combination is often referred to as FC-NVMe (or sometimes NVMe/FC).
Many enterprises have built their entire infrastructure around Fibre Channel because of its performance and reliability, plus its support for fabric-based zoning and name services.
Applications such as databases run much faster when using the NVMe/FC protocol than FCP (SCSI protocol with an underlying Fibre Channel connection). ONTAP NVMe/FC traffic can co-reside with FCP traffic on the same Fibre Channel fabric, so it’s easy to get started with NVMe/FC. For many customers with ONTAP AFF systems, this is simply a non-disruptive software upgrade.