NVM Express, NVMe, or Non-Volatile Memory Host Controller Interface Specification (NVMHCI), is a specification for accessing solid-state drives (SSDs) attached through the PCI Express (PCIe) bus. NVM is an acronym for non-volatile memory, as used in SSDs.
A major application of NVMe is SATA Express, which is a new and backward-compatible interface specification supporting either SATA or PCI Express storage devices. SATA Express can use either legacy AHCI or new NVMe as the logical device interface.
Historically, most SSDs used buses such as SATA, SAS or Fibre Channel for interfacing with the rest of a computer system. Since the SSDs became available in mass markets, SATA has been the most typical way for connecting SSDs in personal computers; however, SATA was designed for mechanical hard disk drives, and has become increasingly inadequate as SSDs have improved. For example, unlike hard disk drives, some SSDs are limited by the maximum throughput of SATA.
High-end SSDs have been made using the PCI Express bus before, but using non-standard specification interfaces. By standardizing the interface of SSDs, operating systems only need one driver to work with all SSDs adhering to the specification. It also means that each SSD manufacturer does not have to use additional resources to design specific interface drivers. This is similar to how USB devices are built to follow the USB specification and work with all computers, with no per-device drivers needed.
The first details of a new standard for accessing non-volatile memory emerged at the Intel Developer Forum 2007, when NVMHCI was shown as the host-side protocol of a proposed architectural design that had ONFI on the memory (flash) chips side. A NVMHCI working group led by Intel was formed that year. The NVMHCI 1.0 specification was completed in April 2008 and released on Intel's web site.
Technical work on NVMe began in the second half of 2009. The NVMe specifications were developed by the NVM Express Workgroup, which consists of more than 90 companies; Amber Huffman of Intel was the working group's chair. Version 1.0 of the specification was released on 1 March 2011, while version 1.1 of the specification was released on 11 October 2012. Major features added in version 1.1 are multi-path I/O (with namespace sharing) and arbitrary-length scatter-gather I/O. It is expected that future revisions will significantly enhance namespace management. Because of its feature focus, NVMe 1.1 was initially called "Enterprise NVMHCI". An update for the base NVMe specification, called version 1.0e, was released in January 2013.
In June 2011, a Promoter Group led by seven companies was formed. Cisco, Dell, EMC, IDT, Intel, NetApp, and Oracle have permanent seats in this group, while six other seats are held by representatives elected from the other member companies of the workgroup.
The first commercially available NVMe chipsets were released by Integrated Device Technology (89HF16P04AG3 and 89HF32P08AG3) in August 2012. The first NVMe drive, Samsung's XS1715 enterprise drive, was announced in July 2013; according to Samsung, this drive supported 3 GB/s read speeds, six times faster than their previous enterprise offerings. The LSI SandForce SF3700 controller family, released in November 2013, also supports NVMe. Sample engineering boards with the PCI Express 2.0 ×4 model of this controller found 1,800 MB/sec read/write sequential speeds and 150K/80K random IOPS. A Kingston HyperX "prosumer" product using this controller was showcased at the Consumer Electronics Show 2014 and promised similar performance. As of March 2014, none of the announced NVMe drives are commercially available.
Comparison with AHCI
While Advanced Host Controller Interface (AHCI) interface has the benefit of legacy software compatibility, it does not deliver optimal performance when an SSD is connected via PCI Express bus. This is because AHCI was developed back at the time when the purpose of a host bus adapter (HBA) in a system was to connect the CPU/memory subsystem with a much slower storage subsystem based on rotating magnetic media. Such an interface has some inherent inefficiencies when applied to SSD devices, which behave much more like DRAM than like spinning media.
NVMe has been designed from the ground up, capitalizing on the low latency and parallelism of PCI Express SSDs, and fulfilling the parallelism of contemporary CPUs, platforms and applications. At a high level, the basic advantages of NVMe over AHCI relate to its ability to exploit parallelism in host hardware and software, manifested by differences in depth of command queues, interrupts processing, the number of uncacheable register accesses etc., resulting in various performance improvements.:p. 17–18
The table below summarizes high-level differences between the basic NVMe and AHCI device interfaces.
|Maximum queue depth||1 command queue;
32 commands per queue
65536 commands per queue
|Uncacheable register accesses
(2000 cycles each)
|6 per non-queued command;
9 per queued command
|2 per command|
and interrupt steering
|2048 MSI-X interrupts|
and multiple threads
|requires synchronization lock
to issue a command
for 4 KB commands
|command parameters require
two serialized host DRAM fetches
|gets command parameters
in one 64 Bytes fetch
Operating system support
- The "NVMe Windows Working Group" is an initiative from the OpenFabrics Alliance to maintain software for Microsoft Windows to use PCI Express solid state devices. The baseline Windows driver contributed to the open-source initiative was developed by several promoter companies in the NVMe workgroup, specifically IDT, Intel, and LSI.
- Intel published an NVM Express driver for Linux. It was merged into the Linux kernel mainline on 19 March 2012, with the release of version 3.3 of the Linux kernel.
- A scalable block layer for high-performance SSD storage, developed primarily by Fusion-io engineers, was merged into the Linux kernel mainline in kernel version 3.13, released on 19 January 2014. This leverages the performance offered by SSDs and NVM Express, by allowing much higher I/O submission rates. With this new design of the Linux kernel block layer, internal queues are split into two levels (per-CPU and hardware-submission queues), thus removing bottlenecks and allowing much higher levels of I/O parallelization. With the release of Linux kernel version 3.13, only the virtioblk driver has been modified to actually use this new interface; other drivers will be ported in the following releases.
- Development work required to support NVMe in OpenBSD has been started in April 2014 by a senior developer formerly responsible for USB 2.0 and AHCI support.
- M.2 (connector and card format specification, supporting either SATA or PCI Express storage devices)
- Dave Landsman. "AHCI and NVMe as Interfaces for SATA Express™ Devices - Overview" (PDF). SanDisk. Retrieved 2013-10-02.
- Walker, Don H. "A Comparison of NVMe and AHCI". 31 July 2012. The Serial ATA International Organization. Retrieved 3 July 2013.
- "NVM Express Explained".
- "Speeding up Flash... in a flash". The Inquirer. 2007-10-13. Retrieved 2014-01-11.
- "Flash new standard tips up". The Inquirer. 2008-04-16. Retrieved 2014-01-11.
- "New Promoter Group Formed to Advance NVM Express". Press release. June 1, 2011. Retrieved September 18, 2013.
- Amber Huffman editor (October 11, 2012). "NVM Express Revision 1.1". Specification. Retrieved September 18, 2013.
- David A. Deming (2013-06-08). "PCIe-based Storage" (PDF). snia.org. Retrieved 2014-01-12.
- Amber Huffman editor (January 23, 2013). "NVM Express Revision 1.0e". Specification. Retrieved September 18, 2013.
- "IDT releases two NVMe PCI-Express SSD controllers". The Inquirer. 2012-08-21. Retrieved 2014-01-11.
- "IDT Shows Off The First NVMe PCIe SSD Processor and Reference Design - FMS 2012 Update". The SSD Review. 2012-08-24. Retrieved 2014-01-11.
- "Samsung Announces Industry’s First 2.5-inch NVMe SSD | StorageReview.com - Storage Reviews". StorageReview.com. 2013-07-18. Retrieved 2014-01-11.
- "LSI SF3700 SandForce Flash Controller Line Unveiled | StorageReview.com - Storage Reviews". StorageReview.com. 2013-11-18. Retrieved 2014-01-11.
- LSI Introduces Blazing Fast SF3700 Series SSD Controller, Supports Both PCIe and SATA 6Gbps
- Kingston Unveils First PCIe SSD: 1800 MB/s Read Speeds
- Kingston HyperX Predator PCI Express SSD Unveiled With LSI SandForce SF3700 PCIe Flash Controller
- "Solid-state drives jump on PCI Express". EE Times. Retrieved 2014-01-11.
- Andy Herron (2013). "Advancements in Storage and File Systems in Windows 8.1" (PDF). snia.org. Retrieved 2014-01-11.
- "Windows NVM Express". Project web site. Retrieved September 18, 2013.
- "Windows 8.1 to support hybrid disks and adds native NVMe driver". Myce.com. 2013-09-06. Retrieved 2014-01-11.
- Matthew Wilcox (2011-03-03). "NVM Express driver". LWN.net. Retrieved 2013-11-05.
- Keith Busch (2013-08-12). "Linux NVMe Driver" (PDF). flashmemorysummit.com. Retrieved 2013-11-05.
- "Hands-on Lab: Compiling the NVM Express Linux Open Source Driver and SSD Linux Benchmarks and Optimizations" (PDF). IDF13. activeevents.com. 2013. Retrieved 2014-01-11.
- "Merge git://git.infradead.org/users/willy/linux-nvme". kernel.org. 2012-01-18. Retrieved 2013-11-05.
- "1.1 A scalable block layer for high-performance SSD storage". Linux 3.13. kernelnewbies.org. 2014-01-19. Retrieved 2014-01-25.
- Jonathan Corbet (2013-06-05). "The multiqueue block layer". LWN.net. Retrieved 2014-01-25.
- Matias Bjørling; Jens Axboe; David Nellans; Philippe Bonnet (2013). "Linux Block IO: Introducing Multi-queue SSD Access on Multi-core Systems" (PDF). kernel.dk. ACM. Retrieved 2014-01-25.
- "Log of /head/sys/dev/nvme". FreeBSD source tree. The FreeBSD Project. Retrieved 16 October 2012.
- "Log of /stable/9/sys/dev/nvme". FreeBSD source tree. The FreeBSD Project. Retrieved 3 July 2013.
- ChangeLog/1.6 - QEMU
- "First NVMe SSDs Announced – What’s NVMe?". StorageNewsletter. 2013-09-20. Retrieved 2014-01-11.
- "Download EDK II from". SourceForge.net. Retrieved 2014-01-11.
- David Gwynne (2014-04-16). "non volatile memory express controller (/sys/dev/ic/nvme.c)". BSD Cross Reference. Retrieved 2014-04-27.