NAND memory caches could clash Data Center power consumption
The advent of cloud computing, deep learning, and AI could revolutionize modern computing, but they’ve also created scaling problems. The vast majority of databases use a similar architecture: DRAM cache servers are used to store the most common queries. These cache servers are backed up by an SSD storage array and possibly a mechanical disk array behind the SSDs. With companies like Intel and AMD offering chips that can address up to 1.5TB (Intel) or 2TB (AMD) of RAM per socket, it might seem ridiculous to imply that a server can still run out of DRAM — but they can. And a new paper from MIT shows how NAND flash could be used within a key-value store caching server while matching or exceeding DRAM’s performance.
Why Use NAND flash ?
There are several reasons to prefer NAND flash over DRAM for caching servers. First, DRAM is volatile memory and must be continually refreshed in order to retain data. NAND flash uses roughly 5 percent as much power, which means it radiates far less heat. Both heat and power scaling are critically important to the United States’ efforts to deploy exascale computing, and DRAM is a major contributor to both problems.
Second, NAND flash is far more dense than DRAM, with up to 100x the storage capacity of the largest DRAM IC. This is a straightforward advantage; the denser the storage medium, the larger the cache server can be. Third, NAND flash is much less expensive than DRAM, at roughly 1/10 the cost per bit.
So far, this seems like a win-win scenario for everyone, but there’s been one major flaw holding NAND back from eating the entire data center: It’s slow. As the chart below illustrates, even PCI Express-connected NAND has a fraction of the bandwidth and is hundreds of times slower than DRAM. So how can a DRAM cache ever be replaced by a NAND-based product? As the MIT research team shows, it takes a bit of clever engineering, but it can be done.
Their NAND cache server, dubbed BlueCache, uses a mixture of software and hardware to accomplish its goals. Memory accesses are pipelined, allowing the system to execute queries more efficiently. BlueCache also eschews the use of a Flash Translation Layer (FTL), favoring direct access to the raw NAND. The FTL is entirely a black box and companies like Samsung keep its FTL implementations close to its metaphorical chest. Since a KVS database has its own pattern for storing, retrieving, and managing data, the two can often conflict with each other, lowering total NAND performance. Managing the “raw” NAND eliminates this bottleneck.
A BlueCache node doesn’t completely eliminate DRAM, it just dramatically reduces it. Each BlueCache node consists of 1TB of NAND with 8GB of DRAM parked in front of it.
The DRAM is used to cache an index table of where specific key values are stored within the NAND array. An 8GB DDR3 module is sufficient to cache key value locations for a 1TB NAND flash array (more information on how this is accomplished is available in this paper).
One caveat to BlueCache is that it only works if your working data set is significantly larger than the total DRAM that would be installed in a conventionally designed cache server. If the entire database can be stored in RAM, there is no performance advantage to using BlueCache. As the cache hit rate decreases — meaning a larger amount of information isn’t found in the cache and must be pulled from secondary storage — the benefits of BlueCache go up. A workload with a DRAM cache miss rate higher than 7.7 percent begins to favor BlueCache. The larger the miss rate, the greater the value.