At the International Solid State Circuits Conference (ISSCC) last week a new “Last Level Cache” was introduced by a DRAM company called “Piecemakers Technology,” along with Taiwan’s ITRI, and Intel.
The chip was designed with a focus on latency, rather than bandwidth. This is unusual for a DRAM.
Presenter Tah-Kang Joseph Ting explained that, although successive generations of DDR interfaces has increased DRAM sequential bandwidth by a couple of orders of magnitude, latency has been stuck at 30ns, and it hasn’t improved with the WideIO interface or the new TSV-based High Bandwidth Memory (HBM) or the Hybrid Memory Cube (HMC). Furthermore, there’s a much larger latency gap between the processor’s internal Level 3 cache and the system DRAM than there is between any adjacent cache levels. The researchers decided to design a product to fill this gap.
Many readers may be familiar with my bandwidth vs. cost chart that the Memory Guy has used to introduce SSDs and 3D XPoint memory. The gap that needs filling is indicated by the large red arrow that appears in this post’s graphic, which is an updated rendition of this graph.
Piecemakers’ new High-Bandwidth Low-Latency (HBLL) DRAM is an eight-channel design, with each 72-bit channel accessing 32 banks of RAM. This high bank count allows a lot of interleaving to support faster access. One of the reasons its latency is low is because much of the complexity of the DDR interface has been stripped away — the chip uses an SRAM, rather than a DRAM, interface. The combination of these gives the part an 17ns latency and a random bandwidth 75% higher than HBM and about ten times that of WideIO and LPDDR4.
Although it would seem that this would require a huge die area to accomplish, the design shares address decoders across bank groups, so the die area penalty is only about 10%. Each bank has local latches to allow the upstream circuitry to progress to the next address without waiting.
So what has all this to do with Intel? Intel was named as a co-author of the paper because the company provided funding for the project. From this it appears that that Intel is not only interested in filling the speed gap between DRAM and SSDs with 3D XPoint memory, but it also wants to fill the red-arrow gap with Piecemakers’ HBLL or something like it.
Of course, we won’t know this for certain until Intel announces its plans, but the Piecemakers development certainly appears to indicate that this is Intel’s intent.