On April 2nd SNIA hosted a CXL webinar to discuss where the technology currently is, and where it is headed. The Memory Guy was honored to play a part in this event.
Other speakers were Arthur Sainio, on behalf of SNIA, Mahesh Natu, on behalf of the CXL Consortium, and Torry Steed of SMART Modular, a company that has been one of the earliest providers of Continue reading “SNIA CXL Webinar Available On-Demand”
High-end xPUs (GPUs, TPUs, and other AI processors) use HBM memories to get the absolute highest memory bandwidth for training the Large Language Models (LLMs) used in today’s generative AI systems. These processors, used in the thousands by hyperscale datacenters, can consume a kilowatt each. As a result, they must dissipate a phenomenal quantity of heat. This has become a very challenging problem for these datacenters. (The Memory Guy should mention here that Objective Analysis plans soon to release a report on HBM.)
The problem has attracted the attention of Continue reading “AI xPUs to be Cooled by HBM”
All of a sudden HBM is the leading topic of conversation. Why is that? A good number of people ask the Memory Guy that question.
Sure, the recent explosion of generative AI has been the key part of that interest, but there was even more that encouraged competition into this market at an accelerated pace. The simple answer is that HBM, a DRAM used in AI applications, remained Continue reading “Why is Everyone Jumping Into the HBM Market?”
Samsung has been strongly promoting its “Aquabolt-XL” Processor-In-Memory (PIM) devices for the past year. In this second post of a two-part series The Memory Guy will present other companies’ similar PIM devices, and will discuss the PIM approach’s outlook for commercial success.
Part 1 of this series explains the concept of Processing in Memory (PIM), details Samsung’s Aquabolt-XL design, and shares some performance data. It can be found HERE.
This is not at all the first Continue reading “Samsung’s Aquabolt-XL Processor-In-Memory (Part 2)”
During his December 15 IEDM keynote speech, Samsung Electronics Chairman Kinam Kim really surprised me. He spoke favorably of the approach that YMTC is using to produce 3D NAND flash.
This approach, which YMTC named “Xtacking,” involves the use of two separate wafers to manufacture a 3D NAND chip. The brief way to describe it is to say that Continue reading “Did Samsung Just Endorse YMTC’s Xtacking?”
Intel has recently announced a technology that the company calls PowerVia that could inadvertently help reduce the cost of HBM – high-bandwidth memory.
HBM is a stack of up to twelve DRAM chips that are interconnected using over one thousand TSVs – Through-Silicon Vias. These are metal-filled holes etched right through the DRAM die to allow signals to move vertically through the chip. It’s an alternative to more conventional wire bonding.
HBM sells for significantly more than Continue reading “Could Intel’s PowerVia Lower HBM Costs?”
The Memory Guy has found that some people get confused about the terminology surrounding flash “Layers” and “Levels,” Sometimes confusing the two, and often misunderstanding what each one means. This post is meant to be a low-level primer to address that confusion.
There are actually three places where such terminology is used: The number of chips in a package, the number of conductor/insulator pairs in 3D NAND, and the number of voltage levels stored on any single bit cell within the chip. I will address them in that order.
CHIP STACKING: Since the 1990s Both NAND and NOR flash chip makers have been stacking chips within a single plastic package. Originally this approach was used to reduce the size of thin flip phones like the Motorola Razr by stacking an SRAM chip on top of NOR flash, but soon afterwards NAND chips began to use the same approach to get incredible storage capacities into a single IC package or eMMC, or into a microSD card format. What began as 2-die stacks became 4, then 8, and now 16 high. This post’s photo illustrates an 8-high stack.
Since the height of a standard plastic package for a chip is smaller than a stack of 16 full-thickness dice the wafers had to be Continue reading “NAND Flash’s Layers of Layers of Layers”
Since I am the Memory Guy I hate learning that I missed something new and cool in the world of memories, but somehow I was unaware of last week’s Memsys conference in Washington DC until a participant notified me on Saturday that his paper: “Reverse Engineering of DRAMs: Row Hammer with Crosshair,” had been given the the best paper award.
Upon looking at the Memsys website it looks like a very intriguing academic conference. about sixty papers were presented in eight interesting sessions:
in addition to a few apparently-fascinating keynotes.
Fortunately, all of the papers are Continue reading “Memsys: A New Memory Conference”
In a November 25 press release Samsung introduced a 128GB DDR4 DIMM. This is eight times the density of the largest broadly-available DIMM and rivals the full capacity of mainstream SSDs.
Naturally, the first question is: “How do they do that?”
To get all the chips into the DIMM format Samsung uses TSV interconnects on the DRAMs. The module’s 36 DRAM packages each contain four 8Gb (1GB) chips, resulting in 144 DRAM chips squeezed into a standard DIMM format. Each package also includes a data buffer chip, making the stack very closely resemble either the High-Bandwidth Memory (HBM) or the Hybrid Memory Cube (HMC).
Since these 36 packages (or worse, 144 DRAM chips) would overload the processor’s address bus, the DIMM uses an RDIMM protocol – the address and control pins are buffered on the DIMM before they reach the DRAM chips, cutting the processor bus loading by an order of magnitude or more. RDIMMs are supported by certain server platforms.
The Memory Guy asked Samsung whether Continue reading “Samsung’s Colossal 128GB DIMM”
Wiley has recently published a new book by Betty Prince titled Vertical 3D NAND Technologies that is one to consider if you want to bring yourself up to speed on recent research behind today’s and tomorrow’s 3D memory technologies.
For those who haven’t previously encountered Dr. Prince, she is the author of a number of key books covering memory design and holds memory patents written over her 30-year career in the field.
The book provides capsule summaries of over 360 papers and articles from scholarly journals on the subject of 3D memories, including DRAM, NAND flash, and stacked chips.
These papers are organized into Continue reading “New Book: Vertical 3D Memory Technologies”
