Memory Applications - The Memory Guy Blog

AI xPUs to be Cooled by HBM

High-end xPUs (GPUs, TPUs, and other AI processors) use HBM memories to get the absolute highest memory bandwidth for training the Large Language Models (LLMs) used in today’s generative AI systems. These processors, used in the thousands by hyperscale datacenters, can consume a kilowatt each. As a result, they must dissipate a phenomenal quantity of heat. This has become a very challenging problem for these datacenters. (The Memory Guy should mention here that Objective Analysis plans soon to release a report on HBM.)

The problem has attracted the attention of Continue reading “AI xPUs to be Cooled by HBM”

Why Do Emerging Memories Only Sell into Niches?

Every so often an investor will ask The Memory Guy: “Why does MRAM only sell into niche markets?”

These questions aren’t only about MRAM — some ask about ReRAM, FRAM, or PCM as well. It’s the same question for all of them. The answer is pretty simple: They’re expensive. All of these technologies command prices per gigabyte that are Continue reading “Why Do Emerging Memories Only Sell into Niches?”

Announcing Our New CXL Report

The Memory Guy is pleased to announce general availability of a new Objective Analysis Detailed Report: CXL Looks for the Perfect Home.

Since Compute Express Link (CXL) is so important to members of the DRAM market and the hyperscale computing community, the report focuses on those sectors, but it offers Continue reading “Announcing Our New CXL Report”

FMS – New Focus and a Heavy AI and CXL presence

Last week’s FMS conference, originally the Flash Memory Summit, but now the Future of Memory and Storage, saw a strong return of the show’s original excitement and attendance, with numerous presentations on key new technologies, largely AI and CXL, and crowds similar to those before the pandemic. FMS is held in early August every year in Santa Clara, California.

The new name is meant to match the show’s content better than Continue reading “FMS – New Focus and a Heavy AI and CXL presence”

Finally! A Real Virtual Memory Chip

At long last a chip has been embodied to simplify the use of virtual memory, a computing architecture that has been in use since the 1960s. The Memory Guy was given a sneak peek at this new approach by no other than Olly Mugwump, a highly-respected computer architect and longstanding member and fellow of several important technical societies. The new design promises to accelerate memory accesses while Continue reading “Finally! A Real Virtual Memory Chip”

What is “Stranded Memory?”

CXL has been embraced by hyperscale data center managers as a means of solving the problem of “Stranded Memory.” If you don’t know what stranded memory is, then this post is intended to help you to understand. It’s based on a few slides from a presentation that IEEE President Tom Coughlin and The Memory Guy made at the Storage Networking Industry Association (SNIA) CMS Summit in April 2023. SNIA has posted a video of the entire Continue reading “What is “Stranded Memory?””

Where is Micron’s QuantX?

For more than a year The Memory Guy has been fielding questions about Micron’s QuantX products.

First announced at the 2016 Flash Memory Summit, this brand name has been assigned to Micron SSDs and DIMMs that use the Intel/Micron 3D XPoint Memory. Originally QuantX products were scheduled to ship in 2017, but Micron is currently projecting availability in 2019. My clients wonder why there have been these delays, and why Micron is not more actively marketing this product.

The simple answer is that it doesn’t make financial sense for Micron to ship these products at this time.

Within two weeks of the first announcement of 3D XPoint Memory, at the 2015 Flash Memory Summit, I knew and explained that the technology would take two years or more to reach manufacturing cost parity with DRAM, even though Intel and Micron loudly proclaimed that it was ten times denser than DRAM. This density advantage should eventually allow XPoint manufacturing costs to drop below DRAM costs, but any new technology, and even old technologies that are in low-volume production, suffer a decided scale disadvantage against DRAM, which sells close Continue reading “Where is Micron’s QuantX?”

Super-Cooled DRAM for Big Power Savings

Recently Rambus announced that it was using cryogenic temperatures to boost computer performance in large datacenters. This research is being done in a joint project with Microsoft who is developing a processor based on Josephson Junctions.

This is an effort to provide a performance increases greater than can be attained through standard semiconductor scaling. The research project aims to attain improvements in cycle time, power consumption, and compute density, leading to better energy efficiency and cost of ownership (COO). The companies hope to gain side benefits of being able to squeeze more bits onto a DRAM chip thus reducing cost per bit, improving performance, and making DRAM chips less costly to produce.

The system these two companies are researching uses a memory system that is cooled to 77 degrees Kelvin (77°K) with a processor that operates at 4°K. To do this the memory system is bathed in liquid nitrogen while the processor is cooled by liquid helium. The temperatures are the boiling points of these two liquids.

Surprisingly, the fact that these two subsystems are in different Continue reading “Super-Cooled DRAM for Big Power Savings”

How NAND Flash Can Reduce DRAM Requirements

In a comment to a recent Memory Guy post I stated that NAND flash can reduce DRAM requirements, even in PCs. Some readers have told me that they wonder how this could be, so I will write this post to explain.

Some years ago Objective Analysis noticed that clever server administrators were able to use SSDs to reduce their systems’ DRAM requirements. Not only did this save them money, but it lowered power and cooling requirements as well.

Thinking that this might work on other kinds of computers, we commissioned a number of benchmarks to be performed on a PC.

These benchmarks found that after a system already has a certain minimum amount of DRAM, users can get a bigger performance boost by adding a dollar’s worth NAND flash than they can get by adding a dollar’s worth of DRAM.

In every case the minimum amount of DRAM was very small.

This benchmark data was compiled, written up, and explained in depth in the report: How PC NAND Will Undermine DRAM, which can Continue reading “How NAND Flash Can Reduce DRAM Requirements”