Putting the Brakes on Added Memory Layers

Close-up of a part of the blog post's main graphicFor some time two sides of the computing community have been at odds.  One side aims to add layers to the memory/storage hierarchy while other side is trying to halt this growth.

This has been embodied by recent attempts to stop using objective nomenclature for cache layers (L1, L2, L3) and moving to more subjective names that aim to limit any attempt to add another new layer.

This is a matter close to my heart, since The Memory Guy wrote a book to explain cache memory design in the 1990s.

Intel may have been the first to try to stop the creation of additional layers when the company named the last cache layer within the processor the “Last-Level Cache” or LLC.  But this didn’t stop anybody!  Caches added externally to the LLC then became “After-Last-Level” caches (ALL), which wasn’t a term that met with endearment.

Marvell, a company that wanted to add an external ALL layer, but then prevent the addition of any others, created a new name with the term “Final-Level Cache” or FLC.

With a name like that, who could justify adding another  layer?

But new layers kept coming.  And, why not?  After all, Virtual Memory, invented in the 1960s, was a cache to the HDD.  And when SSDs started to gain popularity Fusion-io added ioTurbine, which was probably the first cache layer to be placed between DRAM and HDD.

When Intel and Micron announced 3D XPoint memory in 2015, they explained that it was a new memory layer destined to fit between DRAM and NAND flash SSDs.  This made it a cache in front of the SSD cache!

Those opposing the addition of all these new layers argued that the system was becoming unnecessarily complex, or that it was nondeterministic, since execution time would vary, depending on how often a new layer would experience a cache miss.  Little did these people realize that determinism can’t exist in any system that includes a virtual memory, much less the caches that appear in every system based upon an Intel or AMD processor!  Even the mere presence of interrupts destroys the myth of determinism!

Other advocates for layer reduction took a different approach, effectively saying: “You can add a layer, but you must, in turn, remove another one.”  One such suggestion is the Flape model, where HDDs are abandoned as the layer between flash and tape (thus the name).  Intel advocated the use of its Optane (3D XPoint) SSDs to support SSD speeds in an HDD-based system without actually using a flash SSD.  (You have to imagine that the SSD team at Intel wasn’t very fond of this campaign.)  The campaign disappeared from Intel’s website when Intel  discontinued sales of consumer Optane SSDs.

With all of these new levels, a revised nomenclature seems in order.  Out of the goodness of my heart, The Memory Guy has elected to name the next few layers as a service to the industry.  They are illustrated in the Memory/Storage Hierarchy chart that appears in the diagram below.

Memory Storage Hierarchy with too many levelsWhile I would rather use numbers than names (partly because my mother always admonished me against name-calling) the industry has embarked in a different direction, and I cannot stem the tide.

The whole business of name-calling brings to mind the foibles of RF engineers in the early 20th century, when they used the names Long Wave, Medium Wave, and Short Wave, and naively expected for everything to stop there.  It wasn’t too long before they required new names for higher frequencies, and coined the terms Very-High Frequency (VHF), and Ultra-High Frequency (UHF), hoping that these would put an end to it.  But frequencies continued to increase, and a new convention became necessary, to create the exotic terms, x-band, k-band, and ku-band.  Today all of these names have become well beyond obsolete, as significantly-higher frequencies serve such mundane purposes as CPU clocks, police radar, electronic door openers, and collision avoidance systems.

I have chosen a naming system that prevents such worries.  There will always be an abundance of names, each of which will be as subjective as all of its predecessors, and each of which will indicate that there can never be any additional layers.

While the names in my new chart might sound odd, they are an extension of the disciplinary words used by parents to let their children know that they have crossed the line.  Since that line continually moves, parents have devised more sayings than can be contained in a single blog post.  This means that there will always be an endless number of new terms that can be used to indicate that the next layer in the memory/storage hierarchy is the very last, even though new names are indeed available when that fact becomes fiction.

In future posts I plan to create a memory/storage hierarchy pyramid with numerous layers, each too tiny to discern from the next.   There will be one layer for each of the categories in the chart above.  As with the currently-popular pyramid, it will be pretty, but will convey little useful information.

To learn which of these names is the current ultimate cache layer, please check back on The Memory Guy blog every year on the first of April.


One thought on “Putting the Brakes on Added Memory Layers”

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.