Why DRAM is Threatened by SSDs

Conventional wisdom holds that SSDs will someday displace all HDDs, but in reality SSDs are proving to be more of a challenge to the DRAM market than to the HDD market.

Right now you are probably reviewing the date of this post to make sure it’s not dated April 1.  I assure you that this is the truth.  To understand it, though, you must look at a computer as a computer architect would, or, in other words, the way that an application program sees the memory/storage hierarchy.

To the application program there is no HDD and memory, there is only memory.  The Virtual Memory system, a part of the operating system, hides the difference between the two by moving code and data into DRAM as it is needed and back onto the HDD when it is no longer important, without telling the application program that it is moving anything around.  I like to tell people that the DRAM makes the HDD look fast, and the HDD makes the DRAM look big.

If you think of the DRAM as something that makes the HDD look fast, then additional DRAM should help to make the HDD look even faster.  This has long been the standard approach to accelerating a computer’s performance.

Oddly enough, the SSDs used in servers are doing more to displace DRAM than they are HDDs.  System Administrators (SysAdmins) have found that, although their software runs somewhat faster if they add more DRAM, it runs a whole lot faster if they don’t add DRAM but instead add an SSD that costs the same as the additional DRAM would have cost.

This post shares two slides that The Memory Guy presented at Cadence’s Memcon conference back in 2015.  It’s a phenomenon that has been understood by the server community for a number of years, but that still mystifies other members of the DRAM/SSD/HDD community.  I’ll explain it in a way that should help anyone to understand.

The first slide is a graphic build, so it will make up the next four figures in this post.  This is followed by the table from the following slide that puts some simple math around the general concepts illustrated by the graphs.

First, let’s assume that there are two programs that are profiled in the  chart below.  The chart shows how many accesses the programs perform to certain addresses.  These are drawn as bell curves.  The black line represents a program whose code and data accesses largely occur within a relatively tight range, while the red curve represents a program whose accesses are more broadly distributed.

All programs behave in a similar manner – most accesses occur within a certain range, and fewer fall outside of that range.  This is called “locality” and it’s the reason that computers can take advantage of cache memories and virtual memory configurations.  If you want to understand this better I will naturally recommend The Cache Memory Book, which provides a lot of detail on this phenomenon.

Now a typical computer without an SSD will store some of these accesses in DRAM, while others will need to be fetched from the HDD.  The DRAM is represented by the yellow box in the figure below.  The width of the box represents the share of the accesses that are serviced by the DRAM, and the accesses that lie outside of the box force the operating system to go to the HDD, which takes about one million times (10^6) as long as a DRAM access.

SysAdmins who wanted to improve the speed of their computers before SSDs would add more DRAM to capture more accesses.  The chart below represents the added DRAM by showing two boxes that cover twice the width of the original box.

It’s pretty clear that the black-line program benefits more from this than the  red-line program.  Still, very many accesses fall outside of the boxes, incurring that 10^6 HDD penalty.

An alternative to adding this second amount of DRAM would be to spend the same amount of money on an SSD.  Today you can buy about twenty times as many gigabytes of SSD NAND as you can of DRAM: An 8GB DRAM module sells for about the same price as a 160GB SSD.  The SSD is considerably slower than the DRAM, taking about 1,000 (10^3) times as long as the DRAM to provide the data, but it’s a lot larger so it captures significantly more accesses for either one of the curves.  Does this help?  Well the diagram below tries to provide a visual representation of this, but we will have to use math to figure out how it impacts the system.

Clearly very few of the accesses must now go to the slow HDD.

There’s no real precision in the illustration: The SSD box should be 20 times the width of the DRAM box, but it’s actually less than ten times as wide.  The slower speed of the SSD is indicated by the smaller height of the box, but this isn’t an accurate way to represent this, so let’s do some math.

For the black line a single block of DRAM satisfies about 55% of all accesses and a double-DRAM system can fulfill around 90% of accesses.  All other accesses will take one million times as long while the system waits for the HDD.  If we add a SSD it will provide the data for nearly all of the remaining accesses, so we will give that a 44% “Hit Rate” for the accesses outside of the 55% that the single DRAM supplies.  That means that the SSD will provide data 10^3 times as slowly as the DRAM for 44% of the accesses while the HDD, which takes 10^6 as long as DRAM, is only accessed 1% of the time.

Using this we can calculate the average latency of two systems that cost roughly the same, the Double-DRAM system (labeled “No SSD”) vs. the Single-DRAM system that includes an SSD:

The average latency for the  SSD-based system is clearly better by an order of magnitude.  This is why SSDs have become a part of nearly every server installation today.  This has also limited the growth of the DRAM market to some extent.

Remember that this is just a calculation for the black-line program.  The red-line program benefits less from the added DRAM, yet nearly all of its accesses are covered by the SSD, so it will see an even better improvement by going from a Double-DRAM configuration to a Single-DRAM/SSD configuration.

So, if you have a fixed budget, SSDs can help you get the most out of your system and are a better alternative than additional DRAM.

What the table DOESN’T show is what would happen if you spent more money to double the DRAM while also adding an SSD.  In this case 90% of the accesses would go to the DRAM, and still only 1% to the HDD, leaving the other 9% to the SSD.  The difference is only about 3% of a performance improvement over the Single-DRAM system, so it’s a small benefit in return for a significant cost increase.

There’s a compelling  argument to add SSDs as an alternative to increasing the DRAM size in most systems, but the curves above don’t represent real programs.  Today few software tools are available to help SysAdmins optimize their configurations, so they have to use a “Cut and Try” approach to learn what’s the best configuration for their system.  As long as they remember to try using an SSD as an alternative to additional DRAM, though, they should be able to get the greatest performance out of their equipment budget.