Why DRAM is Threatened by SSDs

Memcon Slide on FlashConventional wisdom holds that SSDs will someday displace all HDDs, but in reality SSDs are proving to be more of a challenge to the DRAM market than to the HDD market.

Right now you are probably reviewing the date of this post to make sure it’s not dated April 1.  I assure you that this is the truth.  To understand it, though, you must look at a computer as a computer architect would, or, in other words, the way that an application program sees the memory/storage hierarchy.

To the application program there is no HDD and memory, there is only memory.  The Virtual Memory system, a part of the operating system, hides the difference between the two by moving code and data into DRAM as it is needed and back onto the HDD when it is no longer important, without telling the application program that it is moving anything around.  I like to tell people that the DRAM makes the HDD look fast, and the HDD makes the DRAM look big.

If you think of the DRAM as something that makes the HDD look fast, then additional DRAM should help to make the HDD look even faster.  This has long been the standard approach to accelerating a computer’s performance.

Oddly enough, the SSDs used in servers are doing more to displace DRAM than they are HDDs.  System Administrators (SysAdmins) have found that, although their software runs somewhat faster if they add more DRAM, it runs a whole lot faster if they don’t add DRAM but instead add an SSD that costs the same as the additional DRAM would have cost.

This post shares two slides that The Memory Guy presented at Cadence’s Memcon conference back in 2015.  It’s a phenomenon that has been understood by the server community for a number of years, but that still mystifies other members of the DRAM/SSD/HDD community.  I’ll explain it in a way that should help anyone to understand.

The first slide is a graphic build, so it will make up the next four figures in this post.  This is followed by the table from the following slide that puts some simple math around the general concepts illustrated by the graphs.

First, let’s assume that there are two programs that are profiled in the  chart below.  The chart shows how many accesses the programs perform to certain addresses.  These are drawn as bell curves.  The black line represents a program whose code and data accesses largely occur within a relatively tight range, while the red curve represents a program whose accesses are more broadly distributed.

Two Different Programs' Accesses

All programs behave in a similar manner – most accesses occur within a certain range, and fewer fall outside of that range.  This is called “locality” and it’s the reason that computers can take advantage of cache memories and virtual memory configurations.  If you want to understand this better I will naturally recommend The Cache Memory Book, which provides a lot of detail on this phenomenon.

Now a typical computer without an SSD will store some of these accesses in DRAM, while others will need to be fetched from the HDD.  The DRAM is represented by the yellow box in the figure below.  The width of the box represents the share of the accesses that are serviced by the DRAM, and the accesses that lie outside of the box force the operating system to go to the HDD, which takes about one million times (10^6) as long as a DRAM access.

Accesses serviced by basic DRAM

SysAdmins who wanted to improve the speed of their computers before SSDs would add more DRAM to capture more accesses.  The chart below represents the added DRAM by showing two boxes that cover twice the width of the original box.

Accesses serviced with twice as much DRAM

It’s pretty clear that the black-line program benefits more from this than the  red-line program.  Still, very many accesses fall outside of the boxes, incurring that 10^6 HDD penalty.

An alternative to adding this second amount of DRAM would be to spend the same amount of money on an SSD.  Today you can buy about twenty times as many gigabytes of SSD NAND as you can of DRAM: An 8GB DRAM module sells for about the same price as a 160GB SSD.  The SSD is considerably slower than the DRAM, taking about 1,000 (10^3) times as long as the DRAM to provide the data, but it’s a lot larger so it captures significantly more accesses for either one of the curves.  Does this help?  Well the diagram below tries to provide a visual representation of this, but we will have to use math to figure out how it impacts the system.

Accesses serviced with base DRAM and an SSD

Clearly very few of the accesses must now go to the slow HDD.

There’s no real precision in the illustration: The SSD box should be 20 times the width of the DRAM box, but it’s actually less than ten times as wide.  The slower speed of the SSD is indicated by the smaller height of the box, but this isn’t an accurate way to represent this, so let’s do some math.

For the black line a single block of DRAM satisfies about 55% of all accesses and a double-DRAM system can fulfill around 90% of accesses.  All other accesses will take one million times as long while the system waits for the HDD.  If we add a SSD it will provide the data for nearly all of the remaining accesses, so we will give that a 44% “Hit Rate” for the accesses outside of the 55% that the single DRAM supplies.  That means that the SSD will provide data 10^3 times as slowly as the DRAM for 44% of the accesses while the HDD, which takes 10^6 as long as DRAM, is only accessed 1% of the time.

Using this we can calculate the average latency of two systems that cost roughly the same, the Double-DRAM system (labeled “No SSD”) vs. the Single-DRAM system that includes an SSD:

Memcon Tale comparing DRAM and NAND flash

The average latency for the  SSD-based system is clearly better by an order of magnitude.  This is why SSDs have become a part of nearly every server installation today.  This has also limited the growth of the DRAM market to some extent.

Remember that this is just a calculation for the black-line program.  The red-line program benefits less from the added DRAM, yet nearly all of its accesses are covered by the SSD, so it will see an even better improvement by going from a Double-DRAM configuration to a Single-DRAM/SSD configuration.

So, if you have a fixed budget, SSDs can help you get the most out of your system and are a better alternative than additional DRAM.

What the table DOESN’T show is what would happen if you spent more money to double the DRAM while also adding an SSD.  In this case 90% of the accesses would go to the DRAM, and still only 1% to the HDD, leaving the other 9% to the SSD.  The difference is only about 3% of a performance improvement over the Single-DRAM system, so it’s a small benefit in return for a significant cost increase.

There’s a compelling  argument to add SSDs as an alternative to increasing the DRAM size in most systems, but the curves above don’t represent real programs.  Today few software tools are available to help SysAdmins optimize their configurations, so they have to use a “Cut and Try” approach to learn what’s the best configuration for their system.  As long as they remember to try using an SSD as an alternative to additional DRAM, though, they should be able to get the greatest performance out of their equipment budget.

3 Responses to Why DRAM is Threatened by SSDs

  • Tim Carpenter says:

    You do know that an SSD bit can only be written to (or more precisely erased) a few thousand times, right? DRAM is virtually unlimited write cycles

    • Jim Handy says:

      Tim, Thanks for the comment.

      I get this question a lot, so perhaps I need to be more clear. Trading off SSDs against DRAM is a pretty abstract concept.

      The SSDs in this kind of system are managed by caching or tiering software and thus experience low write loads. Software minimizes the number of writes that are sent to the HDD as well. Imagine how slow would your system would be if every CPU write went to an HDD!

      Remember, too, that the HDD and SD are not only written to when you save a file. There are countless other operations that aren’t visible to the user, but that access the HDD or SSD.

      What’s important is to look at the DRAM/SSD/HDD complex as a single entity. How do you get the highest performance per dollar from this? DRAM’s the fastest and most expensive part, and HDD is the slowest and cheapest, but by careful balancing of the amount of DRAM, SSD, and HDD you can get the most out of your cash outlay. Usually you don’t need all that much DRAM.

      The people who REALLY understand this and use it to their benefit are the hyperscale datacenter companies: Facebook, Google, Amazon, Microsoft, Baidu, Tencent… They do this all the time, but they don’t share a lot of details since it provides a competitive edge.

      Hope that helps.

      Jim

  • Joe Chang says:

    DRAM and SSD might have been a trade-off 4-8 years ago. Back then, some storage vendors had the idea of tiered SSD/HDD storage. Today it might be tiered hi-lo SSD. In any case, now we probably have more DRAM and more NAND flash than we really need.
    The issue with NAND flash today is that manufacturers seem overly focused on price. I can live with TLC, but I wish that for an enterprise product that could reset it to operate as MLC or even SLC. Also good would be a settable over-provisioning value.
    Ideally, there should also be an enterprise NAND product with smaller page size than the consumer oriented 16KB. Whether this is 4KB to match the new disk sector size or even 1KB with the intent of having 5 NAND chips form a RAID 5 stripe of 4KB. The purpose is achieve faster response time, I don’t want to wait 85us to read 1 page.
    Even so, NAND is not the product with which to go after DRAM. For that, it should be NOR. Ten years ago, the higher density of NAND was important to achieve a cost viable product. Now with NAND at $0.50/per GB versus $8/GB for DRAM, I think there is room in between. So the question is: can NOR be manufactured at 2X the cost of NAND? If so, then it would be an awesome DRAM alternative.
    And then as I have argued before, I am willing to give up DRAM capacity. But I want low memory latency, with one example being RL-DRAM, and I am not averse to SRAM either. Though both would require getting closer to the processor.

    ps, I would not get hung up on forecast DRAM and NAND prices. It a way, they are like oil, consumers must have it, manufacturers must make it according to the capacity of the their fabs. Any moderate imbalance of supply or demand can result wild swings in price.
    So just look at price + vendor gross margin to estimate cost.

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.