CMU Computer Systems: The Memory Hierarchy

Memory

• Storage technologies and trends
• Locality of reference
• Caching in the memory hierarchy

Random-Access Memory (RAM)

• Key features

  • RAM is traditionally packaged as a chip
  • Basic storage unit is normally a cell (one bit per cell)
  • Multiple RAM chips form a memory

• RAM comes in two varieties

  • SRAM (Static RAM)
  • DRAM (Dynamic RAM)

Nonvolatile Memories

• DRAM and SRAM are volatile memories

  • Lose information if powered off

• Nonvolatile memories retain value even if powered off

  • Read-only memory (ROM): programmed during production

  • Programmable ROM (PROM): can be programmed once

  • Erasable PROM (EPROM): can be bulk erased

  • Electrically erasable PROM (EEPROM): electronic erase capability

  • Flash memory: EEPROMs. with partial (block-level) erase capability

    • Wears out after about 100,000 erasings

• Uses for Nonvolatile Memories

  • Firmware programs stored in a ROM
  • Solid state disks
  • Disk caches

Traditional Bus Structure Connecting CPU and Memory

• A bus is collection of parallel wires that carry address
• Buses are typically shared by multiple devices

Solid State Disks (SSDs)

• Pages and Blocks
• Data read/written in units of pages
• Page can be written only after its block has been erased
• A block wares out after about 100,000 repeated write

SSD Performance Characteristics

• Sequential access faster than random access
• Random writes are somewhat slower

Locality

• The key to bridging the CPU-Memory gap is a fundamental property of computer programs.

• Principle

  • Programs tend to use data and instructions with address near or equal to those they have used recently

• Temporal locality

  • Recently referenced items are likely to be referenced again in the near future

• Spatial locality

  • Items with nearby addresses tend to be referenced close together in time

Qualitative Estimates of Locality

Claim: Being able to look at code and get a qualitative sense of its locality is a key skill for a professional programmer

Memory Hierarchy

• Regs: CPU registers hold words retrieved from the L1
• L1 cache: hold cache lines retrieved from the L2
• L2 cache: hold cache lines retrieved from the L3
• L3 cache: hold cache lines retrieved from main memory
• Main memory (DRAM)
• Local secondary storage
• Remote secondary storage

Caches

• A smaller, faster storage device that acts as a staging area for a subset of the data in a larger, slower device

• Fundamental idea

  • For each k, the faster, smaller device at level k serves as a cache for the larger, slower device at level k+1

Types of Cache Misses

• Cold (compulsory) miss

  • the cache is empty

• Conflict miss

  • Most caches limit blocks at level k+1 to a small subset

• Capacity miss

  • the set of active cache blocks is larger than the cache

Summary

• The speed gap between CPU, memory and mass storage continues to widen,
• Well-written programs exhibit a property called locality.
• Memory hierarchies based on caching close the gap by exploiting locality.