Storage 2: Caches, costs and benefits, stdio

Overview

In this lecture, we describe caches in general, including access costs, metrics, and benefits. We also use strace to explore the default standard I/O cache.

Full lecture notes on storage — Textbook readings

What is a cache?

• Cache: a small amount of fast storage used to speed up access to slower storage
• Computer systems are rife with caches
  • One of the most important patterns you learn in this class
  • Registers as a cache for primary memory
  • Memory as a cache for flash/hard disk
• Life is rife with caches

Write speeds

• w-syncbyte: Write one byte at a time, make write persistent before returning
• w-syncblock: Write one block at a time, make write persistent before returning
• w-osbyte: Write one byte at a time using system calls
• w-osblock: Write one block at a time using system calls
• w-stdiobyte: Write one byte at a time using the standard I/O cache
• w-stdioblock: Write one byte at a time using the standard I/O cache

Investigating the standard I/O cache

• strace
  • Powerful Linux program snoops on another program’s system calls
  • Prints a human-readable summary of those system calls to a file
• strace -o strace.out PROGRAM ARGUMENTS…
  • -f: follow forks (subprocesses)
  • -s N: Print more bytes of string arguments/return values
  • -e trace=open,openat,close,read,write: Only print those system calls
  • man strace for more

Cache abstraction

Storage access costs

• Caches optimize access to storage
• Storage accesses are reads and writes
  • Read: copy N units of data from storage
  • Write: copy N units of data to storage
• Break down the cost of a read or write operation

Cost model

• A storage access involves two costs
  • Per-request cost R (measured in units of time)
  • Per-unit cost U (measured in units of time per unit of data)
  • Different from operation to operation
• Cost of an access operation accessing N units of data: C = R + NU

Question 1

• What are some examples of per-request and per-unit costs in storage access operations?

Metrics: Latency and throughput

• Latency: The time it takes to access a unit of data
  • Measured in seconds
  • Smaller numbers are better
• Throughput: The rate at which data can be accessed
  • Measured in units per second (e.g., bytes per second)
  • Larger numbers are better

Relationships between latency and throughput

• For some storage technologies, latency = 1/throughput
  • True random access
• For others, these metrics can diverge
  • Hard disk drive throughput: ~1–250 MiB/s
  • SSD (flash drive) throughput: ~50–3000 MiB/s

High-latency, high-throughput I/O

Latency, throughput, and access costs

• C = R + NU
• When will 1/latency be close to throughput?
  • When R \ll U
• When might 1/latency and throughput diverge?
  • When R \gg U