Storage 1: OS input/output, memory hierarchy

Overview

In this bridge lecture to the storage unit, we discuss how files are read and written, starting with a WeensyOS example and transitioning to Linux.

Full lecture notes on storage — Textbook readings

System calls for reading and writing

• Process isolation means processes don’t directly interact
• They can interact via explicit, checked input/output (I/O)
• Often this is to persistent storage
  • “Persistent” means it doesn’t go away when the process, or the computer, dies
• It uses special I/O system calls

WeensyOS reading

• ssize_t sys_read(void* buf, size_t sz)
  • Read up to sz bytes from kernel “file”
  • Bytes read are consumed
• Returns number of bytes successfully read
  • Always ≤ sz
  • If 0, “file” can no longer be read
  • If -1, process should try again

Example reader

int readc() {
    unsigned char ch;
    ssize_t r = -1;

    while (r < 0) {
        r = sys_read(&ch, 1);
    }

    assert(r == 1);
    return ch;
}

WeensyOS writing

• ssize_t sys_write(const void* buf, size_t sz)
  • Write up to sz bytes to kernel “file”
• Returns number of bytes successfully written
  • Always ≤ sz
  • If 0, then “file” can no longer be written
  • If -1, process should try again

Example writer

int writec(unsigned char ch) {
    ssize_t r = -1;

    while (r < 0) {
        r = sys_write(&ch, 1);
    }

    assert(r == 1);
    return ch;
}

Relationship to Unix

• ssize_t read(int fd, void* buf, size_t sz)
• ssize_t write(int fd, const void* buf, size_t sz)
• Change: addition of int fd
  • Allows reading from/writing to more than one file
  • More later!

WeensyOS p-reader and p-writer

• Reader loop: read a line, print it
• Writer loop: write a line, print it, wait 1–3 sec

Speed?

Moving to Unix

Write a file one byte at a time using system calls

    size_t n = 0;
    while (n < size) {
        ssize_t r = write(fd, buf, 1);
        if (r != 1) {
            perror("write");
            exit(1);
        }
        ++n;
    }

• w-syncbyte: Open the file in a mode so that every write is made persistent before the write system call returns
• w-osbyte: Open the file in normal mode

Write a file one byte at a time using the stdio library

    size_t n = 0;
    while (n < size) {
        int ch = fputc(buf[0], f);
        if (ch == EOF) {
            perror("write");
            exit(1);
        }
        ++n;
    }

• fputc(buf[0], f) ≈ fwrite(buf, 1, 1, f), with different error return convention
  • Check the manual!

Question 1

• How long will it take each program to write data one byte at a time?
• Will the results differ by operating system? How much?

Storage hierarchy

• Also called “memory hierarchy”

Expense of storage

• Expense is a major factor in the construction of computer systems
  • Faster technologies are often more expensive
  • Older, slower technologies are often cheaper
• The relative and absolute expenses of storage technologies may be the single thing about computer systems that has changed the most dramatically over time

Question 2

• How much more did a megabyte of memory cost in 1955 relative to now?
• How much more did a megabyte of hard disk storage cost in 1955 relative to now?

Historical costs of storage

Absolute costs ($/MB):

Relative costs (relative to hard disk storage in 2022):

(Processor speed has also increased a lot—from 0.002 MIPS in 1951 to 750,000 MIPS now—but this is “only” 375,000,000x.)