Kernel 1: Processes, isolation, robustness

Overview

In this first lecture in the kernel unit, we use our knowledge of assembly to investigate sanitizers, buffer overflow attacks, and interactions with the operating system.

Full lecture notes on kernel — Textbook readings

On registers

• C++: Each function has its own, possibly-infinite set of local variables
• Processor: One set of registers (per core)
  • Indefinite local variables are emulated by the compiler by saving and restoring registers to memory

On memory

• C++: Each program runs on its own memory
  • Memory is an array of bytes accessed by address
  • Accessing an unallocated address causes undefined behavior
• Processor: One primary memory

So…

• How can a computer run multiple programs at once??
• How can a running program communicate with other programs or with hardware (like a keyboard, disk, or screen)?
• How can multiple running programs be protected from each other’s mistakes?

Terminology

• A program is a set of instructions and data corresponding to a complete set of C++ source code (or source code in another programming language)
• A process is a running program (a program in execution)

Process isolation

• Processes interact only in limited ways as allowed by operating system policy
  • For instance, by reading and writing files
  • Not, for example, by overwriting each other’s memory
• A critically important feature of computer systems
  • Without it, any bug in any software could cause a “blue screen of death”

Consequences of process isolation

• Process control over computer resources must be limited
  • If a process had full control, it could run forever
  • If a process had full control, it could monopolize hardware (like the screen or the disk)
  • If a process had full control, it could destroy other processes’ memory
• Still want software to have control over computer resources
  • Policy is too fluid to bake into silicon
• Other software running on the computer must have full control over computer resources
  • We call that code the kernel
  • Specifically, the kernel is the software running on a machine that has full machine privilege
  • Different operating systems have different kernels (Linux vs. Mac OS X vs. Windows); processes running on the same operating system interact with the same kernel

Kernel responsibilities

• Safely and fairly sharing machine resources among processes
• Provide convenient abstractions for machine resources
• Ensure robustness and high performance

Process isolation’s consequences for hardware design

• There are at least two classes of software running on a computer
  • The kernel has full control over computer resources
  • Processes do not
    • We often call processes unprivileged processes or user processes to highlight this distinction
• Restrictions on processes cannot be enforced by software alone
  • The processor and other computer hardware must have a notion of privilege
  • The processor and other computer hardware must handle privilege violations
• Exception: Some experimental systems have fully-trusted source code chains, where, for example, the compiler has been proven correct, and all code running on the machine passes through the trusted compiler. In these systems, it’s theoretically possible to implement process isolation without hardware support.

Some processor features we’ll investigate

• Special instructions for transferring control to the operating system
  • System calls and the syscall instruction
• Processor privilege modes
• Hardware features that prevent monopolization of resources
• Protection of memory
  • Virtual memory

Why learn about kernels?

• It is natural for programmers to think about performance and functionality
• Harder to think about security and robustness
  • Keeping a program safe from attackers
  • Limiting the impact of bugs (which are inevitable)
  • Kernels sharpen these issues
• Understand how software and hardware interact
• Kernels are super powerful and fun

System calls from a process

• asm4/print.cc

Our own kernel!!!!