Ancient CS 61 Content Warning!!!!!1!!!
This is not the current version of the class.
This site was automatically translated from a wiki. The translation may have introduced mistakes (and the content might have been wrong to begin with).

Lecture 11 Thoughts

Memory error exploits

Wikipedia of course has tons of information on buffer overflows.

Famous buffer overflow exploits include the Morris worm (1988), Code Red (2001–2002), and SQL Slammer (2003).

Buffer overflows still happen, but relatively less often: US-CERT Vulnerability Database search for “buffer overflow”. One reason is that they’re not as profitable any more. Compiler, language, library, OS, and processor designers have implemented a lot of cool techniques for neutering buffer overflow attacks.

Buffer overflow protection

The earliest buffer overflow attacks were pretty direct. An attacker’s buffer would contain actual x86 instructions for the attack. The exploit address (that is, the value that replaced the return address on the stack) would point at those instructions within the buffer.

This worked because of two weaknesses: (1) CPUs could execute code anywhere in memory, and (2) operating systems placed stacks at predictable addresses. Because of (2), the address of a vulnerable stack buffer was stable—every time a program ran, the buffer would have the same address. Thus, an attacker could figure out the address of a vulnerable stack buffer (for instance, using gdb), write exploit code designed to execute at that address, and replace the vulnerable function’s return address with that of the buffer. The vulnerable function’s ret instruction would then set the processor’s program counter to the buffer, and thanks to (1), the CPU would then execute the exploit code.

Attack protection strengthened each of these weaknesses.

First, operating system designers addressed weakness (2) by placing stacks at unpredictable addresses. This technique is an example of address space layout randomization. Run a program a couple times and use gdb to print the stack pointer at the beginning of main(); you’ll notice that the value changes! When a buffer’s address is unpredictable, the simple attack described above is unlikely to work—there’s only a small chance that the vulnerable buffer is actually located at the address expected by the exploit.

Unfortunately, addresses can’t be totally randomized. The stack is always somewhere near the top of a program’s memory space. Attackers developed techniques to get around the relatively low amount of randomness ASLR could introduce, such as padding the exploit instructions with many NOP instructions so that a wide range of exploit addresses could work (see NOP slides).

The more fundamental weakness is (1), and fixing it required help from computer architects. The key feature is non-executable (NX) memory. The processor should refuse to execute instructions located in the stack, the heap, or global variables. Then direct buffer overflow attacks must always fail with a segmentation fault. Since the processor doesn’t know what addresses are in the stack or the heap, operating systems must cooperate by explicitly marking regions as non-executable.

AMD first introduced non-executable memory for some of its 64-bit processors; Intel followed suit soon after. Non-executable memory support has been in Linux in 2004 (see executable space protection and non-executable memory (the NX bit)). Several NX “hacks” are available for older x86 processors, but nowadays almost all computers have the necessary architectural support built in.

NX is a great idea that neutralizes many dangerous attacks. Some programs, though, actually need to run code located on the heap or stack. For example, just-in-time compilation is a powerful optimization strategy that generates specialized machine code at run time, based on program inputs. Your browser most likely contains a just-in-time compiler! Operating systems therefore provide a mechanism to turn off NX for specific regions of memory. See Lecture 2’s l02/mysum-hello.c for an example.

Clever attackers soon discovered buffer overflow attacks that got around NX memory. We saw one in class, a return-to-libc attack. Defenders therefore developed techniques that protect against even these attacks. GCC’s “ProPolice” stack protector is on by default; we saw it in class. Stack protection adds random canary values to the stack that are verified before a function exits. If a canary value has changed, there was a buffer overflow, and GCC quits the program. GCC (and other compilers) also include interesting techniques that can detect even heap attacks. Here’s a detailed technical description of GCC’s object size detection. Address space layout randomization, as applied to heap addresses and standard library functions, helps too.

The Morris worm

This is totally worth reading about—for historical reasons, for the gallery of security bugs it exploited, and because it shows how the lines between “good” and “bad” hacking can blur. (The worm was apparently intended for a benign purpose—either to measure the size of the Internet or to demonstrate the danger posed by the bugs it exploited. But the worm, due to bugs in the worm itself, ended up spreading far more quickly and causing far more damage than its creator intended.)

The Morris worm on Wikipedia, including a free lesson on how to make computer science boring

Detailed technical description of the worm from Eugene H. Spafford

The news report I mentioned in class: “It arrived at MIT in the middle of the night. The students were safe. Their computers weren’t.” 80sTASTIC