Section 12: Threads and lock granularity

In the first part of section, we’ll spend some time on some simple thread examples, working through ways of enforcing ordering through atomics, mutexes, and condition variables. Then we turn to the problem set, discussing different coarse-grained and fine-grained synchronization designs. After a discussion of fancy mutexes, there are some poop jokes.

Three threads

Recall that std::thread's constructor takes the name of the function to run on the new thread and any arguments, like std::thread t(func, param1, ...). Given a thread t, t.join() blocks the current thread until thread t terminates. t.detach() sets the t thread free—it can no longer be joined. Each thread object must be detached or joined before it goes out of scope (is destroyed).

QUESTION. What can this code print, and how many threads might run concurrently during its execution? (sample01.cc)

int main() {
    for (int i = 0; i != 3; ++i) {
        std::thread t(printf, "%d\n", i + 1);
        t.join();
    }
}

QUESTION. What can this code print, and how many threads might run concurrently during its execution? (sample02.cc)

int main() {
    for (int i = 0; i != 3; ++i) {
        std::thread t(printf, "%d\n", i + 1);
        t.detach();
    }
}

QUESTION. What can this code print? (sample03.cc)

int main() {
    for (int i = 0; i != 3; ++i) {
        std::thread t(printf, "%d\n", i + 1);
    }
}

Counting to three

We now turn to this code, in counting.cc.

#include <thread>
#include <mutex>
#include <condition_variable>
#include <cstdio>

/* G1 */

void counter(int n) {
    /* C1 */
    printf("%d\n", n + 1);
    /* C2 */
}

int main() {
    /* M1 */
    std::thread t1(counter, 0);
    std::thread t2(counter, 1);
    std::thread t3(counter, 2);
    /* M2 */
    t3.join();
    t2.join();
    t1.join();
}

EXERCISE. What can this code print, and how many threads might run concurrently during its execution?

In the remaining exercises, we will synchronize this code so that it always prints 1\n2\n3\n, in that order. (The resulting code might as well run in a single thread—all useful work is performed by one thread at a time—but that’s OK for our purposes.)

Counting with atomics

EXERCISE. Use atomic variables to ensure that counting.cc always prints 1\n2\n3\n. There must be no data races. You should add code only in the marked locations (G1, C1, C2, M1, and/or M2). In particular, you should not reorder the join calls.

This is a synchronization problem. We ensure that threads execute in a specific order by introducing shared state on which the threads coordinate. But when shared state exists, accessing it safely requires synchronization.

As you design your synchronization state and code, think about the purpose served by the shared state that coordinates these threads. Discussion questions:

  1. Describe that purpose at a high level.
  2. How many atomic variables are required to represent this state? Are there different coordination designs with different numbers of atomic variables?
  3. Can these atomic variables be local variables within counter, or must they be located elsewhere? Why?

QUESTION. Does your atomics-based solution use blocking or polling?

(Recall that blocking is when a thread waits in a non-runnable state. This can happen, for example, if the thread calls a system call like usleep, or waitpid without WNOHANG; the calling thread will not run until the system call completes, leaving the operating system free to do other work. Polling is when a thread waits in a runnable state—it continually loops back and checks to see if it can continue. This can happen, for example, if the thread calls waitpid with WNOHANG, in a loop.)

Counting with mutexes

QUESTION. The following code using std::mutex is suspicious—very likely to be incorrect. Describe why, and suggest how to fix this problem.

std::mutex m;
int phase;
...
m.lock();
while (phase != 1) {
}
m.unlock();

EXERCISE. Change counting.cc to use synchronize using mutexes. Do not use condition variables or atomics.

EXERCISE. Consider the solutions for the last exercise. Discuss different ways the synchronization can be moved around or changed without breaking functionality.

QUESTION. Does your mutex-based solution use blocking or polling?

Counting with condition variables

EXERCISE. Change counting.cc to use synchronize using mutexes and condition variables. Do not use atomics.

QUESTION. Does your condition-variable-based solution use blocking or polling?

QUESTION. Which of these solutions use fine-grained synchronization (as opposed to coarse-grained synchronization)?

Recall that fine-grained synchronization is when a program involves many synchronization objects, each protecting a relatively small amount of state (or, in the case of condition variables, representing a specific condition). Coarse-grained synchronization is when a program involves relatively few synchronization objects, each protecting a larger amount of state (or representing a more general condition). In programs with coarse-grained synchronization, only a small number of threads can execute concurrently (because each thread typically acquires a coarse-grained mutex that locks out most other threads). In programs with fine-grained synchronization, a larger number of thread can execute concurrently (each thread may acquire several mutexes, but other threads can proceed if they operate on different state and therefore acquire different mutexes).

Breakout synchronization

In the problem set, your task involves implementing non-coarse-grained synchronization for a “breakout” game, in which many balls, each handled by a single thread, carom around a rectangular field.

To design a synchronization strategy for a task, it’s necessary to enumerate the shared state required for that task—especially the modifiable shared state. To avoid data races, any modifiable shared state must be accessed in a synchronized manner—generally while protected by a mutex, though potentially through atomics.

The shared state in the breakout game consists of:

A breakout game with B threads and W warp-tunnel ends has (B + W) threads. The ball threads move balls around the board, while the warp-tunnel threads warp balls from position to position.

QUESTION. What parts of this state are read-only after initialization (i.e., they are only modified while the game is being initialized, before any ball or warp threads begin)? Do these parts of the state require synchronization to access? (You can figure this out by examining the code!)

QUESTION. Which threads might modify a ball’s x,y position? (Again, you can figure this out by examining the code.)

QUESTION. Which threads might modify a ball’s dx,dy direction?

QUESTION. Assume you were implementing a coarse-grained locking strategy where a single mutex protected all shared state. How would you modify ball_thread to implement correct synchronization? Would you need to pulse the mutex?

DISCUSSION QUESTION. Discuss in general terms how you might build a fine-grained synchronization strategy for the breakout board. Don’t worry about warps at first. More specifically:

  1. Would it make more sense to implement one mutex per ball, or one mutex per board position (x,y)? (Recall that the goal of fine-grained synchronization is for threads to acquire a small number of mutexes, and for different threads to typically acquire different mutexes.)

  2. How would it be possible to implement a strategy with fewer than one mutex per board position, but two or more mutexes for the board as a whole?

  3. Would it make more sense to implement one condition variable per ball, or one condition variable per board position?

DISCUSSION QUESTION. In the handout code, a warp thread can change a ball’s x,y position. Can you change the code so that the warp thread does not change ball positions—instead, the ball’s own thread changes its position? Why might this be useful for synchronization purposes?

Fancy mutexes

std::recursive_mutex is like a normal mutex, but a thread that holds a lock may recursively acquire it again (as long as they recursively release it the same number of times). This might be useful if you need to acquire several locks, and sometimes some of those locks might actually be the same lock: with recursive locks, you don't need to keep track of which ones you already hold, to avoid acquiring the same lock twice.

std::shared_mutex is an instantiation of a different kind of lock altogether: a readers-writer lock, or a shared-exclusive lock. The idea behind this kind of lock is that rather than just have one level of access—mutual exclusion—there are two levels of access: shared access and exclusive access. When a thread holds the lock with shared access, that guarantees that all other threads that hold the lock have shared access. When a thread holds the lock with exclusive access, that guarantees that it is the only thread to have any kind of access. We call this a readers-writer lock because the two levels of access often correspond to multiple reader threads, which require the shared data not to change while they're accessing it, but don't modify it themselves; and a single writer thread, which needs to be the only thread with access to the data it's modifying. You do not need the functionality of std::shared_mutex for the problem set, and in fact it is not particularly helpful for this particular application.

std::mutex will likely suffice for this problem set, though std::recursive_mutex may simplify some logic.

std::lock avoids deadlock

C++ also defines some handy tools to acquire locks while avoiding deadlock. Instead of calling .lock() on the mutex itself, you can pass several mutexes as arguments to the function std::lock, which magically acquires the locks in an order that avoids deadlock. For example:

#include <mutex>

std::mutex a;
std::mutex b;

void worker1() {
    std::lock(a, b);
    ...
    a.unlock(); // the order of these doesn't matter,
    b.unlock(); // since unlocking can't cause deadlock
}

void worker2() {
    std::lock(b, a); // look ma, no deadlock!
    ...
    b.unlock();
    a.unlock();
}

...

std::unique_lock and std::scoped_lock unlock for you

The std::scoped_lock and std::unique_lock objects automate some aspects of locking, because they are capable of automatically unlocking. This is really convenient—it’s very easy to forget a call to unlock if a function has multiple return statements in it.

To use a scoped lock, you declare an object at the point where you would normally call lock(). This object is typically called a guard, since it exists to “guard” the lock. You must pass in a reference to the std::mutex that you intend to lock. For example, here’s a version of the increase function from above:

#include <mutex>

int i;
std::mutex m; // protects write access to i

void increase() {
    std::unique_lock guard(m);
    i++;
}

The declaration of guard will block until m can be locked, and then m is automatically unlocked when the function returns and guard goes out of scope (the variable stops existing). This is an instance of an idiom called Resource acquisition is initialization (RAII), which is the idea that an object's resources are automatically acquired and released by its constructor and destructor, which are triggered by scope.

Here’s how std::scoped_lock (which is the multi-lock version of std::unique_lock) would simplify the code above:

#include <mutex>

std::mutex a;
std::mutex b;

void worker1() {
    std::scoped_lock guard(a, b);
    ...
}

void worker2() {
    std::scoped_lock guard(b, a);
    ...
}

...

The std::unique_lock has .lock(), .try_lock(), and .unlock() member functions, in addition to the initialization/deinitialization style, and these can be useful sometimes. (The std::scoped_lock has no member functions at all: you can only use it by declaring it and then letting it go out of scope later.)

Bathroom synchronization

Ah, the bathroom. Does anything deserve more careful consideration? Is anything more suitable for poop jokes?

Bathroom 1: Stall privacy

In this exercise, the setting is a bathroom with K stalls numbered 0 through K–1. A number of users enter the bathroom; each has a preferred stall. Each user can be modeled as an independent thread that repeatedly calls the following code:

void user(int preferred_stall) {
    poop_into(preferred_stall);
}

Each user has a preferred stall that never changes, and two users should never poop_into the same stall at the same time.

You probably can intuitively smell that this bathroom model has a synchronization problem. Answer the following questions without referring to the solutions first.

Question: What synchronization property is missing?

Question: Provide synchronization using a single mutex, i.e., using coarse-grained locking. Your solution should be correct (it should avoid gross conditions [which are race conditions in the context of a bathroom-themed synchronization problem]), but it need not be efficient. Your solution will consist of (1) a definition for the mutex, which must be a global variable, and (2) new code for user(). You don’t need to sweat the syntax.

Question: Provide synchronization using fine-grained locking. Your solution should be more efficient than the previous solution. Specifically, it should allow the bathroom to achieve full utilization: it should be possible for all K stalls to be in use at the same time. You will need to change both the global mutex—there will be more than one global mutex—and the code for user().

Bathroom 2: Stall indifference

The setting is the same, except that now users have no predetermined preferred stall:

void user() {
    int preferred_stall = random() % K;
    poop_into(preferred_stall);
}

Question: Adapt your solution from Bathroom 1 to work for this variant.

Question: As written, the user code picks a preferred stall and then waits for that stall to become available. This isn’t as efficient as possible, since multiple users might pick the same stall by chance. Write a synchronization solution where each user chooses any available stall. First, use a single mutex coupled with a global data structure representing stall occupancy.

Question: Repeat this function, but this time use multiple mutexes and no other global data structure.

Question: Compare these two implementations in terms of the number of std::mutex lock attempts it might take to poop.

Question: These solutions implement a polling style. Why?

Question: Implement a blocking implementation. You will need a condition_variable.

Bathroom 3: Pooping philosophers

To facilitate the communal philosophical life, Plato decreed that philosophy should be performed in a bathroom with K thrones and K curtains, arranged like so:

Philosophers’ bathroom

Each philosopher sits on a throne with two curtains, one on each side. The philosophers normally argue with one another. But sometimes a philosopher’s got to poop, and to poop one needs privacy. A pooping philosopher must first draw the curtains on either side; philosopher X will need curtains X and (X+1) mod K. For instance, here, philosophers 0 and 2 can poop simultaneously, but philosopher 4 cannot poop until philosopher 0 is done (because philosopher 4 needs curtain 0), and philosopher 1 cannot poop until philosophers 0 and 2 are both done:

Philosophers’ bathroom with curtains drawn

Question: In high-level terms, what is the mutual exclusion problem here—that is, what resources are subject to mutual exclusion?

A philosopher could be described like this:

void philosopher(int x) {
    while (true) {
        argue();
        draw curtains x and (x + 1) % K;
        poop();
        release curtains x and (x + 1) % K;
    }
}

Question: Write a correctly-synchronized version of this philosopher. Your solution should not suffer from deadlock, and philosophers should block (not poll) while waiting to poop. First, use std::scoped_lock—which can (magically) lock more than one mutex at a time while avoiding deadlock.

Question: Can you do it without std::scoped_lock?