Process control exercises

Exercises not as directly relevant to this year’s class are marked with ⚠️.

See also question KERN-9.

PROC-1. Rendezvous and pipes

This question builds versions of the existing system calls based on new abstractions. Here are three system calls that define a new abstraction called a rendezvous.

int newrendezvous()
Returns a rendezvous ID that hasn’t been used yet.

int rendezvous(int rid, int data)
Blocks the calling process P1 until some other process P2 calls rendezvous() with the same rid (rendezvous ID). Then, both of the system calls return, but P1’s system call returns P2’s data and vice versa. Thus, the two processes swap their data. Rendezvous acts pairwise; if three processes call rendezvous, then two of them will swap values and the third will block, waiting for a fourth.

void freezerendezvous(int rid, int freezedata)
Freezes the rendezvous rid. All future calls to rendezvous(rid, data) will immediately return freezedata.

Here’s an example. The two columns represent two processes. Assume they are the only processes using rendezvous ID 0.

This code will print

About to rendezvous
Process B got 5
Process A got 600

(the last 2 lines might appear in either order).

QUESTION PROC-1A. How might you implement pipes in terms of rendezvous? Try to figure out analogues for the pipe(), close(), read(), and write() system calls (perhaps with different signatures), but only worry about reading and writing 1 character at a time.

Here’s one mapping.

• pipe(): newrendezvous(). We use a rendezvous ID as the equivalent of a pipe file descriptor.
• close(p): To close the “pipe” p, call freezerendezvous(p, -1). Now all future read and write calls will return -1.
• read(p, &ch, 1): To read a single character ch from the “pipe” p, call ch = rendezvous(p, -1).
• write(p, &ch, 1): To write a single character ch to the “pipe” p (that is, the rendezvous with ID p), call rendezvous(p, ch).

Most mappings will have these features.

QUESTION PROC-1B. Can a rendezvous-pipe support all pipe features?

No. For example, a rendezvous-pipe doesn’t deliver a signal when a process tries to write to a closed pipe. Since the rendezvous-pipe doesn’t distinguish between read and write ends, and since rendezvous aren’t reference-counted like file descriptors, if a “writer” process exits without closing the rendezvous-pipe, a reader won’t get EOF when they read—it will instead block indefinitely. Unlike pipes, which like all file descriptors are protected from access by unrelated processes, rendezvous aren’t protected; anyone who can guess the rendezvous ID can use the rendezvous. Etc.

PROC-2. Process management

Here’s the skeleton of a shell function implementing a simple two-command pipeline, such as “cmd1 | cmd2”.

void simple_pipe(const char* cmd1, char* const* argv1, const char* cmd2, char* const* argv2) {
    int pipefd[2], r, status;

    [A]

    pid_t child1 = fork();
    if (child1 == 0) {
        [B]
        execvp(cmd1, argv1);
    }
    assert(child1 > 0);

    [C]

    pid_t child2 = fork();
    if (child2 == 0) {
        [D]
        execvp(cmd2, argv2);
    }
    assert(child2 > 0);

    [E]
}

And here is a grab bag of system calls.

[1] close(pipefd[0]);
[2] close(pipefd[1]);
[3] dup2(pipefd[0], STDIN_FILENO);
[4] dup2(pipefd[0], STDOUT_FILENO);
[5] dup2(pipefd[1], STDIN_FILENO);
[6] dup2(pipefd[1], STDOUT_FILENO);
[7] pipe(pipefd);
[8] r = waitpid(child1, &status, 0);
[9] r = waitpid(child2, &status, 0);

Your task is to assign system call IDs, such as “1”, to slots, such as “A”, to achieve several behaviors, including a correct pipeline and several incorrect pipelines. For each question:

• You may use each system call ID once, more than once, or not at all.
• You may use zero or more system call IDs per slot. Write them in the order they should appear in the code.
• You may assume that no signals are delivered to the shell process (so no system call ever returns an EINTR error).
• The simple_pipe function should wait for both commands in the pipeline to complete before returning.

QUESTION PROC-2A. Implement a correct foreground pipeline.

[A]	[B]	[C]	[D]	[E]
7	6, 1, 2 or 6, 2, 1 or 1, 6, 2		3, 1, 2	1, 2, 8, 9 or 2, 1, 8, 9 etc. (1, 2 come first)

or

[A]	[B]	[C]	[D]	[E]
7	6, 1, 2 or 6, 2, 1 or 1, 6, 2	2	3, 1	1, 8, 9

QUESTION PROC-2B. Implement a pipeline so that, given arguments corresponding to “echo foo | wc -c”, the wc process reads “foo” from its standard input but does not exit thereafter. For partial credit describe in words how this might happen.

Anything that doesn’t close the pipe’s write end will do it. Below we leave both ends of the pipe open in the shell. We could also enter just “3” in slot [D].

[A]	[B]	[C]	[D]	[E]
7	6, 1, 2		3, 1, 2	8, 9

QUESTION PROC-2C. Implement a pipeline so that, given arguments corresponding to “echo foo | wc -c”, “foo” is printed to the shell’s standard output and the wc process prints “0”. (In a correctly implemented pipeline, “wc” would print 4, which is the number of characters in “foo\n”.) For partial credit describe in words how this might happen.

Anything that doesn’t redirect the left-hand side’s standard output will do it. It is important that the read end of the pipe be properly redirected, or wc would block reading from the shell’s standard input.

[A]	[B]	[C]	[D]	[E]
7	1, 2 (anything without 6)		3, 1, 2	1, 2, 8, 9

QUESTION PROC-2D. Implement a pipeline that appears to work correctly on “echo foo | wc -c”, but always blocks forever if the left-hand command outputs more than 65536 characters. For partial credit describe in words how this might happen.

This happens when we execute the two sides of the pipe in series: first the left-hand side, then the right-hand side. Since the pipe contains 64KiB of buffering, this will often appear to work for left-hand sides that emit relatively few characters.

[A]	[B]	[C]	[D]	[E]
7	6, 1, 2	8	3, 1, 2	1, 2, 9

QUESTION PROC-2E. Implement a pipeline so that, given arguments corresponding to “echo foo | wc -c”, both echo and wc report a “Bad file descriptor” error. (This error, which corresponds to EBADF, is returned when a file descriptor is not valid or does not support the requested operation.) For partial credit describe in words how this might happen.

Given these system calls, the only way to make this happen is to redirect the wrong ends of the pipe into stdin/stdout.

[A]	[B]	[C]	[D]	[E]
7	4, 1, 2		5, 1, 2	1, 2, 8, 9

PROC-3. Processes

Consider the two programs shown below.

// Program 1
#include <cstdio>
#include <unistd.h>
int main() {
    printf("PID %d running prog1\n", getpid());
}

// Program 2
#include <cstdio>
#include <unistd.h>
int main() {
    char* argv[2];
    argv[0] = (char*) "prog1";
    argv[1] = nullptr;
    printf("PID %d running prog2\n", getpid());
    int r = execv("./prog1", argv);
    printf("PID %d exiting from prog2\n", getpid());
}

QUESTION PROC-3A. How many different PIDs will print out if you run Program 2?

One. exec doesn’t change the process’s PID.

QUESTION PROC-3B. How many lines of output will you see?

Two, as follows:

PID xxx running prog2
PID xxx running prog1

Those lines always appear in that order.

Now, let's assume that we change Program 2 to the following:

// Program 2B
#include <cstdio>
#include <unistd.h>
int main() {
    char* argv[2];
    argv[0] = (char*) "prog1";
    argv[1] = nullptr;

    printf("PID %d running prog2\n", getpid());
    pid_t p = fork();
    if (p == 0) {
        int r = execv("./prog1", argv);
    } else {
        printf("PID %d exiting from prog2\n", getpid());
    }
}

QUESTION PROC-3C. How many different PIDs will print out if you run Program 2B?

Two: the child has a different PID than the parent.

QUESTION PROC-3D. How many lines of output will you see?

Three, as follows:

PID xxx running prog2
PID yyy running prog1
PID xxx exiting from prog2

Or, because the system doesn’t define the order in which children and parents run:

PID xxx running prog2
PID xxx exiting from prog2
PID yyy running prog1

Finally, consider this version of Program 2.

// Program 2C
#include <cstdio>
#include <unistd.h>
int main() {
    char* argv[2];
    argv[0] = (char*) "prog1";
    argv[1] = nullptr;

    printf("PID %d running prog2\n", getpid());
    pid_t p = fork();
    pid_t q = fork();
    if (p == 0 || q == 0) {
        int r = execv("./prog1", argv);
    } else {
        printf("PID %d exiting from prog2\n", getpid());
    }
}

QUESTION PROC-3E. How many different PIDs will print out if you run Program 2C?

Four!

1. The initial ./prog2 prints its PID.
2. It forks once.
3. Both the initial ./prog2 and its forked child fork again, for four total processes.
4. All processes except the initial ./prog2 exec ./prog1, which print their distinct PIDs.

QUESTION PROC-3F. How many lines of output will you see?

Five.

PROC-4. Be a CS61 TF!

You are a CS61 teaching fellow. A student working on A4 is having difficulty getting pipes working. S/he comes to you for assistance. The function below is intended to traverse a linked list of commands, fork/exec the indicated processes, and hook up the pipes between commands correctly. The student has commented it reasonably, but is quite confused about how to finish writing the code. Can you help? Figure out what code to add at points A, B, and C.

#include "sh61.hh"

struct command {
    command *next; // Next in sequence of commands
    int argc;      // number of arguments
    int ispipe;    // pipe symbol follows this command
    char** argv;   // arguments, terminated by NULL
    pid_t pid;     // pid running this command
};

void do_pipes(command* c) {
    pid_t newpid;
    int havepipe = 0;   // We had a pipe on the previous command
    int lastpipe[2] = {-1, -1};
    int curpipe[2];

    do {
        if (c->ispipe) {
            int r = pipe(curpipe);
            assert(r == 0);
        }

        newpid = fork();
        assert(newpid >= 0);
        if (newpid == 0) {
            if (havepipe) {
                // There was a pipe on the last command; It's stored
                // in lastpipe; I need to hook it up to this process???
                // **** PART A ****
            }
            if (c->ispipe) {
                // The current command is a pipe -- how do I hook it up???
                // **** PART B ****
            }

            execvp(c->argv[0], c->argv);

            fprintf(stderr, "Exec failed\n");
            _exit(1);
        }

        // I bet there is some cleanup I have to do here!?
        // **** PART C ****

        // Set up for the next command
        havepipe = c->ispipe;
        if (c->ispipe) {
            lastpipe[0] = curpipe[0];
            lastpipe[1] = curpipe[1];
        }
        c->pid = newpid;
        c = c->next;
    } while (newpid != -1 && havepipe);
}

QUESTION PROC-4A. What should go in the Part A space above, if anything?

close(lastpipe[1]);
dup2(lastpipe[0], STDIN_FILENO);
close(lastpipe[0]);

QUESTION PROC-4B. What should go in the Part B space above, if anything?

close(curpipe[0]);
dup2(curpipe[1], STDOUT_FILENO);
close(curpipe[1]);

QUESTION PROC-4C. What should go in the Part C space above, if anything?

if (havepipe) {
    close(lastpipe[0]);
    close(lastpipe[1]);
}

PROC-5. Spork

Patty Posix has an idea for a new system call, spork. Her system call combines fork, file descriptor manipulations, and execvp. It’s pretty cool:

struct spork_file_action_t {
    int type; // equals SPORK_OPEN, SPORK_CLOSE, or SPORK_DUP2
    int fd;
    int old_fd;            // SPORK_DUP2 only
    const char* filename;  // SPORK_OPEN only
    int flags;             // SPORK_OPEN only
    mode_t mode;           // SPORK_OPEN only
};

pid_t spork(const char* file, const spork_file_action_t* file_actions, int n_file_actions, char* argv[]);

Here’s how spork works.

1. First, spork forks a child process.
2. The child process loops over the file_actions array (there are n_file_actions elements) and performs each file action in turn. A file action fa means different things depending on its type. Specifically:
   • fa->type == SPORK_OPEN: The child process opens the file named fa->filename with flags fa->flags and optional mode fa->mode, as if by open(fa->filename, fa->flags, fa->mode). The opened file descriptor is given number fa->fd. (Note that this requires multiple steps, since the file must be first opened and then moved to fa->fd.)
   • fa->type == SPORK_CLOSE: The child process closes file descriptor fa->fd.
   • fa->type == SPORK_DUP2: The child process makes fa->fd a duplicate of fa->old_fd.
3. Finally, the child process execs the given file with argument list argv.
4. If all these steps succeed, then spork returns the child process ID. If any of the steps fails, then either spork returns –1 and creates no child, or the child process exits with status 127. In particular, if a file action fails, then the child process exits with status 127 (and does not call exec).

This function uses spork to print the number of words in a file to standard output.

void print_word_count(const char* file) {
    spork_file_action_t file_actions[1];
    file_actions[0].type = SPORK_OPEN;
    file_actions[0].fd = STDIN_FILENO;
    file_actions[0].filename = file;
    file_actions[0].flags = O_RDONLY;
    const char* argv[2] = {"wc", nullptr};
    pid_t p = spork("wc", file_actions, 1, argv);
    assert(p >= 0);
    waitpid(p, NULL, 0);
}

QUESTION PROC-5A. Use spork to implement the following function.

// Create a pipeline like `argv1 | argv2`.
// The pipeline consists of two child processes, one running the command with argument
// list `argv1` and one running the command with argument list `argv2`. The standard
// output of `argv1` is piped to the standard input of `argv2`.
// Return the PID of the `argv2` process or -1 on failure.
pid_t make_pipeline(char* argv1[], char* argv2[]);

pid_t make_pipeline(char* argv1[], char* argv2[]) {
    int pipefd[2];
    if (pipe(pipefd) < 0) {
        return -1;
    }
    spork_file_actions_t fact[3];
    fact[0].type = SPORK_DUP2;
    fact[0].fd = STDOUT_FILENO;
    fact[0].old_fd = pipefd[1];
    fact[1].type = SPORK_CLOSE;
    fact[1].fd = pipefd[0];
    fact[2].type = SPORK_CLOSE;
    fact[2].fd = pipefd[1];
    if (spork(argv1[0], fact, 3, argv1) < 0) {
        // this is optional:
        close(pipefd[0]);
        close(pipefd[1]);
        return -1;
    }
    close(pipefd[1]);
    fact[0].fd = STDIN_FILENO;
    fact[0].old_fd = pipefd[0];
    // fact[1] is already set up
    pid_t p = spork(argv2[0], fact, 2, argv2);
    close(pipefd[0]);
    return p;
}

QUESTION PROC-5B. Now, implement spork in terms of system calls you already know. For full credit, make sure you catch all errors. Be careful of SPORK_OPEN.

pid_t spork(const char* file, const spork_file_action_t* fact, int nfact, char* argv[]) {
    pid_t p = fork();
    if (p == 0) {
        for (int i = 0; i < nfact; ++i) {
            switch (fact[i].type) {
            case SPORK_OPEN: {
                int fd = open(fact[i].filename, fact[i].flags, fact[i].mode);
                if (fd < 0) {
                    _exit(127);
                }
                if (fd != fact[i].fd) {
                    if (dup2(fd, fact[i].fd) < 0
                        || close(fd) < 0) {
                        _exit(127);
                    }
                }
                break;
            }
            case SPORK_DUP2:
                if (dup2(fact[i].old_fd, fact[i].fd) < 0) {
                    _exit(127);
                }
                break;
            case SPORK_CLOSE:
                if (close(fact[i].fd) < 0) {
                    _exit(127);
                }
                break;
            default:
                _exit(127);
            }
        }
        execvp(file, argv);
        _exit(127);
    }
    return p;
}

Errors that don't need to be handled: close, close in SPORK_OPEN case, type is unknown (we didn't say what to do in that case).

QUESTION PROC-5C. Can fork be implemented in terms of spork? Why or why not?

No, it can’t, because fork makes a copy of the process’s current memory state and file descriptor table, while spork always calls execvp (which creates a fresh process) or exits.

QUESTION PROC-5D. At least one of the file action types is redundant, meaning a spork caller could simulate its behavior using the other action types and possibly some additional system calls. Say which action types are redundant, and briefly describe how they could be simulated.

SPORK_OPEN is redundant: it can be implemented by running open in the parent (before calling spork), creating a new actions list with a SPORK_DUP2/SPORK_CLOSE pair (to dup2 the opened fd into place and then close the opened fd), calling spork with that new actions list, and then closeing the opened fds in the parent.

SPORK_CLOSE is also redundant, because you could set the close-on-exec bit in the parent. However, you cannot fully simulate an arbitrary file-actions list using just close-on-exec, because a SPORK_CLOSE can cause a later file action to deterministically fail before the exec.

PROC-6. File descriptor facts

Here are twelve file descriptor-oriented system calls.

⚠️ The accept, bind, connect, listen, and socket system calls are covered in the synchronization unit.

QUESTION PROC-6A. Which of these system calls may cause the number of open file descriptors to increase? List all that apply.

accept, dup2, open, pipe, socket

QUESTION PROC-6B. Which of these system calls may close a file descriptor? List all that apply. Note that some system calls might close a file descriptor even though the total number of open file descriptors remains the same.

close, dup2

QUESTION PROC-6C. Which of these system calls can block? List all that apply.

accept, connect, read, select, write

The following system calls can also block but only in rare situations, such as closing a file descriptor on an NFS file system, or for short times, such as opening a file on disk or binding a Unix-domain socket on a disk file system: bind, open, close.

I don’t believe the others—dup2, listen, pipe, socket—ever meaningfully block.

QUESTION PROC-6D. Which system calls can open at least one file descriptor where that file descriptor is suitable for both reading and writing? List all that apply.

open (O_RDWR), accept, socket.

connect is also OK to mention, even though it doesn’t create a new file descriptor.

dup2 is also OK to mention, even though it doesn’t really “open” a file descriptor (though it does unambiguously cause the number of open file descriptors to increase).

QUESTION PROC-6E. ⚠️ Which system calls must a network server make in order to receive a connection on a well-known port? List all that apply in order, first to last. Avoid unnecessary calls.

socket, bind, listen, accept

QUESTION PROC-6F. ⚠️ Which system calls must a network client make in order to (1) connect to a server, (2) send a message, (3) receive a reply, and (4) close the connection? List all that apply in order, first to last. Avoid unnecessary calls.

socket, connect, write, read, close

PROC-7. Duplication

Mark Zuckerberg hates duplicates (because Winklevii). He especially hates the dup2 system call, because he can’t remember its order of arguments.

⚠️ Some parts of this question rely on material from the synchronization unit.

QUESTION PROC-7A. What is the order of arguments for dup2? Is it (A) dup2(oldfd, newfd) or (B) dup2(newfd, oldfd)? (Here, oldfd is the pre-existing file descriptor.)

oldfd, newfd

Mark wants to make dup2 obsolete by changing other system calls. He wants to:

• Replace open(const char* path, int oflag, [mode_t mode]) with openonto(int fd, const char* path, int oflag, [mode_t mode]). This system call behaves like open, but rather than choosing a previously-unused file descriptor, it uses file descriptor number fd.

• Replace pipe(int fd[2]) with pipeonto(readfd, writefd), which uses the specified file descriptor numbers for the pipe’s read and write ends.

• Add nextfd(), which returns the lowest-numbered currently-unused file descriptor.

These system calls can fail, meaning they return –1 and set errno to an error code.

• If openonto or pipeonto fails, the process’s file descriptor table is unchanged.
• If openonto succeeds, it returns its fd argument.
• If an fd argument is out of range (e.g., less than 0), openonto and pipeonto return –1 and set errno to EBADF.
• nextfd can return –1 and set errno to EMFILE if too many file descriptors are open.

QUESTION PROC-7B. Assuming a single-threaded process, show how to implement open’s functionality in terms of these new system calls. Don’t worry about mode. Unix’s open cannot set errno to EBADF; neither should yours.

int open(const char* path, int oflag) {

    int fd = nextfd();
    if (fd != -1) {
        fd = openonto(fd, path, oflag);
    }
    return fd;

QUESTION PROC-7C. ⚠️ Your open implementation likely has a race condition if used in a multithreaded process: a bug can occur if two threads call open at about the same time. Explain this race condition briefly (two or three sentences max).

Two threads can use the same nextfd.

QUESTION PROC-7D. ⚠️ Solve this race condition using synchronization objects. You may introduce global variables, which we’ll assume are initialized. Again, be careful to handle error conditions properly.

int open(const char* path, int oflag) {

    extern std::mutex m;
    std::scoped_lock guard(m);
    ...previous code...
    return fd;

QUESTION PROC-7E. Can these system calls (without dup or dup2) implement a shell pipeline? Why or why not? Be brief.

Nope: there is no way to shift the pipe ends onto stdin/stdout without replacing the shell's stdin/stdout.

Sheryl Sandberg is sympathetic to Mark’s psychological issues, but suggests instead that he replace dup and dup2 with:

• fdswap(int fd1, int fd2). This swaps the meanings of two file descriptors. Thus, after fdswap(fd1, fd2), fd1 references the file structure previously referenced by fd2, and vice versa.

QUESTION PROC-7F. Complete the following function, using at least pipe, fork, execvp, fdswap, and close. Do not use dup, dup2, or pipeonto, and don’t worry too much about error conditions. Make sure to implement pipe hygiene. Hint: The programs in cs61-lectures/synch2 might be useful references.

// simplepipeline_fdswap(cmd1, cmd2)
//    Fork and execute the pipeline `cmd1 | cmd2`. Return the `pid_t` corresponding to `cmd2`
//    (or return -1 with an appropriate error code if the pipeline could not be created).

pid_t simplepipeline_fdswap(const char* cmd1, const char* cmd2) {
    char* const cmd1_argv[] = { (char*) cmd1, nullptr };
    char* const cmd2_argv[] = { (char*) cmd2, nullptr };

    int pfd[2];
    if (pipe(pfd) != 0) {
        return -1;
    }
    int pid1 = fork();
    if (pid1 == 0) {
        fdswap(pfd[1], 1);
        close(pfd[1]);
        close(pfd[0]);
        execvp(cmd1, cmd1_argv);
    }
    int pid2 = fork();
    if (pid2 == 0) {
        fswap(pfd[0], 0);
        close(pfd[0]);
        close(pfd[1]);
        execvp(cmd2, cmd2_argv);
    }
    close(pfd[0]);
    close(pfd[1]);
    return pid2;

PROC-8. Tripe

This question concerns a new pipe-like feature called a tripe. A tripe has one write end and two read ends. Any data written on the write end is duplicated onto both read ends, which observe the same stream of characters.

QUESTION PROC-8A. A tripe can be emulated using regular pipes and a helper process, where the helper process runs the following code:

     void run_tripe(int fd1, int fd2, int fd3) {
/*1*/    while (true) {
/*2*/        char ch;
/*3*/        ssize_t n = read(fd1, &ch, 1);
/*4*/        assert(n == 1);
/*5*/        n = write(fd2, &ch, 1);
/*6*/        assert(n == 1);
/*7*/        n = write(fd3, &ch, 1);
/*8*/        assert(n == 1);
/*9*/    }
     }

How many regular pipes are required?

3

QUESTION PROC-8B. If run_tripe encounters end of file on fd1, it will assert-fail. It should instead close the write ends and exit. Change the code so it does this, using line numbers to indicate where your code goes. You may assume that read and write never return an error.

Change line 4 to say:

        if (n == 0) {
            _exit(0);
        }

QUESTION PROC-8C. “Pipe hygiene” refers to closing unneeded file descriptors. What could go wrong with the tripe if its helper process had bad pipe hygiene (open file descriptors referring to unneeded pipe ends)? List all that apply.

1. Undefined behavior.
2. read from either tripe read end would never return end-of-file.
3. write to the write end would never block.
4. write to the write end would never detect the closing of a read end.
5. None of the above.

2, 4

PROC-9. Process control explanations

QUESTION PROC-9A. Match each system call with the shell feature that requires that system call for implementation. You will use each system call once.

1d, 2e, 3b, 4c, 5a

QUESTION PROC-9B. What is the best explanation for the function of zombie processes in Unix? Select one.

1. Zombie processes automatically kill other processes that leak too much memory, keeping the OS as a whole more stable.
2. Zombie processes ensure that process IDs are never reused.
3. Zombie processes allow parent processes to reliably collect the statuses of their exited children.
4. Zombie processes cannot be killed by most signals.
5. None of the above.

3 is the best answer. 1 and 2 are false; 4 is true—zombie processes are already dead, so they cannot be killed—but the function of zombie processes is not to not be killable.

QUESTION PROC-9C. What is the best explanation for why signal handlers in Unix should be kept simple? Select one.

1. Signal handlers run with interrupts disabled, so a complex signal handler could violate process isolation.
2. Signal handlers can be invoked at any moment, including in the middle of another function, so a complex signal handler is likely to violate assumptions on which the program and/or C++ libraries depend.
3. The kernel ensures that signal handlers have no effect on primary memory.
4. Signal handlers are called by the kernel rather than being called explicitly by other parts of the program.
5. None of the above.

2 is the best answer. 1 and 3 are false. 4 is true, but does not address why signals need to be simple.

QUESTION PROC-9D. What is the best explanation for why the pipe system call precedes the fork system call when setting up a pipeline in a shell? Select one.

1. The fork system call creates a new, isolated process whose file descriptor table cannot be influenced by other processes.
2. In the shell grammar, a command is part of a pipeline, so the pipe is encountered first, before the command is forked.
3. Pipe hygiene requires that all extraneous ends of a pipe are closed in all processes, including the parent shell.
4. The read end of a pipe is created after the write end.
5. None of the above.

1 is the best answer. 3 has nothing to do with the ordering of pipes and forks. 2 and 4 are false.

QUESTION PROC-9E. What is the best explanation for the difference between foreground conditional chains and background conditional chains? Select one.

1. In a foreground conditional chain, the shell waits for each pipeline in sequence, while in a background conditional chain, all commands in the pipelines run in parallel.
2. Foreground conditional chains never create zombie processes.
3. Background conditional chains do not write to the shell’s standard output.
4. Foreground conditional chains do not create subshells.
5. None of the above.

4 is the best answer. However, some students will have implemented subshells even for foreground conditional chains, so they might put 5; we’ll have to decide what to do there.

There is something to be said for 2 if the parent shell waits for both sides of each pipeline, because zombies will be reaped immediately. However, strictly speaking, every process becomes a zombie process for at least a split second after it exits.

1 is not true: if the conditional chain has more than one condition, then later pipelines in that chain will run after the earlier pipelines complete. 3 is not true; we saw how background commands can write to the shell’s standard output.

PROC-10. Piping hot

QUESTION PROC-10A. Fill in the blanks so that each command line prints 61, and only 61, to the shell’s standard output. The resulting command lines should be valid according to the shell grammar. At most one of your fragments may contain the string 61.

1. _________
2. _________ || echo 61
3. false && _________ && echo 61
4. echo ab | tr _________
5. yes _________ echo 61

1. echo 61
2. false
3. true || true
4. ba 16
5. | echo > /dev/null ;

In the remaining questions, we provide strace output for attempts at a shell running a two-process pipeline, echo foo | wc -c. For each question, you are to characterize the shell. This is Shell X1.

58797 pipe([3, 4])                      = 0
58797 clone(child_stack=NULL, flags=CLONE_CHILD_CLEARTID|CLONE_CHILD_SETTID|SIGCHLD, child_tidptr=0x7f892b19ea10) = 58798
58797 close(4)                          = 0
58798 close(3)                          = 0
58797 clone(child_stack=NULL, flags=CLONE_CHILD_CLEARTID|CLONE_CHILD_SETTID|SIGCHLD, child_tidptr=0x7f892b19ea10) = 58799
58798 dup2(4, 1)                        = 1
58798 close(4)                          = 0
58798 execve("/bin/echo", ["/bin/echo", "foo"], 0x7ffdea26fe98 /* 57 vars */ <... detached ...>
58797 close(3)                          = 0
58797 wait4(58798,  <... unfinished ...>
58799 dup2(3, 0)                        = 0
58799 close(3)                          = 0
58799 execve("/usr/bin/wc", ["/usr/bin/wc", "-c"], 0x7ffdea26fe98 /* 57 vars */ <... detached ...>
58797 <... wait4 resumed> NULL, 0, NULL) = 58798
58797 --- SIGCHLD {si_signo=SIGCHLD, si_code=CLD_EXITED, si_pid=58798, si_uid=1000, si_status=0, si_utime=0, si_stime=0} ---
58797 wait4(58799, NULL, 0, NULL)       = 58799
58797 --- SIGCHLD {si_signo=SIGCHLD, si_code=CLD_EXITED, si_pid=58799, si_uid=1000, si_status=0, si_utime=0, si_stime=0} ---
58797 exit_group(0)                     = ?
58797 +++ exited with 0 +++

QUESTION PROC-10B. What is the process ID of the parent shell?

58797.

QUESTION PROC-10C. Does Shell X1 wait for the right-hand process, the left-hand process, or both?

Both (there are wait4 lines for each child).

QUESTION PROC-10D. Does Shell X1 appear to implement two-process pipelines correctly?

Yes it does. The shell creates the pipe before cloning; the left-hand child dup2s the write end of the pipe, file descriptor 4, onto standard input, fd 1, then closes both ends of the pipe. Meanwhile, the parent shell closes the write end of the pipe and clones again; the right-hand child dup2s the read end of the pipe onto stdin, then closes the read end. Finally, the parent closes the read end before waiting for both children. The children run in parallel and there are no extraneous pipe ends open in any process (the history of each process does both close(3) and close(4)).

QUESTION PROC-10E. This is Shell X2. It is incorrect.

58969 pipe([3, 4])                      = 0
58969 clone(child_stack=NULL, flags=CLONE_CHILD_CLEARTID|CLONE_CHILD_SETTID|SIGCHLD, child_tidptr=0x7f6d566b0a10) = 58970
58970 close(3)                          = 0
58970 dup2(4, 1)                        = 1
58970 close(4)                          = 0
58970 execve("/bin/echo", ["/bin/echo", "foo"], 0x7ffcc0a30220 /* 57 vars */ <... detached ...>
58969 clone(child_stack=NULL, flags=CLONE_CHILD_CLEARTID|CLONE_CHILD_SETTID|SIGCHLD, child_tidptr=0x7f6d566b0a10) = 58971
58971 dup2(3, 0)                        = 0
58971 close(3)                          = 0
58971 execve("/usr/bin/wc", ["/usr/bin/wc", "-c"], 0x7ffcc0a30220 /* 57 vars */ <... detached ...>
58969 close(3)                          = 0
58969 close(4)                          = 0
58969 wait4(58970,  <... unfinished ...>
58969 <... wait4 resumed> NULL, 0, NULL) = 58970
58969 --- SIGCHLD {si_signo=SIGCHLD, si_code=CLD_EXITED, si_pid=58970, si_uid=1000, si_status=0, si_utime=0, si_stime=0} ---
58969 wait4(58971, NULL, 0, NULL)       = ? ERESTARTSYS (To be restarted if SA_RESTART is set)
58969 --- SIGINT {si_signo=SIGINT, si_code=SI_KERNEL} ---
58969 +++ killed by SIGINT +++

Give an strace line, including process ID, that, if added in the right place, would make Shell X2 appear to implement two-process pipelines correctly.

58971 close(4) = 0, which would go somewhere after 58971 is created and before execve (but they don’t need to name the place).

QUESTION PROC-10F. This is Shell X3. It is incorrect, though it appears to run correctly on this specific pipeline.

59026 pipe([3, 4])                      = 0
59026 clone(child_stack=NULL, flags=CLONE_CHILD_CLEARTID|CLONE_CHILD_SETTID|SIGCHLD, child_tidptr=0x7f0dfa757a10) = 59027
59026 close(4)                          = 0
59027 close(3)                          = 0
59026 wait4(59027,  <... unfinished ...>
59027 dup2(4, 1)                        = 1
59027 close(4)                          = 0
59027 execve("/bin/echo", ["/bin/echo", "foo"], 0x7ffe0672b1b0 /* 57 vars */ <... detached ...>
59026 <... wait4 resumed> NULL, 0, NULL) = 59027
59026 --- SIGCHLD {si_signo=SIGCHLD, si_code=CLD_EXITED, si_pid=59027, si_uid=1000, si_status=0, si_utime=0, si_stime=0} ---
59026 clone(child_stack=NULL, flags=CLONE_CHILD_CLEARTID|CLONE_CHILD_SETTID|SIGCHLD, child_tidptr=0x7f0dfa757a10) = 59028
59026 close(3)                          = 0
59026 wait4(59028,  <... unfinished ...>
59028 dup2(3, 0)                        = 0
59028 close(3)                          = 0
59028 execve("/usr/bin/wc", ["/usr/bin/wc", "-c"], 0x7ffe0672b1b0 /* 57 vars */ <... detached ...>
59026 <... wait4 resumed> NULL, 0, NULL) = 59028
59026 --- SIGCHLD {si_signo=SIGCHLD, si_code=CLD_EXITED, si_pid=59028, si_uid=1000, si_status=0, si_utime=0, si_stime=0} ---
59026 exit_group(0)                     = ?
59026 +++ exited with 0 +++

Give a two-process Unix pipeline that Shell X3 will not appear to run correctly, and/or briefly describe problem with Shell X3.

The parent shell waits for the left-hand command to finish before starting the right-hand command, so yes | echo foo would fail.

PROC-11. System calls

QUESTION PROC-11A. Which of the following system calls are likely to block? List all that apply.

1. read on a pipe whose pipe buffer is empty
2. write to a pipe whose pipe buffer is full
3. fork when there are no available process slots
4. waitpid when the current process has no children yet
5. None of the above

1 and 2 only. fork and waitpid return errors in the given situations.

QUESTION PROC-11B. Which of the following statements about execv are true? List all that apply.

1. It must always be paired with fork
2. It does not return unless it encounters an error
3. It replaces the current process’s program image with a new program
4. It closes the current process’s file descriptors except for standard input, standard output, and standard error
5. None of the above

2 and 3. 1 is false; execv is almost always combined with fork, but it can be used independently. 4 is false; it leaves all file descriptors open (except those explicitly marked as “close-on-exec”).

QUESTION PROC-11C. When will the return value of a read system call equal 0? List all that apply.

1. When the byte of data at the file position equals '\0'
2. When the file descriptor is the read end of a pipe and either the pipe buffer is empty or the write end of the pipe is closed
3. When the file descriptor is the read end of a pipe and both the pipe buffer is empty and the write end of the pipe is closed
4. When the file descriptor is the write end of a pipe
5. When the read system call is interrupted by a signal before it can read any data
6. None of the above

Only 3.

1. False: read returns the number of bytes read, not the value of a byte.
2. False: When the pipe buffer is empty but the write end of a pipe is closed, read will block.
3. True!
4. False: read will return an error in this situation (EINVAL).
5. False: read will return an error in this situation (EINTR).

QUESTION PROC-11D. In class, we explored some racer programs whose goal is to wait until a child process exits or a timeout has elapsed, whichever comes first. The racer-block program attempted to use system call interruption to solve this problem. Here’s a version of its code, compressed to omit error checking.

    static volatile sig_atomic_t got_signal = 0;
    void signal_handler(...) {
S1:     got_signal = 1;
    }

    int main() {
M1:     set_signal_handler(SIGCHLD, signal_handler);
M2:     pid_t p = fork();
M3:     if (p == 0) { /* child process code */ }
M4:     usleep(timeout_in_microseconds);
M5:     if (elapsed_time_in_microseconds() >= timeout_in_microseconds) {
M6:         printf("SLOW\n");   /* child did not exit before timeout */
M7:     } else {
M8:         printf("quick\n");  /* child exited before timeout */
M9:     }
    }

Which line contains a system call that the programmer expects to be interrupted whenever a child process exits before the timeout?

Line M4.

QUESTION PROC-11E. In what circumstance might this program print an incorrect result (SLOW when the child exited meaningfully before the timeout, or quick when the child exited meaningfully after the timeout)? Explain briefly, referring to specific lines of code.

If the child exits right away, the parent process might receive a signal between line M2 and line M4. That will mean the usleep system call begins after the child has exited. usleep will not be interrupted by the signal, and will run to completion; the program will print SLOW instead of quick.

QUESTION PROC-11F. The racer race condition can be solved by designing new system calls. In this question, we add system calls that can temporarily mask signals, causing them to be delayed until a future time. The system calls are:

• sigmask(int signum)
  Mask signal signum. If the process is sent this signal, the kernel records the signal as pending, but does not interrupt the process.

• sigunmask(int signum)
  Unmask signal signum and immediately deliver any pending signum to the process.

• pusleep(useconds_t timeout)
  Unmask all signals and sleep for timeout. This happens effectively atomically, with no sleep–wakeup race conditions, so if the process has a pending signal when pusleep is called, the kernel immediately delivers the pending signal and pusleep returns with error code EINTR. The original set of masked signals is restored before return.

Use these system calls to fix the race condition in the racer-block code above, or explain briefly how this could be done.

Replace line M4 with:

sigmask(SIGCHLD);
if (!got_signal) {
    pusleep(timeout_in_microseconds);
}
sigunmask(SIGCHLD);

This would work too:

sigmask(SIGCHLD);
pid_t wp = waitpid(p, &status, WNOHANG);
if (wp < 0) {
    pusleep(timeout_in_microseconds);
}
sigunmask(SIGCHLD);

The important part is to mask the SIGCHLD signal before calling pusleep.

PROC-12. Testing a shell

Your shell problem set was tested with a check.pl script that runs shell commands and examines their output (including both standard output and standard error). In this problem you will reason about some of check.pl’s checks.

QUESTION PROC-12A. Check REDIR17 and its expected output are:

• Command: echo > out.txt the redirection can occur anywhere && cat out.txt
• Expected output: the redirection can occur anywhere

Match the actual outputs, on the left, with the implementations that might generate those outputs, on the right. You might use an implementation more than once or not at all.

(When a feature is “not understood”, that means the implementation treats the feature’s tokens like normal command arguments; for instance, an implementation that does not understand conditionals would treat && tokens like normal arguments. The actual outputs translate newlines into spaces, which is what check.pl does.)

• 1. (empty)
  2. > out.txt the redirection can occur anywhere && cat out.txt
  3. > out.txt the redirection can occur anywhere cat: out.txt: No such file or directory
  4. > out.txt the redirection can occur anywhere
• 1. Conditionals incorrectly implemented, redirection correctly implemented
  2. Conditionals incorrectly implemented, redirection not understood
  3. Conditionals correctly implemented, redirection not understood
  4. Conditionals and redirection not understood
  5. None of the above

1—A, 2—D, 3—C, 4—B

If redirection is not understood, then the echo command will print > out.txt the redirection can occur anywhere (including the redirection character). So implementations B–D cannot match output 1.

1. If there’s no output, then redirection seems to have worked well enough: the file out.txt contains the redirection can occur anywhere. However, the command that reads out.txt, cat out.txt, might not execute if conditionals are incorrectly implemented. The best match is A.
2. Since the && token was printed like it was normal, the best match is D (conditionals not understood).
3. This output contains text printed by cat, so the shell tried to run cat. The best match is C.
4. This output has no evidence of any attempt to run cat. The best match is B.

QUESTION PROC-12B. Check PIPE5 runs the command yes | head -n 5.

Match the actual outputs, on the left, with the implementations that might generate those outputs, on the right. You might use an implementation more than once or not at all.

• 1. y y y y y y y y … forever
  2. yes: invalid option -- 'n' Try 'yes --help' for more information.
  3. y y y y y, but shell hangs rather than printing another prompt
  4. y y y y y
• 1. Pipes correctly implemented
  2. Pipes not understood
  3. Pipes incorrectly hooked up to command standard input and/or output
  4. Pipes correctly hooked up to command standard input and/or output, but parent shell process does not implement pipe hygiene
  5. None of the above

1—C, 2—B, 3—D, 4—A

The expected output is y y y y y: the first five lines of yes’s output, which is an infinite stream of ys. So output 4 matches with implementation A.

If pipes aren’t understood, then yes will be given arguments like yes "|" "head" "-n" "5", and it might complain that it doesn’t understand option -n. So output 2 matches with implementation B.

If pipes are correctly hooked up but pipe hygiene is missing, then the yes command might block forever rather than exiting with head. The reason yes exits prematurely is that it tries to write to a closed pipe buffer (where no readers exist). That attempt causes a SIGPIPE signal, which by default causes yes to exit. So output 3 matches with implementation D.

Finally, output 1 looks like yes is writing its standard output directly to the terminal, rather than to head. This matches with implementation C.

QUESTION PROC-12C. Check BG7 runs this command:

sh -c "sleep 0.1; /bin/echo Second" & sh -c "sleep 0.05; /bin/echo First" & sleep 0.15

List the true statements about this command.

1. sleep 0.15 is executed in the foreground.
2. sleep 0.15 ensures that there is enough time for both First and Second to be printed before the parent sh61 returns to the prompt.
3. sh -c "sleep 0.1; /bin/echo Second" will finish executing before sh -c "sleep 0.05; /bin/echo First" begins executing.
4. sleep 0.05 will finish executing before /bin/echo First begins executing.
5. This command line should take a minimum of 0.30 seconds to execute.
6. None of the above.

1, 2, 4

1. True: The last command in this command line has no & background marker.
2. True: Assuming the background processes start instantaneously, the first one will run for about 0.1 seconds and the second one (in parallel) for about 0.05 seconds.
3. False: The two background commands will start almost simultaneously.
4. True: The ; sequence connector ensures sleep 0.05 finishes before /bin/echo First begins.
5. False: The command line will take about 0.15 seconds.

QUESTION PROC-12D. How many different sh61 descendant processes (children, children of children, etc.) will be created as sh61 executes check BG7? Explain briefly.

A good answer is nine.

• A background sh61 to manage sh -c "sleep 0.1; /bin/echo Second"
  • A child process to run sh -c "sleep 0.1; /bin/echo Second"
    • A grandchild process to run sleep 0.1
    • A grandchild process to run /bin/echo Second
• A background sh61 to manage sh -c "sleep 0.05; /bin/echo First"
  • A child process to run sh -c "sleep 0.05; /bin/echo First"
    • A grandchild process to run sleep 0.05
    • A grandchild process to run /bin/echo First
• A child process to run sleep 0.15

Other answers are possible. For instance, an optimized shell might not fork a background sh61 for the two background conditionals, because those background conditionals are simple (they execute a single command). That would leave 7 descendant processes.

QUESTION PROC-12E. This code is from a student’s incorrect implementation of background commands.

// This function runs the conditional `sec` to completion.
void run_conditional(shell_parser sec);

void run_list(shell_parser sec) {
    shell_parser condsec = sec.first_conditional();
    while (condsec) {
        if (condsec.op() != TYPE_BACKGROUND) {
            run_conditional(condsec);
        } else {
            pid_t p = fork();
            assert(p >= 0);
            if (p == 0) {
                run_conditional(condsec);
                _exit(0);
            }
            [[maybe_unused]] int status;
            waitpid(p, &status, 0);
        }
        condsec.next_conditional();
    }
}

What would the associated shell print on check BG7, and how long will it take to print it? Explain briefly.

It will print Second First after 0.3 seconds, because it waits for background conditionals as well as foreground conditionals.

QUESTION PROC-12F. How could the code in the previous question be fixed?

Remove the waitpid. The comment indicates that run_conditional already contains the relevant waitpid (since it “runs the conditional sec to completion”).

QUESTION PROC-12G. This similar code has a different problem.

// This function runs the conditional `sec` to completion.
void run_conditional(shell_parser sec);

void run_list(shell_parser sec) {
    shell_parser condsec = sec.first_conditional();
    while (condsec) {
        if (condsec.op() != TYPE_BACKGROUND) {
            run_conditional(condsec);
        } else {
            pid_t p = fork();
            assert(p >= 0);
            if (p == 0) {
                run_conditional(condsec);
            }
        }
        condsec.next_conditional();
    }
}

What will this code print on check BG7? Explain briefly.

Something like First First Second or First Second First, followed by lots of shell prompts. Since the subshells created for background commands don’t exit, they proceed through the rest of the command line and then return to (a forked copy of) the main shell loop.