File Descriptors

Here’s a brief introduction to file descriptors for CS 61.

For another presentation of this material, see CS:APP3e chapter 10, particularly through section 10.5. Section 10.4.2 may be particularly interesting for Assignment 3!

A file descriptor is the Unix abstraction for an open input/output stream: a file, a network connection, a pipe (a communication channel between processes), a terminal, etc.

A Unix file descriptor thus fills a similar niche as a stdio FILE*. However, whereas a FILE* (like stdin or stdout) is a pointer to some object structure, a file descriptor is just an integer. For example, 0, 1, and 2 are the file descriptor versions of stdin, stdout, and stderr, respectively.

(Integers are used because they’re easier for the operating system kernel to verify than arbitrary pointers. Although the kernel has objects somewhat similar to FILE*s, it doesn’t give applications direct access to those objects. Instead, an array called the file descriptor table stores an array of such objects. The file descriptors that applications manipulate are indexes into this table. It’s very easy to check that an integer is in bounds. We’ll hear more about this in the second half of the course.)

Logically, a file descriptor comprises a file object, which represents the underlying data (such as /home/kohler/grades.txt), and a position, which is an offset into the file. There can be many file descriptors simultaneously open for the same file object, each with a different position. For disk files, the position can be explicitly changed: a process can rewind and re-read part of a file, for example, or skip around, as we saw with strided I/O patterns. These files are called seekable. However, not all types of file descriptor are seekable. Most communication channels between processes aren’t, and neither are network channels.

File descriptor system calls

You will use the following system calls in Assignment 3. You may read about them in detail using man: for instance, man 2 open, man 2 read, man 2 lseek. The “2” means “tell me about the system call.” Or you can check the book.

ssize_t read(int fd, char *buf, size_t sz)

Read at most sz bytes from file descriptor fd into buffer buf. Returns the number of bytes read, if any. Returns 0 at end of file and -1 on error.

Normally, read returns sz, but it can return less. For instance, there might be just sz - 2 bytes left in the file, or there might only be sz - 10 bytes available to read at the moment. A read that returns less than the requested number of bytes is called a short read.

If sz > 0, then the return value 0 is a reliable end-of-file indicator. For instance, when reading a pipe, 0 means the other end of the pipe has closed. Other short reads are not reliable end-of-file indicators. For instance, when reading from the terminal, a read of 1024 bytes might return 1 byte because the user has only typed 1 byte so far; the user might still type more bytes in the future.

The return value -1 indicates an error and means that no bytes were read. If any bytes were read, the return value will be greater than 0. However, not all errors are equally serious; the EINTR error, for example, is generally masked by I/O libraries. The read(2) manual page lists all errors that can occur. (That notation means “the page for read in section 2 of the manual”; run [wo]man 2 read.)

ssize_t write(int fd, const char *buf, size_t sz)

Write at most sz bytes to file descriptor fd from buffer buf. Returns the number of bytes written, if any. Returns -1 on error.

Normally, write returns sz, but as with read, it might return less: a short write. For instance, there might only be room for sz - 2 bytes on the disk, or the process reading from a pipe might be behind, or a signal might interrupt the write partway through.

The return value -1 indicates an error and means that no bytes were written. If any bytes were written, the return value will be greater than 0.

off_t lseek(int fd, off_t pos, int whence)

Change file descriptor fd’s position. Normally whence == SEEK_SET. Then the file’s position is set to pos; so pos == 0 sets the position to the beginning of the file, pos == 1 sets it one byte in, and so forth. You may also set whence == SEEK_CUR, which changes the position relative to the current position, or whence == SEEK_END, which sets the position relative to the file’s size (i.e., lseek(fd, -1, SEEK_END) sets the position to the file’s last byte). Returns the new position, measured in bytes past the beginning of the file. Returns -1 on error, which can happen, for example, if the file is not seekable or the new file position is out of range for the file.

int close(int fd)

Close the file descriptor.

Understanding errors

The Unix error convention is that system calls return -1 on error. A global variable, int errno, is then set so the program can tell what kind of error occurred. The <errno.h> header file defines symbolic names for specific error conditions. Each name starts with E. For example, the system calls above “return EBADF if fd is not an open file descriptor.” This actually means that the system call returns the value -1 (cast to the appropriate type), and the global errno variable is set to the constant EBADF.

The const char* strerror(int errnum) library function returns a textual string describing an error constant. For instance, strerror(EINVAL) returns "Invalid argument". This might be useful for debugging.

A system call’s manual page will list the errors it might return.

Additional system calls

The following system calls might also be useful for problem set 3, depending on your implementation strategy. Read their manual pages, consult CS:APP3e or our handout code, or contact Piazza for more.

void* mmap(void* addr, size_t len, int prot, int flags, int fd, off_t offset) Memory-map a portion of a file, returning the mapped address. Returns MAP_FAILED == (void*) -1 on error. Doesn’t work for all file types.

int munmap(void* addr, size_t len) Unmap a previously-mapped memory region.

int madvise(void* addr, size_t len, int advice) Provide prefetching advice for a portion of a memory-mapped region.

int posix_fadvise(int fd, off_t pos, off_t len, int advice) Provide prefetching advice for a portion of a file descriptor.