Part 6 of 7 · 3 min read
Ways of waiting
Blocking, non-blocking and asynchronous I/O
Think of three ways to order food. In one you wait at the counter until the food is ready. In another you ask "is it ready?" and go away if not. In the third you give your phone number and the shop calls you. The analogy stops in one place: a shop's call always comes, while a program must prepare a way to receive the answer.
In plain words: In blocking I/O, the call does not return until the operation is done, and the kernel puts the process to sleep meanwhile. This is the default, and it is the simplest to write. In non-blocking I/O the call returns at once. If the data is not ready, it returns -1 with the error EAGAIN, and the program must try again later or ask poll or epoll which descriptors are ready. In asynchronous I/O the program submits the request and gets a notification when it is done, and it can do other work. Linux offers io_uring for this.
A worked example. This program makes a pipe, sets the read end to non-blocking, and reads from it while it is empty and again after a write.
#include <errno.h>
#include <fcntl.h>
#include <stdio.h>
#include <string.h>
#include <unistd.h>
int main(void) {
int p[2];
if (pipe(p) < 0) return 1;
fcntl(p[0], F_SETFL, O_NONBLOCK); /* reads on p[0] no longer block */
char buf[16];
ssize_t n = read(p[0], buf, sizeof buf);
printf("empty pipe: read returned %zd, errno is EAGAIN: %s\n", n, errno == EAGAIN ? "yes" : "no");
if (write(p[1], "hello", 5) != 5) return 1;
n = read(p[0], buf, sizeof buf);
printf("after a write: read returned %zd, data \"%.*s\"\n", n, (int)n, buf);
return 0;
}
Output on the test machine:
empty pipe: read returned -1, errno is EAGAIN: yes
after a write: read returned 5, data "hello"
With blocking mode, the first read would have put the program to sleep and, as nobody writes, it would never have woken.
The key lines. A server that handles thousands of connections cannot give each one a sleeping thread. It sets its descriptors to non-blocking and waits for readiness with epoll, which names the ready ones.
Blocking and non-blocking I/O
Blocking I/O is an I/O call that does not return until the operation is complete or has failed. Non-blocking I/O is an I/O call that returns at once, and reports with an error such as EAGAIN that the operation could not be done without waiting.
Example: a read on an empty pipe sleeps in blocking mode and returns -1 with EAGAIN in non-blocking mode.
Check yourself:
1. What does a blocking read on an empty pipe do?
2. What does non-blocking read return when the pipe is empty?
3. Why would a web server use epoll and non-blocking sockets?
Words you will meet
| Word | Meaning |
|---|---|
| VFS | Kernel layer that gives all file systems one interface. |
| Page cache | Memory that holds file data. |
| Major number | Identifies the driver of a device file. |
| Minor number | Identifies the unit of that driver. |
| EAGAIN | Error that means "try again later". |
Quick notes
- A read passes the VFS, the file system, the page cache, the block layer, the driver and the controller.
- Block devices have numbered blocks. Character devices are byte streams. Both are files in
/dev. - A driver hides one device behind the same calls as all the others.
- A buffer matches speeds and sizes. A cache keeps copies for reuse. The page cache does both.
- Written data may stay in the cache for a while.
fsyncforces it to the device. - Blocking calls sleep. Non-blocking calls return
EAGAIN. Asynchronous calls report later.