Part 5 of 7 · 4 min read
Buffering and caching
Think of a kitchen and a pantry. The pantry holds things that you use often, so you do not walk to the shop for each one. The kitchen counter is a smaller place where you set out the things for one meal. The analogy stops in one place: the shop never changes your pantry, but a disk write can fail after the program thinks it is done.
In plain words: A buffer is a memory area that holds data while it moves between two places that work at different speeds or in different sizes. A program that writes one byte at a time still produces whole blocks on disk, because the kernel gathers the bytes in a buffer. A cache is a memory area that keeps a copy of data that was used, so a second use does not go to the device. Linux keeps both in the page cache, which uses almost all free memory. Reads that hit the cache are about as fast as memory. Writes go to the cache first and reach the disk later. This is quick, and it means that a crash can lose the last writes unless the program asks for fsync.
A worked example. The program below reads a file three times. It first removes the file from the page cache with posix_fadvise. The first read must go to the device. The second read finds the data in the page cache. The third uses O_DIRECT to bypass the cache.
#define _GNU_SOURCE
#include <fcntl.h>
#include <stdio.h>
#include <stdlib.h>
#include <time.h>
#include <unistd.h>
static double now(void) {
struct timespec t; clock_gettime(CLOCK_MONOTONIC, &t);
return t.tv_sec + t.tv_nsec / 1e9;
}
static double read_all(const char *path, int flags, long long *bytes) {
int fd = open(path, O_RDONLY | flags);
if (fd < 0) { perror("open"); exit(1); }
char *buf; if (posix_memalign((void **)&buf, 4096, 1 << 20)) exit(1);
double t0 = now(); long long n = 0; ssize_t r;
while ((r = read(fd, buf, 1 << 20)) > 0) n += r;
double t = now() - t0;
close(fd); free(buf); *bytes = n;
return t;
}
int main(int argc, char **argv) {
if (argc < 2) return 1;
long long b;
int fd = open(argv[1], O_RDONLY);
posix_fadvise(fd, 0, 0, POSIX_FADV_DONTNEED); /* drop this file from the page cache */
close(fd);
double t1 = read_all(argv[1], 0, &b);
printf("buffered, cold cache: %.3f s, %.0f MB/s\n", t1, b / t1 / 1e6);
double t2 = read_all(argv[1], 0, &b);
printf("buffered, warm cache: %.3f s, %.0f MB/s\n", t2, b / t2 / 1e6);
double t3 = read_all(argv[1], O_DIRECT, &b);
printf("O_DIRECT (no cache): %.3f s, %.0f MB/s\n", t3, b / t3 / 1e6);
return 0;
}
On the test machine, with a 1 GiB file of random bytes:
buffered, cold cache: 0.374 s, 2870 MB/s
buffered, warm cache: 0.146 s, 7343 MB/s
O_DIRECT (no cache): 0.460 s, 2333 MB/s
The warm read is 0.374 / 0.146 = 2.6 times quicker than the cold one. The gap is small because the virtual disk is itself fast and cached by the host. On a spinning disk the warm read would be hundreds of times quicker. The very first read of a new file on this machine took 43 seconds, since the host had to fetch it, which shows how far a cache can hide a slow device.
The key lines. A cache that holds writes is a write-back cache. It is fast and has the risk of losing data. A write-through cache writes to the device at once. The call fsync(fd) forces the data of one file to the device, and page 8 measures its cost.
Page cache
The page cache is the part of main memory in which the kernel keeps the contents of files and block devices, in page-sized units. Reads are served from it when the data is there, and writes go to it first and are written to the device later.
Example: the second read of a file in the program above is served from the page cache and takes 0.146 s instead of 0.374 s.
Common mistake
free shows the page cache as "buff/cache" and some people think that memory is used up. That memory is released at once when a program needs it. A small "free" value on a healthy Linux machine is normal.
Check yourself:
1. What is the difference between a buffer and a cache?
2. Why can a crash lose data that a program has written?
3. What does O_DIRECT do?