CSE Roadmap

Common tier · about 3 hours · 7 parts

Part 3 of 7 · 3 min read

Deleted but open

Think of a bookshop that takes a book off the shelf and removes it from the catalogue while a customer is still reading it at a table. The book is still in the building and takes space until the customer leaves. The analogy stops in one place: a customer is visible, while an open file is visible only in /proc.

In plain words: unlink removes a directory entry and lowers the link count (page 1). The inode and its blocks are freed only when the count is 0 and no process has the file open. A program that writes a log file and keeps its descriptor open keeps the blocks, even if somebody deletes the log. df counts the blocks, because they are allocated. du cannot count them, because no name leads to them. This is the usual reason for a disk that is full while the directories look small.

A worked example. The program below writes 4 MiB to a file, calls unlink, and then sleeps for 20 seconds with the file still open.

#include <fcntl.h>
#include <stdio.h>
#include <string.h>
#include <unistd.h>

int main(int argc, char **argv) {
    if (argc < 2) return 1;
    int fd = open(argv[1], O_CREAT | O_RDWR | O_TRUNC, 0644);
    char blk[4096]; memset(blk, 'x', sizeof blk);
    for (int i = 0; i < 1024; i++)                    /* 4 MiB */
        if (write(fd, blk, sizeof blk) < 0) return 1;
    fsync(fd);
    unlink(argv[1]);                                  /* the name is gone, the file is still open */
    printf("holding %s open and unlinked\n", argv[1]); fflush(stdout);
    sleep(20);
    return 0;
}

I ran it in the background on the image and read the numbers while it slept:

before:      df used 24 KiB     du 20 KiB
while held:  df used 4120 KiB   du 20 KiB    ls shows only lost+found
after exit:  df used 24 KiB
/proc/PID/fd: 3 -> /tmp/l8/mnt/log.txt (deleted)

df rose by 4,096 KiB, the size of the file, while du and ls showed nothing new. The descriptor in /proc showed the path with the word (deleted). When the program ended, the blocks were freed at once and df returned to 24 KiB. The cure on a real system is to make the program close or reopen its log file (many services reload on a signal), or to restart it. To free the space without a restart, you can truncate the open file through its descriptor, with : > /proc/PID/fd/N, if the program can cope with that.

The key lines. A log-rotation tool that deletes or renames a log without telling the writing program creates this situation. Programs that rotate their own logs, or that reopen the file on a signal, avoid it.

Deleted but open file

A deleted but open file is a file whose last directory entry has been removed while at least one process still holds a file descriptor for it. Its inode and blocks stay allocated until the last descriptor is closed, so it uses space that no path leads to.

Example: a service writes to a log that was deleted by hand. The disk fills until the service is restarted.

Common mistake

Deleting a large log file by hand does not always free space. If a program still has it open, only the name goes. Check df after deleting, and look for (deleted) in /proc/*/fd if the number did not fall.

Check yourself: 1. Why did du not change while the program held the file? 2. What did /proc/PID/fd show? 3. When are the blocks freed?