Part 3 of 6 · 2 min read
Pipelines
In plain words: The shell command ls /etc | wc -l starts two programs and connects the output of the first to the input of the second. Neither program knows about the pipe. ls writes to descriptor 1, and wc reads from descriptor 0.
The steps for the shell:
- Create a pipe.
- Fork the first child. Use
dup2to make descriptor 1 refer to the write end, close both pipe descriptors, andexecls. - Fork the second child. Use
dup2to make descriptor 0 refer to the read end, close both pipe descriptors, andexecwc. - In the parent, close both ends, and wait for both children.
#include <stdio.h>
#include <unistd.h>
#include <sys/wait.h>
int main(void) { /* the same as: ls /etc | wc -l */
int fd[2];
pipe(fd);
if (fork() == 0) {
dup2(fd[1], STDOUT_FILENO);
close(fd[0]); close(fd[1]);
execlp("ls", "ls", "/etc", (char *)NULL);
_exit(127);
}
if (fork() == 0) {
dup2(fd[0], STDIN_FILENO);
close(fd[0]); close(fd[1]);
execlp("wc", "wc", "-l", (char *)NULL);
_exit(127);
}
close(fd[0]); close(fd[1]);
wait(NULL); wait(NULL);
return 0;
}
On the test machine it printed 166, and typing ls /etc | wc -l in a shell also printed 166. The two ways agree.
Closing matters in every step. After dup2, the child has the pipe end under two numbers, so it closes the original. The parent also closes both ends, because otherwise wc would never see end of file: the parent itself would still be a writer. The most common error in pipeline code is a missing close, and the symptom is a program that hangs at the end.
Common mistake
Forgetting to close the write end in the parent or in the reading child. The reader then waits for ever, because the kernel counts the unclosed copy as a living writer.
Check yourself:
1. Which descriptor of ls must be connected to the pipe?
2. Why does the parent close both ends?
3. How many wait calls does the parent need?