Part 3 of 6 · 2 min read
The map step
Think of the map step as fifty clerks, each reading one shelf of a huge filing cabinet. Each clerk writes a small index card for every document they see. The card holds just a label and a value, nothing more.
In plain words: The map function is called once for every input record, and its job is only to extract a key and a value from that record. A mapper does not need to know what other mappers are doing, where its input came from, or where its output is going; it only transforms one record into zero or more key-value pairs. Because every mapper's work is independent of every other mapper's, the framework can run thousands of mappers in parallel across many machines with no coordination between them.
A worked example. To count how many times each book title was viewed, the mapper reads one log line at a time and emits the book title as the key and the number 1 as the value. A log line for "Database Systems" produces the pair ("Database Systems", 1). A mapper on a different machine, reading a different block of the log, does the same thing independently.
The key lines.
# A mapper, called once per input record (one log line)
def map(log_line):
fields = log_line.split("\t")
book_title = fields[3]
emit(book_title, 1)
Map function
The map function is a callback invoked once per input record, which extracts or computes a key and a value from that record and emits zero or more such key-value pairs. It operates on one record in isolation, with no access to any other record or any shared state.
Example: the mapper above emits one pair per log line, with the book title as key and the count 1 as value.