Part 2 of 5 · 1 min read
Why partition at all
A single machine has a limit on storage and throughput; partitioning splits data across several machines to go past that limit.
Think of a library whose single building has run out of shelf space. The city opens several branch libraries instead, each holding a portion of the overall collection.
In plain words: partitioning splits a dataset across multiple machines, with each machine, called a partition or shard, holding only a portion of the total data. This lets the dataset's total size and total read-or-write throughput grow beyond what any single machine could hold or serve, since each partition independently stores and serves only its own share of the data. The central question partitioning raises is how to decide which row goes to which partition, which is exactly what the next two parts address.
A worked example. A dataset of 100 million customer records, too large for one machine's disk, is split across 10 partitions, each holding roughly 10 million records.
The key lines. No new SQL; which partition a row lives on is a storage-layer decision, invisible to an ordinary SELECT or INSERT statement.
Partitioning, partition
Partitioning is splitting a dataset across multiple machines so that each one holds only a portion of the total; each such portion, and the machine holding it, is called a partition or shard.
Example: the 100-million-record dataset split into 10 partitions of roughly 10 million records each is a partitioned dataset.