Part 4 of 5 · 2 min read
Skew and hot partitions
A poorly chosen key sends a disproportionate share of reads or writes to one partition, overloading it while others sit idle.
Skew and hot partitions
Think of one branch library receiving far more visitors than any other. Its alphabet range happens to cover the most common last names, leaving that branch overloaded while the others sit nearly empty.
In plain words: skew is an uneven distribution of data or load across partitions, where one or a small number of partitions, called hot partitions or hot spots, handle a disproportionate share of reads or writes compared to the rest. Skew commonly arises from a partitioning key with a few very common values, such as a celebrity account on a social platform, or from a key that increases monotonically over time, such as a timestamp, which concentrates all new writes on whichever partition currently covers the latest range. The practical fix is choosing a partitioning key that spreads activity evenly, such as adding a random or hashed component to an otherwise skewed key.
A worked example. A social platform partitioning posts by author ID sends every post from one extremely popular account to a single partition, making that partition a hot spot while partitions holding less popular accounts stay underused.
The key lines. No new SQL; detecting and fixing skew is an operational and schema-design task, not something expressed in an individual query.
Skew, hot partition
Skew is an uneven distribution of data or load across partitions; a hot partition, or hot spot, is a partition receiving a disproportionate share of reads or writes because of skew.
Example: the branch library overloaded by common last names is a hot partition caused by skew in the chosen partitioning key.
Remember this
Range partitioning keeps related keys together, which helps range queries but risks hot partitions on sequential keys. Hash partitioning spreads keys evenly, which avoids that risk but scatters range queries across every partition.
Quick notes
- Partitioning splits a dataset across multiple machines, each holding only a portion of the total.
- Range partitioning keeps nearby keys together, favoring range queries but risking hot spots on sequential keys.
- Hash partitioning spreads keys evenly, avoiding hot spots but scattering range queries.
- Skew is an uneven distribution of load across partitions; a hot partition bears a disproportionate share of it.
- A good partitioning key choice depends on the workload's actual query and write patterns.