Part 7 of 8 · 2 min read
Trade-offs
What to say out loud.
Trade-offs and what to say
| Choice | Gain | Cost |
|---|---|---|
| Queues around SMTP | Independent scaling, buffering | More parts, latency of seconds |
| Attachments in object storage | Cheap and large | A second store to keep in step |
| Partition by user | Fast mailbox reads | Shared mail stored per user |
| Denormalised read and unread tables | Fast queries | Complex writes |
| Single primary per mailbox | Correct data | Short unavailability on failover |
| Search engine | Simple and quick to add | Sync and scale limits |
| Multi-data-centre replicas | Availability, nearby access | Cost, conflict avoidance |
Extra topics if time allows. Fault tolerance of each part. Compliance, such as rules about personal data in some regions. Security features such as phishing protection. An optimisation: when one message with a large attachment goes to many recipients, store the attachment once and share a reference.
Interview angle
Answer in your head first, then open it.
What is the main challenge?
Storage at petabyte scale, and deliverability.
Why queues?
To scale sending and receiving apart from the web tier and to absorb surges.
How do you model mail?
Partition by user, sort by time, and denormalise for read and unread.
Which store?
A wide-column store or a custom one. Say the needed properties, not a brand.
How do you avoid spam folders?
Dedicated IPs, warm-up, authentication and feedback loops.
One follow-up to try. "Delete an email and later undo within 30 days." A good reply: do not remove the row. Move it to a trash folder with a deletion time, and a daily job removes rows past 30 days from the mail store, the search index and the object store (only when no other mail refers to the attachment).
Check yourself: 1. What does a daily job do for the undo follow-up? 2. Why share one stored attachment among many recipients? 3. Which choice trades availability for correctness?
Words you will meet
| Word | Meaning |
|---|---|
| SMTP | The protocol for sending mail between servers |
| IMAP | A protocol that reads mail on the server from many devices |
| MX record | A DNS record that names a domain's mail servers |
| Metadata | Data about a message, apart from attachments |
| Denormalisation | Storing the same data in several shapes to speed queries |
| Hard and soft bounce | A permanent or a temporary delivery failure |
| SPF, DKIM, DMARC | Standards that help prove who sent a message |
| LSM tree | A write-friendly disk structure that merges sorted levels |