Part 7 of 8 · 5 min read
Using object storage in a design
Upload paths, signed links, cost tiers and the numbers an interviewer expects.
Using object storage in a design
Think of a valet service at a theatre. Guests hand over their keys at one desk and collect their cars at another. The theatre does not carry cars through its lobby.
In a design, large files should not travel through application servers more than needed. The usual pattern:
- The client asks the API server for permission to upload.
- The API server checks who the user is and creates a pre-signed URL, a temporary address for one object, valid for a short time, such as 10 minutes.
- The client uploads straight to the object store with that URL.
- The store reports completion (a callback or an event), and the API server updates the metadata.
For downloads the same trick works: the API server returns a pre-signed URL, or the object is served through a CDN (Level 1, module 4) with signed links for private files.
Pre-signed URL
A pre-signed URL is an address to an object that includes a signature and an expiry time. Anyone who holds it can read or write that one object until it expires.
Example: https://files.example.com/u7/91?expires=1731400000&sig=ab12... lets the holder download one file for ten minutes.
A worked example of why. A service has 10,000 uploads a minute of 5 MB each.
- Through the application servers: 10,000 × 5 MB = 50,000 MB a minute, about 833 MB per second, all of which passes through servers, which must be sized for that bandwidth and keep a connection open for each slow client.
- With pre-signed URLs: the servers handle 10,000 small requests a minute, about 167 per second, and the object store takes the bandwidth.
Storage classes and cost. The book mentions cold storage, a tier for inactive data. Object stores offer several classes: hot (fast, dearer), infrequent access and archive (cheap to store, slow or costly to read). A lifecycle rule moves objects automatically after a number of days.
A worked example. Using the book's cloud drive estimate: 50 million users with 10 GB each is 500 PB allocated. Suppose actual use is 20 percent of the allocation, 100 PB (assumed), and 80 percent of that has not been read in 90 days.
- Cold data: 80 PB. Hot data: 20 PB.
- If hot storage costs 1 unit per GB-month and cold costs 0.25 (made up), moving 80 PB saves 80 PB × 0.75 = 60 PB worth of cost units, about 60 percent of the former total of 100 (so the new cost is 20 + 80 × 0.25 = 40).
- The cost falls from 100 units to 40, a 60 percent saving. Reads from cold storage are slower and charged, so the rule must fit real access patterns.
The book's numbers for request rates are a useful model for the interview:
- 10 million daily users × 2 uploads ÷ 86,400 seconds ≈ 231, rounded to 240 uploads per second.
- Peak at twice the average ≈ 480 per second.
- Storage added per day: 20,000,000 files × 500 KB = 10 TB.
- Storage added per year: 10 TB × 365 = 3.65 PB.
Object stores handle those rates easily. The harder parts are metadata consistency and sync (Levels 5 and 11).
A final point on choosing between a file system and an object store. A distributed file system suits workloads that need appends, in-place edits, directories and POSIX behaviour, such as big data jobs or a shared home directory. Object storage suits whole-object reads and writes through HTTP at huge scale. Most web designs choose object storage, with a metadata database for names and permissions.
Interview angle
Answer in your head first, then open it.
Where do you store user-uploaded images and videos?
In object storage, with a metadata row in the database, uploaded through pre-signed URLs and served through a CDN. Name the clean-up job for orphans.
How do you make large uploads reliable and cheap?
Chunk the file, hash the chunks, upload in parallel, resume from the last confirmed chunk, send only changed chunks on edits.
How do you keep the data from being lost?
Copies across zones and regions or erasure coding, checksums with repair, versioning, and separate backups. Say the difference from availability.
Why not store files in the database?
The database would hold terabytes of rarely changed bytes that slow backups and replicas. Object storage is cheaper and scales better. The cost is the loss of one transaction across both.
Check yourself:
- Why do pre-signed URLs keep large uploads off the application servers?
- What does a lifecycle rule do, and what does it risk?
Words you will meet
| Word | Plain meaning |
|---|---|
| Block storage | A raw disk in numbered blocks |
| File storage | A tree of folders and files that can be edited in place |
| Object storage | Whole objects found by key, read over HTTP, replaced not edited |
| Bucket | A named container of objects |
| Blob | A large binary object such as an image or video |
| Metadata | Data about data: name, size, owner, location |
| Reference plus blob | Keep the blob in an object store and a pointer in the database |
| Chunk, block | A fixed-size piece of a file |
| Content hash | A fingerprint of a chunk's bytes |
| Delta sync | Upload only the chunks that changed |
| Deduplication | Storing equal chunks only once |
| Resumable upload | An upload that continues from where it stopped |
| Durability | The chance that data is not lost over time |
| Replication | Keeping copies on several machines or sites |
| Erasure coding | Pieces plus parity pieces, any k of k + m rebuild the data |
| Pre-signed URL | A temporary address to one object, with a signature |
| Cold storage | A cheap tier for rarely read data |