CSE Roadmap

Core tier · about 5 hours · 8 parts

Part 2 of 8 · 2 min read

Scope and numbers

Upload and watch, at a cost.

Think of a television station that receives film from many people. Each film arrives in its own format. The station converts it into several versions, stores them, and sends copies to local relay towers near viewers. Viewers then watch from the nearest tower, never from the studio.

Questions and the answers assumed here (from the book's sample talk).

  1. Main features? Upload a video and watch a video.
  2. Clients? Mobile apps, web browsers and smart TVs.
  3. Users? 5 million daily active users (DAU), spending 30 minutes a day.
  4. International users? Yes, many.
  5. Resolutions and formats? Most of them.
  6. Encryption? Yes.
  7. File size? Up to 1 GB, small and medium videos.
  8. Use cloud services? Yes, building everything is unrealistic.

Requirements.

  • R1. Upload fast.
  • R2. Stream smoothly, and change quality when the network changes.
  • R3. Low infrastructure cost.
  • R4. High availability and reliability.

Numbers (from the book). Each user watches 5 videos a day. 10 per cent of users upload one video a day, at an average of 300 MB. Daily storage for new uploads is 5,000,000 × 10% × 300 MB = 150 TB. This is only the original files, and every video is also stored in several converted versions.

For the delivery cost, take a CDN price of $0.02 per GB sent (the book's figure for one provider, with all traffic in one country). The daily cost is 5,000,000 × 5 videos × 0.3 GB × $0.02 = $150,000. Big customers get discounts, but the cost remains large. Much of the design is about cutting it.

CDN (content delivery network)

A CDN is a set of servers around the world that keep copies of content and send it from the one nearest the viewer.

Example: a viewer in Mumbai gets a video from a server in Mumbai, not from a data centre in another continent.

Check yourself: 1. Why is the CDN the main cost? 2. Is 150 TB the total storage need? Why not? 3. Why use cloud services for storage and the CDN?