CSE Roadmap

Core tier · about 5 hours · 8 parts

Part 7 of 8 · 2 min read

Errors and trade-offs

Playbook.

Errors, trade-offs and what to say

Errors are either recoverable (retry a few times, then report) or not (stop the work for that video and report). The book gives a playbook.

Failure Response
Upload fails Retry a few times
Old client cannot split Server splits the whole file
Transcode fails Retry
Preprocessor fails Rebuild the DAG
DAG scheduler fails Reschedule the task
Task worker down Hand the task to another worker
Resource manager queue down Switch to a replica
API server down Stateless, so another one takes over
Metadata cache down Replicas serve, and a new node replaces it
Metadata database primary down Promote a replica
Choice Gain Cost
Use cloud storage and CDN Fast to build Per-GB bills
Many encoded versions Smooth play on any device More storage
Chunked upload Resume and parallel speed More client work
DAG pipeline Flexible and parallel A scheduler to run
CDN for popular videos only Lower cost Slower start for rare ones

Extras (from the book). Scaling the API tier is easy because servers are stateless. Databases scale by replication and sharding. Live streaming has tighter delay needs, so it uses different protocols and less parallelism. Video takedowns handle copyright and illegal content, found at upload or by user reports.

Interview angle

Answer in your head first, then open it.

Why transcode?

Many devices, many networks, and smaller files.

Why a DAG?

Different creators need different steps, and unlinked steps can run at once.

How is cost cut?

CDN for popular videos only, fewer versions for rare ones, regional placement.

How is upload made safe?

Pre-signed URLs and encryption or DRM.

How do failures recover?

Retry the piece, rebuild the DAG, use replicas.

One follow-up to try. "Add live streaming." A good reply: the same upload, encode and stream path, but chunks are tiny and flow without waiting for a full file. Use a protocol suited to low delay. Less parallel splitting is needed, because the chunks are already small. Error handling must be quick, since a retry that takes seconds is already a visible gap.

Check yourself: 1. What happens when a task worker dies? 2. Which choice trades storage for smooth playback? 3. How does live streaming differ?

Words you will meet

Word Meaning
Transcoding Converting a video to other formats and qualities
Bitrate Bits processed per second of video
Container The file wrapper holding video, audio and metadata
Codec A method to compress and decompress video
GOP A short chunk of frames that can play by itself
DAG Steps joined by one-way arrows with no loops
Pre-signed URL A link with time-limited permission to upload or fetch
Long tail Few items get most use, many get little