Part 7 of 8 · 2 min read
Errors and trade-offs
Playbook.
Errors, trade-offs and what to say
Errors are either recoverable (retry a few times, then report) or not (stop the work for that video and report). The book gives a playbook.
| Failure | Response |
|---|---|
| Upload fails | Retry a few times |
| Old client cannot split | Server splits the whole file |
| Transcode fails | Retry |
| Preprocessor fails | Rebuild the DAG |
| DAG scheduler fails | Reschedule the task |
| Task worker down | Hand the task to another worker |
| Resource manager queue down | Switch to a replica |
| API server down | Stateless, so another one takes over |
| Metadata cache down | Replicas serve, and a new node replaces it |
| Metadata database primary down | Promote a replica |
| Choice | Gain | Cost |
|---|---|---|
| Use cloud storage and CDN | Fast to build | Per-GB bills |
| Many encoded versions | Smooth play on any device | More storage |
| Chunked upload | Resume and parallel speed | More client work |
| DAG pipeline | Flexible and parallel | A scheduler to run |
| CDN for popular videos only | Lower cost | Slower start for rare ones |
Extras (from the book). Scaling the API tier is easy because servers are stateless. Databases scale by replication and sharding. Live streaming has tighter delay needs, so it uses different protocols and less parallelism. Video takedowns handle copyright and illegal content, found at upload or by user reports.
Interview angle
Answer in your head first, then open it.
Why transcode?
Many devices, many networks, and smaller files.
Why a DAG?
Different creators need different steps, and unlinked steps can run at once.
How is cost cut?
CDN for popular videos only, fewer versions for rare ones, regional placement.
How is upload made safe?
Pre-signed URLs and encryption or DRM.
How do failures recover?
Retry the piece, rebuild the DAG, use replicas.
One follow-up to try. "Add live streaming." A good reply: the same upload, encode and stream path, but chunks are tiny and flow without waiting for a full file. Use a protocol suited to low delay. Less parallel splitting is needed, because the chunks are already small. Error handling must be quick, since a retry that takes seconds is already a visible gap.
Check yourself: 1. What happens when a task worker dies? 2. Which choice trades storage for smooth playback? 3. How does live streaming differ?
Words you will meet
| Word | Meaning |
|---|---|
| Transcoding | Converting a video to other formats and qualities |
| Bitrate | Bits processed per second of video |
| Container | The file wrapper holding video, audio and metadata |
| Codec | A method to compress and decompress video |
| GOP | A short chunk of frames that can play by itself |
| DAG | Steps joined by one-way arrows with no loops |
| Pre-signed URL | A link with time-limited permission to upload or fetch |
| Long tail | Few items get most use, many get little |