Part 3 of 5 · 5 min read
Interview questions and limits
Covers: Interview questions, with the follow-ups; Common mistakes; What the example does not do.
Interview questions, with the follow-ups
How do you make an agent survive a crash or a deploy?
"I keep state outside the process, keyed by a run id, and load it on every request so any worker can resume; this is called the stateless design. I checkpoint after each step, and because a crash can fall between a side effect and its checkpoint, I give every effect an idempotency key. I log model outputs so I can replay." Follow-up: "Snapshot or event log?" Snapshots resume instantly but write quadratically: 6.1 MB against 64 kB at 200 steps in my model. A log writes little but replays 100 events on average. I'd use a hybrid and pick the interval from the resume time I can tolerate.
Why can't you just re-run the agent from the log?
"Because a model call isn't deterministic. In my test a live re-run diverged from the original in essentially every run, even over five steps. Replay has to re-apply the recorded outputs, so I store them, and apply has to be a pure function." Follow-up: "Then how do you debug a bad run?" Load a checkpoint, fork, and change one input, with recorded results for the steps before it.
What happens when you change the state schema?
"Old checkpoints no longer match the code. I store a version number and keep a migration chain, so a v1 state is upgraded step by step to the current one, and state from a newer version is refused. Without it, an old run fails on resume, as the KeyError in my test shows." Follow-up: "Do you migrate stored runs or at load time?" Usually at load; a batch migration is an option when the old code must be removed.
An agent paused for approval and resumes a day later. What can go wrong?
"The world changed. With a 2 percent hourly chance in my model, 38 percent of day-long pauses resumed on a false premise. So before the irreversible step I re-read the facts it depends on, and re-plan or ask again if they differ; I also expire very old pauses." Follow-up: "Isn't that expensive?" One read per resume, against a wrong refund or a repeated email.
Common mistakes
- Keeping the run state in process memory (lost at a deploy; not for production, as the in-memory saver's documentation says).
- Checkpointing but not making effects idempotent.
- Replaying by calling the model again.
- No schema version, so any state change breaks old runs.
- Never pruning checkpoints.
- Resuming without re-checking the premise.
- Putting secrets or full personal data in checkpoints without a retention rule (Level 10 covers it).
- Unbounded waits with no expiry.
What the example does not do
- No real storage: no database, transactions, locks, or concurrent writers (module 3 covers conflicts).
- Sizes are JSON length, with no compression or serialisation overhead.
- Crash points are uniform over steps and happen between steps.
- One migration path with two simple changes; real migrations can be lossy.
- A single hazard rate for staleness; real changes depend on the fact.
- Optional real experiment (I did not run this): use a real checkpointer with a SQLite file, run 200 steps, kill the process at random steps, resume, and measure file size and resume time against the simulation.
Check yourself
A run crashes at step 40 of 200. With a snapshot every step, an event log, or a snapshot every 20 events, which writes least and which resumes fastest? (The log writes least and replays the most; snapshots resume instantly and write the most; the hybrid sits between. Choose by bytes written and events replayed.)
For the curious A saved state can go stale
A paused run that resumes a day later may act on facts that changed. With an assumed 2 percent per hour chance, 38 percent of resumes after 24 hours acted on a false premise unless the agent re-checked before the irreversible step. A part-hour pause used to count as zero hours; it now rounds up and is tested.
What you will use daily at work
| Practice | What it looks like |
|---|---|
| State outside the process | Keyed by a run id so any worker can resume. |
| Record model outputs | Replaying with a live model call can diverge from the original run. |
| Version the state | A migration for each schema change. |
Draft: confirm which store your platform uses for checkpoints and how long it keeps them.
What you must be able to say (one page)
- Durable state means a run's progress lives outside the process, so a crash, a deploy or a day-long wait does not lose it. In a stateless design, when input arrives a new
agent.run()call resumes execution, any server can handle the resume, and nothing is held in memory. - Three storage models, chosen by two numbers, bytes written and events replayed on resume. In my simulation (200 steps, 300 bytes each): snapshot every step wrote 6,115 kB and replayed 0 events; a pure event log wrote 64 kB and replayed 100.5 events on average; a hybrid (snapshot every 20 events) wrote 399 kB and replayed 9.5.
- Snapshots grow with the square of the run (50 steps: 389 kB; 800 steps: 97 MB), logs linearly (16 kB; 257 kB). The hybrid interval is the dial: k=5 wrote 1,311 kB and replayed 2.0; k=50 wrote 216 kB and replayed 24.5.
- Replay is only safe if you replay recorded results, not live calls. Re-calling a model with a fresh draw diverged from the original in 99.6 to 100 percent of runs, even over 5 steps. Event-sourced replay re-applies logged outputs and is identical by construction.
- Old checkpoints meet new code. A state saved under schema v1 fails to load under v3 (
KeyError) unless a migration chain upgrades it; state newer than the code must be refused. - Resuming after a wait is acting on old information. With an assumed 2 percent per hour chance that the premise changes, 38 percent of resumes after 24 hours and 77 percent after 72 hours act on a false premise unless the agent re-validates before the irreversible step.
- Documented fact (31 October 2026): LangGraph checkpointers save per-thread state keyed by
thread_idand enable human-in-the-loop, time travel and fault tolerance; the in-memory saver is not for production, and checkpoints accumulate and need pruning. - Honest limit: storage numbers are byte counts of an in-memory model, not measurements of any database; I did not test a real checkpointer.