CSE Roadmap

Common tier · about 4 hours · 12 parts

Part 8 of 12 · 1 min read

Answering portfolio questions in an interview

Walk me through your project's README and what a reader should take from it.

"One sentence and the user, a 90-second demo, then a results table with the baseline, the system and the gain. The table carries a 95 percent interval, the number of items, cost and latency, plus a quickstart of three commands and a limitations section. It is generated from the evaluation output so it cannot drift." Follow-up: "How do I know the numbers are real?" One command regenerates them, and the evaluation report lists the data and the commit.

How do you document your design decisions?

"An architecture note with a table: decision, what I chose, alternatives, evidence and what I gave up. It also covers failure modes, permissions, approval points and a cost and latency model. It lets me defend each choice and shows where I would change it." Follow-up: "Give me one decision you would reverse." Pick one with evidence (module 6 of this level).

How do you report evaluation results honestly?

"A table with a baseline and an interval for each number, the size and construction of the evaluation set, whether tuning touched it, slices, ten failure examples with causes, and the commands to reproduce. Contamination is disclosed, and results are reported on the clean subset too." Follow-up: "What do you leave out?" Nothing that changes the conclusion. Limits go in the limitations section.

What do you do when the README and the code disagree?

"The code and the last evaluation run win; the README is regenerated from them. In the example's example, a rerun made 3 to 5 of 9 numbers stale. So I check numbers against the results file before every release." Follow-up: "How do you prevent it?" Generate the tables and run a provenance check in continuous integration.