Part 3 of 4 · 2 min read
Exercises
Easy
Easy 1: Run it (30 minutes)
Run python -m pytest -q and python run_all.py in code/mini-small.
Done when: you can explain each table in the results.
Easy 2: Will it fit? (30 minutes)
For three models you care about, compute weight size in 16-bit, 8-bit and 4-bit and name the smallest device from your own list that fits each.
Done when: your numbers match weight_gb and fits.
Easy 3: Licence checklist (30 minutes)
Read the licence of one open model and answer three questions (commercial use, restrictions, output use).
Done when: you quote the clause for each answer.
Medium
Medium 1: Your break-even (3 hours)
Replace the assumed GPU price, API price and engineering cost with your real ones and recompute break-even and the utilisation needed.
Done when: you report whether one GPU can reach break-even at all.
Medium 2: Router noise (3 hours)
Sweep the router noise from 0 to 1 and the small model skill from 0.3 to 0.8 and plot the cost to reach within 2 points of the large model.
Done when: you mark where routing stops paying.
Medium 3: Margin test (3 hours)
Change the question to non-inferiority: the cheap model is acceptable if it is no more than 2 points worse. Compute the items needed and check by simulation.
Done when: your calculated and simulated powers agree within a few points.
Hard
Hard 1: Real small model on your task (8 hours)
Evaluate an open model of 7B to 13B parameters on at least 500 of your real requests against a hosted model with a fixed judge. (I did not run this, so your numbers are the ground truth.)
Done when: you report the paired difference with a confidence interval and the cost per request of each.
Hard 2: Trained router (8 hours)
Train a small classifier to predict which requests the small model gets wrong and compare it with the simulated noise levels. (I did not run this.)
Done when: you report its accuracy and the cost-quality curve it produces.
Hard 3: Measured decode speed (8 hours)
Measure tokens per second for 16-bit, 8-bit and 4-bit builds on your hardware and compare with the bandwidth bound. (I did not run this.)
Done when: you explain the gap between the bound and the measurement.