Part 6 of 8 · 2 min read
Working like a professional
What teams add to this client in practice, and how this topic comes up in interviews.
What you will use daily at work
| Practice | What it looks like |
|---|---|
| Use the provider's SDK, but know what it does | Official SDKs for model providers usually retry on 429 and 5xx with backoff for you. Read how many times and how long, and set your own limits. |
| One shared client | Create one httpx.Client and reuse it, so connections are reused instead of opened for every request. |
| A total deadline | Cap the time of the whole call, not only each attempt, so that five retries cannot take minutes. |
| Log each attempt | Record status, attempt number, wait time and request ID, but never the key. |
| Circuit breakers | After several failures in a row, stop calling a broken service for a while and fail fast. |
| Idempotency keys on every write | Especially anything that costs money or sends a message. |
| Test with a fake transport | httpx.MockTransport returns scripted responses, so you can test 429s and timeouts without a network. |
Draft: to be checked against the current documentation of your chosen provider's SDK before this page goes live.
Interview angle
Answer in your head first, then open it.
How would you retry a failing HTTP call safely?
Retry only transient errors, use backoff with jitter, obey Retry-After, cap the number of attempts, and make the call idempotent (or send an idempotency key) before retrying a POST.
What is the difference between 401 and 403?
401: we do not know who you are. 403: we know, and you are not allowed.
Why is cursor pagination preferred?
It stays correct when rows are added or removed while you page.
What does idempotent mean?
Doing it several times has the same effect as doing it once. That is why GET, PUT and DELETE are safe to repeat and a plain POST is not.
Why do requests need a timeout?
Without one, a hung server freezes your program. Set connect and read limits deliberately.