Part 4 of 7 · 2 min read
Model APIs, and verifying what you read
What to look up in a model provider's reference, why to test the docs with real requests, and how to read examples.
Model providers use different names but share the same ideas. In each provider's reference, find these and write down the current names, because they change:
- Model identifier. Which names exist, which are aliases that move to newer models, and which are fixed versions. Use a fixed name in production so behaviour does not change underneath you.
- Messages and roles. How the system instructions, user turns and tool results are sent.
- Output limit and context limit. Two different numbers (Level 1).
- Sampling controls. Temperature and related settings, and whether some models fix them.
- Tools and structured output. How to declare tools and ask for JSON (Levels 2 and 6).
- Streaming. How to receive the answer as it is produced.
- Usage fields. The input and output token counts in each reply, for tracking cost.
- Errors and rate limits. Often headers that tell you how much budget remains.
Some libraries offer one interface over many providers (for example LiteLLM). That saves you from learning each client, but you still need each provider's documentation for limits and model names.
Verify, do not assume
Documentation can be wrong, old, or describe what should happen. Check with the smallest real request. This is the shape of the command; the address is the fictional API from section 4 and will not answer:
curl -i "https://api.notes.example/v1/notes?limit=2" \
-H "Authorization: Bearer $APP_KEY"
-i prints the status line and the headers, which is where you find rate-limit headers and request IDs. Then try the failure cases on purpose (no key, a bad parameter, too many requests) and compare with what the documentation says. Keep these requests in a file: they become your first tests (module 5). Use the key from an environment variable, as in module 6, never typed into the command.
Read examples critically
Examples in docs and blogs often leave out error handling, timeouts and retries, use an old library version, and may even put a key in the source. Take the shape, not the habits.
Check yourself
- Why use a fixed model name in production?
- What do you learn from
curl -ithat plaincurlhides?