Keys, REST, pagination and timeouts: HTTP, REST and JSON

How you prove who you are, how web APIs are organised, how to read a long list one page at a time, and why a request must never wait forever.

Authentication, REST and pagination

Authentication headers

Most APIs identify you by a secret key sent in a header. The usual form is:

Authorization: Bearer <your key>

Some providers use their own header name, such as x-api-key. Always check the documentation (module 8). Two rules apply everywhere:

Send keys only over HTTPS, never plain HTTP. HTTPS encrypts the message, so nobody on the way can read the key.
Never put a key in a URL, and never log it. URLs end up in logs, browser history and error reports.

Level 0, module 6 covers where keys should live.

REST

A style for designing web APIs around resources, which are things with names, such as conversations or tickets. Each resource has a URL, and you act on it with the standard methods. GET /conversations/42 reads one, POST /conversations creates one and DELETE /conversations/42 removes one.

Good habits when you build or use such an API:

Nouns in paths (/tickets), and the verb comes from the method.
Plural resource names and stable IDs.
Error bodies with one consistent shape, for example {"error": {"type": "...", "message": "..."}}.
A version in the path (/v1/), so that later changes do not break existing users.

Pagination

A list endpoint rarely returns everything at once. If a service has a million tickets, it sends them in pages. There are two common styles:

Style	Request	Notes
Offset	`?limit=50&offset=100`	Simple, but rows can shift while you page
Cursor	`?limit=50&cursor=abc`	The server gives you a `next_cursor`. Stable, and usually preferred

With a cursor, each response says where the next page starts: {"items": [...], "next_cursor": "abc"}. You keep asking until the cursor is missing. Always loop until there is no next cursor. Forgetting this means your agent silently sees only the first page, which is a quiet and common bug.

For the curious Why offset paging can go wrong

With offset paging, "give me 50 rows starting at row 100" means counting from the start each time. If someone adds a row at the top while you page, every row shifts down by one, so the last row of one page appears again at the start of the next. If a row is deleted, one is skipped. A cursor points at a specific place instead of a count, so it does not shift.

Timeouts

Sometimes a server never answers. If your code waits forever, your whole agent run freezes. A timeout is a limit on how long you are willing to wait, after which the call fails and you can decide what to do.

Libraries differ, so always set one yourself

The popular requests library has no timeout by default: a hung server can block your program forever. httpx, used in this course, defaults to 5 seconds, which is safe but too short for a model that writes a long answer (I checked: httpx.Client().timeout prints Timeout(timeout=5.0)). Choose your own values deliberately, and give model calls longer ones than ordinary APIs.

A request has stages, and you can limit each one: connecting to the server, sending the request, waiting to read the reply, and the total. In httpx you set them together:

import httpx

timeout = httpx.Timeout(10.0, connect=3.0, read=60.0)   # 10s default, 3s to connect, 60s to read
client = httpx.Client(timeout=timeout)

A short connect limit finds a dead server quickly. A long read limit gives a slow model time to finish.