The API remembers nothing

A test proves the point (real output). Asked "What is my name?" with no history, the stand-in answers I do not know your name. Send the earlier exchange along, and it answers Your name is Asha.

The class Conversation in the example owns that list and enforces the rules that many real APIs enforce:

• A request starts with the system prompt, then user and assistant messages in turn.
• You may not add two user messages in a row, or an assistant message with no question before it. Some providers reject such lists, and a lost reply is the usual cause. Real output of the two errors:

two user messages in a row: the previous one has no reply
an assistant message must follow a user message

• The last message must be from the user when you ask for a new reply.

The history is re-sent, so every turn costs more

If every call carries all earlier messages, the request grows on every turn. In the demo, eight turns of about 60 tokens each send 62, 121, 180, 239, 298, 357, 416 and 475 tokens (using the rough 4-characters-per-token rule).

This is the same effect you met for agents in Level 1 (module 6): the total tokens over a whole conversation grow with the square of the number of turns. You control it with:

• Trimming: drop old turns.
• Summarising: replace old turns with a short summary written by the model.
• Memory: store facts outside the conversation (Level 7).

The example's trim(budget) drops the oldest user and assistant pair until the request fits, and never touches the system prompt or the newest message. Trimming has a cost: the model forgets what you dropped, so do not trim facts that the next answer needs. A safer pattern keeps a short running summary at the top of the history.

A bug I found and fixed. The old trim had two problems. If the budget was impossible (smaller than the system prompt plus the newest message), it dropped everything it could, returned quietly, and left a request that was still over budget. And called after a complete exchange, it could leave a single assistant message as the whole history, which is an invalid request. Now it works out the minimum first: if the system prompt and newest message alone do not fit, it raises ValueError: budget 5 is too small: ... and changes nothing. Otherwise it removes whole pairs only, so the history always starts with a user message. Two new tests cover this (12 tests pass).