CSE Roadmap

Core tier · about 6 hours · 7 parts

Part 2 of 7 · 4 min read

Describe the shape once

Five ways to get structure from a model, then the one habit that makes the rest easy: say what you want once, as a small Python class.

Five ways to get structure

There is a ladder of methods. Each one costs more effort and gives more reliability, and you can combine them.

Method What it is Cost How reliable
Prompting Describe the format in the instructions and show the schema Free Good, not guaranteed
Post-processing Code that finds the JSON in the reply and tidies it Free Fixes harmless slips
Retry with feedback Send the error back to the model and ask again Extra calls Fixes most content mistakes
Constrained decoding The provider restricts which tokens the model may produce, so output must match your schema (you met sampling in Level 1, module 4) A provider feature Guarantees the shape
Fine-tuning Train a model on your format (Level 13) High effort Strong for fixed formats

This module builds the first three, which work with any model. Provider features such as "structured output" or forced tool calls (module 4) are the fourth method. Use them when you have them, but still validate: a reply can have the right shape and the wrong meaning, and providers support only part of what a schema can express.

For the curious Why JSON mode is not enough

Many providers offer a JSON mode. It usually promises that the output is valid JSON, and nothing more. It does not promise your field names, your allowed values, or that the reply was not cut off before it finished. A schema-based mode goes further and restricts the output to your schema. Even then, the value of priority can be "high" when the right answer was "low". Valid and correct are different things, and the section on measuring comes back to this.

Describe the shape once

Before writing any checking code, you need a way to say what a valid reply looks like. In Python the most common tool for this is a library called Pydantic. You met it in Level 0, module 1, and here it is again from the beginning.

Pydantic

A Python library for describing data and checking it. You write a class that lists the fields and their types. Pydantic can then check any dictionary against that class, and it can also produce a schema from it.

Here is the form for a support ticket:

class Priority(str, Enum):
    LOW = "low"
    NORMAL = "normal"
    HIGH = "high"

class Ticket(BaseModel):
    category: str = Field(min_length=1)
    priority: Priority
    summary: str = Field(min_length=3, max_length=120)
    needs_human: bool

Read it line by line:

  • class Ticket(BaseModel) says: "Ticket is a form, and it can check itself." Ticket is just the name we chose for "one support ticket". BaseModel is what Pydantic provides to make a class checkable.
  • category: str means the field category must be text. Field(min_length=1) adds a rule: at least one character.
  • priority: Priority means the value must be one of the three words listed in Priority. A class like that, which lists the only allowed values, is called an enum (short for enumeration).
  • summary is text between 3 and 120 characters.
  • needs_human: bool must be true or false.

Enum

A short, fixed list of allowed values. Use one whenever a field can only be one of a few words. It removes a whole kind of mistake: the model can no longer invent "urgent" or "HIGH!!".

Now the class does two jobs. First, it checks replies. Here it accepts a good reply:

>>> Ticket.model_validate({"category": "billing", "priority": "high",
...                        "summary": "Charged twice", "needs_human": True})
Ticket(category='billing', priority=<Priority.HIGH: 'high'>, summary='Charged twice', needs_human=True)

model_validate checks a dictionary and, if it is fine, gives you a real Ticket object whose fields have the right types. If it is not fine, it raises an error that lists every problem at once:

>>> Ticket.model_validate({"category": "x", "priority": "low", "summary": "ab"})
2 validation errors for Ticket
summary
  String should have at least 3 characters [type=string_too_short, input_value='ab', input_type=str]
needs_human
  Field required [type=missing, input_value={'category': 'x', ...}, input_type=dict]

Second, the class writes the instructions for the model. Ticket.model_json_schema() turns the class into a JSON Schema, a standard way to write a schema as JSON. Here is the part that describes priority and summary:

"priority":   {"$ref": "#/$defs/Priority"},
"summary":    {"type": "string", "minLength": 3, "maxLength": 120},
"needs_human": {"type": "boolean"}

and "Priority": {"enum": ["low", "normal", "high"], "type": "string"} further up. The function schema_prompt in the example code pastes this into the system prompt, together with the sentence "Reply with ONE JSON object and nothing else". Because the same class writes the prompt and does the check, the two can never disagree. That is the main idea of this part: describe the shape once, and use it twice.

Check yourself

  • What does priority: Priority forbid that priority: str would allow?
  • Which method checks a reply, and which method writes the instructions?
  • Why is it useful that one class does both jobs?