Part 1 of 6 · 1 min read
Overview
In this module you will
- Explain why retrieval exists and what a chunk is in the retrieval pipeline.
- Parse HTML and PDF-style text into clean blocks, dropping navigation, scripts, headers and footers.
- Cut text into chunks four ways (fixed size, sentences, recursive, by structure) and say what each one breaks.
- Choose chunk size and overlap by measuring on questions you wrote, instead of copying a default.
- Keep each chunk's heading path so a piece still makes sense when it is read alone.
An agent that answers from your documents is only as good as the pieces it can fetch. If the answer was cut in half at ingestion, or a table row lost its header, no later step (a better embedding model, a clever reranker, a stronger LLM) can bring it back. Parsing and chunking happen once, early, and every later module depends on them. They are also the cheapest place to find bugs: you can read the chunks with your own eyes.
About the examples
The example corpus is a small invented router manual (about 20 blocks, 12 test questions) and the retriever is a keyword scorer written for these notes. The numbers show the method and the direction of the effects. They are not benchmarks, and a real corpus with an embedding model will give different figures. The harder exercises ask you to repeat the experiment on your own documents. No real model or service was called.