CSE Roadmap

Common tier · about 4 hours · 12 parts

Part 2 of 12 · 2 min read

A linter catches missing pieces, not a good README

A reviewer decides whether to keep reading in the first few seconds, so the README has to carry proof, not just a claim.

Think of a store window: most passers-by decide whether to walk in within a few seconds, based on what's visible from the sidewalk, not on what's stocked in the back room.

In plain words: a README is the first and often only thing a reviewer reads. So a text-rule checker scans it for the handful of things that make a claim checkable: a results section, a baseline, an interval, a quickstart short enough to actually run. It flags their absence. Even passing every check still doesn't make the README good.

A worked example. A deliberately weak fictional README ("a revolutionary, state-of-the-art AI agent... 95% accurate and never fails") raises 15 flags. These are 7 missing sections, 4 overclaim words, an 8-command install, no baseline, no interval, and a number that matches nothing in the results file. A partially-trimmed README — missing cost/latency and limitations, with two numbers that don't match results — raises 4 flags. The README generated straight from the evaluation's own results file raises zero.

The key lines. From code/mini-portfolio/readme_check.py:

# from code/mini-portfolio/readme_check.py
OVERCLAIM = ("state-of-the-art", "state of the art", "production-ready", "production ready", "revolutionary", "best-in-class",
             "human-level", "100% accurate", "never fails", "blazing", "seamless", "cutting-edge", "game-changing")

Read it as: the overclaim list is a fixed set of phrases, matched as plain substrings. It will miss any overclaiming the author phrases differently. That's exactly why a passing lint result is weak evidence of quality rather than proof of it: it catches the specific things it was told to look for, nothing more.

README

The README is the first file a visitor reads in a repository; it should say what the project is, how well it works, and how to run it.

Example: the weak fictional README's 15 flags are mostly absence: no results section, no baseline, no quickstart short enough to run. That is the practical definition of a README that fails its one job — letting a stranger decide in seconds whether the project does what it claims.