CSE Roadmap

Specialist tier · about 5 hours · 12 parts

Part 10 of 12 · 1 min read

Quick notes

  • Data is the lever, judged on quality, coverage and quantity. A small set of carefully chosen examples can beat a large noisy one, and "high quality" means aligned with the task, not merely correct.
  • Deduplicate before anything else: duplicates skew the distribution and leak test data into training. Normalised exact matching misses near duplicates; MinHash plus LSH banding catches them, with bands and rows chosen from the similarity you want and candidates verified.
  • Find wrong labels with cross-validated predictions and review the flagged rows, and measure every cleaning step on the model you will train: a robust model hides data defects, a memoriser exposes them.
  • Split by duplicate group and check contamination with n-grams, then report results on the clean subset; leakage inflates a memoriser most, which is why large models make contamination a real risk.
  • Synthetic data needs verification, and a student inherits its teacher's blind spots. Mixing in real data stops model collapse in the toy, though there is no settled ratio.