CSE Roadmap

Specialist tier · about 3 hours · 12 parts

Part 4 of 12 · 2 min read

A severity rule written before the incident decides who gets woken up

A fixed formula over impact, reach and reversibility grades an incident the same way at 3 a.m. as at 3 p.m., instead of leaving the grade to whoever is on shift.

Think of hospital triage: a patient's colour tag comes from a fixed rule applied to their vital signs, not from whichever clinician happens to be on duty that night.

Severity should come from a fixed formula over impact, how many people or records are affected and whether the action is reversible, decided in advance. That means two different people grade the same incident the same way.

A worked example. Scoring critical impact, 5,000 records, irreversible gives SEV1. Serious impact, 500 records, irreversible gives SEV2. Minor impact, 50 records, reversible drops all the way to SEV3, the same grade as serious impact on only 5 records reversible -- because once reach moves from "under 10" to "10 and over", the reach band outweighs a one-point difference in raw impact.

The key lines. From code/mini-accountability/accountability.py:

# from code/mini-accountability/accountability.py
def severity(impact, reach, reversible):
    score = impact * 2 + (2 if reach >= 1000 else 1 if reach >= 10 else 0) + (0 if reversible else 2)
    return "SEV1" if score >= 8 else "SEV2" if score >= 6 else "SEV3" if score >= 3 else "SEV4"

Read it as: the grade is arithmetic over three inputs. It is reproducible from the incident's facts rather than from who is paged. The 2-point bonus for irreversibility is what pushes a SEV3-looking incident into SEV2 territory.

Severity, escalation, on-call

  • Severity: a rated seriousness of an incident that decides response speed and who is told.
  • Escalation to a person: passing an issue to the next level of people when the first does not respond or lacks authority.
  • On-call: the person responsible for responding at a given time.

Example: severity(3, 5000, False) scores 8 and returns SEV1, the grade that pages the on-call engineer immediately rather than waiting for the next stand-up.