CSE Roadmap

Core tier · about 3 hours · 8 parts

Part 3 of 8 · 1 min read

Precision and recall

Too noisy, or too quiet.

UDS names a trade-off. Precision is the share of alerts that point to a real, significant event. Recall is the share of real, significant events that raised an alert. An alert with low precision is noisy and often not worth acting on. One with low recall stays silent during real outages. Both at 100 percent would be ideal, but improving one usually lowers the other.

A made-up example of the counts. In one month there are 10 real outages. An alert fires 40 times, and 8 of the firings fall on real outages. Precision is 8 ÷ 40 = 20 percent. Recall is 8 ÷ 10 = 80 percent. The alert is noisy, and it also missed two outages.

Precision and recall

Precision is real events caught divided by all alerts fired. Recall is real events caught divided by all real events.

Example: 8 caught out of 40 alerts is precision 0.2. 8 caught out of 10 events is recall 0.8.

Check yourself: 1. Which of the two is low when the alert is noisy? 2. Which is low when an outage passes without a page? 3. In the example, what would precision be if only the 8 real firings had fired?