Part 10 of 13 · 2 min read
Mistakes that undercut a trade-off argument
Common mistakes, and what the example does not do.
Mistakes that undercut a trade-off argument
- "It depends" with no number behind it.
- Weighting before filtering by hard requirements.
- Hiding the argument in the weights and presenting the score as objective.
- Rescaling by the options present and getting rank reversals.
- Presenting an assumed number as measured.
- No reversal trigger, so the decision can never be wrong.
- Counting only the benefits of your choice.
- Choosing the complex option without a measured benefit.
What the example does not do
About this example What the example does not do
- No stakeholders: one weight vector, no negotiation.
- Few criteria: three in case B; real decisions include risk, team skill and maintenance.
- Additive scoring: it assumes criteria trade off linearly, which fails for hard limits (that is why they are filtered first).
- Assumed inputs: quality values and prices are assumed; the case A numbers are illustrative and not from any measurement.
- No uncertainty on latency and cost beyond the optional score noise.
- Optional experiment: measure your own tool-heavy share of traffic from a month of logs and place yourself on the break-even chart.
Check yourself
What are the four slots of a decision sentence?
Show answer
Choose X over Y because [the criterion that mattered most]; it costs [Z]; the evidence is [number, with its source]; you would switch if [trigger]. If you cannot fill all four, you have an opinion, not a decision.
For the curious A weighted matrix hides the argument in the weights
The winner of the two surviving options won 55 to 77 percent of the time as confidence in the weights grew, because the two scored 0.520 against 0.515. The decision flipped when the p95 weight fell below 0.37; asking how wrong the weights would have to be beats asking for the right weights. With min-max scaling, adding a dominated option flipped the winner; scaling against fixed targets did not. Case A uses illustrative numbers and case B uses simulated latency and assumed quality.
What you will use daily at work
| Practice | What it looks like |
|---|---|
| Check constraints before weights | Remove options that miss a hard requirement. |
| Find the critical weight | How wrong would the weights have to be? |
| Measure when the margin is thin | Quality 0.73 lost at 0.728 in the example. |
| Write a decision record with a revisit trigger | A measurable reason to reverse. |