CSE Roadmap

Specialist tier · about 5 hours · 12 parts

Part 5 of 12 · 2 min read

Errors compound across steps, and the DOM doesn't cover everything

A 95 percent per-step success rate sounds solid until you multiply it by itself twenty times, and a DOM that's missing some elements leaves exactly those elements unreachable.

Think of a factory assembly line with twenty stations, each 95 percent reliable on its own. A single station rarely fails. But the chance that all twenty in a row succeed is far lower than any one station's own reliability suggests.

In plain words: task success is the per-step success rate raised to the power of the number of steps, so even a solid-looking per-step rate collapses over a long task — verifying each action and retrying once recovers much of that loss, and falling back to a screenshot when the DOM doesn't expose a target recovers coverage the same way.

A worked example. At a 95 percent per-step success rate, a 5-step task succeeds 77 percent of the time, but a 20-step task succeeds only 36 percent of the time — the same per-step rate, a very different outcome. Adding a check that catches 80 percent of failures and retries once brings that 20-step task back up to 79 percent — more than doubling the raw success rate. Separately, when the DOM only exposes 50 percent of the page's actual interactive targets, DOM-only success falls to 0.49; a hybrid approach that falls back to a screenshot for the missing half holds at 0.93 instead.

The key lines. From code/mini-multimodal/costs.py:

# from code/mini-multimodal/costs.py
def success_with_retry(p_step, v, n):
    """Per step: succeed first time with p; else with probability v the failure is noticed and the step retried once."""
    p = p_step + (1 - p_step) * v * p_step
    return p ** n

Read it as: the retry term (1 - p_step) * v * p_step only fires on the branch where the first attempt already failed. Verification does not make any single step inherently more reliable. It gives failed steps a second independent chance. That is why it compounds so favorably over a long sequence.

Explicit and implicit planning

Explicit planning works out the whole action sequence from the first state, good for static forms where the layout won't change mid-task. Implicit planning chooses each action after seeing the latest state, needed for dynamic pages where earlier assumptions may no longer hold.

Example: the retry mechanism above is really only coherent with implicit planning — an explicit plan committed up front has no natural place to insert "check what actually happened and try again," since it never looks at the latest state between steps.