← Research
Field observations

When the agent decides

Ten lessons on coverage, verification and silent failures from operating an offensive security agent.

An available tool is not necessarily an executed tool. Coverage needs evidence for each phase.

Of about 60 attack capabilities, fewer than 20 ran every time. In the 9-stage pipeline, 8 are deterministic and only exploit chaining is left to the model. Of 7 critical or high findings, 1 had fresh evidence and 6 were discarded: the verifier caught 86% of the noise. 24-second animation, no audio.
Key data

Coverage and verification, measured

Why it matters

The report connects observations from one system to engineering patterns: a deterministic testing core, verification separate from the original reasoning, bounded retries, and scope validation at each action. Minimal examples make each pattern explorable without the original system.

Method and limits

This is a report from one system across several sessions, not a comparative benchmark. Figures describe historical runs; they do not guarantee current coverage, complete detection or absence of false positives.