The Cliff Was the Context Window
A small model stopped calling its tools. It looked like a tool-count limit; it was a silent prompt truncation.
Ten lessons on coverage, verification and silent failures from operating an offensive security agent.
An available tool is not necessarily an executed tool. Coverage needs evidence for each phase.
The report connects observations from one system to engineering patterns: a deterministic testing core, verification separate from the original reasoning, bounded retries, and scope validation at each action. Minimal examples make each pattern explorable without the original system.
This is a report from one system across several sessions, not a comparative benchmark. Figures describe historical runs; they do not guarantee current coverage, complete detection or absence of false positives.