Testing the Risk Engine Before It Cost Us a Sprint

For a PMO, the scariest line item in a risk-platform build is the one that looks deceptively simple and then consumes three sprints in rework. On BlackSwan, that line item was the Risk Engine — the surface where admins configure module weights, geographic rules, and PEP/sanctions thresholds. I treated usability testing as schedule insurance, not a polish step. We tested the engine restructured into four core engines with drill-down rules while it was still cheap to change: clickable flows, not finished code. The questions were concrete. Could an admin predict what a threshold change would do? Could they trace a configuration decision well enough to defend it in an audit? Early sessions exposed where confidence broke down, and we fixed those moments before engineering committed to them. We ran lighter testing across Explore, the Entity page, the Dashboard, and Horizon Scanning too, sized to the risk each surface carried. The discipline that mattered for delivery was sequencing: test the highest-uncertainty surface first, so the findings that could move a deadline arrived early enough to actually move it — not after handoff, when changes are most expensive.