Blog / Analytics & attribution
Check a warning against a known answer before a manager sees it
Pipeline warnings that nobody has checked are just noise with confidence. With no real deals to test on, we built a test account where we knew every right answer, and the product had to match all 23.
An AI sales product we built watches a company’s CRM and warns managers about risky deals: overdue close dates, large deals that have gone quiet. A warning a manager acts on had better be right. The problem: the client’s CRM had no deals in it yet, so 15 of the 16 warning rules had nothing to check.
Plant the answers first
Waiting for real data would have meant shipping rules nobody had seen fire. Instead we built a test account with 24 sample deals, plus contacts and companies, where we planted every problem on purpose and wrote down which warnings should appear.
Then we synced it end to end, the same way a customer’s CRM would be synced, and compared the product’s warnings with the planted answers.
Open pipeline
$518,000
Found across 21 open deals
Flagged
19
9 overdue deals and 10 large deals gone quiet
Correct
23 of 23
Warnings matched what was planted
What the known answers caught
The run found and fixed 15 defects along the way, the kind that only show when real-looking data flows through every step. It also confirmed the behaviour a manager actually wants: a deal leaves the warning list the day it gets a call.
What it proves
The rules do exactly what they should on a known answer, before any customer relies on them. The same test stays in place for every change, so real deals always meet rules that have already been proven.
How we read data
We check a warning against a known answer before a manager sees it.
The examples in this article come from real engagements. Client details withheld; every figure comes from the client’s own data.
Read the case study →