The gap is the warning: when a measure becomes the target

Tuning the test

Each week you tune the suite. The gold score climbs toward a perfect 36 out of 36. Keep one eye on the sealed hold-outs.
tuned set sealed hold-outs the gap
60% 70% 80% 90% 100% 0 1 2 3 4 5 6 7 8 weeks spent tuning pass rate
Week 0 of 8
TUNED SET
33/36
looks like success
SEALED HOLD-OUTS
90%
the real behavior
THE GAP
2
points apart
Week zero: the tuned set and the sealed hold-outs agree.
When a measure becomes a target, it stops being a good measure. The tuned score climbed to 36 out of 36 while the real behavior fell to 70%. The gap to the sealed hold-outs is the earliest honest warning you can buy.
The fix: seal a fresh batch of hold-outs, then re-baseline.