When a checker stays green from the day it goes live, there are only two explanations: nothing is broken, or it isn’t looking.
On screen, the two look exactly the same.
How we found out
The script was supposed to catch design tokens written out as raw hex colors. We ran it the day we wrote it: green. We ran it every day after that: green. Two weeks later someone stumbled on an obvious raw hex value by hand, and we went back to ask why the script hadn’t flagged it. The regex was missing a character class, and the scan scope resolved to an empty list. Every day it had carefully scanned zero files and reported a pass.
The cost wasn’t the two weeks. It was that for those two weeks we believed someone was watching, so we stopped looking ourselves.
Fixing the regex isn’t the fix
A corrected regex only fixes this one case. The real fix: every checker has to ship with an input that makes it fail.
Every validator we run now has a --selftest. It feeds the checker deliberately broken sample data, and if the checker doesn’t complain, --selftest itself exits with code 1.
That’s when the green light started to mean something. Now it says, “I looked, and I’ve proven I can see.”
It isn’t only about code
The same shape turns up elsewhere.
Design guidelines without examples of misuse are just a pile of suggestions. Nobody can tell whether the image in front of them breaks the rules. A handoff document that no one has followed from scratch is just a well-written memoir.
⛔ The test is simple: can you point to something that would make it say no? If you can’t, it isn’t guarding anything.
