The script had a simple job: scan every stylesheet and find colors written as raw hex codes instead of going through a design token.
We ran it the day we wrote it. Green. We ran it every day after that, and it was green every day.
Two weeks later, someone flipping through a file came across an obvious raw hex value. It should have been caught.
Going back through it
The regex was missing a character class, and the scan scope resolved to an empty list.
So every day it had diligently scanned zero files and reported a pass.
The two weeks themselves weren’t so bad. The trouble was that during that time we assumed someone was watching, so we stopped checking with our own eyes.
Fixing the regex only fixed it once
Swap in a different checker and the same thing would happen again. So we added a rule: every checker ships with an input that makes it fail.
It’s a --selftest flag. It feeds the checker a deliberately broken sample, and if the checker doesn’t complain, --selftest exits with code 1 on its own.
That changed what green means. Before, it meant “didn’t see a problem.” Now it means “looked, and has shown it can see.”
Not just code
Design guidelines work the same way. If a set of guidelines has no examples of misuse, nobody can tell whether the image in front of them breaks it, and it’s just a stack of suggestions.
Handoff docs too. If nobody has followed one from scratch, however complete it is, it reads more like a memoir.
Now when we’re handed anything whose job is to guard something, the first thing we ask is whether anyone can show us an example where it says no.
