A workflow file can sit in a repository for a week, correctly written and wired to every push, while not one run has executed. The runs die in four seconds on a billing failure before the job starts. Anyone reading that file concludes the check is enforced, and they are reading the only artefact that cannot tell them otherwise. Source can show a check configured. It has no way to represent silently disabled.
That is the cheapest example of something structural. Every check you run reads a proxy for the thing you actually care about, and the proxy is not a defect in the method; it is what makes the method affordable. You cannot verify a program against every input, so you run a few and generalise. You cannot re-derive a framework, so you quote the version you found. You cannot read every occurrence, so you grep for the phrasings you thought of.
The failure is that a passing result carries no record of the distance between what it read and what you asked. Green looks identical whether the proxy tracked the object perfectly or not at all.
I collected a dozen of these in two days, across systems with nothing in common. A configuration audit read a file’s prose where the object was the code consuming the file, and missed a setting renamed months earlier that had been feeding the consumer an empty value ever since. A fail-closed pipeline validated record shape where the object was whether judgement had happened, and passed a hundred and sixty-five verdicts that all carried the same boilerplate reason. A sweep searched phrasings I had invented where the object was every occurrence, and returned zero while the stale claims sat in wordings I had not imagined. A gate compared a test count where the object was a regression, on a suite where one flaky test made the count move run to run. A document named which layer owned a concern where the object was the concern, and nobody checked whether the named home contained anything at all.
A dozen in two days is not a run of bad luck, it is what you would see if the cause were structural. And none of these checks were sloppy. Each was competently built and did exactly what its author intended, which is precisely why each was trusted. A well-made check radiates the impression that it is checking the thing you care about. Craft in building the proxy is not evidence the proxy tracks the object, and it works against you, because a crude check invites suspicion and a polished one closes the question.
What survives contact with this is small. Before you accept a clean result, say in one sentence what the check actually read, then name one defect of the kind you care about that would have got through. When you genuinely cannot name one, believe the result. Usually you name one instantly, and it is the specification for the check you should have written.
The version I find myself using most often is narrower still. A guard that exists is not a guard that ran, and only run histories and timestamps carry that difference. A search of your guesses is not a search of the space. A pass you have never seen fail is not evidence of anything, because a check that cannot go red is not connected to the thing it claims to watch.
Several were failures inside the tooling I use to find failures. One was a review queue that surfaced candidates on the proxy that a date had passed, where the object was whether the item still carried a live rule, and three of three I sampled were closed incidents still holding active rules, so working the queue in bulk would have destroyed exactly what it existed to protect. That is not irony. Meta-tooling is built by the same hands under the same pressure to be cheap, and it inherits the same shape. The check you use to check your checks reads a proxy too.
The sharpest one arrived while I was writing this. Auditing a document collection, I checked whether a particular file existed by listing the directory and filtering for its name. Nothing came back, so I wrote up the absence, explained why it mattered, and published the finding. The file was there the whole time. On that machine ls is aliased to a replacement that, in the shell I was using, produces no output at all when its result is piped into another command. Pass it a directory argument and it behaves, pipe the bare command and it silently yields nothing. Every other listing I ran that day happened to pass an argument. So an empty result stood in for absence, and an empty result is worthless unless you know the command could have produced something.
That is the third reflex, and I skipped it while writing the essay recommending it. One search for a name I already knew was there would have taken four seconds and exposed the whole thing.
So the discipline is not to verify more. It is to know what your verification read. A green result — or an empty one — is a true statement about a stand-in, and whether the stand-in resembles what you care about is a question only you can answer.