What is a positive control in software verification?

A signal the system would necessarily produce if the path under investigation had been taken: a log line before the send, a batch id, a connection attempt. Without it, an empty log does not distinguish "it didn't happen" from "I never got there".

Why isn't a zero or a silence enough as proof?

Because they have two possible causes with the same output: the event didn't happen, or the mechanism that would detect it didn't run. A skipped test, a counter that is never incremented, a command whose exit code is swallowed by a pipe and a dashboard that stopped being fed all produce exactly the same result as the healthy situation.

Why look again after the first success?

Because you investigate when something fails, and the behaviour that matters usually only appears when the new code runs end to end. The first green is the most dangerous moment in an integration: it is when everything finally executes and nobody is watching, because it has already been filed as solved.

How does this apply to designing a monitor or a watcher?

By making it speak every cycle even when it finds nothing: "412 entries checked, 0 violations" instead of silence. A process that only speaks when there is a problem is indistinguishable from a dead process, and silence reads as good news. Same for a suite reporting "0 failures" without saying how many tests it ran: zero out of zero is green.

I said no data was leaving. On the first good run, two records left

I was asked whether the system was sending patient data to an external body while the integration was half-built. I went and read the logs of every run. They all died early: some with a 415 because the content type wasn't what the other end expected, others with a 500. Not one showed an outbound call.

I answered that nothing was going out.

The first run that got past the 500 sent two requests carrying real clinical data.

My answer had been false from the start, and the worst part is that it was false in a way that felt rigorous: I had looked. I had evidence. The evidence was logs of real executions, not assumptions.

A negative says nothing on its own

The mistake wasn't misreading the logs. It was not noticing what produced that silence.

The runs died before reaching the code that sends. The log didn't say "I didn't send"; it said "I never got to the part that sends". Those are two different statements and they produce exactly the same output: nothing.

That's the general shape of the problem, and it turns up everywhere once you look for it:

A counter at zero can mean "it didn't happen" or "the counter was never incremented". A "not found" can mean "it doesn't exist" or "I looked in the wrong place". A green test can mean "it passed" or "it skipped itself". An exit 0 can mean "it worked" or "the command was strangled by a pipe that swallowed the exit code". A silent dashboard can mean "everything is fine" or "the process feeding it has been dead for three weeks".

In all five, the evidence is identical. And in all five, the optimistic reading is the reassuring one, so it's the one chosen without thinking.

The positive control

The fix isn't to be more suspicious. It's to demand one specific thing before accepting any negative:

Find something the log MUST show if the path was actually taken.

If the system had reached the part that sends, something would have to appear in the log: the "preparing request" line, the batch identifier, the connection attempt. Any signal that is impossible without having gone through there.

If that signal isn't there, you haven't demonstrated that it doesn't send. You've demonstrated that you don't know whether it sends. And "I don't know" is a perfectly respectable answer; "it doesn't send" was a lie.

When that witness exists, the negative starts to count, because now it does separate the two worlds: got there and didn't send, versus never got there.

And you have to look again after the first success

This is the second half, and in my case it was the expensive one.

I looked after the failures. That's the natural thing: you investigate when something goes wrong. But the behaviour I'd been asked to check only appeared when things went right, and that doesn't happen until someone fixes the 415 and the 500.

So the complete rule has two moments: check that the path executed, and check again after the first successful run, not only after the failed ones. The first green is the most dangerous moment in an integration, because that's when the new code finally runs end to end and nobody is watching: it has already been filed as solved.

Anything that can stay silent must say why it is silent

From this comes a design consequence that changed how I write anything that watches.

A process that only speaks when it finds a problem is indistinguishable from a dead process. Both produce the same silence, and silence is exactly what we read as "all good".

So a watcher that can legitimately stay quiet must emit, every cycle, why it is quiet: "412 entries checked, 0 violations" instead of saying nothing. It costs one line. And it turns "healthy and quiet" into something that looks different from "dead", which is exactly what the watcher exists to distinguish.

The same goes for tests: a suite that reports "0 failures" without saying how many tests it ran has said nothing. Zero out of zero is green.

What's left

Before claiming something does not happen, check that the code capable of doing it actually ran. If you can't check that, the answer is "cannot be determined", not "no".
Find the witness: the signal the log would have to show if the path had been taken.
Look again after the first success, not only after the failures.
Anything that can legitimately stay silent must say why it is silent, every cycle.

None of the four costs money or any appreciable time. The one I was missing was the first, and the bill was two transmissions of patient data that I had certified did not exist.