Validation Is a Loop, Not an Assertion: Why Your AI Agent Reports Success It Never Achieved

The most common production AI agent failure is treating validation as an assertion, a one-shot pass/fail check, instead of a loop that validates, scores, and decides whether to accept, retry, or escalate. Deterministic code can assert. Non-deterministic model output needs a closed loop, or your agent will report success while doing the wrong thing.

An agent I was reviewing was asked to rename a function across a repository. It ran for a while, made its edits, and returned a clean summary: Done. Renamed parseUser to parseAccount everywhere. The summary was confident, well formatted, and wrong. Six of eight call sites were updated. Two were not, one buried in a string that built the name dynamically and one in a test helper the search had skipped.

The agent did not lie. It returned what its model believed was true at the moment it returned. The problem is that nothing sat between I believe I succeeded and I report success. There was no step that walked back out into the world, looked at what had actually changed, and compared it to what the task asked for. The agent’s confidence and the agent’s correctness were two completely unrelated numbers, and only one of them made it into the summary.

This is the single most common way I see production agents fail, and it is almost never a model problem. It is a validation problem, and specifically it is validation built in the wrong shape.

In a deterministic system, validation is an assertion: check once, pass or fail. In a system driven by non-deterministic model output, validation has to be a loop: observe the real outcome, score how close it is, and decide whether to accept, retry, route, or stop. The most common production agent failure is treating the second case like the first.

This piece sits alongside two companions. Determinism where you can, judgement where you must is about deciding which technique owns each part of a system. A wrong ruler is worse than no ruler is about verifying the checks you trust before you let them hold authority. This piece is about the shape of the check itself, once you have decided the output is non-deterministic enough to need one.

What an assertion assumes

assert(result == expected) is one of the most useful lines in deterministic software, and it is useful because of a set of assumptions that are almost always true in that world. The operation you just ran is deterministic, so it either did the thing or it did not. The check is far cheaper than the operation, so running it once at the end costs nothing worth counting. And the answer is binary, because in a deterministic system there is no meaningful state between success and failure. You parsed the integer or you threw. You wrote the row or you got an error.

Every one of those assumptions breaks the moment the operation is a model call.

The operation is no longer deterministic, so it did the thing is now a probability, not a fact. The check is no longer trivially cheaper than the operation, because verifying whether a model actually accomplished a fuzzy goal can be almost as hard as the goal. And the answer is no longer binary, because the whole texture of model failure is partial. The rename that got six of eight. The summary that captured four of the five key points. The extraction that found every date except the one written in words. Partial success is not an edge case in these systems. It is the main case, and an assertion has no vocabulary for it. 6 of 8 and 8 of 8 collapse to the same false, which throws away exactly the information you needed.

The shape of the loop

A loop keeps the information an assertion discards. It does not ask did it pass. It asks how close did it get, and what should happen because of that. Concretely it is four moves and a stopping rule.

1. Validate by observation. After the action, look at the real, observable outcome, not the model’s report of it. For the rename, that means searching the codebase for the old and new names and seeing which call sites actually changed. The validator’s input is the world, not the transcript.
2. Score with partial credit. Turn the observation into a number or a structured verdict that can express most of the way there. Six of eight is 0.75, and that number carries the two misses that a boolean would have erased.
3. Decide against thresholds. Is the score high enough to call the task done? Low enough to abandon and report failure honestly? Or in the band between, where more work might close the gap?
4. Route the in-between case. When the score is neither clearly done nor clearly hopeless, choose the next action deliberately. Retry only the two missed call sites. Ask the user whether the dynamic-string case should even be touched. Escalate to a stronger model. The router is where the loop earns its keep, because it is the part an assertion cannot do at all.
5. Iterate to a terminal condition. Repeat until one of three things is true: the score clears the accept threshold, a retry budget is exhausted, or a human steps in. The budget matters as much as the score. A loop with no budget does not converge, it hangs.

Drawn out, the control flow is small but it is a cycle, and the cycle is the point.

flowchart LR
  A[Agent action] --> V[Validate
observe real outcome]
  V --> S[Score
partial credit]
  S --> D{Decide}
  D -->|clears accept| DONE([Report success])
  D -->|below abandon| STOP([Halt, surface partial progress])
  D -->|in between| R[Route
retry / clarify / escalate]
  R --> A

Notice where control lives. The model reasons inside this loop and it can even do the scoring when the judgement is subjective. But it does not own the loop. The decision of what counts as done, and what happens when the task is not done, stays with the orchestrator. This is the same boundary the technique piece draws: the model may reason under the control, it may not direct the control itself. An agent that gets to declare its own task finished and exit on its own authority has no loop. It has an assertion it wrote for itself and then graded.

The scoring function is the hard half

If you have designed retry-with-backoff for a flaky network call, you have already built most of this loop. Observe, decide, retry with a budget, give up gracefully. That machinery ports over almost unchanged. The part that does not port, the part that makes this genuinely a production AI problem rather than a distributed-systems problem you already solved, is the scoring function.

For the rename, scoring is easy, because the outcome is observable by deterministic means. You can enumerate call sites and count. Whenever you can score by observation like that, do, because a count does not hallucinate. The hard cases are the ones where the goal is subjective: is this summary faithful, is this explanation coherent, does this answer satisfy the brief. There the score has to come from judgement, often a model judging another model’s output, and now you have a new problem stacked on the first one.

That scorer is itself a check, and a check that can be wrong. If you let a broken scorer drive the loop, you do not merely fail to catch errors. You manufacture confident wrong verdicts and feed them straight back into the agent’s next action. A loop built on a bad ruler is worse than no loop, for exactly the reasons the companion piece lays out: a wrong check with authority does not leave you uncertain, it leaves you confidently wrong, and here it does so in a cycle that compounds. So the discipline from that piece applies in full to the scorer: verify it at least as hard as the claims it will act on, score by comparison against anchors rather than emitting a bare number, and know the ceiling of human agreement before you trust a model to beat it.

Two places the loop lives

In a real agent the loop shows up at two scales, and mature systems run both.

The first is per-step, as middleware. A validator runs after every tool call, before the model sees the result. What gets fed back into the next prompt is not the raw API response. It is the validated, scored, sometimes rewritten version of it. This is what stops a single bad step from silently poisoning everything downstream, because the model never gets to reason on top of an unchecked result. The loop here is tight and cheap and runs constantly.

The second is end-to-end, as a harness. At the boundary of the whole task, you check the full resulting state against the intended outcome and treat the score as a circuit breaker. If the end-to-end score is below threshold, the agent does not report success. It halts and surfaces the partial progress honestly: here is what changed, here is what did not, here is where it stopped. This is the layer that would have caught the rename. Per-step middleware might have passed each edit individually while the aggregate still missed two sites, and only a check against the whole goal sees that gap.

You want both because they catch different failures. Middleware catches the step that went wrong. The harness catches the task that came out wrong even though every step looked fine. Skip the middleware and errors compound before you notice. Skip the harness and you are back to an agent that reports success it never achieved, which is where we started.

The distributed-systems reframe

If all of this feels familiar, it should. It is eventual consistency wearing different clothes.

An assertion-style check on a freshly written row in a distributed store fails not because the write was wrong but because it has not propagated yet. The correct pattern was never a single assertion. It was a loop: poll until the read converges with the write, or until a timeout says stop waiting. Model output validation is the same pattern with cognitive uncertainty standing in for replication lag. The output is not yet known-good, so you observe, you score, you decide whether to wait for a better one or accept what you have, and you bound the whole thing with a budget so it terminates.

Which is why teams from a distributed-systems background tend to get this right faster once it is named. You already know that in any system where the truth arrives late and imperfectly, a one-shot check is the wrong instrument. You built loops for that years ago. An LLM is just another source where the truth arrives late and imperfectly. The retry-with-backoff instinct is half the answer. The scoring function is the half that is new.

Where an assertion is still right

The loop is not a universal upgrade, and reaching for it everywhere is its own mistake. The whole reason the loop exists is that the output is non-deterministic. Where the output is deterministic, an assertion is not just adequate, it is correct, and wrapping a deterministic step in a scoring loop adds cost and latency to buy nothing.

The schema either validates or it does not. The parser either succeeds or it throws. The row count either matches or it does not. Those are assertions, and they should stay assertions, sitting at the boundaries of your system doing exactly the one-shot job they are good at. The technique boundary is the tool for deciding which is which: put as much of the system as you can onto deterministic ground where a plain assertion holds, and reserve the loop for the parts that are genuinely non-deterministic and genuinely need judgement. An agent that scores and retries its way through a JSON schema check is making the opposite mistake to the one this piece is about, and it is just as wasteful.

The one honest question

The next time an agent hands you a confident done, ask it the question the loop asks and the assertion cannot: not did it pass, but how do you know, and what did you observe to know it. If the answer is that the model believed it, you have an assertion the model wrote for itself. If the answer is a real observation of the world, scored against the goal, with a decision that followed from the score, you have a loop. Only one of those two agents can be trusted to tell you when it failed, and it is the one that was built to look.

This piece extends two companions. The boundary it assumes: Determinism Where You Can, Judgement Where You Must. The check-verification discipline its scorer depends on: A Wrong Ruler Is Worse Than No Ruler.

I occasionally advise small teams on backend reliability, Go performance, and production AI systems. Learn more: /services