Testing AI You Cannot Fully Predict: Adapting ATF and QA for Agentic Workflows

Every QA practice you have on the platform rests on a quiet assumption: the same input produces the same output, so you can assert exact results. The Automated Test Framework is built on that assumption. Generative AI breaks it. Ask a skill the same question twice and you may get two differently-worded correct answers, or a correct one and a subtly wrong one. So the question every serious team eventually hits is real: how do you QA something non-deterministic with tooling designed for the deterministic? Here is how the practice has to adapt.

First, stop asserting exact matches on generative output and start asserting properties and ranges. You are no longer checking "did it return this exact string." You are checking "did it return a response that is grounded in the source, addresses the request, stays within the required format, and contains no prohibited content." These are property checks, not equality checks. Some are mechanically verifiable, format and schema adherence can be asserted programmatically, and those absolutely still belong in deterministic ATF. The structured scaffolding around your AI, the workflow that triggers a skill, the branching on its output, the record updates, is deterministic and should be tested in ATF exactly as you test any flow. Do not abandon ATF; aim it at the parts that are still deterministic.

For the generative core itself, the regression suite becomes an evaluation set, the labeled collection of representative inputs with reference answers we have discussed elsewhere. You run that set on every change, prompt edits, model swaps, and critically knowledge-base updates, and you score the outputs on your quality dimensions rather than diffing them against a golden string. Grounded-versus-hallucinated is the metric that matters most for knowledge skills; format adherence, completeness, and safety round it out. Where manual grading does not scale, a model-based evaluator can do first-pass scoring against your references with humans reviewing the borderline cases. The point is that "passing" means "quality stayed above threshold across the distribution," not "output byte-identical to last time."

The third piece people forget: test the guardrails and the escalation paths, not just the happy outputs. Construct adversarial inputs designed to tempt the agent across a boundary or into a prohibited action, and assert that the guardrail holds and the escalation fires every time. Those are deterministic requirements, the agent must not delete the record, must route to a human below a confidence threshold, must require approval for an irreversible action, and you should test them with the rigor of a unit test. The mental model: deterministic ATF for the scaffolding and the guardrails, evaluation sets for the generative quality, adversarial tests for the boundaries. Test all three and you can ship AI you cannot fully predict, responsibly.