Agentic testing is real: systems that decide what to test, generate the tests from plain-language stories, run them, and analyze the results with little human input. The pitch is genuine. The trap is quieter. Pointing an agent at an unhealthy suite does not fix the suite, it scales the mess. An agent that decides what to test needs a signal it can trust, and if a quarter of your builds flake, the agent is learning from noise. This checklist scores whether your suite is ready before you turn one loose.
What happens by default: you point the agent at the app and tell it to maximize coverage. Thousands of generated tests arrive with no priority and no risk weighting. Self-healing quietly rewrites assertions and masks real regressions. Flake gets absorbed into the pile until nobody can find the signal, and maintenance cost explodes because the agent generates faster than humans can prune.
What good quality engineering does: you give the agent a risk model, so what breaks worst gets tested first. Coverage is earned against risk, not chased as a raw percentage. Self-healing is scoped to locators and never to assertions, flake is controlled before the agent arrives, and a human owns the oracle and the merge. The agent proposes, a person decides.
Agentic testing rests on a signal the agent can believe. That means stable, seedable test data and isolated environments that reset cleanly between runs, plus a measured flake rate with the worst offenders quarantined and retries logged as policy, not reflex. If the agent cannot tell a real failure from a flaky one, neither of you can, and everything downstream inherits the noise.
The agent needs to know where to spend and what “correct” means. Map the critical user journeys, rank areas by blast radius and change frequency, and decide what “enough” coverage looks like per area. Then fix the oracle: assertions that check behavior rather than pixels, a defined source of truth per behavior, and self-healing scoped to locators so a test can never quietly bless a regression.
Autonomy without bounds is a liability. Give the agent bounded permissions, route generated tests through human review, log every decision, and keep a kill switch you can hit in one step. Then name an owner and be explicit that QA is moving from writing tests to designing oracles, modeling risk, and reviewing agent output. The judgment is where the value moves.
An agent that decides what to test cannot separate a real failure from a flaky one if you cannot either. Measure the flake rate, quarantine the worst offenders, and make retries a logged policy instead of a habit. Trustworthy signal is the precondition for everything else.
List the critical user journeys and rank areas by blast radius and change frequency. This is the map the agent follows. Without it, the agent optimizes for a coverage number and tests the trivial as hard as the critical.
Decide what “correct” means for each behavior and make assertions check behavior, not exact-match snapshots. Scope self-healing to locators only. An agent that can rewrite its own assertions can silently approve a regression.
Generated tests enter through review. The agent proposes, a person decides. This one guardrail stops the quiet accumulation of junk, duplicates, and tests that pass for the wrong reason.
One team owns the agentic system and its output quality. Be explicit that QA is shifting from authoring tests to designing oracles, modeling risk, and reviewing agent work. The headcount stays relevant because the judgment does.
On a healthy suite with trustworthy signal and a risk model, an agent is a force multiplier. On a suite that already spends half its budget on maintenance and lets a quarter of its builds flake, the agent multiplies the bill. A 12% flake rate on a 500-minute-per-day pipeline for a 20-engineer team runs to roughly $120k a year, and each flaky end-to-end test costs about 3.7 engineering hours to fix. Score yourself honestly, fix the weakest dimension first, then let the agent test the things that matter.
Volume is not coverage. Thousands of tests with no risk weighting bury the signal and blow up maintenance, and your readiness is capped by your weakest dimension, not your average. A smaller suite aimed by a risk model catches more of what matters than a huge one that treats the trivial like the critical.
On teams that already have trustworthy signal, mapped critical journeys, and a human on the merge. In that setting the agent takes over the repetitive authoring and triage so people move up to oracle design and risk modeling, which is where quality signal actually comes from.
Directors of QA, QA leads, and test automation owners deciding whether and how to introduce agentic or self-healing testing, and who need an honest readiness verdict before they commit.
Only if you scope it correctly. Self-healing is safe for locators, updating a selector when the UI moves. It is dangerous for assertions, because a test that rewrites what “correct” means can approve a regression and never tell you. Allow the first, review the second.
Score the six dimensions, one point per checked box, out of thirty, and find your band. Under 13 you are not ready and should fix fundamentals first. 13 to 19 supports a narrow, fenced pilot. 20 to 25 supports rollout. Above 25 you can push the frontier. Fix the lowest-scoring dimension before you widen scope.
