A flaky test is an automated test that produces inconsistent results, sometimes passing and sometimes failing, against the exact same code with no meaningful change in between runs. It's not a test that correctly fails because of a real bug; it's a test whose outcome depends on something unrelated to the correctness of the code it's supposed to be verifying, like timing, execution order, or environment state. The defining trait is unpredictability: rerun it enough times without touching the code and it will eventually flip, which is exactly what makes it so hard to catch with a single glance at a build report.
The reason flaky tests exist is that automated tests run inside real systems with real timing, network calls, shared state, and concurrency, and none of those things are perfectly deterministic even when the underlying application logic is correct. A test that waits a fixed half-second for a page to load will pass most of the time and fail occasionally when the network or the CI runner happens to be slower that run. Flaky tests exist not because anyone wants them, but because writing a test that accounts for every source of non-determinism in a real system takes more discipline than most test authors apply by default, especially under deadline pressure. That gap between how tests are ideally written and how they actually get written under time pressure is where most flakiness quietly accumulates.
What distinguishes a flaky test from a genuinely broken test is that a broken test fails consistently because something is actually wrong, while a flaky test fails intermittently for reasons disconnected from the code's correctness. Common root causes include race conditions where a test checks a result before an asynchronous operation has finished, reliance on shared or leftover state from a previous test run, dependence on real time or real random values, and tests that assume a specific order of execution that isn't actually guaranteed. Identifying which of these is causing a specific test's flakiness is usually the hardest part of fixing it, since the failure often looks identical on the surface regardless of which underlying cause produced it.
By 2026, flaky tests remain one of the most common complaints engineering teams have about their test suites, and if anything the problem has grown alongside the shift to distributed systems, microservices, and cloud infrastructure, all of which introduce more asynchronous behavior and more sources of timing variability than a single monolithic application typically has. Tooling has improved for detecting flaky tests automatically, usually by rerunning failed tests and flagging ones with an inconsistent history, but detection alone doesn't fix the underlying causes, which still require someone to dig in and address them directly. Automated flakiness detection has made the problem more visible, which is progress, but visibility and remediation are two different things, and plenty of teams still struggle with the second part.
This page covers what makes a test flaky, the common root causes behind it, how flaky tests differ from tests that are simply broken, why flakiness is corrosive to a team's trust in its test suite even when no individual flaky test seems that important, and how a team systematically finds, prevents, and fixes flaky tests rather than just living with them. The durable idea underneath it is that a test suite is only valuable if its results are trusted, and a single unpredictable test corrodes that trust for the entire suite, not just for itself, which is why the fix for even one flaky test is worth more than its size would suggest.
Race conditions are probably the single most common source of flakiness, especially in tests involving anything asynchronous: a UI action that triggers a network call, a background job that needs to finish before a result can be checked, or a database write that hasn't been fully committed when the test tries to read it back. A test that checks for a result immediately, or after a fixed short wait, will usually pass because most of the time the operation finishes quickly, but it will occasionally fail when that operation happens to take slightly longer than usual, which has nothing to do with whether the underlying feature actually works. These failures tend to cluster around periods when the underlying system is under heavier load than usual, which is exactly when a test suite's results matter most and can least afford to be second-guessed.
Shared or leftover state between tests is another major cause, particularly in suites that don't carefully isolate each test's data. If one test creates a record in a database and a later test assumes that record doesn't exist, or if two tests running in parallel both modify the same shared resource, the outcome can depend on the order tests happen to run in or whether they're executed in parallel that particular time. This kind of flakiness is especially frustrating because a test can pass reliably in isolation and only fail intermittently when run as part of a larger suite, which makes it hard to reproduce on demand. A developer trying to debug it by running the single failing test on their own machine often can't reproduce the failure at all, since the conditions that triggered it only show up when the full suite runs together.
Reliance on real time, real randomness, or real external services introduces flakiness that's directly tied to things outside the test's control. A test that checks behavior around midnight, or one that calls a live third-party API instead of a mock, is at the mercy of whatever that external factor happens to be doing at the moment the test runs. These tests often pass for long stretches and then fail in clusters when the external dependency has a bad day, which can make root-causing them confusing if nobody remembers that a live dependency was ever part of the equation. A test that's passed reliably for six months and suddenly starts failing is a strong hint to check whether something it depends on outside the codebase has changed.
Execution order dependencies round out the common causes: a test that only passes because an earlier test happened to leave the system in a particular state, without that dependency being explicit or guaranteed. These are especially dangerous because most test runners don't guarantee execution order will stay the same forever, particularly once a team introduces parallel test execution to speed up a growing suite, and a test suite that worked reliably in serial can suddenly start flaking once parallelization exposes hidden order dependencies that were always there but never triggered. Teams that adopt parallel test execution purely for speed sometimes get an unpleasant surprise here, discovering hidden dependencies they never knew existed because the tests had simply never run in any order but the original one.
The practical distinction matters because the fix is completely different depending on which one you're dealing with. A genuinely broken test fails consistently because the code it's testing has an actual defect, or because the test itself has a bug, like an incorrect assertion or outdated expected value. The fix is straightforward in principle: find the discrepancy between what the test expects and what the code does, and correct whichever one is wrong. That straightforwardness is exactly what a flaky test lacks, since there's no single, reproducible discrepancy to point at in the first place.
A flaky test, by contrast, might pass ninety-five times out of a hundred runs against identical code, which makes the discrepancy much harder to pin down because there's no consistent failure to compare against a consistent cause. Debugging a flaky test often requires deliberately rerunning it many times, sometimes under artificially adverse conditions like a slowed-down network or a heavily loaded CI machine, just to reproduce the failure often enough to observe what's actually going wrong.
Confusing the two categories leads teams astray in both directions. Treating a genuinely broken test as merely flaky, and just rerunning it until it passes, lets a real defect slip through, since the "fix" of rerunning does nothing to address whatever's actually wrong with the code. Treating a flaky test as genuinely broken, and spending hours trying to find a deterministic bug in code that's actually fine, wastes investigation time chasing a defect that doesn't exist, when the real problem is in the test's assumptions about timing or state.
The most reliable way to tell them apart is history: a test with a failure rate that hovers around some percentage over many runs against unchanged code is flaky; a test that fails every single time until the underlying code or test logic gets changed is broken. Teams that track test result history over time, rather than just looking at the most recent run, have a much easier time telling these two apart quickly instead of guessing.
The most serious cost of a flaky test isn't the occasional false failure itself; it's the effect that false failure has on how developers treat the entire test suite going forward. Once developers learn that a particular test, or a whole class of tests, fails intermittently for reasons unrelated to real bugs, they start reflexively rerunning failed builds rather than investigating them, and eventually they start ignoring red results altogether on the assumption that it's "probably just flaky." That assumption is sometimes wrong, and a real bug slips through wearing the same red X as a hundred false alarms before it. By the time anyone notices, the bug has often already reached customers, at which point the cost of the original flakiness has multiplied many times over.
This erosion of trust compounds over time in a way that's hard to reverse. A test suite that's ninety-five percent reliable feels, in practice, much worse than ninety-five percent as good as a fully reliable suite, because a handful of flaky tests scattered across a large suite means almost every build has at least one red result that needs to be triaged, and triaging red results that turn out to be nothing, repeatedly, teaches people to stop caring about red results at all. The damage isn't proportional to the flaky test count; it's driven by how often any flaky test shows up in any given run, which is why even a small number of frequently run, frequently flaky tests can do more damage to trust than a much larger number of tests that flake rarely.
Flaky tests also cost real engineering time directly, independent of the trust problem. Someone has to notice the failure, investigate whether it's a real issue, often rerun the build to check, and eventually either shrug it off or file a ticket to look into it later, a ticket that competes for attention with everything else on the backlog and often loses. Multiply this across a team running dozens of builds a day and the cumulative time spent triaging false failures becomes a real, if often uncounted, drain on the team's capacity.
There's also a slower, more insidious cost: flaky tests that get quietly disabled or skipped, rather than fixed, to stop the noise, without anyone tracking that the disabling happened. Over months, a test suite can accumulate a growing number of silently disabled tests, each one representing coverage that everyone assumes still exists because the test file is still sitting in the repository, even though it hasn't actually run in a meaningful way for a long time. This creates a false sense of security about coverage that's more dangerous than having no test at all, because at least the absence of a test is visible.
Most flaky tests are cheaper to prevent at authoring time than to root-cause months later, once they've already accumulated a history and a reputation for being ignorable. A team that adopts a short set of authoring conventions, always wait for an explicit condition rather than a fixed delay, always set up and tear down a test's own data rather than relying on what's already in the environment, always mock external services rather than calling them live, catches a large share of common flakiness sources before a test is ever merged, simply because the pattern that would have caused it never gets written in the first place.
Code review is an underused line of defense against flakiness specifically because most reviewers focus on the application logic being tested and skim past the test's own mechanics. Adding a habit of scanning new or modified tests for the known red flags, a fixed sleep call, a test that creates data without cleaning it up, an assertion checking something that depends on wall-clock time, catches a meaningful share of future flakiness for the cost of a few extra minutes per review, well before that test has run enough times in CI to build up a track record either way.
Running new tests repeatedly before merging them, sometimes called a stability check, is a more mechanical prevention technique that some teams build directly into their CI process: any newly added or modified test has to pass some number of consecutive runs, occasionally under artificially varied conditions like randomized ordering or simulated latency, before it's allowed into the main suite. This catches race conditions and order dependencies that might not show up on a single run but reveal themselves reliably after a few dozen repetitions, and it shifts the cost of finding flakiness onto the person who just wrote the test, while the context is still fresh in their head, rather than onto whoever happens to be on call when it starts failing intermittently months later.
None of these prevention techniques are complicated individually, but they require consistent discipline across a team, which is harder to sustain than it sounds, especially under release pressure when a slightly flaky-but-passing test is tempting to just merge and move on from. Teams that treat these conventions as a genuine gate rather than a suggestion tend to see meaningfully lower flake rates over time than teams that only address flakiness reactively after it's already caused problems.
The first step in dealing with flaky tests well is measurement, not intuition. Teams that rely on a general sense of "that test flakes sometimes" without tracking actual pass and fail rates over time end up guessing at which tests are the worst offenders and often prioritize the wrong ones. Test result history, ideally tracked automatically and visible to the whole team, turns flakiness from a vague complaint into a ranked list of specific tests worth fixing first, usually starting with whichever ones fail most often and touch the most business-critical functionality.
Once a specific flaky test is identified, isolating and reproducing the failure reliably is usually the hardest technical step. Running the test many times in a row, running it under load or with artificial network delay, and running it both in isolation and as part of the full suite in parallel are all standard techniques for surfacing the specific condition that triggers the failure. It's tedious work, and there's no shortcut around actually reproducing the failure a meaningful number of times before you can be confident you've found and fixed the real cause rather than a coincidental one. Rushing this step and applying a fix based on a guess rather than a confirmed reproduction is a common way teams end up "fixing" a flaky test that flakes again a few weeks later, having addressed a symptom rather than the actual cause.
Fixing the underlying cause usually falls into one of a few patterns depending on the root cause identified: replacing fixed waits with explicit waits for a specific condition to become true (rather than a flat sleep of a fixed duration), properly isolating test data so tests don't share or depend on state from one another, mocking external dependencies instead of calling real services during automated test runs, and removing implicit ordering dependencies by making any genuinely required setup explicit rather than accidental. None of these fixes are exotic, but applying them consistently across a large existing suite takes real, sustained effort.
For flaky tests that resist a quick fix, quarantining them, meaning moving them out of the main suite so their failures no longer block builds or erode trust in the rest of the suite, while tracking them separately for a dedicated root-cause effort, is often more productive than leaving them in place to keep poisoning developer trust in the meantime. Quarantining isn't the same as ignoring; a quarantined test should still have an owner and a deadline to either get fixed or get deleted, since a permanently quarantined test with no plan is just a slower, quieter version of the same trust problem it was meant to solve.
A flaky test is an automated test that produces inconsistent pass or fail results against the exact same code, with the outcome depending on something unrelated to the actual correctness of the code being tested, like timing or shared state. Rerunning it enough times against unchanged code will eventually reveal the inconsistency.
Common causes include race conditions with asynchronous operations, shared or leftover state between tests, reliance on real time or randomness, dependence on live external services, and unguaranteed execution order between tests. Most flaky tests trace back to one of these five patterns rather than something more exotic.
A broken test fails consistently because of a real defect in the code or the test itself, while a flaky test passes and fails inconsistently on unchanged code, which makes it harder to diagnose using a single failed run. Looking at a test's failure history over many runs, not just the latest one, is usually the fastest way to tell them apart.
Because they erode developer trust in the entire test suite; once people learn a test fails for reasons unrelated to real bugs, they start ignoring red results generally, which lets genuine failures slip through unnoticed. The damage compounds over time as more of the suite gets treated as background noise.
No. Rerunning can get a build to pass for that one instance, but it doesn't address the underlying race condition, shared state, or timing issue causing the inconsistency, so the same test will likely flake again later. Treating a rerun as a fix rather than a workaround is one of the most common mistakes teams make.
By tracking pass and fail history over time rather than relying on impression, which turns flakiness into a ranked, measurable list rather than a vague sense that "some tests are unreliable." That ranked list makes it much easier to prioritize fixing the worst offenders first.
Replacing a fixed-duration wait with an explicit wait for a specific condition to be true is one of the most common fixes, since it directly addresses the race-condition pattern behind a large share of flaky tests. It's also usually one of the cheaper fixes to apply once the root cause has been correctly identified.
Not immediately. Quarantining it, meaning removing it from the main build gate while assigning an owner and a deadline to root-cause or replace it, is usually better than an unowned deletion or an indefinite disable. Deleting outright should be reserved for tests that turn out to no longer be testing anything meaningful.
Not always. Sometimes a flaky test is correctly surfacing a real, intermittent bug in the application itself, such as an actual race condition in production code, rather than being a problem with the test's own logic. Dismissing every flaky result as a test problem risks missing a genuine, if inconsistent, defect worth investigating.
