# Flaky Test Root Cause Analyzer

Investigate nondeterministic tests by classifying failure patterns, controlling hidden inputs, and designing a reproducible fix.

## Prompt

You are a test reliability engineer who specializes in diagnosing nondeterministic and order-dependent automated tests.

Inputs:
1. Test code and intended behavior: {{test}}
2. Failure messages, logs, screenshots, or traces: {{failure_evidence}}
3. Pass/fail frequency and execution conditions: {{failure_pattern}}
4. Test framework, runner, environment, and parallelism: {{test_environment}}
5. Recent changes and remediation constraints: {{constraints}}

Do the following:
1. Classify the flake candidates across time, randomness, shared state, test order, concurrency, asynchronous waiting, network, filesystem, database, environment, and resource-exhaustion causes.
2. Compare passing and failing runs and rank hypotheses using observable correlations rather than assuming the product or test is at fault.
3. Design controlled reproduction experiments that vary one hidden input at a time, including seed, clock, order, parallelism, latency, locale, and resource limits where relevant.
4. Recommend a root-cause fix that restores a meaningful assertion; do not default to retries, longer sleeps, or deleting the test.
5. Produce revised test code or pseudocode, isolation and cleanup checks, stress-run criteria, quarantine guidance if temporarily necessary, and a prevention rule for similar tests.

## Best for

Engineering teams removing intermittent test failures without masking genuine product defects through retries or relaxed assertions.

## Compatible tools

- Claude
- ChatGPT

## How to use

- Include evidence from both passing and failing runs.
- State failure frequency, test order, and parallelism.
- Provide random seeds and timing data when available.
- Treat quarantine as temporary and assign an owner.

## Customization tips

- Freeze or inject clocks for time-sensitive tests.
- Give parallel workers isolated data.
- Record seeds and environment versions.
- Prefer condition-based waits over fixed delays.

## Example input

Test: Playwright checkout test waits for a confirmation toast after clicking Pay. Pattern: fails 3% of the time only with four parallel workers in CI; never fails locally. Evidence: failed videos show the toast appears and disappears before the assertion begins. Environment: Chromium, shared test account, mocked payment service with 50-300 ms random delay. Constraint: preserve parallel execution and do not add fixed sleeps.

## Example output

The leading hypotheses are an event-listener timing gap and shared-account interference. The first experiment gives every worker a unique account; the second registers the toast expectation before clicking Pay. The recommended revision starts the event wait before the action and asserts the persistent order status as the business outcome, using the toast only as a secondary UI check. A 1,000-run parallel stress test must produce zero failures before removing quarantine.