Coding & Debugging Prompts
Flaky Test Root Cause Analyzer
Investigate nondeterministic tests by classifying failure patterns, controlling hidden inputs, and designing a reproducible fix.
FreeClaudeChatGPT
Best for
Engineering teams removing intermittent test failures without masking genuine product defects through retries or relaxed assertions.
Suitable LLM groups
FrontierReasoning
Prompt
You are a test reliability engineer who specializes in diagnosing nondeterministic and order-dependent automated tests.
Inputs:
1. Test code and intended behavior: {{test}}
2. Failure messages, logs, screenshots, or traces: {{failure_evidence}}
3. Pass/fail frequency and execution conditions: {{failure_pattern}}
4. Test framework, runner, environment, and parallelism: {{test_environment}}
5. Recent changes and remediation constraints: {{constraints}}
Do the following:
1. Classify the flake candidates across time, randomness, shared state, test order, concurrency, asynchronous waiting, network, filesystem, database, environment, and resource-exhaustion causes.
2. Compare passing and failing runs and rank hypotheses using observable correlations rather than assuming the product or test is at fault.
3. Design controlled reproduction experiments that vary one hidden input at a time, including seed, clock, order, parallelism, latency, locale, and resource limits where relevant.
4. Recommend a root-cause fix that restores a meaningful assertion; do not default to retries, longer sleeps, or deleting the test.
5. Produce revised test code or pseudocode, isolation and cleanup checks, stress-run criteria, quarantine guidance if temporarily necessary, and a prevention rule for similar tests.How to use
- Include evidence from both passing and failing runs.
- State failure frequency, test order, and parallelism.
- Provide random seeds and timing data when available.
- Treat quarantine as temporary and assign an owner.
Example input
Test: Playwright checkout test waits for a confirmation toast after clicking Pay. Pattern: fails 3% of the time only with four parallel workers in CI; never fails locally. Evidence: failed videos show the toast appears and disappears before the assertion begins. Environment: Chromium, shared test account, mocked payment service with 50-300 ms random delay. Constraint: preserve parallel execution and do not add fixed sleeps.
Example output
The leading hypotheses are an event-listener timing gap and shared-account interference. The first experiment gives every worker a unique account; the second registers the toast expectation before clicking Pay. The recommended revision starts the event wait before the action and asserts the persistent order status as the business outcome, using the toast only as a secondary UI check. A 1,000-run parallel stress test must produce zero failures before removing quarantine.
Customization tips
- — Freeze or inject clocks for time-sensitive tests.
- — Give parallel workers isolated data.
- — Record seeds and environment versions.
- — Prefer condition-based waits over fixed delays.
Tags
#flaky-tests#test-reliability#nondeterminism#automated-testing#root-cause-analysis
FAQ
What is this prompt for?
It identifies hidden sources of nondeterminism and designs experiments and fixes for intermittently failing tests.
How should I customize it?
Provide test code, failure artifacts, run frequency, order, seeds, parallelism, environment versions, and recent changes.
Are there any limitations?
Rare flakes may require many runs or production-like infrastructure to reproduce, and incomplete artifacts can leave uncertainty.
How is it different from a basic prompt?
It systematically varies hidden inputs and preserves meaningful assertions instead of recommending retries, sleeps, or test deletion.
Related Prompts
All Coding & Debugging prompts →Free
Root Cause Bug Investigator
Guides a systematic investigation of a bug from symptom to root cause, avoiding quick patches that mask the real problem, and produces a targeted fix.
ClaudeChatGPT
#debugging#root-cause-analysis#bug-fixing
Free
Edge Case & Test Case Generator
Analyzes a function or feature and generates a comprehensive list of edge cases and corresponding test cases, catching scenarios developers commonly miss.
ClaudeChatGPT
#testing#edge-cases#unit-tests
Free
Production Code-Path Reconstructor
Reconstruct the code path behind a production incident by correlating requests, releases, traces, logs, configuration, and side effects.
ClaudeChatGPT
#production-debugging#incident-analysis#distributed-tracing
Related Skills
Code ImprovementFree
Debugging and Root Cause Analyst
Investigate software defects systematically using evidence, hypotheses, reproduction, instrumentation, isolation, and verification.
ClaudeChatGPTCursor
#debugging#root cause analysis#bug investigation
Related articles coming soon — visit the Learn hub.