# Prompt A/B Variant Generator

Generates meaningfully different variants of a prompt for systematic A/B testing, each changing one specific variable so you can isolate what actually improves output quality.

## Prompt

You are a prompt engineering expert who designs systematic A/B tests for prompts, isolating one variable at a time so results are actually interpretable.

Baseline prompt: {{baseline_prompt}}
What you're trying to improve (e.g., accuracy, creativity, conciseness, consistency): {{improvement_goal}}
What you suspect might be causing the current limitation (if you have a hypothesis): {{hypothesis}}

Do the following:
1. Identify 3 distinct variables in the prompt that could plausibly be changed to affect the stated improvement goal (e.g., instruction ordering, level of specificity, use of examples, persona framing, output format constraints).
2. For each variable, create one variant prompt that changes ONLY that variable, keeping everything else identical to the baseline, so any difference in output can be attributed to that specific change.
3. Label each variant clearly (Variant A: [variable changed], Variant B: [variable changed], Variant C: [variable changed]) with a one-sentence hypothesis for why this change might help.
4. Suggest a simple test methodology: how many times to run each variant against the same set of test inputs, and what to look for when comparing outputs (specific, not just 'see which is better').
5. Note which variant you'd predict is most likely to improve the stated goal, and why, while being clear this is a hypothesis to test, not a guaranteed result.

## Best for

Prompt engineers and developers who want to systematically test prompt changes instead of randomly tweaking prompts and guessing which change actually caused an improvement.

## Compatible tools

- Claude
- ChatGPT

## How to use

- Paste your baseline prompt along with what specific aspect of output quality you're trying to improve.
- Share any hypothesis you already have about what's causing the current limitation.
- Run each variant against the same set of test inputs using the suggested methodology.
- Compare results specifically against the suggested comparison criteria, not just a general 'which looks better' impression.

## Customization tips

- If you have real failure examples from your baseline prompt, include 1-2 so the generated hypotheses are grounded in actual observed problems.
- For high-stakes prompts, ask for a larger test set size recommendation (e.g., 20-30 runs per variant) for more statistically meaningful results.
- Once you've identified a winning variant, run this prompt again using the winner as the new baseline to keep iterating systematically.

## Example input

Baseline prompt: 'Summarize this article in 3 sentences.' Improvement goal: Accuracy (summaries sometimes miss the main point). Hypothesis: Maybe the model needs to identify the main argument before summarizing.

## Example output

3 variables identified: instruction ordering (add a reasoning step before summarizing), specificity (define what 'main point' means explicitly), and example inclusion (add one example summary). Variant A adds 'First identify the article's core argument, then summarize it in 3 sentences,' Variant B adds an explicit definition of what to prioritize (main claim over supporting details), Variant C adds one example input/output pair, each with a hypothesis, a suggested methodology to run each variant against the same 10 test articles and compare whether the core argument survives into the summary, and a prediction that Variant A is most likely to help since it directly addresses the reasoning gap suggested by the observed failure pattern.