# Agent Cost and Latency Optimizer

Reduce an agent's response time and operating cost while protecting task quality, safety controls, and critical reasoning steps.

## Prompt

You are an AI performance architect who specializes in optimizing tool-using agents for cost, latency, and reliability.

Inputs:
1. Agent workflow and task classes: {{workflow_and_tasks}}
2. Current models, prompts, tools, and routing logic: {{current_architecture}}
3. Baseline latency, token, tool-call, and cost data: {{baseline_metrics}}
4. Quality, safety, and reliability requirements: {{protected_requirements}}
5. Infrastructure and implementation constraints: {{constraints}}

Do the following:
1. Build a stage-by-stage cost and latency budget, separating model inference, retrieval, tools, orchestration, retries, and human review; identify the largest contributors and variance.
2. Segment tasks by complexity and risk, then propose routing, caching, parallelization, prompt reduction, context pruning, batching, and tool changes appropriate to each segment.
3. For every optimization, state the expected gain, implementation effort, quality or safety risk, rollback signal, and experiment needed to validate it.
4. Identify steps that must not be removed or downgraded, including authorization checks, critical verification, provenance, and high-impact reasoning.
5. Deliver a ranked optimization backlog, target architecture, experiment matrix, guardrail metrics, and a 30-day rollout plan with stop conditions.

## Best for

AI engineering teams facing high inference costs or slow agent responses that cannot afford silent quality or safety regressions.

## Compatible tools

- Claude
- ChatGPT

## How to use

- Provide measured stage-level baselines instead of estimates where possible.
- State quality and safety metrics that cannot regress.
- Separate routine, complex, and high-risk task classes.
- Run controlled experiments before changing production routing.

## Customization tips

- Include p50 and p95 values to expose tail latency.
- Add provider-specific token and cache pricing.
- Mark checks that must remain model-independent.
- Define rollback thresholds before each experiment starts.

## Example input

Workflow: A procurement agent compares supplier quotes, checks policy, requests missing details, and drafts a recommendation. Architecture: frontier model for all steps, four sequential searches, full 18,000-token policy context, and up to three retries. Baseline: median 31 seconds, p95 58 seconds, 27,000 input tokens, and EUR 0.74 per case. Requirements: recommendation agreement with reviewers above 95%, all policy citations verified, and no reduction in conflict-of-interest checks. Constraints: must use current vector store and two approved model providers.

## Example output

The budget shows policy-context loading and sequential retrieval produce 61% of cost and 46% of latency. The top experiments are section-level policy retrieval with citation verification, parallel supplier searches, a smaller model for quote normalization, and a single bounded retry based on error type. Conflict-of-interest checks and final citation validation remain on the frontier model. The plan targets EUR 0.38 median cost and 18-second latency, with automatic rollback if reviewer agreement falls below 95% or unsupported citations exceed 0.5%.