# Evaluator-Optimizer Refinement Loop

> One model generates, another grades against a rubric, and the loop repeats until the rubric passes or a limit is hit.

Category: AI Agent Workflows  
Tags: reflection, critique, quality  
Setup time (estimate): 30-60 minutes  
Estimated cost: Each iteration is one generation plus one evaluation call; cap iterations.  
Last verified: not yet independently verified

Useful when quality criteria are clear and checkable (translations, code that must pass tests, copy with constraints).

## Steps

1. Generator produces a draft.
2. Evaluator scores it against an explicit rubric and returns structured feedback.
3. If `pass` is false, feed the feedback to the generator and regenerate.
4. Stop at `pass: true` or after N rounds; return the best-scoring draft.

## Evaluator prompt

```text
You are a strict reviewer. Grade the DRAFT against each criterion.
Return JSON only:
{"scores":{"accuracy":0-5,"constraints":0-5,"clarity":0-5},
 "pass": true|false,
 "fixes": ["specific, actionable change", ...]}
pass is true only if every score >= 4.
CRITERIA: {{rubric}}
DRAFT: {{draft}}
```

## Config

```yaml
max_rounds: 3
keep: best_score
prefer_programmatic_checks: true   # run tests/linters before the LLM judge
```

## Failure modes

- Evaluator and generator share blind spots when they are the same model.
- Loops oscillate without converging; enforce a max-iteration limit.
- Vague rubrics give inconsistent scores; use concrete pass/fail criteria.