LLM-as-Judge#

Core principle: Same model = same blind spots. Different model = fresh perspective. Cross-model review catches ~85% of issues vs ~60% for self-reflection.

Activation Criteria#

Use this pattern when:

• Architecture or system design decisions
• Multi-file changes affecting >5 files or >500 LOC
• Security-critical code (auth, payments, crypto/DeFi)
• Financial/trading systems (market making, quant strategies)
• Planning documents that will drive weeks of work
• Stuck after 3+ failed attempts on same problem

Skip when:

• Simple edits, config tweaks, bug fixes with obvious cause
• Documentation updates
• Single-file changes under 100 LOC
• Tasks where self-review is sufficient

The Pattern#

text

Executor (Model A) → Output → Judge (Model B) → Verdict → Action

Verdicts: APPROVE | REVISE (with specific feedback) | REJECT (restart)

Model Pairing#

Use a different provider than the executor to avoid shared blind spots:

• Executor: Claude → Judge: kimi or grok or gemini-pro
• Executor: Kimi/Gemini → Judge: opus
• Principle: Different provider, similar capability tier

Judge Prompt Templates#

Plan/Architecture Review#

See references/judge-prompts.md for full templates covering:

• Plan completeness, feasibility, risk, testing strategy
• Architecture review with scoring (0-10 per dimension)
• Code review checklist (correctness, design, safety, maintainability)

Integration Points#

• With adversarial review: This IS the formalized version of "spawn a separate model to review"
• With planning-protocol: Judge reviews the plan before the Execute phase
• With coding workflows: Code → cross-model review → fix findings → test → build → push

Quick Decision#

text

Simple task?           → Self-review
Complex / high stakes? → LLM-as-Judge
Stuck after retries?   → LLM-as-Judge (fresh perspective)
Financial/security?    → LLM-as-Judge (mandatory)

Gotchas#

• Same provider defeats the purpose — Claude Opus judging Claude Sonnet shares the same training distribution. Use a different provider (Grok judging Claude, Gemini judging GPT, etc.).
• Vague judge output is useless — If the judge says "looks good" without specifics, the prompt is too weak. Always require the judge to produce scored dimensions + specific actionable items, even if approving.
• Judge scope creep — Judges sometimes rewrite the entire plan instead of reviewing it. Constrain the verdict to APPROVE / REVISE / REJECT with specific feedback, not a replacement solution.
• Approval rate drift — If the judge approves >80% of submissions, the model pairing is too similar or the prompts are too lenient. Target 60-70% approval rate.
• Don't judge trivial tasks — A 50-line CSS fix doesn't need cross-model review. Use the activation criteria in this skill strictly.