Self-Refine Reflection Skill#

An AI agent skill for systematic self-reflection and iterative output refinement. Based on Madaan et al. (2023), Shinn et al. (2023), and Andrew Ng's Reflection design pattern.

Platform Auto-Detection#

At skill load time, detect your runtime environment and adjust capabilities:

CapabilityHow to CheckFallbackFile systemCan you read references/reflection-templates.md?Use the inline templates below insteadPersistent memoryCan you write to memory/?Store reflection notes in conversation context onlyLong contextIs your context window > 32K tokens?Cap at Level 2 (skip adversarial review)Tool accessCan you call external tools?Use mental verification only

Detection rules:

• If you can read this file's references/ directory → full mode (all levels + memory)
• If you can read files but not write → full levels, in-conversation memory only
• If you cannot read files at all → use inline templates (copied below), cap at Level 2
• If context is limited (< 8K usable) → default to Level 1, max Level 2

This means every platform gets the best possible experience automatically — no manual configuration needed.

Inline Reflection Templates (for environments without file access)#

If you cannot read references/reflection-templates.md, use these directly:

Level 1 Internal Prompt#

text

Review drafted response: (1) Did I answer everything? (2) Logic gaps? (3) Can I cut 20%? (4) User can act on this? Fix → Deliver.

Level 2 Internal Prompt#

text

For each dimension (Logic / Facts / Completeness / Conciseness / Actionability / Consistency):
  - Quote exact sentences with issues
  - Rate: Critical / Minor / Pass
Fix ALL Critical. Fix Minor if straightforward. Re-read once. Deliver.

Level 3 Internal Prompt#

text

Round 1: Level 2 review.
Round 2: "If a domain expert attacked this, what would they target?" Fix valid attacks.
Round 3: Did fixes introduce new issues? If stable → deliver. If not → fix and deliver.

When to Activate#

Activate when you are about to deliver a final response to the user, after you've gathered all information and formed your answer. Reflection happens before delivery, not instead of work.

Core Loop: GENERATE → CRITIQUE → REFINE → CHECK#

text

1. GENERATE  — Produce your initial response as normal
2. CRITIQUE  — Apply the reflection framework (depth-dependent)
3. REFINE    — Fix every issue found in CRITIQUE
4. CHECK     — If issues remain AND within budget, loop to step 2
              — If clear OR budget exhausted, deliver

Source: Madaan et al., "Self-Refine: Iterative Refinement with Self-Feedback" (2023) — the same LLM acts as generator, critic, and refiner.

---

Reflection Dimensions#

Every critique examines the response across these dimensions. Not all apply to every response — skip irrelevant ones silently.

#DimensionWhat to CheckSignal Words1Logical CompletenessDoes the reasoning chain have gaps? Are conclusions supported by premises?"therefore" without prior evidence, jumps in logic2Factual AccuracyAre there unverified assertions? Claims that could be wrong?Specific numbers, dates, names, "everyone knows..."3Response CompletenessDid I address every part of the user's question? Are there sub-questions I ignored?Multiple questions in user message, implicit needs4ConcisenessIs there redundancy? Can paragraphs be merged? Are there filler phrases?"It's important to note that", "In conclusion", repeated points5ActionabilityCan the user act on this immediately? Or do they need to ask follow-ups?Vague advice without steps, missing specifics6Internal ConsistencyDo any parts of my response contradict each other?Conflicting recommendations, contradictory statements

---

Reflection Depth Levels#

The depth is determined by task complexity, not user preference. The agent auto-selects.

Level 0: Quick Scan (Skip formal reflection)#

Trigger: Simple factual lookups, greetings, trivial questions, single-sentence answers.

Action: Do nothing extra. Just respond. Cost: 0 additional tokens.

Examples: "What time is it?", "Thanks", simple formatting requests.

Level 1: Standard Review#

Trigger: Medium-complexity tasks — explanations, how-to guides, code snippets, multi-paragraph responses.

Action: One pass through all 6 dimensions mentally. Fix issues. Deliver.

Budget: 1 refinement round. ~15-20% overhead on response tokens.

Internal process:

text

After drafting your response, ask yourself:
- Did I answer everything they asked?
- Any logical gaps or contradictions?
- Can I cut 20% of the words without losing meaning?
- Would the user know exactly what to do next?
Fix → Deliver.

Level 2: Deep Audit#

Trigger: Complex tasks — technical architectures, multi-step plans, research summaries, anything with 5+ distinct claims.

Action: Explicit dimension-by-dimension review. One full refinement round with documented issues.

Budget: Up to 2 refinement rounds. ~30% overhead.

Internal process:

text

Draft response.
For each dimension (1-6):
  - Identify specific issues (quote the exact sentence)
  - Rate severity: Critical / Minor / Pass
If any Critical: refine entire response, then re-check
If only Minor: fix inline, deliver

Level 3: Adversarial Review#

Trigger: High-stakes tasks — production code, security decisions, medical/legal adjacent, public-facing content, or user explicitly requests thorough review.

Action: Full audit PLUS an adversarial pass where you actively try to find the weakest point in your response.

Budget: Up to 3 refinement rounds. ~50% overhead.

Internal process:

text

Draft response.
Round 1: Standard dimension-by-dimension review.
Round 2: Adversarial pass — "If I wanted to prove this response wrong,
         what would I attack?" Check those attack vectors.
Round 3 (if needed): Verify fixes from Round 2 didn't introduce new issues.

---

Convergence Rules#

Based on the empirical finding from Madaan et al. that "most gains are in the initial iterations":

1. Maximum 3 refinement rounds regardless of depth level.
2. Stop early if no Critical or Minor issues found in a round.
3. Stop early if the changes between rounds are purely stylistic (no substantive improvement).
4. Diminishing returns rule: If Round N fixes fewer issues than Round N-1, stop after N.

Anti-pattern — DO NOT:

• Refine forever trying to reach perfection
• Make changes just to justify another round
• Re-introduce previously fixed issues

---

Cost Control Strategy#

DepthMax RoundsApprox. Token OverheadWhen to UseLevel 000%Simple Q&ALevel 1115%Most conversationsLevel 2230%Complex technicalLevel 33~50%High-stakes only

Principle: Reflection should cost less than the cost of delivering a wrong answer. For low-stakes responses, skip reflection entirely.

---

Trigger Conditions Summary#

Auto-Trigger (always on)#

• Response exceeds 3 paragraphs → at least Level 1
• Response makes 3+ factual claims → at least Level 1
• Response includes code → at least Level 1 (check logic + actionability)

Skip Reflection#

• User is in a hurry (explicit: "quick", "brief", "just tell me")
• Response is under 2 sentences
• Pure social/chat exchange

User Manual Trigger#

• User says "think carefully", "double-check", "make sure this is right" → Level 2+
• User says "this is important", "critical", "production" → Level 3
• User says "reflect" or "self-refine" → Level 2+

---

Output Format#

Reflection is internal — the user should not see the raw critique. However:

After refinement, you MAY append a subtle note:#

Level 1: No note (keep it invisible). Level 2: Optionally: _(response refined via self-review)_ Level 3: Optionally: _(refined through [N]-round adversarial self-review)_

NEVER:#

• Show the actual critique/feedback text to the user
• Make the note prominent or distracting
• Add notes for Level 0 or Level 1 responses

---

Reflexion Memory (Cross-Session Learning)#

Inspired by Shinn et al. (2023) — Reflexion stores verbal self-reflections in persistent memory for future tasks.

When to use: After completing a Level 2 or Level 3 reflection where you discovered a significant pattern in your own errors (e.g., "I tend to forget edge cases in X", "I consistently over-explain Y").

How to use: Write a brief note to memory/ capturing the pattern:

text

Self-reflection: When writing about [topic], I tend to [pattern].
Fix: [specific behavior change].
Date: [today].

This creates a growing corpus of self-corrections that improve future responses.

---

Relationship to Other Reflection Techniques#

This skill synthesizes multiple academic approaches:

TechniqueSourceWhat We BorrowSelf-RefineMadaan et al. 2023Core GENERATE→CRITIQUE→REFINE loopReflexionShinn et al. 2023Persistent memory of self-reflectionsChain-of-VerificationDhuliawala et al. 2023Verification questions for factual claims (Level 2-3)Self-CalibrationKadavath et al. 2022Confidence assessment of own outputsAndrew Ng's ReflectionNg 2024Design pattern: agent critiques and improves its own output iterativelyCRITICGou et al. 2023Tool-interactive critiquing (use tools to verify when possible)

---

Quick Reference Card#

text

REFLECT? ──→ Simple/trivial? ──→ NO → Just respond
    │
    YES
    │
    ├─ Medium complexity → LEVEL 1 (1 round, mental check)
    ├─ Complex / multi-claim → LEVEL 2 (2 rounds, explicit review)
    └─ High-stakes / user requested → LEVEL 3 (3 rounds, adversarial)

For each round:
  1. Check 6 dimensions
  2. Fix all issues
  3. Converged? → Deliver
  4. Budget left? → Next round
  5. Budget exhausted → Deliver current version