Production Agent Design#

Core Principle#

The LLM is the reasoning engine. Your code is the execution engine. The loop is the contract between them.

Every production concern — safety, cost, retries, logging, permissions — lives in the harness, not the prompt. A prompt that says "be careful with deletions" is a suggestion. A GuardedToolNode that intercepts delete_* calls is a guarantee.

When to Use This Skill#

• Designing a new multi-agent system from scratch
• Adding safety, cost controls, or observability to an existing agent
• Debugging runaway cost, infinite loops, or context window exhaustion
• Choosing between single-agent vs multi-agent topology
• Implementing human-in-the-loop (HITL) for irreversible actions
• Setting up session persistence and resumption

Architecture at a Glance#

text

INGRESS (HTTP / CLI / Webhook / Schedule)
    │
ROUTER LAYER          — classify intent, dispatch cheaply
    │
ORCHESTRATOR          — decompose tasks, delegate to specialists
    ├── Agent A (scoped tools)
    └── Agent B (scoped tools)
         │
TOOL LAYER            — validate schema → check permission → execute → truncate
         │
CROSS-CUTTING CONCERNS
    ├── MEMORY         (short-term / working / long-term)
    ├── OBSERVABILITY  (traces, cost, session replay)
    └── RESILIENCE     (retry, circuit breaker, loop guard)
         │
PERSISTENCE           — checkpoints (Redis / Postgres) + audit log

Single Agent vs Multi-Agent#

text

Task scoped to ONE domain?
  YES → Single ReAct agent with appropriate tools
  NO  → Independent subtasks?
          YES → Parallel multi-agent (supervisor + specialists)
          NO  → Sequential / hierarchical orchestrator
                  │
              Any irreversible step requiring human review?
                YES → Plan-then-execute with HITL interrupt
                NO  → Orchestrator with auto-delegation

Rule: Start with a single agent. Add multi-agent complexity only when you hit a concrete limit — context window size, tool set sprawl, latency, or accuracy.

Framework Selection#

NeedUseComplex branching, HITL, durable persistence, fine-grained controlLangGraphSimple loop, minimal boilerplate, rapid prototype, leaf agentsStrandsOrchestration graph + simple leaf agentsLangGraph + Strands hybrid

Reference Files#

Load these on demand using the triggers listed below. Do not load all of them upfront.

FileLoad when...references/router-layer.mdDesigning intent routing, building a classifier node, handling misroutingreferences/orchestrator-layer.mdDecomposing tasks, spawning subagents, implementing plan-then-executereferences/tool-safety-layer.mdDesigning tools, adding permission rules, implementing HITL or killswitchreferences/memory-layer.mdContext window approaching limit, adding long-term memory, injecting project contextreferences/observability-layer.mdAdding tracing, tracking token cost, debugging agent behavior, setting up alertsreferences/resilience-layer.mdAdding retry logic, circuit breakers, preventing infinite loopsreferences/persistence-layer.mdChoosing a checkpointer, implementing session resume, session branchingreferences/production-checklist.mdBefore deploying to production — full ~40-point readiness checklist

Quick Reference#

PatternKey implementationReferenceIntent routingconditional_edges + confidence thresholdrouter-layer.mdScoped subagentscreate_react_agent with tool subsetorchestrator-layer.mdPlan-then-executeTwo nodes, read-only tools in plan phaseorchestrator-layer.mdTool schemaargs_schema=PydanticModel on @tool``tool-safety-layer.mdPermission guardGuardedToolNode with PermissionRule listtool-safety-layer.mdHITL interruptinterrupt() + Command(resume=...)``tool-safety-layer.mdRuntime concurrencyis_concurrency_safe(input) per tool calltool-safety-layer.mdAbort hierarchyQuery-level abort + sibling-level child aborttool-safety-layer.mdTiered compactionbudget → snip → microcompact → autocompactmemory-layer.mdAuto-compactionSummarization node at 80% contextmemory-layer.mdContext injectionAGENT.md loaded into system promptmemory-layer.mdFull traceBaseCallbackHandler + structured eventsobservability-layer.mdCost trackingPer-turn token accounting in callbackobservability-layer.mdConfig snapshotFreeze all feature flags at query entryobservability-layer.mdDiminishing returnsTrack token deltas; stop if delta < 500 × 2resilience-layer.mdOutput limit escalationEscalate to 64k tokens before compactionresilience-layer.mdStreaming cleanupTombstone partial messages on fallbackresilience-layer.mdError-as-observationtry/except → ToolMessage``resilience-layer.mdCircuit breakerState machine wrapping tool fnresilience-layer.mdSession resumeCheckpointer + stable thread_id``persistence-layer.md

Gotchas#

• Safety rules must be code, not prompts. A prompt saying "don't delete production data" is not a safety control.
• Never dump the full parent message history into a subagent. Pass only the specific task and relevant data — context pollution degrades performance and wastes tokens.
• InMemorySaver is for development only. Use Redis or Postgres checkpointers in production.
• interrupt() pauses the graph. Resume it by calling graph.invoke(Command(resume=...), config=config) — forgetting this leaves the agent stuck.
• Tool result truncation is mandatory. Large tool outputs (file reads, search results) will exhaust the context window if not truncated before returning.
• Always set max_iterations. Without a loop guard, a miscalibrated agent runs indefinitely and incurs unbounded cost.
• Apply compaction in tiers. Budget tool results → snip → microcompact → autocompact. Jumping straight to full summarization wastes tokens when a cheaper step would suffice.
• Track diminishing returns, not just token budget. An agent can burn through its iteration budget producing nearly empty continuations. Stop when the last 2 deltas are both below ~500 tokens.
• Snapshot config at query entry. Never re-read feature flags or env vars mid-turn — a remote config change during a 30-second response causes inconsistent behavior within a single turn.
• Concurrency safety must be checked at runtime. Schema metadata cannot determine if a bash command is safe — inspect the actual input string at call time. Fail conservatively (serial) if parsing fails.