Tool Calling (Deep Workflow)#

Tool calling is contract design between a probabilistic planner (the model) and deterministic systems. Failures are usually schema, permissions, or ambiguity—not the LLM “being dumb.”

When to Offer This Workflow#

Trigger conditions:

• Designing OpenAI/Anthropic-style functions, MCP tools, or internal JSON tool protocols
• Debugging wrong arguments, hallucinated parameters, or unsafe side effects
• Building agents with many tools—selection and routing problems

Initial offer:

Use six stages: (1) define tool surface, (2) schema & validation, (3) authz & safety, (4) execution semantics, (5) errors & observability, (6) evaluation & regression. Confirm side-effect class (read-only vs write).

---

Stage 1: Define Tool Surface#

Goal: Minimize tools; maximize clarity per tool.

Principles#

• One action per tool when possible—avoid mega-tools with mode flags unless necessary
• Names descriptive: search_orders not do_stuff
• Prefer idempotent operations where writes exist; separate read vs write clearly

Anti-patterns#

• Exposing raw SQL or shell to the model
• Too many overlapping tools → routing errors

Exit condition: Tool list with purpose, inputs, outputs, side effects table.

---

Stage 2: Schema & Validation#

Goal: Arguments are typed, constrained, and machine-validated before execution.

Practices#

• JSON Schema: enums, min/max, patterns, required fields
• Normalize dates, IDs, currencies server-side— never trust model formatting alone
• Default behaviors explicit in description + schema

Descriptions#

• Tool and parameter docstrings seen by model— precise language; examples of valid args

Exit condition: Validator rejects invalid args with actionable errors back to model or orchestrator.

---

Stage 3: Authorization & Safety#

Goal: Every tool call runs as some principal with least privilege.

Patterns#

• User-scoped credentials carried from session; tool implementation re-checks ownership (e.g., order_id belongs to user)
• Admin tools behind explicit allowlists and human approval when needed
• Rate limits per user + global circuit breakers

Data exfiltration#

• Tools that read sensitive data need output filtering and logging policies

Exit condition: Threat brief: “What if model is tricked into calling tool X?” answered.

---

Stage 4: Execution Semantics#

Goal: Clear transactionality, retries, and idempotency.

Design#

• Idempotency keys for writes; dedupe window
• Timeouts and cancellation propagation
• Ordering: parallel safe vs must be serial

Long operations#

• Async jobs with poll tool vs blocking calls—prefer non-blocking for UX and cost

Exit condition: Semantics documented for retry behavior (at-least-once delivery common).

---

Stage 5: Errors & Observability#

Goal: Model (or orchestrator) can recover from failures without leaking internals.

Error messages#

• Structured error codes: ORDER_NOT_FOUND, PERMISSION_DENIED
• Hints for model on how to fix— without stack traces to end users

Observability#

• Trace IDs across tool calls; audit log for write tools (who/when/args hash)

Exit condition: Dashboards/alerts on tool error rate, latency, denials.

---

Stage 6: Evaluation & Regression#

Goal: Tool changes are tested like APIs.

Harness#

• Golden conversations with expected tool calls (args normalized)
• Adversarial prompts attempting privilege escalation
• Version tools; deprecate with compatibility window

Exit condition: CI or manual eval suite before deploying new tools/schemas.

---

Final Review Checklist#

• Minimal orthogonal tool set
• Strict schema validation on server
• AuthZ enforced per call; sensitive reads controlled
• Idempotency and timeouts defined for writes
• Structured errors + observability + eval harness

Tips for Effective Guidance#

• Treat tool descriptions as API docs the model reads—iterate wording like UX copy.
• Recommend two-step patterns for dangerous ops: propose → confirm (human or policy).
• When using MCP, same discipline— server must validate everything.

Handling Deviations#

• Read-only RAG: fewer semantic risks—still validate query args and injection into search backends.
• Local tools (filesystem): sandbox, path allowlists, size limits.