Prompt Engineering (Deep Workflow)#

Prompts behave like natural-language programs: they need specs, tests, and version control—especially in production.

When to Offer This Workflow#

Trigger conditions:

• Prompt or system message change; quality regressions
• Structured outputs (JSON), tool use, or RAG grounding requirements
• Safety or policy alignment needs

Initial offer:

Use six stages: (1) define task & success, (2) constraints & format, (3) few-shot & style, (4) build eval set, (5) iterate with discipline, (6) ship, monitor, regress). Confirm model family and latency budget.

---

Stage 1: Define Task & Success#

Goal: Clear user-visible outcome and failure modes (hallucination, omission, tone).

Exit condition: Success rubric in plain language; out-of-scope cases listed.

---

Stage 2: Constraints & Format#

Goal: Must/must-not rules; output schema (JSON Schema, bullet structure); length limits.

Practices#

• Separate system (policy, role) from user (task instance)
• Ask model to cite sources when grounding matters

---

Stage 3: Few-Shot & Style#

Goal: Use examples only when they reduce ambiguity—avoid huge prompt bloat.

Practices#

• Diverse examples; avoid overlong negative examples that confuse

---

Stage 4: Build Eval Set#

Goal: Frozen inputs with expected properties (not always exact text match).

Practices#

• Adversarial and multilingual slices if relevant
• Regression suite in CI for critical prompts

---

Stage 5: Iterate With Discipline#

Goal: Change one major variable at a time when debugging quality.

Practices#

• Compare with same temperature settings when A/B testing wording
• Log prompt version id with outputs in production

---

Stage 6: Ship, Monitor, Regress#

Goal: Canary prompt changes; watch implicit signals (thumbs, edits, task completion).

---

Final Review Checklist#

• Task and rubric defined
• Constraints and output format explicit
• Eval set versioned; regression path exists
• Iteration log disciplined; prompt versions tracked
• Production monitoring and rollback plan

Tips for Effective Guidance#

• Clarity beats cleverness—short explicit instructions often win.
• Chain-of-thought: use when reasoning helps; hide chain from end users if needed.
• Align with llm-evaluation skill for larger harness design.

Handling Deviations#

• Chat vs batch: batch can use stricter structure and lower temperature.
• Multimodal: specify how image details may be used or ignored.