Token Cost Optimization#

Use Cases#

User mentions token savings, API cost optimization, prompt compression, cache strategy, model downgrade, cost analysis.

Quick Start#

Token Calculator#

Run the calculation script, input conversation scale, and quickly estimate current token consumption and optimization potential:

bash

python scripts/token_calculator.py

The script will prompt for:

• Number of conversation history items / average length
• Model and pricing used
• Current optimization status

Output: Current cost, optimized cost, savings percentage.

Three-Tier Optimization Strategy#

Ranked by effect / implementation cost:

TierStrategyEffectImplementation CostL1Prompt compression & output truncation10-30%LowL2Conversation summary caching30-50%MediumL3Model downgrade + task routing50-70%High

Priority Recommendation: Implement in order L1 → L2 → L3, verifying results at each stage before proceeding.

Detailed strategies, configuration guides, and pitfalls → See references/tier-strategies.md

Phased Implementation Guide#

Phase 1: L1 Compression (Immediate Effect)#

• Clean up redundant descriptions in system prompt
• Set max_tokens limits for long responses
• Remove outdated/unused messages from conversation history

Phase 2: L2 Caching (1-3 Days)#

• Establish FAQ shortcuts for high-frequency repeat questions
• Add summary compression at the beginning of conversations (execute every N rounds)

Phase 3: L3 Routing (1-2 Weeks)#

• Route simple tasks to cheaper models (e.g., 4o-mini / Haiku)
• Retain strong models for complex tasks
• Configure model routing rules

Quantifiable Comparison Example#

See the "Quantified Comparison" section in references/tier-strategies.md for details.