3. Implement Prompt Caching

Both OpenAI and Anthropic offer prompt caching — identical prefix tokens are cached and charged at 90% discount. This is free money.

• OpenAI: Automatic for prompts >1024 tokens (GPT-4o, GPT-5). Cache hits cost 0.1x input price
• Anthropic: Automatic for prompts >2048 tokens (Claude Sonnet/Opus). Cache hits cost 0.1x input price
• Structure your prompts: Put static content (system prompt, context) at the beginning, dynamic content at the end

4. Batch Your Requests

Batch processing lets you send multiple requests in one API call, reducing overhead and often getting bulk discounts.

• OpenAI Batch API: 50% discount on all models for non-real-time workloads
• Group similar tasks: Send 10 summarization requests in one batch call instead of 10 separate calls
• Use for: Data processing, content generation, offline analysis, nightly jobs

5. Route by Complexity

Don't use a sledgehammer for every nail. Implement a router that sends simple requests to cheap models and complex ones to premium models.

• Tier 1 (simple): Gemini Flash Lite, GPT-5 Mini, DeepSeek Flash — $0.03-0.15/1M tokens
• Tier 2 (moderate): Claude Haiku, GPT-5 Mini, Mistral Small — $0.10-0.80/1M tokens
• Tier 3 (complex): GPT-5, Claude Sonnet, Gemini Pro — $2-5/1M tokens
• Tier 4 (premium): Claude Opus, GPT-5.5 — $10-15/1M tokens (rare use only)

Tool: Use the Multi-Model Pipeline Calculator to build cost-optimized routing strategies.

6. Implement Response Caching

Cache identical or similar responses to avoid re-computing. Works best for deterministic outputs and frequently repeated queries.

• Exact-match cache: Hash the prompt, store the response. 100% hit rate for repeated queries
• Semantic cache: Use embeddings to find similar past queries. 70-85% hit rate for natural language
• TTL strategy: Cache for 1-24 hours depending on data freshness needs

7. Trim Output Tokens

You're paying for every output token. Set max_tokens appropriately and use stop sequences to prevent runaway generation.

• Set max_tokens: Classifications need 10-50 tokens, not 4096
• Use stop sequences: Stop at "Answer:" or "\n\n" to prevent elaboration
• Temperature 0: More deterministic = less wasted output tokens

8. Use Streaming for UX, Not Cost

Streaming doesn't save money — it's the same total tokens. But it improves perceived performance, which means you can use smaller models without users noticing.

• First-token latency matters more than total time for user satisfaction
• Consider: stream from a fast cheap model vs. wait for a slow expensive one

9. Negotiate Volume Discounts

If you're spending $500+/month, you're likely eligible for volume pricing. Most providers offer 10-30% discounts at commitment levels.

• OpenAI: Tier 3 ($1K+) and Tier 4 ($10K+) get progressive discounts
• Anthropic: Enterprise pricing available for $1K+/month
• Google: Committed use discounts for Gemini API
• DeepSeek: Already the cheapest — focus on model selection instead

10. Switch Providers for Specific Use Cases

Different providers win at different price points. Mix and match instead of going all-in on one.

• Cheapest chatbot: DeepSeek V4 Flash ($0.07/1M input)
• Cheapest code: DeepSeek V4 Pro ($0.27/1M input)
• Best quality/price: Claude Sonnet 4.6 ($3/1M input) — often matches GPT-5 at 60% of the cost
• Best for agents: Gemini 3.1 Pro ($1.25/1M input) with 1M context

Tool: Use the Cost Migration Report to find cheaper alternatives for your current spend.

11. Implement Token Budgets

Set hard limits per request and per user. Prevents runaway costs from edge cases.

• Per-request limit: 4K tokens max for most use cases
• Per-user daily limit: Prevent abuse and surprise bills
• Circuit breaker: Stop and fallback if token count exceeds threshold

12. Use Smaller Context Windows

A 128K context window costs the same per token as a 4K window — but you're more likely to fill it with irrelevant context. Use the smallest window that fits your use case.

• Chat: 8-16K is usually sufficient
• RAG: 32K covers most retrieval scenarios
• Document analysis: 128K+ only when needed

13. Compress Context with Summarization

Instead of sending full conversation history, periodically summarize it. A 10-message conversation can be compressed to 1-2 summary tokens.

• Summarize every 5 messages into a running summary
• Use a cheap model (GPT-5 Mini) for the summarization step
• Keep last 2 messages verbatim for continuity

14. Monitor and Alert on Anomalies

You can't optimize what you don't measure. Set up cost alerts before you get a surprise bill.

• Daily spend alerts: Get notified if daily cost exceeds 1.5x average
• Per-endpoint tracking: Know which API endpoints cost the most
• Weekly reports: Review cost trends and identify optimization opportunities

Tool: Set up Price Alerts to get notified when model prices change.

15. Evaluate Fine-Tuning for High-Volume Tasks

For tasks you run 10K+ times/month with consistent patterns, fine-tuning a smaller model can be 10x cheaper than using a large model.

• Best candidates: Classification, extraction, structured output, style matching
• Use fine-tuned GPT-5 Mini instead of GPT-5 for specific tasks
• ROI calculator: Fine-tuning cost ÷ monthly savings = payback period