Lowering LLM costs without lowering quality

Send less, send smarter

• Retrieve only the passages that matter, not the whole document.
• Trim chat history and summarise old turns.
• Keep prompts short and move fixed instructions into a cached prefix when your provider supports it.

Use the right model for each step

Routing, classification and extraction rarely need the biggest model. Reserve the strongest one for the final answer, and measure whether a smaller one is good enough.

Measure before you optimise

Log tokens and latency per request, then look at the top few costs. The expensive path is usually one or two places, not everywhere.

Optimise the system, not the prompt wording.