Treat tokens like cloud spend, or the invoice will teach you.

The pattern is consistent across engagements. A team ships an AI feature, usage grows faster than expected because the feature is genuinely useful, and three months in someone in finance asks why the cloud bill has a five-figure line item nobody budgeted for. The team's response is almost always the same: "we didn't realize it would scale like this." That is the tell. Nobody modeled cost per task before launch, so nobody could have realized anything.

The fix is not complicated, but it has to happen before launch, not after. Cost control retrofitted onto a live system means renegotiating behavior your users already depend on. Cost control designed in from day one is a few configuration decisions. If you are past launch and already staring at an unexplained bill, the $1,000 AI Audit gives you the cost breakdown by workflow in 48 hours instead of weeks of log archaeology.

Here is the sequence we run on every engagement before a workflow goes live:

1. Estimate cost per task from your eval set. Run your 20 to 50 regression cases through the real pipeline and record token count and dollar cost per case, including retries.
2. Set a ceiling at roughly 1.5x your median observed cost. This catches runaway context or infinite retry loops without blocking the normal range of legitimate variation.
3. Decide the failure behavior in advance. When a request approaches the ceiling, does it truncate context, downgrade to a cheaper model, or fail closed with a message to the user? Write this down before it happens in production, not while it is happening.
4. Wire the ceiling into the code path, not into a dashboard alert someone has to notice and act on manually.
5. Re-check the ceiling monthly against actual production distribution, since real traffic patterns diverge from your eval set over time.

Teams that skip step three are the ones who find out their failure behavior the hard way, usually because the model silently degraded to truncated context and gave confidently wrong answers instead of failing loudly.

The most expensive habit in production AI systems is sending every request to the most capable, most expensive model available, regardless of what the task actually requires. Classification, extraction, formatting, and simple lookups do not need frontier-model reasoning. They need consistency, and a smaller model at a fraction of the cost delivers that just as reliably when scoped correctly.

• Tier your tasks. Split your workflow into steps and classify each by how much reasoning it genuinely requires, not by how important the workflow feels overall.
• Test the cheaper model on your own eval set, not a public benchmark. Public benchmarks tell you nothing about your specific extraction schema or your specific tone requirements.
• Route dynamically where it matters. A support triage step can start on a small model and escalate to a larger one only when the small model's confidence signal is low or the schema validation fails.
• Keep the swap cheap. Abstract the model call behind one interface so a routing change is a configuration edit, not a code change across a dozen call sites.

Teams that do this well typically cut 30 to 50 percent of spend with no measurable drop in output quality, because most of their volume was never in the category that needed the expensive model in the first place. Our production readiness guide covers how routing fits into the broader hardening checklist before launch.

Context length drives cost more directly than almost anything else, and it is the easiest lever to pull because it requires no model change at all. Most production agents send the full conversation history, every available tool schema whether or not it is relevant to the current step, and retrieved documents that were never trimmed to the parts actually relevant to the query.

A few concrete fixes: summarize conversation history after a fixed number of turns instead of sending the raw transcript. Scope tool schemas to only the tools relevant to the current step of a multi-step workflow, not the full tool registry. Trim retrieved documents to the specific passages that scored above your relevance threshold, not the entire source document. Each of these is a small engineering change. Together they typically cut context size, and therefore cost, by 20 to 40 percent with zero change in the quality of the final output, because none of the trimmed content was contributing to the answer in the first place.

Any workflow with a stable system prompt and stable tool definitions is a candidate for provider-side prompt caching, which discounts the repeated prefix of a request. This is close to free to implement, since it is usually a flag on the API call, and it consistently saves 15 to 30 percent on workflows with any meaningful repetition, which describes almost every production agent.

Layer your own application-level cache on top for genuinely repeated queries, like a support agent answering the same handful of policy questions dozens of times a day. Cache the final validated response, keyed on a normalized version of the query, with a sensible expiry tied to how often the underlying data changes. This is standard software caching discipline. The only thing new is applying it to a cost center that is priced per token instead of per compute-second.

The popular argument this year is that falling per-token prices make cost control a shrinking concern, that the curve will bail teams out the way it has for the last two years. There is truth in the trend, but the argument misses two things that matter operationally.

First, usage grows faster than price falls for any feature that succeeds. A workflow that costs a tenth as much per call but gets called fifty times more often than projected, because the product team correctly built something people want to use constantly, ends up costing more in absolute terms, not less. Second, falling prices remove the pressure to build the discipline, and teams that never build routing, budgets, and caching in one product area do not build it in the next one either. The habit compounds or the debt compounds; falling prices only change the exchange rate, not which one you are accumulating.

Price drops are real and worth capturing. They are not a substitute for the operational discipline of knowing your cost per task and having a ceiling before a workflow ships.

Every engagement that skips a cost dashboard eventually gets surprised by a bill. Every engagement that builds one but assigns it to nobody in particular finds the same surprise, just delayed by a month, because a dashboard nobody is accountable for reading gets built once during launch week and never opened again.

• One dashboard, wired into your existing observability stack, not a separate tool the team has to remember to check.
• Cost per task, per model, and per user segment, broken out weekly, so a spike is visible against a baseline instead of buried in a monthly total.
• One named owner with the authority to change a routing rule or lower a budget cap without a change-approval process getting in the way.
• A standing weekly fifteen minutes where the owner actually looks at the numbers, not a dashboard that exists in theory.

This is the same discipline that made cloud FinOps work: visibility plus a named human with authority, checked on a fixed cadence. The tooling is the easy part. The habit is what most teams skip.

If none of this exists yet for your production AI workflows, here is the order to build it in: pull cost per task from your logs for the last 30 days and find your current baseline. Set a budget ceiling at 1.5x the median for your highest-volume workflow first. Check whether that workflow's hardest steps actually need your most expensive model, and test a cheaper one against your eval set. Trim context on the same workflow and measure the cost delta. Then wire up one dashboard and name one owner.

This sequence, run on one workflow, usually takes a week and pays for itself inside the first billing cycle. If you want it done end to end with your team, this is a standard component of our Claude implementation engagements, alongside the eval and observability work that keeps the savings from silently eroding.