The unit economics of an AI feature

You don't buy tokens. You buy finished tasks — and while the price per token has been falling roughly 50x a year, the number of tokens one task consumes has grown faster. That's the entire reason AI bills keep rising in a market where the raw input gets cheaper every quarter. If your cost model tracks the price list rather than the loop, it will be wrong in the same direction every month.

Why this matters now

The price collapse is real, and it's steeper than most people realise.

• Inference prices have fallen at a median of 50x per year, according to Epoch AI's analysis of state-of-the-art performance across six benchmarks over three years. The spread is enormous — 9x to 900x annually depending on the task — and the trend accelerated: measured from January 2024 onward, the median rate rises to 200x per year. Holding capability fixed, the same answer costs a fraction of what it did.
• And yet gross margins on AI products sit around 53% in 2026 — against the ~80% that defined mature software. That figure comes from ICONIQ's 2026 State of AI report, its third edition, surveying roughly 305 executives at companies building AI products. Margins are improving (45% in 2025, a projected 59% by 2027), but the recovery is being driven by scale and routing, not by pricing power.
• The reason is on the consumption side. A Microsoft Research team's April 2026 paper, How Do AI Agents Spend Your Money?, analysed agent trajectories from eight frontier models on SWE-bench Verified and found agentic tasks consume roughly 1000x more tokens than code chat — with input tokens, not output, driving the cost.

Cheaper tokens, more tokens per job. Those two curves are racing, and for anything agentic the second one is winning.

The mechanism: the meter moved from the token to the loop

Here's the part that pricing spreadsheets miss. Every model API call is stateless — the provider doesn't remember your last turn. So an agent re-sends the accumulated context on every step: the system prompt, the tool definitions, every prior tool call and its result. Step 20 pays for steps 1 through 19 all over again.

That single design fact explains almost everything about AI cost behaviour:

• Input tokens dominate the bill. Most teams instinctively optimise the answer — shorter outputs, terser responses. The money is on the other side of the request, in the context you resend.
• Cost grows super-linearly with steps. A ten-step task doesn't cost ten times a one-step task. It costs closer to the sum of a growing series, because each step carries everything before it.
• The bill is stochastic, not fixed. The same paper found runs on the identical task differing by up to 30x in total tokens. You are not buying a unit price; you're sampling from a long-tailed distribution.
• Nobody can forecast it from the inside, including the model. Frontier models asked to predict their own token usage managed correlations of only up to 0.39, and systematically underestimated real cost.
• Spending more doesn't reliably buy quality. The researchers found accuracy "often peaks at intermediate cost and saturates at higher costs" — past a point, extra tokens buy variance, not correctness.

The token price is the one number on your invoice you don't control and don't need to worry about. It's falling on its own. What you control is how many times you make the model re-read the same context.

How to actually budget for it

Five things we'd do before shipping any metered AI feature:

1. Define the billable unit as the task, not the call. "Cost per resolved ticket," "cost per document processed," "cost per itinerary drafted." If you can't express your AI cost in the same unit you charge in, you can't tell whether the feature makes money.
2. Budget on p95, not the mean. With 30x run-to-run variance, the average is a fiction that hides the runs that hurt. Instrument per-task token counts from day one and set your alerting on the tail.
3. Cap the loop, hard. A step ceiling, a token ceiling, and a wall-clock ceiling per task — with a defined behaviour when the cap trips (escalate to a human, return partial work, ask a clarifying question). An uncapped agent is an uncapped invoice, and the accuracy data says the extra laps rarely earn their keep.
4. Attack input before output. Prune the transcript, summarise completed sub-steps rather than replaying them, trim tool definitions to what the current step can actually use, and cache the stable prefix. This is where the money is.
5. Decide who absorbs the variance — deliberately. ICONIQ found consumption pricing climbing from 35% to 42% of AI companies, and 84% of consumption-priced companies now pass at least part of the token bill through to the customer. Flat pricing on a long-tailed cost means your heaviest users define your margin. That can be the right call, but it should be a decision, not an accident.

Model choice matters too, but it's a different lever and we've written about it separately in the case for small language models. The point here is that right-sizing the model reduces the price of a lap. Capping the loop reduces the number of laps — and on a 1000x multiple, laps are the bigger term.

Our opinion

The industry is optimising the wrong variable. Enormous attention goes to per-million-token price comparisons — a number falling 50x a year without any help from you — and almost none to the loop architecture, which is entirely yours and moves cost by orders of magnitude.

We'd go further: a per-token price list is a misleading way to think about an AI feature at all. It invites you to model AI as a utility with a stable rate, when it behaves much more like a variable-latency, variable-cost job queue with a heavy tail. The teams we've seen get this right stopped asking "which model is cheapest" and started asking "what is the p95 cost of one completed job, and what happens when it exceeds our cap." That's a systems question, not a procurement question.

There's a corollary worth stating plainly: falling prices will bail out a mediocre implementation eventually. They will not bail out an unbounded one, because an unbounded loop can consume any price decline you hand it. Cheap tokens make sloppy architecture affordable right up until an agent gets a task it can't finish.

How Ashvara helps

We build AI features with the cost model attached — per-task token instrumentation, explicit step and spend caps, context pruning, and a defined fallback when a job exceeds its budget. That's the same discipline as evals: you can't manage what you don't measure, and with AI both the quality and the cost are distributions rather than numbers.

It's a core part of our AI solutions work, and it draws on the same instincts as our cloud cost practice — find the mechanism, cap the runaway, measure the tail. If you have an AI feature in production and you can't yet say what one completed task costs you, tell us what it does and we'll help you put a number on it.

Sources: Epoch AI, "LLM inference prices have fallen rapidly but unequally across tasks" (epoch.ai); Bai et al., "How Do AI Agents Spend Your Money? Analyzing and Predicting Token Consumption in Agentic Coding Tasks," arXiv:2604.22750, April 2026 (arxiv.org); ICONIQ, "State of AI: The Builder's Economy," 2026 (iconiq.com).