Why attributing AI cost is still a problem

If you want the reconciled invoice to carry the dimensions you bill back on, the provider has to let you attach them to the billed call. Three clouds, three different mechanisms, three different amounts of coverage, and one with no dollar figure at all.

Give a few teams access to the cloud model providers and the first question from anyone who answers for the budget is simple: who spent the money. Not the total on the cloud invoice, but the breakdown, this app, that team, that one developer running an assistant on the expensive model all week. The honest answer at most organizations I have seen is a shrug and a spreadsheet. The token usage is sitting right there in the logs, and still nobody can slice it cleanly by the dimensions that matter.

When the job of fixing that landed on me, the surprise was not the difficulty. It was the absence. I expected a well-worn path, and there was no clean existing solution to buy or copy. This was my first time working a problem at this scale, three clouds and dozens of clusters, and the lesson it kept teaching was that at this scale the good answers take actual creativity, not assembly. At some point my manager asked the natural question: how do other companies do this? My first thought was honest enough, maybe we are simply behind. The longer I looked, the more the answer seemed to be that most companies never decide this at all; they outsource the decision and go with the flow. So if we wanted attribution that meant something, the choices were ours to make: how far we trust our own developers, and how much complexity we could afford at the speed we needed it.

The goal is one number anyone can trust: what every model call cost, across every provider, every model, every call shape, streaming included, sliced by the dimensions you actually bill back on. What surprised me on the way there is how little of the outcome your own tooling decides. It is decided by whether the provider lets you attach your dimensions to the billed call, and each cloud has formed its own opinion about that. This is a notes-from-the-build piece on those differences, and on where the problem is still genuinely unsolved.

The dimensions, and how they get onto the call

Ask who spent the money and you are really asking two questions, because two kinds of caller sit behind the bill. A service calling models on its own has to answer for the application and for the team that owns it. The obvious upgrade is to take the team from the directory service, so the answer lines up with how the rest of the organization already describes itself. We deliberately did not: that map from applications to identities is one more component that drifts and needs an owner, and our trust model starts from trusting our own developers to say who they are. A person writing a prompt, in a coding assistant or a chat client, has to answer for themselves and for the group they belong to. Those answers are the dimensions, and it helps to treat them as concepts rather than fixed field names. The exact labels are a choice. The shape, who owns the spend and who incurred it, is the part that matters.

Mechanically a dimension is a tag, a label, or a header, and something in the path has to stamp it onto the call. An AI gateway is one common way to do that, since it sees every call and already knows who is calling, but it is one means among several: clients can set the values themselves, an SDK wrapper can inject them, a sidecar can do it. What matters is that the dimension is on the call by the time the provider bills it. The stamp does not always land on the call itself, either: on Bedrock, as the next section covers, it rides on the credentials used to sign the call rather than on the request. The detail that bites regardless of mechanism is streaming. These tools stream by default, and any attribution scheme that only reads token counts from a buffered, non-streaming response will miss most of the traffic. Reading usage off the streamed response is a requirement, not a nicety.

Two ways to a dollar figure, and they disagree

There are two roads to a cost number, and they are not the same number.

The first is computed cost: take the token counts you recorded, multiply by a price book you maintain, and you have a figure that is live, per-call, and sliceable by those dimensions immediately. The catch is that it is your arithmetic. If your price book is stale or your token accounting is slightly off, your number is confidently wrong.

The second is reconciled cost: the provider’s own billing, the authoritative dollars you actually pay. The catch there is the mirror image. Cloud invoices arrive with a delay, and they do not carry your dimensions unless you did specific work upfront to push them into the billing pipeline, and even then only partially. You likely need both: computed cost for the live dashboard, provider billing for the monthly truth-up. The decision that actually matters is which one is the headline number on the dashboard, because the two will diverge and someone will ask why.

Each cloud carries the dimension differently

If you want the reconciled figure to carry your dimensions, the provider has to let you attach them to the billed call. Three clouds, three different mechanisms, and three different amounts of coverage. There is a reason for the mess, and it is not carelessness: different people built these systems, holding different opinions, and all of the results work, some better than others in certain corners. It is the CAP theorem lesson in a different suit: past a certain scale nobody gets everything, only tradeoffs someone chose. Which means the design work here leans on a solid, and above all current, understanding of structures outside your own organization, each of them an ecosystem of its own.

1. 01

   AWS Bedrock: session tags into the cost report

   Bedrock’s opinion is that attribution belongs to identity, not to the request. It lets you attach session tags to an inference call. Once you activate those as cost allocation tags, they surface as columns in the detailed cost export, the newer report format in particular, so the reconciled invoice can be grouped by your dimension. The mechanics are easy to misread: the tags are not headers on the inference request, they ride on the STS call where the caller assumes its IAM role, so whatever stamps them has to own the credential flow, not just the HTTP path. In practice that means hand-rolling the AssumeRole exchange in whatever sits in the path. Streaming calls still have to emit usage for this to be complete, which is the same streaming requirement as everywhere else. This mechanism is also where, fittingly, the token dropped for me.

2. 02

   Google Vertex: labels, on the calls that accept them

   Vertex’s opinion is that attribution belongs on the request itself: labels on the generate-content family of calls, with the usual constraints on how many labels and what characters keys and values may contain. The gap is coverage: a number of call types, the raw-predict variants and several embedding and partner-model paths, do not take labels at all. Those calls cannot carry your dimension into billing, so for them computed cost is the only route you have. An attribution design that assumes labels everywhere will quietly under-count.

3. 03

   Azure AI Foundry: header dimensions, no dollar figure

   Foundry has not formed an opinion yet. The gateway can stamp your dimensions as headers on Foundry calls and you can read token usage back, so computed cost works. A native way to get a reconciled dollar figure broken down by your dimensions, the way the cost export does on AWS and labels do on Google, simply is not there: that is no longer just something I failed to find, it is a fact I have since verified with Microsoft representatives, as of today. Until that changes, Foundry attribution is computed-only. I hope it changes; the platform has everything it needs to close that gap.

Why a token-counting tool is not enough

A reasonable first instinct is to reach for a tool that already tracks LLM cost, LiteLLM being the common example: route your traffic through it and it tallies tokens against a built-in price table. That is genuinely useful, and it is exactly the computed-cost path from above. The problem is that everything it holds is an estimate, and this is where I refuse to budge: an estimate is not good enough for me, not when the real number exists. The moment a team disputes its bill, an estimate is just your arithmetic against their disbelief. The authoritative figure, the dollars the provider actually billed, is the only number that ends that conversation, and an estimation tool by definition does not hold it. I am just looking for the most sensible way to keep quality control over the thing everyone cares about most.

There is also a bigger reason to be stubborn about this, and it connects to a question I mentioned in another article: whether the value of all this spend justifies it. I argued there that the value side is genuinely hard to measure, and that this is not proof it is absent. But that argument only has a chance of being settled if the cost side is a fact. To measure value you have to know what it cost.

The second reason has less to do with accuracy than with where the numbers live, and with whose opinions they embody. A built-in price table is somebody else’s opinion of what the models cost, maintained on somebody else’s schedule. And a bolt-on tool puts your cost of record in a third-party system, separate from the provider’s billing and separate from your own usage records, one more place a FinOps team has to reconcile and trust. The realization that settled it for me is almost embarrassingly simple: if you want the real cost, take it from the place you buy. It is the most logical source, especially when models are not the only thing you buy there, and especially because that number is not an estimate and not a maybe; it has a stamp on it. Funnel it into whatever dashboard you like afterwards, next to every other provider, as long as every dollar sign has a backer behind it. In 2026, with the stack already sprawling, one more disconnected source of truth usually costs more than it looks.

The price book is a first-class asset

Computed cost is only as trustworthy as the price book behind it, and the day someone challenges the dashboard, the price book is where the challenge lands. So it deserves to be treated as real infrastructure rather than a constant in the code: one row per provider, model, and token type, with an effective date and a unit price. Prices change, new token types appear, ratios get corrected, and a versioned table lets you say what a call cost on the day it was made rather than at today’s rate.

Store the raw token counts on every row too, never just the dollar figure. The first time a price turns out to have been wrong, and one eventually will, the difference between the two habits shows. Keep the tokens, and the correction is a recompute. Keep only the computed cost, and it is a permanent gap in the record, because you have thrown away the one input you would need to redo the arithmetic.

Why this usually ends up built

The hard part of this is not any single cloud. It is that the answer has to span all of them at once, and span them at the depth where session tags, the gaps in Vertex label coverage, and the missing Foundry dollar figure each behave differently, all stitched back to your own token accounting. You are not integrating three APIs so much as staying current with three ecosystems, each moving on its own schedule. That particular combination, cross-cloud billing detail plus token metering plus custom attribution taxonomy, is an uncommon shape to find pre-assembled if we’re being honest.

None of it is new thinking, either. Build versus buy, a single source of truth, reconciling an estimate against the authoritative ledger: these are well-worn system design questions, and the AI bill just asks all of them at once. In my experience the assembled answer rarely arrives as a product or a single engagement. Most teams I have seen field deep expertise in one cloud, sometimes two, seldom all three at the granularity this needs, and almost never paired with the token accounting that ties them together. So the work tends to land with the enterprise’s own platform team, less because building is virtuous and more because the thing being built is specific to your traffic, your price book, and your reconciliation tolerance. I would hold that as a pattern I keep running into rather than a law: the real question is usually not whether to build, but how much to build now and how much to defer. That question turned out to be large enough to deserve an essay of its own.

What already works, and what is still on the way

The spine is already standing: dimensions attached to every call by whatever sits in the path, computed cost from tokens and a versioned price book giving you a live and fully sliceable number, and on the clouds that accept tags or labels a reconciled invoice that can be made to agree with it. What has not arrived yet is the ragged edge: Vertex call types that refuse labels, the reconciled dollar figure that Foundry does not seem to expose, the structural lag between a computed number and a settled invoice, and per-request tag values on the session-tag path, something I have not seen any gateway do natively yet. None of those holes are negligence; they are what other people’s tradeoffs look like from the outside, and they move when those people change their minds. The single pane is reachable. It is just honest to say it has holes in it, and to know where they are before you put the dashboard in front of someone who will trust it.

The timing is worth a note, because this layer is young. Vertex label support in the tools arrived in pieces: regular chat calls in September 2025, and the other call types, image generation among them, only in April 2026. Native session-tag support at the gateway layer is younger still. The cloud side has supported all of it for a while; the tooling around it is still filling in.