fix(ai): stop double-counting cache-creation tokens in reported token totals (#23405)
## Context Under AI SDK v6 usage normalization, `usage.inputTokens` is the **full prompt**: fresh (noCache) + cache-read + cache-creation tokens. Our `totalTokens` formulas still added `cacheCreationTokens` (extracted from provider metadata) on top of `inputTokens` — a leftover from the pre-v6 SDK generation, where flat `inputTokens` excluded cache tokens. The v6 upgrade changed the semantics under the formula's feet, so every Claude run using prompt caching reported a `totalTokens` inflated by exactly `cacheCreationTokens`. ## Evidence, traced through AI SDK source **1. The Anthropic provider folds cache tokens into `inputTokens`.** The raw Anthropic API reports `input_tokens` *excluding* cache tokens; the provider sums all three components — [`convertAnthropicMessagesUsage`, `@ai-sdk/anthropic@3.0.84`](https://github.com/vercel/ai/blob/%40ai-sdk/anthropic%403.0.84/packages/anthropic/src/convert-anthropic-messages-usage.ts): ```ts inputTokens: { total: inputTokens + cacheCreationTokens + cacheReadTokens, noCache: inputTokens, cacheRead: cacheReadTokens, cacheWrite: cacheCreationTokens, } ``` **2. ai core surfaces that total as the app-visible `usage.inputTokens`** — [`asLanguageModelUsage`, `ai@6.0.97`](https://github.com/vercel/ai/blob/ai%406.0.97/packages/ai/src/types/usage.ts): ```ts inputTokens: usage.inputTokens.total, ... totalTokens: addTokenCounts(usage.inputTokens.total, usage.outputTokens.total), ``` So the SDK's own `totalTokens` is already "full prompt (incl. cache read + creation) + output". **3. The value we were adding on top is the same one already inside `inputTokens`.** The provider also exposes the raw API field in metadata (`@ai-sdk/anthropic` dist): ```ts const anthropicMetadata = { usage: response.usage, cacheCreationInputTokens: response.usage.cache_creation_input_tokens ?? null, ... ``` `extract-cache-creation-tokens.util.ts` reads exactly `providerMetadata.anthropic.cacheCreationInputTokens` — the same `cache_creation_input_tokens` that step 1 already folded into `inputTokens.total`. Adding it again counts it twice. **Worked example** (matches the new pinning test): API returns `input_tokens: 400, cache_read_input_tokens: 600, cache_creation_input_tokens: 200, output_tokens: 500` → app sees `usage.inputTokens = 1200`, `providerMetadata.anthropic.cacheCreationInputTokens = 200` → old formula reported `1200 + 500 + 200 = 1900`; actual tokens processed: `1700`. All snippets are verbatim from the version tags in `vercel/ai` and match the installed `node_modules` dists. ## Provider independence `inputTokens + outputTokens` is correct for every provider Twenty routes through, not just Anthropic: - The v3 provider spec (`@ai-sdk/provider`) defines `inputTokens.total` as "the total number of input (prompt) tokens used", with `noCache`/`cacheRead`/`cacheWrite` as its components — and all 8 installed provider packages comply (verified in dists): `anthropic` and `amazon-bedrock` sum the components explicitly ([`convertBedrockUsage`, `@ai-sdk/amazon-bedrock@4.0.117`](https://github.com/vercel/ai/blob/%40ai-sdk/amazon-bedrock%404.0.117/packages/amazon-bedrock/src/convert-bedrock-usage.ts): `total: inputTokens + cacheReadTokens + cacheWriteTokens`); `openai`, `azure`, `google`, `mistral`, and `openai-compatible` pass through wire values that already include cached tokens; `xai` even detects which wire convention the API used and normalizes either way. - The removed `cacheCreationTokens` term was already 0 for every provider except Anthropic/Bedrock (`extract-cache-creation-tokens.util.ts` only reads those two metadata namespaces), so this PR is a strict no-op for OpenAI-style providers and only removes the double-count where it existed. Caveat: a custom `AI_PROVIDERS` entry pointing at a legacy V2-spec provider package bypasses this normalization (ai core's shim passes flat usage through verbatim); that path could misreport under any formula, and none of the built-in providers use it. ## What changed Four sites computed the inflated total: - `ai-billing.service.ts` — `quantity` on the emitted AI token usage event - `chat-execution.service.ts` — chat-turn usage event - `agent-async-executor.service.ts` — workflow-agent usage event - `build-ai-agent-step-log.util.ts` — workflow step log (display) The first three now compute `totalTokens = inputTokens + outputTokens`; the step-log util uses the SDK's `usage.totalTokens` directly (it receives the `generateText` usage object, where the field is guaranteed). The explicit sum is used where usage objects are hand-assembled or merged — e.g. the streaming path in `stream-agent-chat.job.ts` builds usage literals with no `totalTokens` field at all, so `usage.totalTokens ?? 0` would silently emit 0. Both forms are definitionally identical where the SDK object exists, since ai core computes `totalTokens` as `input + output` (see evidence above). **Impact: reported/analytics quantities only.** Billed credits (`creditsUsedMicro`) come from `computeCostBreakdown`, which already handles the cache-inclusive convention correctly and is unchanged. **Ops note:** `usageEvent.quantity` for cache-heavy workspaces steps down on deploy — dashboards trending this metric may want an annotation. Historical rows are not backfilled (per-row component fields aren't stored, so mixed-era rows can't be reliably corrected). ## How tested - Updated `build-ai-agent-step-log.util.spec.ts` expectation (155 → 150 with `cacheCreationTokens: 5` still present) - New pinning test in `ai-billing.service.spec.ts`: emitted `quantity` is 1700 (not 1900) for inclusive Anthropic usage with `cacheCreationTokens: 200` - New pinning test in `agent-async-executor.service.spec.ts`: emitted total is 150 (not 180) when steps carry `providerMetadata.anthropic.cacheCreationInputTokens` - 3 suites / 20 tests pass; oxlint, oxfmt, and `nx typecheck twenty-server` clean <!-- This is an auto-generated description by cubic. --> <a href="https://cubic.dev/pr/twentyhq/twenty/pull/23405?utm_source=github" target="_blank" rel="noopener noreferrer" data-no-image-dialog="true"><picture><source media="(prefers-color-scheme: dark)" srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source media="(prefers-color-scheme: light)" srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img alt="Review in cubic" src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a> <!-- End of auto-generated description by cubic. -->
This commit is contained in:
Etienne
+41
@@ -200,6 +200,47 @@ describe('AgentAsyncExecutorService — workflow agent role-scoped tool resoluti
|
||||
expect(result.creditsUsedMicro).toBe(4200);
|
||||
});
|
||||
|
||||
it('emits the token total without re-adding cache-creation tokens', async () => {
|
||||
roleTargetRepository.findOne.mockResolvedValueOnce({
|
||||
roleId: agentRoleId,
|
||||
});
|
||||
aiBillingService.calculateCost.mockReturnValue(0.0042);
|
||||
generateTextMock.mockResolvedValueOnce({
|
||||
text: '',
|
||||
steps: [
|
||||
{
|
||||
toolCalls: [],
|
||||
providerMetadata: {
|
||||
anthropic: { cacheCreationInputTokens: 30 },
|
||||
},
|
||||
},
|
||||
],
|
||||
usage: {
|
||||
...baseUsage,
|
||||
// inputTokens (100) is the full prompt: noCache(60) + cacheRead(10) +
|
||||
// cacheCreation(30) — the emitted total must not add the 30 again
|
||||
inputTokenDetails: {
|
||||
noCacheTokens: 60,
|
||||
cacheReadTokens: 10,
|
||||
cacheWriteTokens: 30,
|
||||
},
|
||||
},
|
||||
} as unknown as Awaited<ReturnType<typeof generateText>>);
|
||||
|
||||
await service.executeAgent({
|
||||
agent: buildAgent(),
|
||||
userPrompt: 'test',
|
||||
workspaceId,
|
||||
});
|
||||
|
||||
expect(aiBillingService.emitAiTokenUsageEvent).toHaveBeenCalledTimes(1);
|
||||
|
||||
const [, , emittedTotalTokens] =
|
||||
aiBillingService.emitAiTokenUsageEvent.mock.calls[0];
|
||||
|
||||
expect(emittedTotalTokens).toBe(150);
|
||||
});
|
||||
|
||||
it('folds native web search dollars into totalCostInDollars and creditsUsedMicro', async () => {
|
||||
roleTargetRepository.findOne.mockResolvedValueOnce({
|
||||
roleId: agentRoleId,
|
||||
|
||||
+1
-2
@@ -426,8 +426,7 @@ export class AgentAsyncExecutorService {
|
||||
);
|
||||
const totalTokens =
|
||||
(accumulatedUsage.inputTokens ?? 0) +
|
||||
(accumulatedUsage.outputTokens ?? 0) +
|
||||
cacheCreationTokens;
|
||||
(accumulatedUsage.outputTokens ?? 0);
|
||||
|
||||
void this.aiBillingService.emitAiTokenUsageEvent(
|
||||
workspaceId,
|
||||
|
||||
+44
@@ -384,5 +384,49 @@ describe('AiBillingService', () => {
|
||||
'workspace-1',
|
||||
);
|
||||
});
|
||||
|
||||
it('should not add cache-creation tokens to the emitted token quantity', async () => {
|
||||
mockAiModelRegistryService.getEffectiveModelConfig.mockReturnValue(
|
||||
anthropicModelConfig as ReturnType<
|
||||
AiModelRegistryService['getEffectiveModelConfig']
|
||||
>,
|
||||
);
|
||||
|
||||
await service.calculateAndBillUsage(
|
||||
'claude-sonnet-4-5-20250929',
|
||||
{
|
||||
usage: {
|
||||
// inputTokens is the full prompt: noCache(400) + cacheRead(600) +
|
||||
// cacheCreation(200), so quantity must be 1200 + 500, not + 200 again
|
||||
inputTokens: 1200,
|
||||
outputTokens: 500,
|
||||
totalTokens: 1700,
|
||||
inputTokenDetails: {
|
||||
noCacheTokens: 400,
|
||||
cacheReadTokens: 600,
|
||||
cacheWriteTokens: 200,
|
||||
},
|
||||
outputTokenDetails: { textTokens: 500, reasoningTokens: 0 },
|
||||
},
|
||||
cacheCreationTokens: 200,
|
||||
},
|
||||
'workspace-1',
|
||||
UsageOperationType.AI_CHAT_TOKEN,
|
||||
'agent-id-123',
|
||||
);
|
||||
|
||||
expect(
|
||||
mockWorkspaceEventEmitter.emitCustomBatchEvent,
|
||||
).toHaveBeenCalledWith(
|
||||
USAGE_RECORDED,
|
||||
[
|
||||
expect.objectContaining({
|
||||
creditsUsedMicro: 9630,
|
||||
quantity: 1700,
|
||||
}),
|
||||
],
|
||||
'workspace-1',
|
||||
);
|
||||
});
|
||||
});
|
||||
});
|
||||
|
||||
+1
-2
@@ -74,8 +74,7 @@ export class AiBillingService {
|
||||
|
||||
const totalTokens =
|
||||
(billingInput.usage.inputTokens ?? 0) +
|
||||
(billingInput.usage.outputTokens ?? 0) +
|
||||
(billingInput.cacheCreationTokens ?? 0);
|
||||
(billingInput.usage.outputTokens ?? 0);
|
||||
|
||||
if (this.billingService.isBillingEnabled()) {
|
||||
await this.billingUsageService.decrementAvailableCreditsInCache({
|
||||
|
||||
+1
-4
@@ -377,10 +377,7 @@ export class ChatExecutionService {
|
||||
);
|
||||
|
||||
const cacheCreationTokens = extractCacheCreationTokensFromSteps(steps);
|
||||
const totalTokens =
|
||||
(usage.inputTokens ?? 0) +
|
||||
(usage.outputTokens ?? 0) +
|
||||
cacheCreationTokens;
|
||||
const totalTokens = (usage.inputTokens ?? 0) + (usage.outputTokens ?? 0);
|
||||
|
||||
const costInDollars = this.aiBillingService.calculateCost(
|
||||
registeredModel.modelId,
|
||||
|
||||
Reference in New Issue
Block a user