From 74260a161d9aae07769cdb5fd4c5496b7ab1a2a6 Mon Sep 17 00:00:00 2001 From: Etienne <45695613+etiennejouan@users.noreply.github.com> Date: Tue, 28 Jul 2026 11:59:57 +0200 Subject: [PATCH] fix(ai): stop double-counting cache-creation tokens in reported token totals (#23405) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit ## Context Under AI SDK v6 usage normalization, `usage.inputTokens` is the **full prompt**: fresh (noCache) + cache-read + cache-creation tokens. Our `totalTokens` formulas still added `cacheCreationTokens` (extracted from provider metadata) on top of `inputTokens` — a leftover from the pre-v6 SDK generation, where flat `inputTokens` excluded cache tokens. The v6 upgrade changed the semantics under the formula's feet, so every Claude run using prompt caching reported a `totalTokens` inflated by exactly `cacheCreationTokens`. ## Evidence, traced through AI SDK source **1. The Anthropic provider folds cache tokens into `inputTokens`.** The raw Anthropic API reports `input_tokens` *excluding* cache tokens; the provider sums all three components — [`convertAnthropicMessagesUsage`, `@ai-sdk/anthropic@3.0.84`](https://github.com/vercel/ai/blob/%40ai-sdk/anthropic%403.0.84/packages/anthropic/src/convert-anthropic-messages-usage.ts): ```ts inputTokens: { total: inputTokens + cacheCreationTokens + cacheReadTokens, noCache: inputTokens, cacheRead: cacheReadTokens, cacheWrite: cacheCreationTokens, } ``` **2. ai core surfaces that total as the app-visible `usage.inputTokens`** — [`asLanguageModelUsage`, `ai@6.0.97`](https://github.com/vercel/ai/blob/ai%406.0.97/packages/ai/src/types/usage.ts): ```ts inputTokens: usage.inputTokens.total, ... totalTokens: addTokenCounts(usage.inputTokens.total, usage.outputTokens.total), ``` So the SDK's own `totalTokens` is already "full prompt (incl. cache read + creation) + output". **3. The value we were adding on top is the same one already inside `inputTokens`.** The provider also exposes the raw API field in metadata (`@ai-sdk/anthropic` dist): ```ts const anthropicMetadata = { usage: response.usage, cacheCreationInputTokens: response.usage.cache_creation_input_tokens ?? null, ... ``` `extract-cache-creation-tokens.util.ts` reads exactly `providerMetadata.anthropic.cacheCreationInputTokens` — the same `cache_creation_input_tokens` that step 1 already folded into `inputTokens.total`. Adding it again counts it twice. **Worked example** (matches the new pinning test): API returns `input_tokens: 400, cache_read_input_tokens: 600, cache_creation_input_tokens: 200, output_tokens: 500` → app sees `usage.inputTokens = 1200`, `providerMetadata.anthropic.cacheCreationInputTokens = 200` → old formula reported `1200 + 500 + 200 = 1900`; actual tokens processed: `1700`. All snippets are verbatim from the version tags in `vercel/ai` and match the installed `node_modules` dists. ## Provider independence `inputTokens + outputTokens` is correct for every provider Twenty routes through, not just Anthropic: - The v3 provider spec (`@ai-sdk/provider`) defines `inputTokens.total` as "the total number of input (prompt) tokens used", with `noCache`/`cacheRead`/`cacheWrite` as its components — and all 8 installed provider packages comply (verified in dists): `anthropic` and `amazon-bedrock` sum the components explicitly ([`convertBedrockUsage`, `@ai-sdk/amazon-bedrock@4.0.117`](https://github.com/vercel/ai/blob/%40ai-sdk/amazon-bedrock%404.0.117/packages/amazon-bedrock/src/convert-bedrock-usage.ts): `total: inputTokens + cacheReadTokens + cacheWriteTokens`); `openai`, `azure`, `google`, `mistral`, and `openai-compatible` pass through wire values that already include cached tokens; `xai` even detects which wire convention the API used and normalizes either way. - The removed `cacheCreationTokens` term was already 0 for every provider except Anthropic/Bedrock (`extract-cache-creation-tokens.util.ts` only reads those two metadata namespaces), so this PR is a strict no-op for OpenAI-style providers and only removes the double-count where it existed. Caveat: a custom `AI_PROVIDERS` entry pointing at a legacy V2-spec provider package bypasses this normalization (ai core's shim passes flat usage through verbatim); that path could misreport under any formula, and none of the built-in providers use it. ## What changed Four sites computed the inflated total: - `ai-billing.service.ts` — `quantity` on the emitted AI token usage event - `chat-execution.service.ts` — chat-turn usage event - `agent-async-executor.service.ts` — workflow-agent usage event - `build-ai-agent-step-log.util.ts` — workflow step log (display) The first three now compute `totalTokens = inputTokens + outputTokens`; the step-log util uses the SDK's `usage.totalTokens` directly (it receives the `generateText` usage object, where the field is guaranteed). The explicit sum is used where usage objects are hand-assembled or merged — e.g. the streaming path in `stream-agent-chat.job.ts` builds usage literals with no `totalTokens` field at all, so `usage.totalTokens ?? 0` would silently emit 0. Both forms are definitionally identical where the SDK object exists, since ai core computes `totalTokens` as `input + output` (see evidence above). **Impact: reported/analytics quantities only.** Billed credits (`creditsUsedMicro`) come from `computeCostBreakdown`, which already handles the cache-inclusive convention correctly and is unchanged. **Ops note:** `usageEvent.quantity` for cache-heavy workspaces steps down on deploy — dashboards trending this metric may want an annotation. Historical rows are not backfilled (per-row component fields aren't stored, so mixed-era rows can't be reliably corrected). ## How tested - Updated `build-ai-agent-step-log.util.spec.ts` expectation (155 → 150 with `cacheCreationTokens: 5` still present) - New pinning test in `ai-billing.service.spec.ts`: emitted `quantity` is 1700 (not 1900) for inclusive Anthropic usage with `cacheCreationTokens: 200` - New pinning test in `agent-async-executor.service.spec.ts`: emitted total is 150 (not 180) when steps carry `providerMetadata.anthropic.cacheCreationInputTokens` - 3 suites / 20 tests pass; oxlint, oxfmt, and `nx typecheck twenty-server` clean Review in cubic --- .../AiChatInitialLoadingIndicator.tsx | 1 + .../agent-async-executor.service.spec.ts | 41 +++++++++++++++++ .../services/agent-async-executor.service.ts | 3 +- .../__tests__/ai-billing.service.spec.ts | 44 +++++++++++++++++++ .../ai-billing/services/ai-billing.service.ts | 3 +- .../services/chat-execution.service.ts | 5 +-- .../build-ai-agent-step-log.util.spec.ts | 2 +- .../utils/build-ai-agent-step-log.util.ts | 4 +- 8 files changed, 91 insertions(+), 12 deletions(-) diff --git a/packages/twenty-front/src/modules/ai/components/AiChatInitialLoadingIndicator.tsx b/packages/twenty-front/src/modules/ai/components/AiChatInitialLoadingIndicator.tsx index d68648310c..184f96b657 100644 --- a/packages/twenty-front/src/modules/ai/components/AiChatInitialLoadingIndicator.tsx +++ b/packages/twenty-front/src/modules/ai/components/AiChatInitialLoadingIndicator.tsx @@ -10,6 +10,7 @@ const StyledLoadingIconContainer = styled.div` display: flex; justify-content: center; padding-inline: ${themeCssVariables.spacing[1]}; + width: fit-content; `; const StyledLoadingIconWrapper = styled.span` diff --git a/packages/twenty-server/src/engine/metadata-modules/ai/ai-agent-execution/services/__tests__/agent-async-executor.service.spec.ts b/packages/twenty-server/src/engine/metadata-modules/ai/ai-agent-execution/services/__tests__/agent-async-executor.service.spec.ts index d647d8d971..b34ee88977 100644 --- a/packages/twenty-server/src/engine/metadata-modules/ai/ai-agent-execution/services/__tests__/agent-async-executor.service.spec.ts +++ b/packages/twenty-server/src/engine/metadata-modules/ai/ai-agent-execution/services/__tests__/agent-async-executor.service.spec.ts @@ -200,6 +200,47 @@ describe('AgentAsyncExecutorService — workflow agent role-scoped tool resoluti expect(result.creditsUsedMicro).toBe(4200); }); + it('emits the token total without re-adding cache-creation tokens', async () => { + roleTargetRepository.findOne.mockResolvedValueOnce({ + roleId: agentRoleId, + }); + aiBillingService.calculateCost.mockReturnValue(0.0042); + generateTextMock.mockResolvedValueOnce({ + text: '', + steps: [ + { + toolCalls: [], + providerMetadata: { + anthropic: { cacheCreationInputTokens: 30 }, + }, + }, + ], + usage: { + ...baseUsage, + // inputTokens (100) is the full prompt: noCache(60) + cacheRead(10) + + // cacheCreation(30) — the emitted total must not add the 30 again + inputTokenDetails: { + noCacheTokens: 60, + cacheReadTokens: 10, + cacheWriteTokens: 30, + }, + }, + } as unknown as Awaited>); + + await service.executeAgent({ + agent: buildAgent(), + userPrompt: 'test', + workspaceId, + }); + + expect(aiBillingService.emitAiTokenUsageEvent).toHaveBeenCalledTimes(1); + + const [, , emittedTotalTokens] = + aiBillingService.emitAiTokenUsageEvent.mock.calls[0]; + + expect(emittedTotalTokens).toBe(150); + }); + it('folds native web search dollars into totalCostInDollars and creditsUsedMicro', async () => { roleTargetRepository.findOne.mockResolvedValueOnce({ roleId: agentRoleId, diff --git a/packages/twenty-server/src/engine/metadata-modules/ai/ai-agent-execution/services/agent-async-executor.service.ts b/packages/twenty-server/src/engine/metadata-modules/ai/ai-agent-execution/services/agent-async-executor.service.ts index 944b5f6715..3efaf642c4 100644 --- a/packages/twenty-server/src/engine/metadata-modules/ai/ai-agent-execution/services/agent-async-executor.service.ts +++ b/packages/twenty-server/src/engine/metadata-modules/ai/ai-agent-execution/services/agent-async-executor.service.ts @@ -426,8 +426,7 @@ export class AgentAsyncExecutorService { ); const totalTokens = (accumulatedUsage.inputTokens ?? 0) + - (accumulatedUsage.outputTokens ?? 0) + - cacheCreationTokens; + (accumulatedUsage.outputTokens ?? 0); void this.aiBillingService.emitAiTokenUsageEvent( workspaceId, diff --git a/packages/twenty-server/src/engine/metadata-modules/ai/ai-billing/services/__tests__/ai-billing.service.spec.ts b/packages/twenty-server/src/engine/metadata-modules/ai/ai-billing/services/__tests__/ai-billing.service.spec.ts index 0d47a1b92e..eef55b946a 100644 --- a/packages/twenty-server/src/engine/metadata-modules/ai/ai-billing/services/__tests__/ai-billing.service.spec.ts +++ b/packages/twenty-server/src/engine/metadata-modules/ai/ai-billing/services/__tests__/ai-billing.service.spec.ts @@ -384,5 +384,49 @@ describe('AiBillingService', () => { 'workspace-1', ); }); + + it('should not add cache-creation tokens to the emitted token quantity', async () => { + mockAiModelRegistryService.getEffectiveModelConfig.mockReturnValue( + anthropicModelConfig as ReturnType< + AiModelRegistryService['getEffectiveModelConfig'] + >, + ); + + await service.calculateAndBillUsage( + 'claude-sonnet-4-5-20250929', + { + usage: { + // inputTokens is the full prompt: noCache(400) + cacheRead(600) + + // cacheCreation(200), so quantity must be 1200 + 500, not + 200 again + inputTokens: 1200, + outputTokens: 500, + totalTokens: 1700, + inputTokenDetails: { + noCacheTokens: 400, + cacheReadTokens: 600, + cacheWriteTokens: 200, + }, + outputTokenDetails: { textTokens: 500, reasoningTokens: 0 }, + }, + cacheCreationTokens: 200, + }, + 'workspace-1', + UsageOperationType.AI_CHAT_TOKEN, + 'agent-id-123', + ); + + expect( + mockWorkspaceEventEmitter.emitCustomBatchEvent, + ).toHaveBeenCalledWith( + USAGE_RECORDED, + [ + expect.objectContaining({ + creditsUsedMicro: 9630, + quantity: 1700, + }), + ], + 'workspace-1', + ); + }); }); }); diff --git a/packages/twenty-server/src/engine/metadata-modules/ai/ai-billing/services/ai-billing.service.ts b/packages/twenty-server/src/engine/metadata-modules/ai/ai-billing/services/ai-billing.service.ts index fef33acdc7..24bdaab1ce 100644 --- a/packages/twenty-server/src/engine/metadata-modules/ai/ai-billing/services/ai-billing.service.ts +++ b/packages/twenty-server/src/engine/metadata-modules/ai/ai-billing/services/ai-billing.service.ts @@ -74,8 +74,7 @@ export class AiBillingService { const totalTokens = (billingInput.usage.inputTokens ?? 0) + - (billingInput.usage.outputTokens ?? 0) + - (billingInput.cacheCreationTokens ?? 0); + (billingInput.usage.outputTokens ?? 0); if (this.billingService.isBillingEnabled()) { await this.billingUsageService.decrementAvailableCreditsInCache({ diff --git a/packages/twenty-server/src/engine/metadata-modules/ai/ai-chat/services/chat-execution.service.ts b/packages/twenty-server/src/engine/metadata-modules/ai/ai-chat/services/chat-execution.service.ts index eb0b814ead..d74109dff5 100644 --- a/packages/twenty-server/src/engine/metadata-modules/ai/ai-chat/services/chat-execution.service.ts +++ b/packages/twenty-server/src/engine/metadata-modules/ai/ai-chat/services/chat-execution.service.ts @@ -377,10 +377,7 @@ export class ChatExecutionService { ); const cacheCreationTokens = extractCacheCreationTokensFromSteps(steps); - const totalTokens = - (usage.inputTokens ?? 0) + - (usage.outputTokens ?? 0) + - cacheCreationTokens; + const totalTokens = (usage.inputTokens ?? 0) + (usage.outputTokens ?? 0); const costInDollars = this.aiBillingService.calculateCost( registeredModel.modelId, diff --git a/packages/twenty-server/src/modules/workflow/workflow-executor/workflow-actions/ai-agent/utils/__tests__/build-ai-agent-step-log.util.spec.ts b/packages/twenty-server/src/modules/workflow/workflow-executor/workflow-actions/ai-agent/utils/__tests__/build-ai-agent-step-log.util.spec.ts index f09dae0c0f..f5820c8b0d 100644 --- a/packages/twenty-server/src/modules/workflow/workflow-executor/workflow-actions/ai-agent/utils/__tests__/build-ai-agent-step-log.util.spec.ts +++ b/packages/twenty-server/src/modules/workflow/workflow-executor/workflow-actions/ai-agent/utils/__tests__/build-ai-agent-step-log.util.spec.ts @@ -60,7 +60,7 @@ describe('buildAiAgentStepLog', () => { reasoningTokens: 10, cacheReadTokens: 20, cacheCreationTokens: 5, - totalTokens: 155, + totalTokens: 150, }); expect(stepLog.details.cost).toEqual({ totalCostInDollars: 0.012, diff --git a/packages/twenty-server/src/modules/workflow/workflow-executor/workflow-actions/ai-agent/utils/build-ai-agent-step-log.util.ts b/packages/twenty-server/src/modules/workflow/workflow-executor/workflow-actions/ai-agent/utils/build-ai-agent-step-log.util.ts index af279bc01b..5677559d91 100644 --- a/packages/twenty-server/src/modules/workflow/workflow-executor/workflow-actions/ai-agent/utils/build-ai-agent-step-log.util.ts +++ b/packages/twenty-server/src/modules/workflow/workflow-executor/workflow-actions/ai-agent/utils/build-ai-agent-step-log.util.ts @@ -31,9 +31,7 @@ export const buildAiAgentStepLog = ({ executionResult.usage.outputTokenDetails?.reasoningTokens, cacheReadTokens: executionResult.usage.inputTokenDetails?.cacheReadTokens, cacheCreationTokens: executionResult.cacheCreationTokens, - totalTokens: - (executionResult.usage.totalTokens ?? 0) + - executionResult.cacheCreationTokens, + totalTokens: executionResult.usage.totalTokens ?? 0, }, cost: { totalCostInDollars: executionResult.totalCostInDollars ?? 0,