Ai Chat - Caching optim (#20126)

EDIT : 
- solving auto-caching from Anthropic by updating ai-sdk/anthropic +
adding providerOption at stream level
- concerning Bedrock, it needs breakpoint


**1. Breakpoints were only on the system prompt**

The code already placed a cache marker on the system prompt (~10K
tokens). But the conversation history — which can grow to hundreds of
thousands of tokens — had no marker, so Anthropic re-read it at full
price on every turn.

The fix adds a prepareStep hook inside streamText that stamps the last
message with a cache breakpoint before every LLM call. Anthropic then
caches the entire conversation prefix, and subsequent turns read it at
$0.30/M instead of $3/M.

prepareStep is used rather than a one-shot pre-processing step because
an agentic turn makes multiple internal LLM calls as tool results
accumulate — the hook refreshes the breakpoint before each one.

**2. Bedrock was using the wrong field**

The system prompt marker for Bedrock was set as cacheControl: { type:
'ephemeral' } — which is the Anthropic wire format. The Bedrock Converse
API expects cachePoint: { type: 'default' }. The system prompt was
silently not being cached on Bedrock at all.

Both the system prompt and the new prepareStep now go through a shared
getCacheProviderOptions helper that returns the correct field per
provider.

**3. Persisted cached token usage to monitor cache strat. efficiency**
This commit is contained in:
Etienne
2026-04-29 16:42:12 +02:00
committed by GitHub
parent 11628d19a3
commit fd6d5f895d
9 changed files with 195 additions and 151 deletions
@@ -18,6 +18,7 @@ import { AddConditionalAvailabilityExpressionToPageLayoutWidgetFastInstanceComma
import { AddIsPreInstalledToApplicationRegistrationFastInstanceCommand } from 'src/database/commands/upgrade-version-command/2-1/2-1-instance-command-fast-1776886452831-add-is-pre-installed-to-application-registration';
import { AddProviderExecutedToAgentMessagePartFastInstanceCommand } from 'src/database/commands/upgrade-version-command/2-1/2-1-instance-command-fast-1777012800000-add-provider-executed-to-agent-message-part';
import { BackfillPageLayoutWidgetPositionSlowInstanceCommand } from 'src/database/commands/upgrade-version-command/2-1/2-1-instance-command-slow-1795000002000-backfill-page-layout-widget-position';
import { AddCacheTokensToAgentChatThreadFastInstanceCommand } from 'src/database/commands/upgrade-version-command/2-2/2-2-instance-command-fast-1777455269302-add-cache-tokens-to-agent-chat-thread';
export const INSTANCE_COMMANDS = [
AddViewFieldGroupIdIndexOnViewFieldFastInstanceCommand,
@@ -38,4 +39,5 @@ export const INSTANCE_COMMANDS = [
AddIsPreInstalledToApplicationRegistrationFastInstanceCommand,
AddProviderExecutedToAgentMessagePartFastInstanceCommand,
BackfillPageLayoutWidgetPositionSlowInstanceCommand,
AddCacheTokensToAgentChatThreadFastInstanceCommand,
];