feat(ai) - add observability (#20850)

**AI Chat - Tool Executions (counters, tagged with model)**
ai-chat/tool-execution-succeeded: number of tool calls invoked by the AI
that completed without error
ai-chat/tool-execution-failed: number of tool calls invoked by the AI
that threw an error
**AI Chat - Token Usage (counters, tagged with model)**
ai-chat/input-tokens: total input tokens sent to the model across all
turns
ai-chat/output-tokens: total output tokens generated by the model
ai-chat/cache-read-tokens: input tokens served from the model's prompt
cache (cheaper)
ai-chat/cache-write-tokens: input tokens written into the prompt cache
for future reuse
**AI Chat - Latency (histograms in ms, tagged with model)**
ai-chat/turn-latency-ms: total duration of a full chat turn (from stream
start to stream end)
ai-chat/step-latency-ms: duration of a single reasoning/tool-call step
within a turn
ai-chat/ttft-ms: time-to-first-token, i.e. how long until the model
starts streaming output
**MCP - Tool Executions (counters)**
mcp/tool-execution-succeeded: number of MCP tool calls that completed
successfully
mcp/tool-execution-failed: number of MCP tool calls that threw an error
This commit is contained in:
Etienne
2026-05-22 17:32:51 +02:00
committed by GitHub
parent de044f4b45
commit eda41b4eba
23 changed files with 169 additions and 33 deletions
@@ -116,7 +116,7 @@ export const useGraphQLErrorHandlerHook = <
setResult,
}) => {
if (!result.errors || result.errors.length === 0) {
void options.metricsService.incrementCounter({
void options.metricsService.incrementCounterForEvent({
key: MetricsKeys.GraphqlOperation200,
});
@@ -178,11 +178,11 @@ export const useGraphQLErrorHandlerHook = <
}
if (metricKey) {
void options.metricsService.incrementCounter({
void options.metricsService.incrementCounterForEvent({
key: metricKey,
});
} else {
void options.metricsService.incrementCounter({
void options.metricsService.incrementCounterForEvent({
key: MetricsKeys.GraphqlOperationUnknown,
});
}
@@ -282,7 +282,7 @@ export const useGraphQLErrorHandlerHook = <
isDefined(currentMetadataVersion) &&
requestMetadataVersion !== `${currentMetadataVersion}`
) {
void options.metricsService.incrementCounter({
void options.metricsService.incrementCounterForEvent({
key: MetricsKeys.SchemaVersionMismatch,
});
@@ -312,7 +312,7 @@ export const useGraphQLErrorHandlerHook = <
isDefined(backendMajor) &&
frontEndMajor < backendMajor
) {
void options.metricsService.incrementCounter({
void options.metricsService.incrementCounterForEvent({
key: MetricsKeys.AppVersionMismatch,
});
throw new GraphQLError(APP_VERSION_MISMATCH_ERROR, {