Etienne c54ad3c0b9 feat(ai-instrumentation): fix AI histogram buckets & add tool-call duration instrumentation (#23173)
## What & why

Two related observability fixes for AI chat / agent / MCP metrics.

### 1. Widen histogram buckets (`widden-ai-histo`)

Latency percentiles for AI chat pinned at exactly **10s** on the
dashboard — not because anything times out, but because the histogram
buckets top out at 10,000.

`recordHistogram` created histograms without explicit bucket boundaries,
so the OTel SDK applied its defaults: `[0, 5, 10, 25, 50, 75, 100, 250,
500, 750, 1000, 2500, 5000, 7500, 10000]`. These were designed for small
generic values; interpreted as **ms**, the last finite bucket is exactly
10s. Every observation above 10s falls into the `+Inf` overflow bucket,
and quantile estimation clamps to the highest finite bound — so any
percentile that lands in the overflow draws a flat line at 10s.

Reality (gpt-5.5, last 30 days): 64% of turns exceed 10s (so even p50
pins at 10s), mean turn latency ~38s, max ~29min. The same 10k cap
affects the token-unit `tool-output-tokens` histograms.

**Changes:**
- `recordHistogram` now accepts an optional `bucketBoundaries`, applied
per-instrument via `advice.explicitBucketBoundaries` (supported in
`@opentelemetry/api@1.9.1`). Colocated with the metric, no
instrument-setup changes.
- Added bucket-boundary constants:
- `AI_LATENCY_MS_BUCKET_BOUNDARIES` — `[250, 500, 1000, 2500, 5000,
10000, 20000, 30000, 60000, 120000, 300000, 600000]` (250ms → 10min)
- `TOOL_OUTPUT_TOKENS_BUCKET_BOUNDARIES` — `[100, 250, 500, 1000, 2500,
5000, 10000, 25000, 50000, 100000, 250000, 500000]` (100 → 500k tokens)
- Wired boundaries into `ai-chat/turn-latency-ms`,
`ai-chat/step-latency-ms`, `ai-chat/ttft-ms` (latency) and
`ai-chat/tool-output-tokens`, `workflow-agent/tool-output-tokens`,
`mcp/tool-output-tokens` (tokens).

### 2. Instrument tool-call duration (`add-tool-duration-monitoring`)

Previously we tracked tool success/failure counts and output tokens, but
not how long each tool call took. Added a `*/tool-execution-duration-ms`
histogram for each execution path.

**Changes:**
- New metric keys: `ai-chat/tool-execution-duration-ms`,
`workflow-agent/tool-execution-duration-ms`,
`mcp/tool-execution-duration-ms`.
- New constant `TOOL_EXECUTION_DURATION_MS_BUCKET_BOUNDARIES` — `[25,
50, 100, 250, 500, 1000, 2500, 5000, 10000, 30000, 60000, 120000]` (25ms
→ 2min).
- **AI chat / agent node**: use the AI SDK's
`experimental_onToolCallFinish` callback (`ai@6.0.97`), which reports an
exact `durationMs` measured around the tool's `execute()`, plus
`toolCall.toolName` and `success`. Attributes `{ model, tool }`.
- **MCP**: measured directly around `tool.execute` with
`performance.now()`, recorded on both success and failure paths.
Attributes `{ tool }`.

## Notes

- Backward-compatible: metrics without `bucketBoundaries` (e.g.
`job/latency-ms`, `sdk-client-generation/duration-ms`) keep OTel
defaults.
- **Historical data stays clamped** — only writes after deploy use the
new buckets, so percentile panels will show a step change at deploy
time. For truth on existing data, use a mean panel (Sum/Count is exact)
or Sentry trace span durations.
- Provider-executed tools (e.g. native web search) run inside the
provider, so they don't emit a local duration — same limitation as the
existing tool counters.

<!-- This is an auto-generated description by cubic. -->
<a
href="https://cubic.dev/pr/twentyhq/twenty/pull/23173?utm_source=github"
target="_blank" rel="noopener noreferrer"
data-no-image-dialog="true"><picture><source
media="(prefers-color-scheme: dark)"
srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source
media="(prefers-color-scheme: light)"
srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img
alt="Review in cubic"
src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a>
<!-- End of auto-generated description by cubic. -->
2026-07-22 17:51:06 +02:00
2026-06-11 11:02:28 +02:00

Twenty logo

The #1 Open-Source CRM

Website · Documentation · Roadmap · Discord · Figma

Twenty banner


Why Twenty

Twenty gives technical teams the building blocks for a custom CRM that meets complex business needs and quickly adapts as the business evolves. Twenty is the CRM you build, ship, and version like the rest of your stack.

Learn more about why we built Twenty


Installation

Cloud

The fastest way to get started. Sign up at twenty.com and spin up a workspace in under a minute, with no infrastructure to manage and always up to date.

Build an app

Scaffold a new app with the Twenty CLI:

npx create-twenty-app my-app

Define objects, fields, and views as code:

import { defineObject, FieldType } from 'twenty-sdk/define';

export default defineObject({
  nameSingular: 'deal',
  namePlural: 'deals',
  labelSingular: 'Deal',
  labelPlural: 'Deals',
  fields: [
    { name: 'name', label: 'Name', type: FieldType.TEXT },
    { name: 'amount', label: 'Amount', type: FieldType.CURRENCY },
    { name: 'closeDate', label: 'Close Date', type: FieldType.DATE_TIME },
  ],
});

Then ship it to your workspace:

npx twenty app:publish --private

See the app development guide for objects, views, agents, and logic functions.

Self-hosting

Run Twenty on your own infrastructure with Docker Compose, or contribute locally via the local setup guide.



Everything you need

Twenty gives you the building blocks of a modern CRM (objects, views, workflows, and agents) and lets you extend them as code. Here's a tour of what's in the box.

Want to go deeper? Read the User Guide for product walkthroughs, or the Documentation for developer reference.

Create your apps

Learn more about apps in doc

Stay on top with version control

Learn more about version control in doc

All the tools you need to build anything

Learn more about primitives in doc

Customize your layouts

Learn more about layouts in doc

AI agents and chats

Learn more about AI in doc

Plus all the tools of a good CRM

Learn more about CRM features in doc


Stack

Thanks

Greptile      Sentry      Crowdin

Thanks to these amazing services that we use and recommend for code review (Greptile), catching bugs (Sentry) and translating (Crowdin).

Join the Community

Star the repo · Discord · Feature requests · Releases · X · LinkedIn · Crowdin · Contribute

S
Description
The open alternative to Salesforce, designed for AI.
Readme AGPL-3.0 1.4 GiB
Languages
TypeScript 79.6%
MDX 17.3%
JavaScript 2.7%
Python 0.2%
SCSS 0.1%