feat(sentry): propagate workspace context to all spans (#20064)

## Summary After the 2026-04-26 token-usage incident, identifying the responsible workspace from a Sentry trace required a Postgres scavenger hunt — Vercel AI SDK auto-instrumentation captures token counts and model name but no twenty-specific identifiers, and that same gap exists for every other auto-instrumented span (HTTP outbound, Postgres queries, GraphQL resolvers, Redis, etc.). This PR plugs that gap globally, not just for AI: - A small utility (`packages/twenty-server/src/engine/core-modules/sentry/utils/sentry-workspace-context.util.ts`) that writes workspace identifiers onto Sentry's active isolation scope as a `twenty` context block plus filterable tags and a `Sentry.setUser` call. - Two hook points covering all server traffic: - **`WorkspaceAuthContextMiddleware`** — already runs after token hydration on the GraphQL, metadata, admin-panel, and REST routes. It now calls the utility once per authenticated request, before delegating to `withWorkspaceAuthContext`. - **`BullMQDriver.work` and `SyncDriver.processJob`** — every queue job now runs inside `Sentry.withIsolationScope` and applies workspace context from `job.data.workspaceId` (skipping silently for system jobs that don't carry one). - A `beforeSendSpan` hook in `instrument.ts` that reads the scope's `twenty` context block back and projects it onto every span as `twenty.workspace.id` and (when available) `twenty.user_workspace.id` — dotted-namespace naming consistent with OTel/Sentry conventions like `user.id` and `http.response.status_code`. Spans without a workspace context (unauthenticated traffic) pass through untouched. ## Why this shape Sentry's docs position `beforeSendSpan` as a per-span hook. The previous iteration set context only at AI-specific call sites, which left non-AI spans (DB queries, outbound HTTP, regular GraphQL queries, workflow steps not touching AI) entirely unenriched. Hooking the two existing global boundaries — auth middleware for HTTP/GraphQL/REST, and the queue driver `work()` callback for background jobs — covers every authenticated span across the app with no per-handler instrumentation. ## What's not in this PR AI-specific identifiers (`twenty.agent.id`, `twenty.thread.id`, `twenty.turn.id`, `twenty.workflow_run.id`) are out of scope here. They're useful additions but require either propagating the IDs through the call stack or a more fine-grained scope (per-step, per-turn) than the request/job boundary, which is best handled in follow-up PRs that target those specific call sites. ## Test plan - [ ] Make any authenticated GraphQL request locally and confirm the resulting span(s) in Sentry carry `twenty.workspace.id` and (for user-authenticated routes) `twenty.user_workspace.id`. - [ ] Make any authenticated REST request and confirm the same. - [ ] Trigger a queue job (chat stream, agent turn evaluation, workflow run, etc.) and confirm spans produced inside the worker carry `twenty.workspace.id`. - [ ] Confirm that DB and outbound HTTP spans produced under the request/job also carry the workspace tag — these previously had no twenty-specific identifiers. - [ ] In the Sentry UI, filter events by the `twenty.workspace.id` tag and confirm matching events appear. ## Notes for review - Sentry init lives in `instrument.ts`, loaded before Nest bootstraps, so `beforeSendSpan` runs outside Nest DI and reads context off the isolation scope rather than holding a service reference. - The middleware change is three lines; the BullMQ wrap is a single `Sentry.withIsolationScope` around the existing job handler body; the SyncDriver wrap mirrors it for the dev/test path. No new modules or DI providers. - The previous iteration's `AiCallContextService` and per-handler `setContext` / `withContext` calls have been removed in favor of these two hooks. 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-27 20:53:50 +02:00
parent 350b835fbc
commit 875795cc30
6 changed files with 126 additions and 17 deletions
@@ -13,6 +13,7 @@ import { buildApiKeyAuthContext } from 'src/engine/core-modules/auth/utils/build
 import { buildApplicationAuthContext } from 'src/engine/core-modules/auth/utils/build-application-auth-context.util';
 import { buildPendingActivationUserAuthContext } from 'src/engine/core-modules/auth/utils/build-pending-activation-user-auth-context.util';
 import { buildUserAuthContext } from 'src/engine/core-modules/auth/utils/build-user-auth-context.util';
+import { applyWorkspaceSentryContext } from 'src/engine/core-modules/sentry/utils/apply-workspace-sentry-context.util';

@Injectable()
 export class WorkspaceAuthContextMiddleware implements NestMiddleware {
@@ -25,6 +26,8 @@ export class WorkspaceAuthContextMiddleware implements NestMiddleware {

    const authContext = this.buildAuthContext(req);

+    applyWorkspaceSentryContext(authContext);
+
    withWorkspaceAuthContext(authContext, () => {
      next();
    });
@@ -4,6 +4,7 @@ import {
  type OnModuleInit,
 } from '@nestjs/common';

+import * as Sentry from '@sentry/node';
 import {
  type JobsOptions,
  MetricsTime,
@@ -28,6 +29,7 @@ import { type MessageQueue } from 'src/engine/core-modules/message-queue/message
 import { getJobKey } from 'src/engine/core-modules/message-queue/utils/get-job-key.util';
 import { type MetricsService } from 'src/engine/core-modules/metrics/metrics.service';
 import { MetricsKeys } from 'src/engine/core-modules/metrics/types/metrics-keys.type';
+import { applyWorkspaceSentryContextFromJobData } from 'src/engine/core-modules/sentry/utils/apply-workspace-sentry-context-from-job-data.util';

 export type BullMQDriverOptions = QueueOptions;

@@ -108,25 +110,28 @@ export class BullMQDriver

    this.workerMap[queueName] = new Worker(
      queueName,
-      async (job) => {
-        // TODO: Correctly support for job.id
-        const timeStart = performance.now();
-        const workspaceId = job.data?.workspaceId;
-        const workspaceSuffix = workspaceId
-          ? ` [workspace=${workspaceId}]`
-          : '';
+      async (job) =>
+        Sentry.withIsolationScope(async () => {
+          applyWorkspaceSentryContextFromJobData(job.data);

-        this.logger.log(
-          `Processing job ${job.id} with name ${job.name} on queue ${queueName}${workspaceSuffix}`,
-        );
-        await handler({ data: job.data, id: job.id ?? '', name: job.name });
-        const timeEnd = performance.now();
-        const executionTime = timeEnd - timeStart;
+          // TODO: Correctly support for job.id
+          const timeStart = performance.now();
+          const workspaceId = job.data?.workspaceId;
+          const workspaceSuffix = workspaceId
+            ? ` [workspace=${workspaceId}]`
+            : '';

-        this.logger.log(
-          `Job ${job.id} with name ${job.name} processed on queue ${queueName} in ${executionTime.toFixed(2)}ms${workspaceSuffix}`,
-        );
-      },
+          this.logger.log(
+            `Processing job ${job.id} with name ${job.name} on queue ${queueName}${workspaceSuffix}`,
+          );
+          await handler({ data: job.data, id: job.id ?? '', name: job.name });
+          const timeEnd = performance.now();
+          const executionTime = timeEnd - timeStart;
+
+          this.logger.log(
+            `Job ${job.id} with name ${job.name} processed on queue ${queueName} in ${executionTime.toFixed(2)}ms${workspaceSuffix}`,
+          );
+        }),
      workerOptions,
    );

@@ -0,0 +1,25 @@
+import { applyWorkspaceSentryFields } from 'src/engine/core-modules/sentry/utils/apply-workspace-sentry-fields.util';
+
+export const applyWorkspaceSentryContextFromJobData = (
+  jobData: unknown,
+): void => {
+  if (typeof jobData !== 'object' || jobData === null) {
+    return;
+  }
+
+  const workspaceId = (jobData as { workspaceId?: unknown }).workspaceId;
+  const userWorkspaceId = (jobData as { userWorkspaceId?: unknown })
+    .userWorkspaceId;
+
+  if (typeof workspaceId !== 'string' || workspaceId.length === 0) {
+    return;
+  }
+
+  applyWorkspaceSentryFields({
+    workspaceId,
+    userWorkspaceId:
+      typeof userWorkspaceId === 'string' && userWorkspaceId.length > 0
+        ? userWorkspaceId
+        : undefined,
+  });
+};
@@ -0,0 +1,27 @@
+import { type WorkspaceAuthContext } from 'src/engine/core-modules/auth/types/workspace-auth-context.type';
+import { applyWorkspaceSentryFields } from 'src/engine/core-modules/sentry/utils/apply-workspace-sentry-fields.util';
+
+export const applyWorkspaceSentryContext = (
+  authContext: WorkspaceAuthContext,
+): void => {
+  const workspaceId = authContext.workspace?.id;
+
+  if (!workspaceId) {
+    return;
+  }
+
+  switch (authContext.type) {
+    case 'user':
+    case 'pendingActivationUser':
+      applyWorkspaceSentryFields({
+        workspaceId,
+        userWorkspaceId: authContext.userWorkspaceId,
+      });
+      return;
+    case 'apiKey':
+    case 'application':
+    case 'system':
+      applyWorkspaceSentryFields({ workspaceId });
+      return;
+  }
+};
@@ -0,0 +1,26 @@
+import * as Sentry from '@sentry/node';
+
+type WorkspaceSentryFields = {
+  workspaceId: string;
+  userWorkspaceId?: string;
+};
+
+export const applyWorkspaceSentryFields = (
+  fields: WorkspaceSentryFields,
+): void => {
+  Sentry.setUser({
+    id: fields.userWorkspaceId ?? fields.workspaceId,
+  });
+
+  Sentry.setTag('twenty.workspace.id', fields.workspaceId);
+  if (fields.userWorkspaceId) {
+    Sentry.setTag('twenty.user_workspace.id', fields.userWorkspaceId);
+  }
+
+  Sentry.setContext('twenty', {
+    workspace_id: fields.workspaceId,
+    ...(fields.userWorkspaceId && {
+      user_workspace_id: fields.userWorkspaceId,
+    }),
+  });
+};
@@ -45,6 +45,29 @@ if (process.env.EXCEPTION_HANDLER_DRIVER === ExceptionHandlerDriver.SENTRY) {
    profilesSampleRate: 0.3,
    sendDefaultPii: true,
    debug: process.env.NODE_ENV === NodeEnvironment.DEVELOPMENT,
+    beforeSendSpan: (span) => {
+      const twentyContext = Sentry.getIsolationScope().getScopeData().contexts
+        ?.twenty as
+        | {
+            workspace_id?: string;
+            user_workspace_id?: string;
+          }
+        | undefined;
+
+      if (!twentyContext?.workspace_id) {
+        return span;
+      }
+
+      span.data = {
+        ...span.data,
+        'twenty.workspace.id': twentyContext.workspace_id,
+        ...(twentyContext.user_workspace_id && {
+          'twenty.user_workspace.id': twentyContext.user_workspace_id,
+        }),
+      };
+
+      return span;
+    },
  });
 }