Commit Graph

518 Commits

Author SHA1 Message Date
nitin 996cdaf3ff refactor(agents): split tool resolution into native and action rails (#20331)
## Summary

Splits AI agent tool resolution into two independent rails:

- **Native tools** — capabilities baked into the model SDK
(Anthropic/OpenAI `web_search`, xAI `web`/`x` provider options). Bound
by `NativeToolBinderService`, controlled by per-agent
`modelConfiguration` toggles. Opaque to Twenty — executed on the model
provider's servers.
- **Action tools** — registry-scoped tools from `ToolRegistryService`
(code interpreter, send email, record CRUD, etc.). Permission-gated via
the agent's role. Executed on Twenty's server.

Both rails merge into a single `ToolSet` at call time. When both
surfaces expose a search tool the model picks at runtime — coexistence
is intentional (relevant once Exa returns as an action, see below).

## Notable changes worth calling out

**Contract change: `AgentAsyncExecutorService.executeAgent` no longer
accepts `rolePermissionConfig`.** Workflow agents now scope exclusively
by the agent's own permission-tab role (`unionOf: [agentRoleId]`). The
previous role-merging path (caller role intersected with agent role) is
removed. No agent role → no registry tools (fail-closed by design).

**`NativeToolBinderService` relocated** from
`core-modules/tool-provider/native/` →
`metadata-modules/ai/ai-models/services/`. The binder needs SDK-package
knowledge, which lives in `ai-models`. Old location created a backwards
module dependency.

**`NATIVE_MODEL_TOOLS_BY_SDK_PACKAGE` is exhaustive over
`AiSdkPackage`** (`Record<>`, not `Partial<Record<>>`). Adding a new SDK
without thinking about native tools now fails the build. SDKs without
native tools (Bedrock, Google, Mistral, Azure, OpenAI-compatible) get
explicit `{}` entries.

**Discriminated union `kind: 'sdk-tool' | 'provider-option'`** lets one
registry describe both function tools (Anthropic/OpenAI) and runtime
sources (xAI). Follows the local `tool-provider` convention from #19321.

## Deferred to follow-ups

- **Exa web search is dropped from this PR** (along with its
`WEB_SEARCH_TOOL` permission flag and the Exa-specific gating). Exa
comes back as an **action/app tool** once apps can define permission
flags through the SDK — ongoing work in #20481.
- **xAI native search currently errors.** xAI deprecated its Live Search
API (the `web`/`x` provider-option sources this rail maps to), so xAI
returns `410` when native search is actually exercised. The code path
itself is clear — it's only hit if you test xAI native tools. Fixed
separately alongside the broader xAI model fixes.

## Conscious non-decisions

- **No "twenty-native" category.** `native` is reserved for
model/provider SDK features; everything Twenty-owned is just a
tool/action.
- **Coexistence over precedence.** No rule forcing an action search tool
to override native search (or vice-versa) — when both exist, it's the
user's choice in workflow agents and the model's choice in chat.

---------

Co-authored-by: Félix Malfait <felix.malfait@gmail.com>
Co-authored-by: Félix Malfait <felix@twenty.com>
2026-05-28 22:08:05 +02:00
Félix Malfait f4ead89956 refactor(twenty-orm): migrate 23 grandfathered entities to WorkspaceScopedRepository (#20987)
## Summary

Follow-up to #20953. Migrates 23 of the 30 entities that were left in
`WORKSPACE_SCOPED_EXEMPTIONS` last time, so the lint rule's
workspaceId-enforcement default now covers most of the core/metadata
schema.

### Migrated (23 entities, 88 files, 22 commits)

| Family | Entities |
|---|---|
| Trivial caches | `NavigationMenuItem`, `Skill`, `DataSource`,
`Webhook`, `CommandMenuItem`, `IndexMetadata` |
| Views | `View`, `ViewField`, `ViewFieldGroup`, `ViewFilter`,
`ViewFilterGroup`, `ViewGroup`, `ViewSort` |
| Layouts | `PageLayout`, `PageLayoutTab`, `PageLayoutWidget` |
| Roles & permissions | `Role`, `RoleTarget`, `PermissionFlag`,
`ObjectPermission`, `FieldPermission`, `RowLevelPermissionPredicate`,
`RowLevelPermissionPredicateGroup` |

For each entity: swap `@InjectRepository(X)` →
`@InjectWorkspaceScopedRepository(X)` (and the field type →
`WorkspaceScopedRepository<X>`); rewrite every call site to pass
`workspaceId` as the first arg (stripped from `where`/criteria — the
wrapper throws if you include it now); register
`provideWorkspaceScopedRepository(X)` in every owning NestJS module;
update affected spec providers to
`getWorkspaceScopedRepositoryToken(X)`.

### Rule update

- `ApplicationRegistrationVariableEntity` was misclassified — moved to
`STRUCTURAL_EXEMPTIONS` (no `workspaceId` column; it's keyed on
`applicationRegistrationId` at the instance level).
- 22 of the 23 migrated entities removed from
`WORKSPACE_SCOPED_EXEMPTIONS` entirely (zero remaining raw
`@InjectRepository` sites).
- `RoleTargetEntity` also removed; one call site in
`user-workspace.service.ts` keeps a raw injection with an
`eslint-disable` + reason because `softRemove(...)` is not on the
wrapper API yet (the migration would require threading `workspaceId`
through `deleteUserWorkspace`'s three callers).

### Still exempted (7 entities, follow-up PRs)

| Entity | Why deferred |
|---|---|
| `ApplicationEntity` | ~50 sites with several cross-workspace lookups
by id (auth, OAuth, file-storage, cleanup) |
| `CalendarChannelEntity` / `MessageChannelEntity` | Use
`.increment(...)` (not on wrapper) and
`repository.manager.transaction(...)` — wrapper needs to grow
`.increment` + the transaction sites need `withManager` or dual-inject |
| `FieldMetadataEntity` / `ObjectMetadataEntity` | The metadata services
`extends TypeOrmQueryService<X>` and `super(rawRepo)` — requires
dual-inject or reworking the inheritance |
| `KeyValuePairEntity` | Allows `workspaceId: IsNull()` for
instance-level config; wrapper rejects null |
| `UpgradeMigrationEntity` | Same — instance-level + cross-workspace
ledger |

## Test plan

- [x] `npx nx typecheck twenty-server` — clean
- [x] `npx nx lint twenty-server` — clean (0/0)
- [x] All 10 affected unit specs pass (115 tests) — api-key, agent-role,
permissions, workspace-roles-permissions-cache, view-filter-group,
workflow-version-step-operations, two-factor-authentication (service +
resolver), user-workspace, file
- [ ] Server integration tests in CI
2026-05-28 20:46:21 +02:00
Félix Malfait 4797d2f270 feat(twenty-orm): introduce WorkspaceScopedRepository for core/metadata workspace-scoped entities (#20953)
## Summary

Adds a third tenancy enforcement layer for entities that live in shared
schemas (`core`, `metadata`) and carry a `workspaceId` column —
previously the only safeguard at this layer was developer discipline
(remembering to put `workspaceId` in every WHERE clause).

### The three layers, after this PR

| Layer | Scope | How it's enforced |
|---|---|---|
| 1. Workspace data | per-workspace schema (companies, people, custom
objects) | `twentyORMManager.getRepository(workspace, E)` — physical
isolation (own data source) |
| 2. Metadata | shared `metadata` schema (objectMetadata, fieldMetadata,
views, roles…) | Flat-entity-maps cache — workspace-scoped in-memory
map, lookups by id within it |
| 3. Core (new) | shared `core` schema (agent threads/turns/messages,
app tokens, etc.) | `WorkspaceScopedRepository<T>` — `workspaceId` is a
required positional argument on every read/write |

## What's in the PR

### The wrapper
(`packages/twenty-server/src/engine/twenty-orm/workspace-scoped-repository/`)
- `WorkspaceScopedRepository<T extends WorkspaceScopedEntity>` — wraps a
TypeORM `Repository<T>`, requires `workspaceId` on every
`find`/`findOne`/`findOneOrFail`/`update`/`delete`/`softDelete`/`insert`/`save`/`count`
call, merging it into the WHERE or stamping it on the entity.
`createQueryBuilder` is an explicit escape hatch (caller scopes
manually).
- Provided via Nest DI with
`@InjectWorkspaceScopedRepository(EntityClass)` and the
`provideWorkspaceScopedRepository(EntityClass)` provider factory.
- 19 unit tests cover the merge behavior, override-on-conflict, and the
array-where (OR) case.

### Lint enforcement
(`packages/twenty-oxlint-rules/rules/prefer-workspace-scoped-repository.ts`)
- New `twenty/prefer-workspace-scoped-repository` rule (level:
**error**).
- Blacklist of entity names: raw `@InjectRepository(E)` is rejected if
`E` is on the list.
- Initial list: `AgentTurnEntity`, `AgentMessageEntity`,
`AgentMessagePartEntity`, `AgentChatThreadEntity`,
`AgentTurnEvaluationEntity`, `AgentEntity`.
- Designed to grow over time as more consumers are migrated.
- 5 rule tests.

### Migration in this PR
All consumers of the six blacklisted entities, including:
- AI agent / chat / monitor resolvers, services, and jobs
- `AgentService`, `AiAgentRoleService`, `AiAgentWorkflowAction`,
`ApplicationService`, `WorkspaceFlatAgentMapCacheService`
- Admin-panel chat (migrated where the lookup is workspace-known; one
documented `eslint-disable` on the threadId-discovery lookup that
necessarily precedes the `allowImpersonation` permission check)
- `AiAgentRoleService` unit spec updated to mock the scoped wrapper

## Future work (deliberately not in this PR)

A standalone audit identified ~14 additional `core`/`metadata` entities
with `workspaceId` that currently use raw `@InjectRepository` and could
be added to the blacklist. Notable candidates: `UserWorkspaceEntity` (42
sites), `AppTokenEntity` (10), `FileEntity` (7),
`BillingCustomerEntity`/`BillingSubscriptionEntity` (~22 combined). Each
should be its own PR — the migration is mechanical but the surface is
wide.

## Test plan
- [x] `npx nx typecheck twenty-server` — clean
- [x] `npx nx lint twenty-server` — 0 warnings, 0 errors
- [x] `npx jest workspace-scoped-repository` — 19/19 pass
- [x] `npx nx test twenty-oxlint-rules` — 215/215 pass
- [x] `npx jest src/engine/metadata-modules/ai` — 44/44 pass
- [ ] Manual smoke: end-to-end AI agent chat send/receive (reviewer)
- [ ] Manual smoke: AI agent monitor — list turns, run evaluation
(reviewer)
- [ ] Manual smoke: admin-panel chat thread inspection (reviewer)
2026-05-27 18:52:53 +02:00
Etienne eda41b4eba feat(ai) - add observability (#20850)
**AI Chat - Tool Executions (counters, tagged with model)**
ai-chat/tool-execution-succeeded: number of tool calls invoked by the AI
that completed without error
ai-chat/tool-execution-failed: number of tool calls invoked by the AI
that threw an error
**AI Chat - Token Usage (counters, tagged with model)**
ai-chat/input-tokens: total input tokens sent to the model across all
turns
ai-chat/output-tokens: total output tokens generated by the model
ai-chat/cache-read-tokens: input tokens served from the model's prompt
cache (cheaper)
ai-chat/cache-write-tokens: input tokens written into the prompt cache
for future reuse
**AI Chat - Latency (histograms in ms, tagged with model)**
ai-chat/turn-latency-ms: total duration of a full chat turn (from stream
start to stream end)
ai-chat/step-latency-ms: duration of a single reasoning/tool-call step
within a turn
ai-chat/ttft-ms: time-to-first-token, i.e. how long until the model
starts streaming output
**MCP - Tool Executions (counters)**
mcp/tool-execution-succeeded: number of MCP tool calls that completed
successfully
mcp/tool-execution-failed: number of MCP tool calls that threw an error
2026-05-22 15:32:51 +00:00
neo773 323e66433e lint: migrate prettier to oxfmt (#20783)
Most changes are `implements` being unwrapped this is not a oxfmt
regression
Prettier in 3.7 (we're on 3.1) changed this behaviour prettier blog
[post](https://prettier.io/blog/2025/11/27/3.7.0#change-18094)

This unifies our linting tooling

---------

Co-authored-by: github-actions <github-actions@twenty.com>
Co-authored-by: Charles Bochet <charles@twenty.com>
2026-05-22 00:21:33 +02:00
Thomas Trompette 138eb5a74a Add empty operands to UUID filter type in workflow filter action (#20821)
## Summary
- Adds `IS_EMPTY` and `IS_NOT_EMPTY` operands to the UUID entry in
`getStepFilterOperands`, aligning the workflow filter action with the
find records (search) action which already includes these operands for
ID-type fields.

## Test plan
- [ ] Open a workflow with a filter action, select an ID-type field, and
verify the operand dropdown now includes "Is empty" and "Is not empty"
- [ ] Open a workflow with a find records action, select an ID-type
field, and verify the operand dropdown is consistent with the filter
action

---------

Co-authored-by: Charles Bochet <charles@twenty.com>
2026-05-21 19:06:10 +00:00
Marie 869f8af4f0 Fix workflow cron trigger cache stuck without TTL (#20812)
## Problem

The cron-trigger cache key (`module:workflow:workflow-cron-triggers`)
can get stuck without a TTL, silently halting **all** cron-triggered
workflows for a whole tenant until the key is manually deleted from
Redis.

Repro path:

1. Cache miss → DB-scan branch runs.
2. Inner loop writes triggers via `hashSet` (creates the key, **no TTL
yet**).
3. Worker crashes / OOMs / gets killed by a deploy between any `hashSet`
and the trailing `expire(1h)` call.
4. Key now exists with TTL = `-1` and a partial set of fields.
5. Next tick: `hashGetValues` returns those fields →
`cachedValues.length > 0` → **cache-hit branch** → `expire` is never
called.
6. Key has no TTL, so it never auto-expires. The DB-scan branch never
runs again. New / missing triggers are never picked up. Workflows go
silent.

Observed in production: cache key with `TTL: no limit` and 121 fields.
Deleting the key restored normal behaviour (next tick rebuilt with TTL
~3600).

## Fix

Set the TTL right after first value is added 

## Monitoring

Added a "Cache miss" log count in workflow dashboard, counted among the
last 6 hours. Turns green if >= 5
<img width="1627" height="721" alt="Screenshot 2026-05-21 at 16 46 13"
src="https://github.com/user-attachments/assets/8262dd5f-fbbd-43c9-aede-c0ce5d6a0f59"
/>

---------

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-05-21 16:03:09 +00:00
Thomas Trompette b454ad2aea fix(workflow): restore initial input fields on code step creation (#20756)
## Summary

- Fixes a regression from #20208 where creating a new CODE workflow step
shows no input fields
- The split-triggers PR removed `SEED_LOGIC_FUNCTION_INPUT_SCHEMA` and
replaced `toolInputSchema` with `workflowActionTriggerSettings`, but
`CodeStepBuildService.createCodeStepLogicFunction` was not updated to
pass the seed schema — causing `logicFunctionInput` to default to `{}`
and no fields to render
- Adds `SEED_WORKFLOW_ACTION_TRIGGER_SETTINGS` constant (matching the
seed template's `{ a: string, b: number }` params) and passes it when
creating the seed logic function

## Test plan

- [x] Unit test updated to assert `logicFunctionInput` contains `{ a:
null, b: null }` on code step creation
- [x] Create a new CODE step in the workflow builder and verify input
fields `a` and `b` appear immediately

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-05-20 11:21:17 +00:00
Abdul Rahman c800eccc65 Slack workflow connector (#20427)
https://github.com/user-attachments/assets/5a746414-988b-473c-9401-b8863a3e1c15


https://github.com/user-attachments/assets/0cdebdb1-f7c8-43cb-beef-f279387b6ce9



https://github.com/user-attachments/assets/df31c631-0781-42d8-8e6e-e5a16573ee3b



https://github.com/user-attachments/assets/6adaeae4-f3c9-4a5f-b0df-50c1f9a78428

---------

Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: martmull <martmull@hotmail.fr>
2026-05-20 10:21:22 +00:00
Charles Bochet 1a9f786e42 refactor(filters): pass fieldMetadataItems array to dispatcher (#20737)
## Summary

Alternative to #20717. Same goal (clean up the filter dispatcher API
after #20670) but smaller and follows the codebase's "pass data, not
behavior" style.

The dispatcher takes a `fieldMetadataItems: FieldShared[]` array
directly instead of a `findFieldMetadataItemById: (id) => FieldShared |
undefined` callback. The util builds the id lookup internally — once per
call, used for both source-field and relation-target-field lookups. No
new types, no separate hydration step.

## What changes

**`twenty-shared`**
- `computeRecordGqlOperationFilter` /
`turnRecordFilterIntoRecordGqlOperationFilter` /
`turnRecordFilterGroupsIntoGqlOperationFilter`: replace
`findFieldMetadataItemById` param with `fieldMetadataItems` /
`fieldMetadataItemById` (internal Map).
- Remove the exported `FindFieldMetadataItemById` type.
- `turnAnyFieldFilterIntoRecordGqlFilter`: rename its internal
`fieldById` Map for consistency.
- Tests updated to pass arrays.

**Frontend (15 call sites)**
- Switch from `fieldMetadataItemByIdMapSelector` to
`flattenedFieldMetadataItemsSelector`.
- Pass `fieldMetadataItems: flattenedFieldMetadataItems` to the
dispatcher.
- `useFindManyRecordsSelectedInContextStore` keeps the Map selector
because it still does a per-filter lookup for the soft-delete check.

**Server (5 call sites)**
- Pass
`Object.values(flatFieldMetadataMaps.byUniversalIdentifier).filter(isDefined)`.

## Why this over #20717

#20717 moves resolution into a separate hydration step + introduces a
`HydratedRecordFilter` type. The bug that #20717 originally surfaced was
Sentry catching 4 critical runtime errors during review
(`fieldMetadataItemByIdMap` declared but not passed). The added type and
the explicit hydration boundary are extra surface area for not much
benefit — the existing API was a callback wrapping a Map at every call
site, and the natural simplification is to just pass the Map (or its
array) directly.

Net diff: **196 insertions, 203 deletions** (~7 lines net removed). 32
files.

## Test plan
- [x] Shared filter unit tests pass (461 tests)
- [x] Frontend filter/context-store tests pass (13 tests)
- [x] Frontend typecheck passes
- [x] Server typecheck passes
- [x] Lint passes (frontend + server)
- [ ] Integration tests on #20670 still pass — workflow find-records +
chart-data with relation-traversal filter still work end-to-end through
the new array param
2026-05-19 22:49:53 +02:00
Abdul Rahman 08e7e4819b use declared outputSchema for logic-function steps (#20679)
When a logic function declares
`workflowActionTriggerSettings.outputSchema`, use it as the step's
initial output schema so downstream steps can pick variables without
first running the Test tab. A successful test run still overrides the
schema with the inferred shape, preserving "test wins" behavior. Falls
back to the existing "Generate Function Output" LINK placeholder when no
schema is declared (custom code steps, older functions).


https://github.com/user-attachments/assets/af9c45ed-d623-4234-be9f-46812fd06e2e
2026-05-19 14:41:00 +00:00
Félix Malfait 291ce5ccdb fix(filters): make filter dispatcher own relation-target resolution (#20670)
## Summary

Two relation-traversal bugs surfaced post-merge of #20533, both rooted
in the same architectural smell: the GraphQL filter dispatcher took a
flat `fields: FieldShared[]` array and silently dropped any filter whose
`relationTargetFieldMetadataId` wasn't in that array. Callers had to
remember to pre-augment the list with relation targets — and 16+ call
sites did not all know this.

This PR fixes both bugs and removes the smell.

### Bug 1 — Save as new view loses the relation target

`useCreateViewFromCurrentView` built the create-filter input without
`relationTargetFieldMetadataId`. The saved view's filter persisted
without the traversal — on reload the chip showed "Company contains
'air'" instead of "Company → Name contains 'air'". Discarded at save
time, not at read time.

Fix: include `relationTargetFieldMetadataId` in the create input.
(Commit 1.)

### Bug 2 — Workflow Search Records drops one-hop traversals

`FindRecordsWorkflowAction` built its fields list from
`flatObjectMetadata.fieldIds` only (source object's fields). The shared
dispatcher then couldn't resolve the relation target field on the
related object and silently dropped the filter — a configured "People
where Company → Name Contains 'Airbnb'" came through as `{ and: [] }`.

This was the same shape as bugs already fixed in 5 other call sites
(chart filters, view filters, record table, etc.). The pattern was:
caller forgets to augment fields → dispatcher silently drops the filter.

Fix (commit 2): change the dispatcher to take a
`findFieldMetadataItemById: (id) => FieldShared | undefined` resolver
callback. Both source-field and relation-target-field lookups go through
the same resolver, so callers no longer need to know about the
augmentation requirement. Frontend callers pass a workspace-wide
resolver built from `flattenedFieldMetadataItemsSelector`; server
callers wrap `findFlatEntityByIdInFlatEntityMaps` on
`flatFieldMetadataMaps`. In both cases relation-target lookups just
work, because the resolver can see fields on related objects.

## Why this matters

Before: "if you call the dispatcher, pre-augment your fields list with
relation targets, or filters get silently dropped." An invariant only
enforceable by code review, broken often enough to ship two user-visible
bugs in one week.

After: the dispatcher resolves field ids itself. There's no list to
forget to augment. The failure mode (filter silently dropped) becomes
structurally impossible at the dispatcher boundary.

Net diff: 240 insertions, 319 deletions. Removed
`augmentFieldsWithRelationTargets` (frontend) and the workflow
whack-a-mole code (server).

## Test plan
- [ ] Save view: create an advanced filter using a one-hop relation
traversal, click "Save as new view", reload, confirm the chip still
reads "Source → Target operator value"
- [ ] Workflow: configure a Search Records action with a
relation-traversal filter, run the workflow, confirm the filter is
actually applied
- [ ] Dashboard chart: configure a chart with a relation-traversal
filter, confirm the chart data respects it
- [ ] Record table, group-by, calendar, total count, footer aggregates:
all continue to work with both plain and relation-traversal filters
2026-05-19 11:55:22 +02:00
Etienne aecfe699f4 feat(ai-chat) - Stop ai thinking if credits exhausted (#20526)
Billing is now decremented per-step, not per-turn. The onStepFinish
callback in chat-execution.service.ts calls a new
decrementAndCheckAvailableCredits method on each model step, so Redis is
debited incrementally as the agent runs rather than all at once at the
end.

Credit exhaustion stops the stream mid-run. When a step depletes the
remaining credits, a hasNoMoreAvailableCredits flag is set and passed
into the stopWhen predicate of streamText, causing the agent to halt
before starting the next step.

A new credits-exhausted event is introduced. After the stream drains and
the response is persisted, if credits ran out the job publishes a
dedicated credits-exhausted event to the frontend instead of the normal
message-persisted event.

The frontend handles this new event. useAgentChatSubscription has a new
credits-exhausted case that sets a BILLING_CREDITS_EXHAUSTED-coded error
on the atom, closes the writer, and stops the streaming state —
triggering the existing AiChatCreditsExhaustedMessage UI.
2026-05-13 15:09:08 +00:00
Etienne 3d90b3882b fix(ai-agent-node) - agent node execution error (#20534)
**Root cause:** getWorkflowRunContext(stepInfos) builds a Record<string,
unknown> from the previous steps' results. There is no workspaceId key
in it, so context.workspaceId as string silently evaluated to undefined.
That undefined was then passed all the way down to
WorkspaceCacheService.getOrRecompute, **which correctly throws** when
workspaceId is not a valid UUID.

Before : 
<img width="525" height="130" alt="Screenshot 2026-05-13 at 14 58 54"
src="https://github.com/user-attachments/assets/0549b4dc-7063-44e5-95a1-00a460a6d7f1"
/>

Introduced with billing v2 yesterday, since then, workspaceId is needed
to bill credit usage
2026-05-13 13:20:27 +00:00
Paul Rastoin a159a68e2c [twenty-server] no floating promises lint rule (#20499)
## Introduction

That's an audit + RFC

## Fire-and-forget (`void`) -- Intentional, correct

These are telemetry, metrics, and audit logging in hot paths or
non-critical contexts. `void` is the right choice.

| File | What was voided |
|---|---|
| `sign-in-up.service.ts` | `metricsService.incrementCounter` (sign-up
metric) + `auditService.insertWorkspaceEvent` (workspace created) |
| `use-graphql-error-handler.hook.ts` | 5x
`metricsService.incrementCounter` (GraphQL operation metrics) |
| `bullmq.driver.ts` | 2x `metricsService.incrementCounter` (job
completed/failed metrics) |
| `call-webhook.job.ts` | 2x `auditService.insertWorkspaceEvent` + 1x
`metricsService.incrementCounter` |
| `custom-domain-manager.service.ts` | `analytics.insertWorkspaceEvent`
(domain activation event) |
| `logic-function-executor.service.ts` |
`auditService.insertWorkspaceEvent` (function execution) |
| `workflow-runner.workspace-service.ts` |
`metricsService.incrementCounter` (throttle metric) |
| `cleaner.workspace-service.ts` | `metricsService.incrementCounter`
(deleted workspace metric) |
| `stream-agent-chat.job.ts` | Detached IIFE for streaming chunks
(intentional concurrent pipeline) |
| `workspace-auth-context.middleware.ts` |
`withWorkspaceAuthContext(...)` (AsyncLocalStorage, returns void anyway)
|

## Top-level script entry points (`void bootstrap()`)

These are module-level calls where the promise has no consumer. `void`
makes the lint rule happy and documents the intent.

| File | What changed |
|---|---|
| `main.ts` | `void bootstrap()` |
| `command.ts` | `void bootstrap()` |
| `queue-worker.ts` | `void bootstrap()` |
| `truncate-db.ts` | `void dropSchemasSequentially()` |
| `codegen/index.ts` | `void generateTests(forceArg)` |

## Now properly awaited -- Real bug fixes

These were floating promises that could silently fail, lose data, or
cause race conditions.

| File | What was fixed |
|---|---|
| `billing-sync-plans-data.command.ts` | `meters.map(async ...)` wrapped
in `Promise.all` -- was returning before upserts finished |
| `cache-storage.service.ts` | `setAdd` and `setPop` had `.then()`
chains that weren't returned/awaited |
| `create-audit-log-from-internal-event.ts` | 4x
`auditService.createObjectEvent` now awaited inside a job |
| `cleaner.workspace-service.ts` | 2x `emailService.send(...)` now
awaited -- emails could silently fail |
| `agent-async-executor.service.ts` | `calculateAndBillUsage` +
`billNativeWebSearchUsage` in `finally` block now awaited |
| `repair-tool-call.util.ts` | `calculateAndBillUsage` now awaited |
| `agent-title-generation.service.ts` | `calculateAndBillUsage` now
awaited |
| `chat-execution.service.ts` | `billNativeWebSearchUsage` now awaited |
| `ai-generate-text.controller.ts` | `calculateAndBillUsage` in
`finally` block now awaited |
| `agent-turn.resolver.ts` | `messageQueueService.add(...)` now awaited
|
| `command.ts` | `app.close()` now awaited (was exiting before graceful
shutdown) |
| `i18n.service.ts` | `loadTranslations()` in `onModuleInit` now awaited
|
| `workspace-query-hook.explorer.ts` | `explore()` in `onModuleInit` now
awaited |
| `message-queue.explorer.ts` | `handleProcessorGroupCollection` in
`onModuleInit` now awaited |
| `ai-billing.service.spec.ts` | Test now properly `await`s the async
call |
| `messaging-messages-import.service.spec.ts` | `expect(...)` now
properly `await`ed for async assertion |
| `archive.finalize()` (3 files) | Voided -- promise resolution already
handled by `pipeline()` / `on('end')` |

## Impersonation & security audit trail -- Upgraded from `void` to
`await`

These were previously fire-and-forget but are
security/compliance-critical events that must be reliably persisted.

| File | What was fixed |
|---|---|
| `impersonation.service.ts` | 4x `auditService.insertWorkspaceEvent`
now awaited (impersonation attempt, token generation
attempt/success/failure) |
| `auth.resolver.ts` | 5x `auditService.insertWorkspaceEvent` now
awaited (impersonation token exchange attempt/success/failure at server
and workspace levels) |
| `auth.service.ts` | 2x `analytics.insertWorkspaceEvent` now awaited
(impersonation attempted/issued) |

## Billing audit -- Upgraded from `void` to `await`

Payment events should be reliably persisted for financial/compliance
reporting.

| File | What was fixed |
|---|---|
| `billing-webhook-invoice.service.ts` |
`auditService.insertWorkspaceEvent(PAYMENT_RECEIVED_EVENT)` now awaited
inside Stripe webhook handler |

## Fire-and-forget with proper error handling -- Upgraded from bare
`void`

These remain non-blocking but now catch and log errors instead of
risking unhandled rejections.

| File | What was fixed |
|---|---|
| `logic-function-executor.service.ts` |
`applicationLogsService.writeLogs` now uses `.catch()` instead of bare
`void` -- user-facing logs should surface errors |

## Systemic infrastructure fixes

| File | What was fixed |
|---|---|
| `metrics.service.ts` | `incrementCounter`: Redis cache write
(`metricsCacheService.updateCounter`) now uses `.catch()` internally
instead of raw `await` -- prevents unhandled rejections across all `void
metricsService.incrementCounter(...)` call sites when Redis is unhealthy
|
| `audit.service.ts` | `preventIfDisabled`: made properly `async` with
`await` and consistent `Promise<{ success: boolean }>` return type.
Removed broken `catch` that returned an `AuditException` as a value
(wrong constructor args, unreachable dead code). Removed unused
`AuditException` import |

## Fixed in this session (beyond original PR)

| File | What changed |
|---|---|
| `telemetry.listener.ts` | Removed misleading `Promise.all` + `void`
combo; replaced with simple `for...of` + `void` |
| `message-queue.explorer.ts` | Changed from `void` to `await` so
startup crashes on registration failure |
2026-05-13 10:08:42 +00:00
Abdul Rahman 93df64b9b0 Show logic function label instead of technical name in workflow UI (#20470)
### Before
<img width="1304" height="812" alt="Screenshot 2026-05-12 at 7 02 32 AM"
src="https://github.com/user-attachments/assets/94ca4b1d-69c0-4059-8c45-dd8eae8e2a29"
/>



### After
<img width="1296" height="782" alt="Screenshot 2026-05-12 at 6 53 26 AM"
src="https://github.com/user-attachments/assets/47f2e291-73df-4471-9174-bd5aca23e228"
/>
2026-05-12 05:46:56 +00:00
Etienne 9fc5be1c4c Billing - Migrate from Stripe metering (#20298)
**Overall strategy**
**1. Introduce “Billing V2” behind a workspace flag**
Gate the new model with FeatureFlagKey.IS_BILLING_V2_ENABLED so existing
workspaces stay on the old behavior until they’re migrated or explicitly
on V2.

**2. Replace workflow metered SKUs with a resource-credit product**
Conceptually, billable “workflow execution” usage is not the primary
subscription line item anymore. Add a RESOURCE_CREDIT product (and keep
WORKFLOW_NODE_EXECUTION as deprecated for the transition). Usage and
limits are expressed through credit buckets (e.g. price metadata like
credit_amount), so one product can represent pooled credits instead of a
narrow workflow-only meter.

**3. Migrate subscriptions in two layers**
Schema/catalog: persist extra price metadata (instance upgrade) so the
server knows credit amounts and can match Stripe prices to the new
model.
Per workspace: the registered workspace command
upgrade:2-2:migrate-to-billing-v2 finds subscriptions that still have
WORKFLOW_NODE_EXECUTION, swaps those items to the right RESOURCE_CREDIT
prices (using existing Stripe schedule +
BillingSubscriptionUpdateService stack), then treats the workspace as V2
(flag). Workspaces without that legacy item or without a subscription
are skipped.

**4. Unify subscription lifecycle + usage on the server**

**5. Refresh the product surface in Settings**

Test : 

- [x] Subscribe v1 +  Update subscribe + Migrate
- [x]  Subscribe v2 + Update subscribe
2026-05-07 15:42:11 +00:00
Charles Bochet 10876138d2 refactor: stop reading joinColumnName from relation field settings (#20304)
## Summary

`joinColumnName` on relation field settings is always derivable from the
field name (and the target object name for morph relations). This PR
stops reading it from settings anywhere in production code; the stored
value is no longer used.

The settings field is **not** removed from data yet — a follow-up can
drop it once we are confident nothing depends on the stored value.

## Helpers

The helpers are split by layer because frontend and backend hold morph
relations differently: the frontend has a base name plus a
`morphRelations[]` array, the backend has one row per target with the
name already morph-resolved.

| Helper | Layer | When to use |
|---|---|---|
| `computeRelationGqlFieldJoinColumnName` | Shared / frontend
(`gqlField`) | Non-morph relation on the frontend. |
| `computeMorphRelationGqlFieldName` | Shared / frontend (`gqlField`) |
Need the per-target morph gqlField name (e.g. `targetCompany`). |
| `computeMorphRelationGqlFieldJoinColumnName` | Shared / frontend
(`gqlField`) | Per-target morph join column on the frontend. Prefer over
the non-morph helper for any morph field — it forces the per-target
inputs. |
| `computeMorphOrRelationFieldJoinColumnName` | Backend
(`FlatFieldMetadata.name`) | Any backend read or write — the flat name
is already morph-resolved, so one helper covers both cases. |
| `computeMorphRelationFlatFieldName` | Backend
(`FlatFieldMetadata.name`) | **Mutation paths only** (create / update /
object rename). Reads consume the stored `field.name` and never call
this. |

## Test plan

- [x] Typecheck and lint (front, server, shared)
- [x] Existing unit tests pass
- [ ] CI green
2026-05-07 09:53:00 +02:00
Félix Malfait 53fdac1417 feat(apps): split AI tool and workflow action triggers in LogicFunction manifest (#20208)
## Summary

Replaces the bolted-on `isTool` + `toolInputSchema` fields on
`LogicFunctionManifest` with two distinct, opt-in triggers that align
with the existing `cron` / `databaseEvent` / `httpRoute` trigger
pattern:

- **`toolTriggerSettings`** — exposes the function as an AI tool (chat /
MCP / function calling). Uses standard JSON Schema (the format LLMs
natively understand).
- **`workflowActionTriggerSettings`** — exposes the function as a step
in the visual workflow builder. Uses Twenty's rich `InputSchema` so the
builder can render proper `FieldMetadataType`-aware editors, variable
pickers, labels, and an optional `outputSchema`.

A function can opt into none, one, or both. Each surface gets the schema
format appropriate for it.

### Why

`isTool: true` previously exposed the function as both an AI tool AND a
workflow node, with the same JSON Schema feeding both — but the workflow
builder really wants Twenty's `InputSchema` (with `CURRENCY`,
`RELATION`, `EMAILS`, etc.) and the AI surface really wants standard
JSON Schema. Today the workflow builder hacks around this by treating
JSON Schema as `InputSchema`, which silently breaks for any
non-primitive field type. Splitting the triggers fixes that and lets
each surface evolve independently.

### Migration

- **Fast** instance command adds the two new nullable columns.
- **Slow** instance command backfills `toolTriggerSettings` +
`workflowActionTriggerSettings` from `isTool=true` rows (preserving
today's both-surfaces behaviour) then drops the legacy columns.

### Stacked

Stacked on top of #20181. Merge that first, then this.

## Test plan

- [ ] CI green (oxlint, typecheck, jest, vitest)
- [ ] Run `--include-slow` upgrade against a workspace with existing
`isTool=true` logic functions; verify both new columns populated and old
columns dropped
- [ ] Verify AI chat sees migrated tool functions (Linear create-issue,
Exa search) and can call them with the JSON Schema
- [ ] Add an AI-tool function from the Settings UI (toggles
`toolTriggerSettings`) and verify it shows up in chat
- [ ] Add a workflow-action function from the Settings UI (toggles
`workflowActionTriggerSettings`) and verify it appears in the workflow
node picker
- [ ] In the workflow builder, edit a `LOGIC_FUNCTION` step and verify
input fields render (no more JSON-Schema-as-InputSchema hack)
- [ ] Try defining a function with no triggers in the SDK and verify
`defineLogicFunction` rejects it

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Co-authored-by: martmull <martmull@hotmail.fr>
2026-05-05 14:56:09 +02:00
Félix Malfait 842e679cc6 fix(billing): gate AI credit-cap at entry points instead of workflow executor (#20096)
## Background

The 2026-04-26 incident saw 716M Sonnet 4.6 tokens consumed in a single
trial workspace. Two causes: failed agent executions weren't billed
(addressed by #20065) and the credit-cap gate had been removed from
`WorkflowExecutorWorkspaceService.executeStep` in #19904, leaving no
enforcement point at all.

## Why not just revert #19904

#19904 was right that gating at the workflow executor is too coarse.
When one user exhausted a workspace's credits via chat, *all* workflows
hard-failed mid-run — including cheap DB/CRUD/branch automations costing
essentially nothing. Reverting would re-introduce that cliff.

## New design: gate at the AI entry points

The chat resolver already gates this way
(`agent-chat.resolver.ts:137-148`). This PR replicates the same pattern
at every other point where the workspace can incur real AI cost:

- `executeAgent` in `agent-async-executor.service.ts`
- the REST handler in `ai-generate-text.controller.ts`
- `generateThreadTitle` in `agent-title-generation.service.ts`

In each, after auth/validation: skip if `IS_BILLING_ENABLED` is false;
otherwise call `BillingService.canBillMeteredProduct(workspaceId,
BillingProductKey.WORKFLOW_NODE_EXECUTION)`; on `false`, throw
`BillingException(BILLING_CREDITS_EXHAUSTED)`. No new method, no new
exception code, no new product key.

This matches industry convention (Lovable/Replit also gate at the
expensive-operation boundary, not at every cheap step).

## Deliberately not gated

- `WorkflowExecutorWorkspaceService.executeStep` — the design choice is
now intentional, so the #19904 TODO is replaced by a one-line
absolute-behavior comment explaining why the gate isn't here. Cheap
workflow steps (DB CRUD, branching, action steps) are not gated, so a
chat-driven cap exhaustion does not block non-AI automations.
- `repair-tool-call.util` — repair is a sub-call inside an already-gated
AI flow. If the parent is gated, repair will naturally not run. Adding a
gate here adds complexity without value.

## Net effect

A workspace that exhausts credits via chat or AI agent stops making AI
calls. Its non-AI workflows continue running normally. A workflow with
both AI and non-AI steps fails at the AI step with
`BILLING_CREDITS_EXHAUSTED`, but downstream non-AI steps that don't
depend on the AI output still run.

## Conflicts

This PR overlaps with three other in-flight PRs in the same files. None
of them touch the gate logic; rebasing on top of any of them is trivial:

- #20065 (agent-async-executor): adds `workspaceId` to `executeAgent`
args and bills in `finally`. The gate at the top of `executeAgent` from
this PR sits naturally above that.
- #20066 (REST controller): adds usage billing to the controller.
- #20067 (title gen): adds usage billing to title generation and
tool-call repair.

Recommend landing #20065/#20066/#20067 first; this PR rebases trivially
on top.

## Tests

Out of scope per the PR series convention. The existing chat-resolver
gate isn't unit-tested either; this PR follows the same precedent.
Follow-up: add integration coverage that exercises a workspace at
`hasReachedCurrentPeriodCap=true` against each of the three new gates
plus the pre-existing chat-resolver gate.

## Future follow-ups

- Per-user soft cap inside a workspace (the Lovable Business-tier
pattern), so one user can't exhaust the workspace's cap.
- Pre-flight cost estimate so the user sees an "approaching cap" warning
before the hard stop.
- Rename `BillingProductKey.WORKFLOW_NODE_EXECUTION` — the name predates
this design choice and is misleading now that it gates AI entry points
rather than workflow nodes.

## Test plan

- [ ] Trigger a workspace into `hasReachedCurrentPeriodCap=true`.
- [ ] Send a chat message — expect failure with
`BILLING_CREDITS_EXHAUSTED`.
- [ ] Run a workflow whose only AI step is an `ai-agent` action — expect
that step to fail with `BILLING_CREDITS_EXHAUSTED`, downstream non-AI
steps still run.
- [ ] POST to `/rest/ai/generate-text` — expect
`BILLING_CREDITS_EXHAUSTED`.
- [ ] Create a new chat thread (which kicks off `generateThreadTitle`) —
expect `BILLING_CREDITS_EXHAUSTED`.
- [ ] Run a workflow with no AI step (only DB CRUD/branching/actions) —
expect it to run unaffected.

---------

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-29 18:48:06 +02:00
Etienne fbaea0639a Billing - optimize usageEvent CH table (#20019)
- Update usageEvent clickhouse table, partitioning, indexing and
projection (auto materialized view) to optimize credit usage queries
- Add caching for available credits and billing subscription


To do in next PR: deprecate enforceCapUsage cron. Bonus : real-time on
billingSubscription
2026-04-28 14:06:49 +00:00
Sai Sathwik P a5cd64daf5 refactor: standardize JsonStringified casing (#20101)
## Summary
- Rename safeParseRelativeDateFilterJSONStringified to
safeParseRelativeDateFilterJsonStringified
- Update the matching utility file, exports, tests, and workflow usages

Part of #19839.

## Validation
- CI passed
2026-04-28 09:01:26 +00:00
Félix Malfait df3e217d64 fix(ai-billing): bill executeAgent in a finally block so failed runs don't leak (#20065)
## Summary

`AgentAsyncExecutorService.executeAgent` consumes Anthropic tokens at
two points (the main `generateText` and the optional structured-output
sub-call). Billing was previously the **caller's** responsibility,
executed only after `executeAgent` returned. If `executeAgent` threw —
e.g. when `structuredResult.output == null` for a schema-mismatched
response, or anything caught by the catch-and-rethrow — we paid
Anthropic but never recorded a `usageEvent`. Likely the dominant source
of the 716M-token-vs-3.27-credits discrepancy seen on the affected
workspace in the 2026-04-26 incident.

## What changed

- Inject `AiBillingService` into `AgentAsyncExecutorService`. Add
`workspaceId` (required), `userWorkspaceId`, and `operationType`
(default `AI_WORKFLOW_TOKEN`) to `executeAgent`'s args.
- Capture `accumulatedUsage`, `cacheCreationTokens`, and
`nativeWebSearchCallCount` into mutable locals as each `generateText`
resolves. A throw between the main and structured-output calls still
bills the first call's tokens; the schema-validation throw still bills
the merged usage.
- Wrap the body in `try { ... } finally { ... }`. The finally calls
`calculateAndBillUsage` and `billNativeWebSearchUsage`, each guarded by
its own `try/catch + logger.error` so a billing exception can't mask the
original execution error or block the second emit.
- `ai-agent.workflow-action.ts`: pass the new args; drop the
now-redundant billing calls and `AiBillingService` injection.
`AiBillingModule` removed from this action's module imports.
- `run-evaluation-input.job.ts`: pass `workspaceId` (already in `data`)
and `userWorkspaceId: null`. **As a side effect, the eval pipeline now
bills correctly** — closing an additional billing leak from the audit
(`RunEvaluationInputJob` previously called `executeAgent` and discarded
`executionResult.usage`).

## Behavior change worth calling out

Previously, failed agent executions were silently free. They will now be
billed for the tokens Anthropic charged us. This is intentional and
correct.

## Test plan
- [ ] Trigger a workflow agent action that succeeds — `usageEvent` count
should match what was previously emitted.
- [ ] Trigger a workflow agent with a JSON response schema and ambiguous
input that produces a non-schema-conforming output
(`structuredResult.output == null`) — verify a `usageEvent` row is now
written for the consumed tokens (was 0 rows previously).
- [ ] Trigger a `runEvaluationInput` GraphQL mutation — verify a
`usageEvent` row is written (was 0 rows previously).

## Notes for review
- Conflicts trivially with the Sentry-context PR on
`run-evaluation-input.job.ts`. Recommend merging Sentry-context first;
this PR's 2-line argument addition rebases inside that PR's
`aiCallContextService.run(...)` callback wrapper.
- A small follow-up after this lands: thread `billingContext` through
the `experimental_repairToolCall` callback in
`agent-async-executor.service.ts:198` (currently marked with a TODO from
the title-gen+repair-tool PR).

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-28 10:43:53 +02:00
Thomas Trompette fa3d0cd4a6 Force uuids in AI workflow tools (#20010)
- Add .uuid() Zod validation to all AI workflow tool schemas
(workflowVersionId, workflowId, stepId, edge target) so the AI model is
constrained to use valid UUIDs instead of human-readable strings like
step-find-stale-leads
- Fixes Sentry noise from Invalid UUID errors triggered when workflows
created with non-UUID step IDs are later edited via GraphQL mutations
that enforce UUID scalars

Will fix
https://twenty-v7.sentry.io/issues/7240446584/events/05c7782655a34f4a8f5fc889164026ab/?project=4507072499810304&referrer=previous-event
2026-04-23 16:27:53 +00:00
Félix Malfait 0c929e7903 refactor(tool-provider): rename web_search to exa_web_search, drop XOR toggle (#19969)
## Summary

- Today `WEB_SEARCH_PREFER_NATIVE` forces a **mutual exclusion**: either
the custom Exa tool preloads as `web_search` or the SDK-native
`web_search` binds. Same name, different backends.
- This PR lets them **coexist**. Custom Exa becomes `exa_web_search`;
native keeps `web_search`. The model picks based on tool descriptions.
- `WEB_SEARCH_PREFER_NATIVE` and `shouldUseNativeSearch()` are deleted.
Exa enablement follows `WEB_SEARCH_DRIVER` (existing). Native enablement
follows the agent's `modelConfiguration.webSearch.enabled` (existing).

## Key changes

**Config / service**
- Deleted `WEB_SEARCH_PREFER_NATIVE` (config-variables.ts)
- Deleted `WebSearchService.shouldUseNativeSearch()`
- `WebSearchService.isEnabled()` unchanged — still gates Exa
availability

**Custom tool rename**
- `ActionToolProvider.toolMap`: `'web_search'` → `'exa_web_search'`
- Descriptor name matches
- `WebSearchTool.description` rewritten to position Exa as
structured/entity-aware, complementary to native

**Native tool binder**
- `NativeToolBinder.bind()` drops the `shouldUseNativeSearch` gate.
Per-agent `modelConfiguration.webSearch.enabled` (inside
`getNativeModelTools`) stays authoritative.

**Chat**
- Preload list now always includes `exa_web_search` —
`ActionToolProvider` silently skips the descriptor when Exa is disabled,
so `getToolsByName` degrades gracefully
- Native tools always attempted; returns empty ToolSet when the model
doesn't support them
- `directTools = { ...preloadedTools, ...nativeSearchTools }` — both
present when both enabled
- `billNativeWebSearchUsage` called unconditionally (the function
already short-circuits on count ≤ 0)

**Workflow agent**
- Same unconditional billing pattern
- `WebSearchService` dependency removed

**System prompt**
- Dropped the special-cased `web_search` branch. Preloaded tools list
uniformly now.

**Frontend**
- `exa_web_search` reuses the same "Searching the web for X" display as
native
- Test coverage added

## Billing isolation (verified)

- `countNativeWebSearchCallsFromSteps` counts `toolName ===
'web_search'` only. After the rename, only native calls match. Exa calls
(`exa_web_search`) are billed separately via
`WebSearchService.emitUsageEvent` inside `search()`.
- No double-billing path.

## Behavior deltas (intended)

| Scenario | Before | After |
|---|---|---|
| Anthropic model + Exa enabled + PREFER_NATIVE=true | native only |
**both** |
| Anthropic + Exa enabled + PREFER_NATIVE=false | Exa only (as
`web_search`) | **both** |
| Non-native model + Exa enabled | Exa as `web_search` | Exa as
`exa_web_search` |
| Any model + Exa disabled + native supported | native only | native
only |
| Workflow agent with `webSearch.enabled=true` + Anthropic + Exa enabled
| native only | **both** |

## Known regression (accepted)

Customers who set `WEB_SEARCH_PREFER_NATIVE=false` to force Exa-only
will now **also** see native `web_search` if the model supports it.
There's no chat-level kill switch after this PR. Per discussion, this is
accepted — future model-level capability gating (in the model JSON) will
be the right place for that control.

## Stats

- 10 files, +63 / −73 (net deletion)
- Typecheck clean (server: 7 pre-existing unrelated, front: 13
pre-existing unrelated — zero new either side)
- Prettier clean

## Test plan

- [ ] `npx nx typecheck twenty-server` and `npx nx typecheck
twenty-front` pass
- [ ] With Anthropic + Exa enabled: chat shows both `web_search` and
`exa_web_search` in preloaded list; model can call either
- [ ] With Anthropic + Exa disabled: chat shows only native `web_search`
- [ ] With non-native model + Exa enabled: chat shows only
`exa_web_search`
- [ ] Workflow agent with `modelConfiguration.webSearch.enabled=true` +
Exa enabled: both available
- [ ] Billing: native calls billed via `billNativeWebSearchUsage`; Exa
calls billed via `WebSearchService.emitUsageEvent`; no double-billing
- [ ] Frontend: `exa_web_search` renders "Searching the web for X" the
same as `web_search`

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-22 14:57:44 +02:00
neo773 62ea14a072 fix email workflow (#19929)
fixes JSON parse crash with proper resolution of variables and tiptap
rich text classification
2026-04-21 13:47:19 +02:00
neo773 a1de37e424 Fix Email composer rich text to HTML conversion (#19872) 2026-04-21 08:52:05 +02:00
Charles Bochet 1b469168c8 chore(workflow): temporarily lift credit-cap gate on workflow steps (#19904)
## Summary

- Removes the per-step `canBillMeteredProduct(WORKFLOW_NODE_EXECUTION)`
gate in `WorkflowExecutorWorkspaceService.executeStep` so workflows keep
running when a workspace reaches `hasReachedCurrentPeriodCap`.
Previously every step failed with
`BILLING_WORKFLOW_EXECUTION_ERROR_MESSAGE` (\"No remaining credits to
execute workflow…\").
- Drops the now-unused `BillingService` injection, related imports, and
the helper `canBillWorkflowNodeExecution`. Updates the spec to drop the
corresponding billing-validation case and mock.
- Leaves the constant file and `BillingService` itself in place, plus a
TODO at the previous gate site, so the behavior can be re-enabled with a
small, reviewable revert.

## Notes

- Usage events are still emitted (`USAGE_RECORDED` /
`UsageResourceType.WORKFLOW`), and `EnforceUsageCapJob` keeps computing
the cap and flipping `hasReachedCurrentPeriodCap` — only the executor
stops consulting that flag.
- The runner-level `canFeatureBeUsed` check in
`WorkflowRunnerWorkspaceService.run` was already log-only (subscription
presence, not credits), so no change there.
- AI chat (`agent-chat.resolver.ts`) keeps its own
`BILLING_CREDITS_EXHAUSTED` gate; this PR does not touch it.

## Test plan

- [x] `npx jest workflow-executor.workspace-service.spec.ts` (17/17
pass)
- [ ] Manual: with billing enabled and the metered subscription item
flagged `hasReachedCurrentPeriodCap = true`, trigger a workflow run and
verify steps execute end-to-end instead of failing with the billing
error.

Made with [Cursor](https://cursor.com)
2026-04-21 01:00:28 +02:00
Etienne 117909e10a Billing - Adapt to new unit (#19886) 2026-04-20 14:28:07 +00:00
Thomas Trompette 446a3923f2 Add workspace id in job logs (#19764)
Cannot currently investigate spikes
2026-04-16 15:28:37 +00:00
Thomas Trompette cd31d9e0de Add test tab to tool step (#19760)
So we can use those as variables.
When the function input is updated, invalidate the input using an
effect.

<img width="770" height="635" alt="Capture d’écran 2026-04-16 à 14 45
41"
src="https://github.com/user-attachments/assets/cbf0b3e4-baad-424d-8f08-06eb1028abdb"
/>
2026-04-16 14:00:21 +00:00
Thomas Trompette 8a968d1e31 Hide workflow manual trigger from command menu on "Select All" (#19718)
https://github.com/user-attachments/assets/e19c6d23-03c1-49c8-8b83-5c8f5f0f1135
2026-04-15 13:55:38 +00:00
Thomas Trompette 46ee72160d Fix infinite recursion in iterator loop traversal when If/Else branch loops back to enclosing iterator (#19714)
Fix a stack overflow (Maximum call stack size exceeded) in
getAllStepIdsInLoop caused by an If/Else branch inside an iterator loop
pointing back to the enclosing iterator. The traversal incorrectly
treated the enclosing iterator as a nested iterator, calling
getAllStepIdsInLoop recursively with fresh visited sets, causing
infinite recursion.

Add the enclosing iterator's own ID to the skip condition in
traverseSteps so back-edges from If/Else branches are handled the same
way as back-edges from regular nextStepIds.

<img width="1054" height="723" alt="Capture d’écran 2026-04-15 à 11 00
42"
src="https://github.com/user-attachments/assets/aee1477b-5059-4552-809e-7c8a34a9ec4a"
/>
2026-04-15 09:23:25 +00:00
Thomas Trompette 31baf52528 Workflow - Avoid billing skipped steps (#19547)
As title
2026-04-10 12:49:14 +00:00
Charles Bochet b284c8323c Remove Favorite and FavoriteFolder from workspace schema (#19536)
## Summary

- Removes all workspace schema definitions for `Favorite` and
`FavoriteFolder` entities, which have been fully migrated to
`NavigationMenuItems`
- Deletes 26 standalone files including workspace entities, NestJS
modules, services, listeners, jobs, standard application builders (field
metadata, views, view fields, view field groups, indexes, page layouts),
mocks, and integration tests
- Cleans up ~40 modified files: removes `favorites` relation from 10
workspace entities and their field metadata utils, removes entries from
all builder maps, shared constants (`STANDARD_OBJECTS`,
`CoreObjectNameSingular`, `DEFAULT_RELATIONS_OBJECTS_STANDARD_IDS`), SDK
default relations, AI tool filtering, and standard object icons
2026-04-10 12:47:06 +02:00
Charles Bochet f6423f5925 Remove DataSourceService and clean up datasource migration logic (#19532)
## Summary

- **Drop the `objectMetadata.dataSourceId` foreign key and index** via a
1-22 fast instance command — column kept nullable for data preservation
- **Delete `DataSourceService`, `DataSourceModule`, and
`DataSourceException`** — all code now uses `workspace.databaseSchema`
directly
- **Remove `IS_DATASOURCE_MIGRATED` feature flag** from default flags
and all branching logic
- **Simplify workspace/object creation pipelines** —
`WorkspaceManagerService`, `DevSeederService`, and the object creation
action handler no longer route through `DataSourceService`
- **Keep `DataSourceEntity` and the `dataSource` table** for historical
data — entity stripped of all ORM relations
2026-04-10 07:34:05 +00:00
Thomas Trompette e048d03872 Improve workflow crons efficiency (#19381)
**Optimize workflow cron jobs: partition workspaces and use raw
queries**

- Split all 3 workflow cron jobs (WorkflowRunEnqueueCronJob,
WorkflowHandleStaledRunsCronJob, WorkflowCleanWorkflowRunsCronJob) to
process only 1/10th of workspaces per invocation using minute-based
partitioning, reducing per-run load
- Replace ORM repository + workspace context loading with raw SQL
queries in WorkflowRunEnqueueCronJob and
WorkflowHandleStaledRunsCronJob, avoiding costly cache/metadata
hydration for a simple existence check
2026-04-07 14:09:11 +00:00
Félix Malfait ea572975d8 feat: generic web search driver abstraction with Exa support and billing (#19341)
## Summary

- Introduces a pluggable `WebSearchDriver` abstraction (interface,
factory, service, module) so web search is no longer tied to native
provider tools (Anthropic/OpenAI)
- **Exa** is the first driver implementation with support for
category-filtered search (company, people, news, research paper, etc.) —
particularly useful for CRM workflows
- Per-query billing for both Exa ($0.007/query) and native provider
surcharges ($0.01/query for Anthropic/OpenAI) via the existing
`USAGE_RECORDED` pipeline
- New config variables: `WEB_SEARCH_DRIVER` (EXA/DISABLED),
`EXA_API_KEY`, `WEB_SEARCH_PREFER_NATIVE` (default false — prefers Exa
over native when both available)
- `WEB_SEARCH` operation type added for usage tracking and Stripe
metering

### Architecture

```
WebSearchDriver (interface)
├── ExaDriver          — Exa neural search with category support
└── DisabledDriver     — throws when search is disabled

WebSearchDriverFactory (extends DriverFactoryBase)
└── creates driver based on WEB_SEARCH_DRIVER config

WebSearchService (facade)
├── search(query, options?, billingContext?)
├── isEnabled()
└── emits USAGE_RECORDED events per query

WebSearchTool (Tool implementation)
└── registered in ActionToolProvider, available via tool catalog
```

### Native search billing gap fixed

Anthropic and OpenAI both charge $0.01/search on top of token costs. The
token costs were already billed, but the per-call surcharge was not.
Added `countNativeWebSearchCallsFromSteps` utility +
`billNativeWebSearchUsage` to `AiBillingService`, wired into both chat
and workflow agent paths.

## Test plan

- [ ] Set `WEB_SEARCH_DRIVER=EXA` + `EXA_API_KEY=...` and verify AI chat
can search the web
- [ ] Verify category parameter works (ask about a specific
company/person)
- [ ] Set `WEB_SEARCH_DRIVER=DISABLED` and verify search tool is not
exposed
- [ ] Set `WEB_SEARCH_PREFER_NATIVE=true` with Anthropic model and
verify native search is used
- [ ] Verify usage events are emitted in ClickHouse for both Exa and
native search paths
- [ ] Verify existing billing tests pass (`npx jest
ai-billing.service.spec.ts`)


Made with [Cursor](https://cursor.com)

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Co-authored-by: claude[bot] <41898282+claude[bot]@users.noreply.github.com>
2026-04-06 12:02:46 +02:00
Thomas Trompette 90597e47ca Add redis cache for cron triggers (#19306)
- WorkflowCronTriggerCronJob was querying every active workspace (~700)
sequentially every minute to find cron triggers, causing regular CPU
spikes to 100% on worker pods

- Added a Redis hashset cache that stores cron triggers. On cache miss
(TTL expired, cold start, or explicit invalidation), a full scan
rebuilds the cache

- Creating/deleting a new cron trigger updates the cache, only if exists
2026-04-03 15:08:31 +00:00
Paul Rastoin cd23a2bc80 Cleaning UpgradeCommand code flow (#19241)
# Introduction
Currently preparing the `UpgradeCommand` refactor, in this way started
by cleaning up the existing avoiding unecessary dependencies to others
services allowing easier readability and concern centralization for
upcoming refactor
The UpgradeCommand was extending up to five classes, overriding
abstracted class and so on. It was also cascade
injecting 3 services

Introducing the `WorkspaceIteratorService` that centralize the commands
set to run over a single workspace logic shared between both atomic
upgrade command call and global upgradeCommand

## Tradeoff
Duplicated `@Option` between both `UpgradeCommandRunner` and
`WorkspaceMigrationRunner`


## Before
```
UpgradeCommand
  └─ extends UpgradeCommandRunner
       └─ extends ActiveOrSuspendedWorkspacesMigrationCommandRunner
            └─ extends WorkspacesMigrationCommandRunner  (owns workspace iteration loop + ORM deps)
                 └─ extends MigrationCommandRunner       (dry-run, verbose, error handling)
                      └─ extends CommandRunner            (nest-commander)
```

## Now

```
UpgradeCommand
  └─ extends UpgradeCommandRunner
       └─ extends CommandRunner                 (nest-commander)
       uses ─► Services         (via composition)
```

## Logging management
At the moment all services are logging, in the best of the world only
the runners should be doing so
2026-04-02 13:24:55 +00:00
Raphaël Bosi 9f95c4763c [COMMAND MENU ITEMS] Remove deprecated code (#19199)
This PR is the first one of a cleanup after upgrading command menu items
to V2.
2026-04-01 15:56:52 +00:00
Charles Bochet 08b041633a chore: remove 1-19 upgrade commands and navigation menu item feature flags (#19083)
## Summary
- Deletes the entire `1-19` upgrade version command directory (7 files,
~1500 lines) and removes all references from the upgrade module and
command runner.
- Removes `IS_NAVIGATION_MENU_ITEM_ENABLED` and
`IS_NAVIGATION_MENU_ITEM_EDITING_ENABLED` feature flags from the enum,
seed data, generated schema files, and test mocks. These flags had no
remaining feature-gated code — navigation menu items are now always
enabled.
2026-03-30 14:15:46 +00:00
Félix Malfait a51b5ed589 fix: resolve settings/usage chart crash and add ClickHouse usage event seeds (#19039)
## Summary

- **Fix settings/usage page crash**: The `GraphWidgetLineChart`
component used on `settings/usage` was crashing with "Instance id is not
provided and cannot be found in context" because it requires
`WidgetComponentInstanceContext` (for tooltip/crosshair component
states) which is only provided inside the widget system. Wraps the
standalone chart usages with the required context provider.
- **Avoid mounting `GraphWidgetLegend` when hidden**: The legend
component calls `useIsPageLayoutInEditMode()` which requires
`PageLayoutEditModeProviderContext` — another context only available
inside the widget system. Since the settings page passes
`showLegend={false}`, the fix conditionally unmounts the legend instead
of always mounting it with a `show` prop. Applied consistently across
all four chart types (line, bar, pie, gauge).
- **Add ClickHouse usage event seeds**: Generates ~400 realistic
`usageEvent` rows spanning the past 35 days with weighted user activity,
weekday/weekend patterns, and gradual ramp-up. Enables developers to see
the usage analytics page with data locally.

## Test plan

- [ ] Navigate to `settings/usage` — page should render without errors
- [ ] Verify the daily usage line chart displays correctly
- [ ] Navigate to a user detail page from the usage list
- [ ] Verify the user detail chart renders without errors
- [ ] Run `npx nx clickhouse:seed twenty-server` and confirm usage
events are seeded
- [ ] Verify chart legend still works correctly on dashboard widgets (no
regression)


Made with [Cursor](https://cursor.com)
2026-03-29 07:31:36 +02:00
Charles Bochet c407341912 feat: optimize hot database queries with multi-layer caching (#19068)
## Summary

Introduces multi-layer caching for the 5 most frequent database queries
identified in production (Sentry data), targeting the JWT authentication
hot path and cron job logic.

### Problem
Our database is under heavy load from uncached queries on the auth hot
path:
- `WorkspaceEntity` lookups: **638 queries/min**
- `ApiKeyEntity` lookups: **491 queries/min**
- `UserEntity` lookups: **147 queries/min**
- `UserWorkspaceEntity` lookups: **143 queries/min**
- `LogicFunctionEntity` lookups: **1800 queries/min** (cron job)

### Solution

**1. New `CoreEntityCacheService`** for non-workspace-scoped entities
(Workspace, User, UserWorkspace):
- Mirrors `WorkspaceCacheService` architecture (in-process Map + Redis
with hash validation)
- Provider pattern with `@CoreEntityCache` decorator
- Keyed by entity primary key (not workspaceId)
- 100ms local TTL, Redis-backed hash validation for cross-instance
consistency
- Three providers: `WorkspaceEntityCacheProviderService`,
`UserEntityCacheProviderService`,
`UserWorkspaceEntityCacheProviderService`

**2. New `apiKeyMap` WorkspaceCache** for workspace-scoped API key
lookups:
- `WorkspaceApiKeyMapCacheService` loads all API keys for a workspace
into a map by ID
- Leverages existing `WorkspaceCacheService` infrastructure
- Cache invalidation on API key create/update/revoke

**3. `CronTriggerCronJob` refactored** to use existing
`flatLogicFunctionMaps` workspace cache:
- Eliminates per-workspace `LogicFunctionEntity` repository queries
(~1800/min)
- Filters cached data in-memory instead

**4. `JwtAuthStrategy` refactored** to use caches for all entity
lookups:
- Workspace, User, UserWorkspace → `CoreEntityCacheService`
- ApiKey → `WorkspaceCacheService` (`apiKeyMap`)
- Impersonation queries kept as direct DB queries (rare path, requires
relations)

**5. Cache invalidation** wired into mutation paths:
- `WorkspaceService` → invalidates `workspaceEntity` on
save/update/delete
- `ApiKeyService` → invalidates `apiKeyMap` on create/update/revoke

### Architecture

```
Request → JwtAuthStrategy
  ├── Workspace lookup → CoreEntityCacheService (in-process → Redis → DB)
  ├── User lookup → CoreEntityCacheService (in-process → Redis → DB)
  ├── UserWorkspace lookup → CoreEntityCacheService (in-process → Redis → DB)
  └── ApiKey lookup → WorkspaceCacheService (in-process → Redis → DB)

CronTriggerCronJob
  └── LogicFunction lookup → WorkspaceCacheService (flatLogicFunctionMaps)
```

### Expected Impact
| Query | Before | After |
|-------|--------|-------|
| WorkspaceEntity | 638/min | ~0 (cached) |
| ApiKeyEntity | 491/min | ~0 (cached) |
| UserEntity | 147/min | ~0 (cached) |
| UserWorkspaceEntity | 143/min | ~0 (cached) |
| LogicFunctionEntity | 1800/min | ~0 (cached) |

### Not included (ongoing separately)
- DataSourceEntity query optimization (IS_DATASOURCE_MIGRATED migration)
- ObjectMetadataEntity query optimization (already partially cached)
2026-03-28 22:53:34 +01:00
Thomas Trompette 34ab72c460 Fix: Cannot create two workflow agent nodes because these have the same name (#19015)
Currently, when trying to create a second step agent, we get the error
agent already exists because the name is using workflow id. Using step
id instead.
2026-03-26 16:00:15 +00:00
nitin 578d990b9c [AI] Match ai chat composer to figma (#18874)
https://www.figma.com/design/xt8O9mFeLl46C5InWwoMrN/Twenty?node-id=93653-368288&t=obTG32NRidXid4lN-0

closes
https://discord.com/channels/1130383047699738754/1480990726442582086

---------

Co-authored-by: Félix Malfait <felix.malfait@gmail.com>
Co-authored-by: Félix Malfait <felix@twenty.com>
2026-03-26 05:43:54 +00:00
Thomas Trompette 6c1db2e7fb Do not run loop when iterator is skipped (#18964)
When an If/Else branch skips an Iterator step (because the branch wasn't
taken), the executor incorrectly entered the iterator's loop body. This
happened because getNextStepIdsToExecute checked !hasProcessedAllItems
on an undefined result, which evaluated to true, causing it to return
initialLoopStepIds instead of the post-loop nextStepIds.

Fix: Add a !executedStepOutput.shouldSkipStepExecution guard to the
iterator condition in getNextStepIdsToExecute, consistent with the
existing shouldFailSafely guard.
2026-03-25 17:35:58 +00:00
Raphaël Bosi 49afe5dbc4 Refactor: Unify command menu item execution (#18908)
- Makes engineComponentKey non-nullable on CommandMenuItem. Adds two new
engine component keys: TRIGGER_WORKFLOW_VERSION and
FRONT_COMPONENT_RENDERER to cover the former standalone cases.
- Unifies the previously separated engine, front component and workflow
runners into a single `CommandRunner` component with a unified
`useMountCommand` hook that handles all command types.
2026-03-24 15:59:52 +00:00
Félix Malfait 77d4bd9158 Add billing usage analytics dashboard with ClickHouse integration (#18592)
## Summary
This PR adds a comprehensive billing usage analytics feature that
provides detailed breakdowns of credit consumption across execution
types, users, resources, and time periods. The implementation includes a
new ClickHouse-backed analytics service, GraphQL API endpoint, and a
frontend dashboard component.

## Key Changes

### Backend
- **New BillingAnalyticsService**: Queries ClickHouse for usage
breakdowns by user, resource, execution type, and time series data
- **BillingEventWriterService**: Writes billing events to ClickHouse for
analytics while maintaining best-effort semantics (never blocks Stripe
billing)
- **ClickHouse Schema**: Added `billingEvent` table with 3-year TTL for
storing detailed billing event data
- **GraphQL Resolver**: New `getBillingAnalytics` query that aggregates
usage data for the current billing period, protected by feature flag and
billing permissions
- **Enhanced BillingUsageEvent**: Added `userWorkspaceId` field to track
per-user credit consumption
- **AI Billing Integration**: Updated AI billing service to pass
`userWorkspaceId` when recording usage events

### Frontend
- **SettingsBillingAnalyticsSection**: New component displaying:
  - Usage breakdown by execution type with progress bars
  - Daily usage time series chart (28-day view)
  - Per-user credit consumption breakdown
  - Per-resource (agent/workflow) credit consumption breakdown
- **SettingsUsage Page**: Dedicated page for viewing usage analytics
- **GraphQL Query**: `GetBillingAnalytics` query with generated hooks
- **Navigation**: Added Usage menu item in settings (feature-flagged)
- **Mock Data**: Included screenshot mock data for preview/testing

### Feature Flag
- Added `IS_USAGE_ANALYTICS_ENABLED` feature flag to control visibility
and access to analytics features

## Implementation Details
- Analytics data is queried in parallel for performance
- ClickHouse writes are non-blocking to ensure billing operations never
fail
- Progress bars use dynamic coloring from a predefined palette
- Time series visualization normalizes bar heights relative to max value
- Empty state handling when no analytics data is available
- Responsive UI with proper text truncation for long names

https://claude.ai/code/session_01Y1EqrX6PFq3EJxJq89h7DF

---------

Co-authored-by: Claude <noreply@anthropic.com>
2026-03-23 10:28:23 +01:00
Raphaël Bosi c107d804d2 Create command menu items for workflows with manual trigger (#18746)
- Automatically create and sync command menu items for all workflows
with a manual trigger
- Refactor `useCommandMenuItemsFromBackend`
- Prefill a _Quick Lead_ workflow command menu item during workspace
setup and dev seeding
- Add a ready prop to `HeadlessEngineCommandWrapperEffect` to prevent
premature execution when async data hasn't loaded yet

---------

Co-authored-by: Charles Bochet <charles@twenty.com>
2026-03-22 23:52:43 +01:00