65155fe50c675ab92f6df2b9838b67b22aae64d9
1430 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 martmull
|
65155fe50c |
feat(apps): add enqueueJob to run a logic function on the workers (#23527)
Closes twentyhq/core-team-issues#2742 A logic function run is capped by its own `timeoutSeconds` (900s max), so anything that can't finish in one run — a full re-sync, a per-record fan-out, a rate-limited third-party API — had no way to continue. This adds a way to hand that work to the workers. ## What it looks like for an app author ```ts import { enqueueJob } from 'twenty-sdk/logic-function'; await enqueueJob({ logicFunctionUniversalIdentifier: '9f1c3d7e-51b8-4a29-8f0d-7c4e2a6b1d33', payload: { cursor: nextCursor }, retryLimit: 3, priority: 2, delayMs: 60_000, }); ``` The target runs in its own process with its own timeout budget. The classic shape is a function that enqueues *itself* with the next cursor until there is nothing left. ## Changes **twenty-shared** — `EnqueueJobInput` / `EnqueueJobOptions` / `EnqueueJobResult` in `application`. **twenty-server** — new `application-job` module under `core-modules/application`, following the `application-key-value` pattern: - `enqueueJob` mutation on the metadata API, `@AuthApplication`-scoped - the lookup is scoped to `applicationId` + `workspaceId` — that's the authorization boundary, an app can only enqueue its own logic functions, anything else is `LOGIC_FUNCTION_NOT_FOUND` - pushes a `LogicFunctionTriggerJob` onto the existing `logicFunctionQueue`, so the enqueued run goes through the same executor (and the same execution throttling) as every other trigger - the queued run inherits the caller's `userId`/`userWorkspaceId`, so its app access token carries the same permissions as the function that queued it **Job options** are range-checked via `ResolverValidationPipe`, since the values come from application code and an unbounded delay or retry count would let an app pin work in the shared queue: | Option | Default | Range | |--------|---------|-------| | `retryLimit` | `0` | `0`–`10` | | `priority` | queue default | `1`–`10` (lower first) | | `delayMs` | `0` | `0`–7 days | `retryLimit` defaults to `0` rather than inheriting the server-route path's `3`: retries re-run the whole handler, so opting in should be the author's explicit choice. **twenty-sdk** — `enqueueJob` in `twenty-sdk/logic-function`, same shape as `runAgent`/`kv`. **Docs** — new "Background Jobs" page under Extend → Apps → Logic, plus nav and overview entries. **Generated** — regenerated `twenty-front/src/generated-metadata` and `twenty-client-sdk/src/metadata/generated` for the new mutation. ## Tests - `application-job.service.spec.ts` — 5 unit tests: job options mapping, defaults, acting-user propagation, application-scoped lookup, not-found - `enqueue-job.integration-spec.ts` — 5 integration tests: rejects a non-`APPLICATION_ACCESS` token, enqueues a function the app owns, rejects a function owned by another application, rejects an unknown identifier, rejects out-of-range options All green locally, along with `typecheck` for `twenty-server`/`twenty-sdk` and oxlint/oxfmt on the touched files. ## Notes for review - The target is addressed by `universalIdentifier`, matching `runAgent({ agentUniversalIdentifier })` and `ServerRouteDispatchResult.targetLogicFunctionUniversalIdentifier`. Addressing by `name` would be friendlier, but logic function names aren't validated for uniqueness within an app — happy to add it as a convenience if you'd rather. - `enqueueJob` returns as soon as the job is accepted; it can't return the target's result, since the queue driver's `add` returns void. Documented, with a pointer to the KV store for handing results back. --- _Generated by [Claude Code](https://claude.ai/code/session_01QrYvGonS3HMdeuMAVjs5hR)_ <!-- This is an auto-generated description by cubic. --> <a href="https://cubic.dev/pr/twentyhq/twenty/pull/23527?utm_source=github" target="_blank" rel="noopener noreferrer" data-no-image-dialog="true"><picture><source media="(prefers-color-scheme: dark)" srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source media="(prefers-color-scheme: light)" srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img alt="Review in cubic" src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a> <!-- End of auto-generated description by cubic. --> --------- Co-authored-by: github-actions <github-actions@twenty.com> |
||
|
Raphaël Bosi
|
a3beea893d |
Revert the external link confirmation popup for front components (#23567)
Reverts #23270 and #23404. Links in front components navigate natively again, with no confirmation popup and no per-app trusted-origins state in localStorage. <!-- This is an auto-generated description by cubic. --> <a href="https://cubic.dev/pr/twentyhq/twenty/pull/23567?utm_source=github" target="_blank" rel="noopener noreferrer" data-no-image-dialog="true"><picture><source media="(prefers-color-scheme: dark)" srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source media="(prefers-color-scheme: light)" srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img alt="Review in cubic" src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a> <!-- End of auto-generated description by cubic. --> |
||
|
Paul Rastoin
|
dbd2eac69c |
Let the instance upgrade version reach releases without instance commands (#23552)
Fixes the CI failure on #23520: An upgrade version sequence has to at least contain one instance or one workspace command Workspaces commands do not run for the instance level and aren't triggered automatically Explaining this PR need <img width="1396" height="954" alt="image" src="https://github.com/user-attachments/assets/5455d05b-b286-482a-8914-808f55f3b0bf" /> ``` Upload failed: App requires Twenty server >=2.26.0 but this server is 2.25.0. ``` The server really is 2.26 (`TWENTY_CURRENT_VERSION = '2.26.0'`), but `validateServerCompatibility` resolves the instance version through `UpgradeMigrationService.getInferredVersion()`, which reads the last row in `core.upgradeMigration` with `workspaceId IS NULL AND isInitial = false` and takes the version prefix off its name. Instance commands are the only ones that write a `workspaceId`-null row, and `2-26/` ships none (only three workspace commands), so the highest instance command in the tree is still `2-25-instance-command-fast-1785230296000-add-is-hidden-to-agent-message`. A fully migrated 2.26 server infers 2.25.0, and any app declaring `engines.twenty: ">=2.26.0"` is unpublishable. Two defects, both of which `getWorkspaceCompletedVersion` already avoids: - **Not sequence-aware.** The workspace path walks the registered sequence and only credits a version once the cursor sits on that version's last step. The instance path just reads the cursor's prefix, so a version contributing zero instance commands is unreachable. - **Not status-aware.** `getLastAttemptedInstanceCommand` filters on `attempt = MAX(attempt)` but not on status, so a *failed* 2.25 command still made the server report 2.25.0. ## What changed `UpgradeStatusService` gains `getInstanceCompletedVersion()`, the instance-scope mirror of `getWorkspaceCompletedVersion`. It walks the sequence filtered to instance steps, requires the cursor to sit on the last instance step of its version *and* be `completed`, then advances through any later supported version that declares no instance command at all. The version-skipping rule is the part that unblocks 2.26: a release with no instance-level work has nothing for the cursor to land on, so it is reached as soon as the last version that does have instance commands is done. A version whose instance command exists but has not run still holds the cursor back. - `validateServerCompatibility` calls the new method; `UpgradeMigrationService` is no longer a dependency of `ApplicationVersionValidationService`. - `getInstanceStatus` reports it as `inferredVersion`, so the upgrade gauge metric and `upgrade:status` CLI stop showing 2.25.0 on a 2.26 server. - `getInferredVersion` is deleted. Its one remaining caller passed a command name, which is just `extractVersionFromCommandName`. - Cursor resolution is extracted to `resolve-completed-version-from-cursor.util`, now shared by both scopes; the skip rule lives in `advance-through-versions-without-instance-commands.util`. The asymmetry between the two scopes is intentional and stays: instance commands record a row per workspace as well, so workspace cursors land on both command kinds and never had this gap. ## Testing - `npx nx typecheck twenty-server` clean, `npx nx lint:diff-with-main twenty-server` clean. - 294 unit tests pass across the upgrade and application modules, including 7 new ones for `getInstanceCompletedVersion`. Two pin the boundary: a trailing workspace-only version is reached, a trailing version whose instance command has not run is not. - The fixture in `upgrade-status.service.spec.ts` used `1.21.0`/`1.22.0`/`1.23.0`, which are real entries in `TWENTY_PREVIOUS_VERSIONS`. With the skip rule in place that sequence read as "every version from 2.0 onward has no instance commands" and walked to the end, so the fixture is renumbered to `0.2x.0` to keep those tests on cursor resolution alone. - `failing-app-installation-workspace-version.integration-spec.ts` already carried a comment describing this bug as a hazard it worked around. The workaround still holds, but integration tests were not run here (no DB in this session) — the stale comment is updated. #23520 stays at `>=2.26.0` and unblocks once this lands. --- _Generated by [Claude Code](https://claude.ai/code/session_012SvBG1BB3jTaZs6LA2Wi9R)_ <!-- This is an auto-generated description by cubic. --> <a href="https://cubic.dev/pr/twentyhq/twenty/pull/23552?utm_source=github" target="_blank" rel="noopener noreferrer" data-no-image-dialog="true"><picture><source media="(prefers-color-scheme: dark)" srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source media="(prefers-color-scheme: light)" srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img alt="Review in cubic" src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a> <!-- End of auto-generated description by cubic. --> |
||
|
Raphaël Bosi
|
38ad13655c |
Auto-start the workspace setup chat with a data model proposal (#23437)
https://github.com/user-attachments/assets/d447372b-c1b4-4c95-bac9-a8f8efa7d4a1 When the workspace creator lands on `/workspace-setup` after onboarding, the AI chat now starts on its own: an invisible first message, built server-side from the company enrichment collected in #23199, asks the assistant to propose a data model tailored to the business. The proposal streams in; the user never sees the prompt. - New `startWorkspaceSetupChat` mutation: creator only, gated on `IS_ONBOARDING_AI_CHAT_ENABLED`, available models and credits. Idempotent per user and workspace via a `keyValuePair` pointing at the thread, so a reload or a second tab joins the same conversation instead of starting a new one. - The thread holds exactly one hidden `USER` message combining the company context and the setup instructions, which keeps the one-hidden-message-per-thread index from #23199 satisfied. It goes through a dedicated streaming path that never queues, so the prompt cannot resurface as a visible message. - The assistant only proposes. It creates nothing until the user approves, then builds the model with the `metadata-building` skill. Objects and fields get English names with labels in the user's language, and the conversation continues in that language. - With no enrichment (consumer email domain, or the integration disabled) the kickoff still runs, and the assistant asks one short question about the business before proposing. - `findLatestSentUserMessage` no longer filters out hidden messages, so a failed kickoff turn stays retryable, and the no-message chat error surface now offers retry for stream errors. <!-- This is an auto-generated description by cubic. --> <a href="https://cubic.dev/pr/twentyhq/twenty/pull/23437?utm_source=github" target="_blank" rel="noopener noreferrer" data-no-image-dialog="true"><picture><source media="(prefers-color-scheme: dark)" srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source media="(prefers-color-scheme: light)" srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img alt="Review in cubic" src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a> <!-- End of auto-generated description by cubic. --> |
||
|
nitin
|
079e9b8e56 |
feat(dashboard): extend number format option to bar, line and pie charts (#23505)
https://discord.com/channels/1130383047699738754/1509604545381142649 Extends the Format option (Short/Full) added for the Number widget in #21521 to bar, line and pie charts. Format controls the numbers printed on the chart face: data labels and the pie center metric. Axis ticks stay abbreviated and tooltips always show the full value. Defaults to Short, so existing charts render unchanged. Server: nullable `numberFormat` on the bar/line/pie configuration DTOs, exposed in the dashboard AI tool schema. No migration, configuration is jsonb. Deferred: - The Format row has no visible effect while data labels are off, since tooltips are always full. - Number widget format defaults differ by field type (CURRENCY defaults to Short, NUMBER to Full). Pre-existing, untouched here. https://github.com/user-attachments/assets/0778f08a-6681-4e7a-8716-fb3026d1e01f <!-- This is an auto-generated description by cubic. --> <a href="https://cubic.dev/pr/twentyhq/twenty/pull/23505?utm_source=github" target="_blank" rel="noopener noreferrer" data-no-image-dialog="true"><picture><source media="(prefers-color-scheme: dark)" srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source media="(prefers-color-scheme: light)" srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img alt="Review in cubic" src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a> <!-- End of auto-generated description by cubic. --> |
||
|
Thomas Trompette
|
4ff9cba76d |
fix(server): stop the global catch-all filter from shadowing typed GraphQL exception filters (#23508)
## Context Sentry [TWENTY-SERVER-60Y](https://twenty-v7.sentry.io/issues/6633503406) ("Permission Denied: Entity performing the request does not have permission") is still firing at full rate on `v2.25.0`: ~10.8k events in the last 7 days, 24k total. #23104 tried to fix it by registering `PermissionsGraphqlApiExceptionFilter` globally via `APP_FILTER`. That registration is correct but **inert in production**, and the integration test added alongside it passes for a reason unrelated to prod behaviour. ## Root cause `main.ts` registered a catch-all filter after bootstrap: ```ts app.useGlobalFilters(new UnhandledExceptionFilter()); ``` Nest builds each resolver's filter list as `[...global, ...class, ...method]`, reverses it, and selects **exactly one** matching filter — there is no chaining. `APP_FILTER` providers are collected during module scan; `useGlobalFilters` appends after that, so the catch-all ended up at the head of the list: ``` 1. UnhandledExceptionFilter @Catch() <- matches everything, wins 2. PermissionsGraphqlApiExceptionFilter <- never reached 3. BillingGraphqlApiExceptionFilter <- never reached ``` On a GraphQL host `UnhandledExceptionFilter` then no-ops: `host.switchToHttp().getResponse()` returns the GraphQL args object, `response.header` is undefined, so it hits `return;`. Nest treats a falsy return as unhandled and rethrows the original `PermissionsException`, which reaches the Yoga error hook as a non-`BaseGraphQLError`, is serialized `INTERNAL_SERVER_ERROR`, and is reported by `shouldCaptureException`. The 28 resolvers carrying `@UseFilters(PermissionsGraphqlApiExceptionFilter)` were unaffected — method-level filters are evaluated before globals. Only the resolvers relying on the global registration leaked, which is exactly the set showing up in Sentry (`findOneApplication`, `uploadFilesFieldFileByUniversalIdentifier`, `UpdatePageLayoutWithTabsAndWidgets`, ...). Two other global filters were shadowed the same way and have never run: `BillingGraphqlApiExceptionFilter` and `FlatEntityMapsGraphqlApiExceptionFilter`. ## Why the existing test did not catch it `test/integration/utils/create-app.ts` builds the app from `AppModule` directly and never executes `main.ts`, so `useGlobalFilters` does not exist in the test process. It registered `MockedUnhandledExceptionFilter` as an `APP_FILTER` on the root testing module, which is collected *first* and therefore evaluated *last* — the exact inverse of production precedence. The `findOneApplication` denial test passed while the same query kept reporting to Sentry. ## Fix Register `UnhandledExceptionFilter` through `APP_FILTER` on `AppModule`. Root-module providers are scanned first, so it is collected first and evaluated last. The filter stays global, stays catch-all, and keeps its CORS-header role for HTTP; it simply no longer cuts in front of the typed filters. Un-shadowing the other two global filters means they now actually run, so `FileStorageExceptionFilter` and `FlatEntityMapsGraphqlApiExceptionFilter` get the `host.getType() !== 'graphql'` rethrow that `Billing` and `Permissions` already had. Without it they would start throwing GraphQL error objects into the REST pipeline. `MockedUnhandledExceptionFilter` is removed: `AppModule` now supplies the real filter in the same position, so the mock was dead weight. ## Test Verified against a real server (not the integration harness), calling the exact document from Sentry event `8d19eb7c` as a member with no permission flags: ``` query ($v1:UUID){findOneApplication(id:$v1){applicationVariables{key,value}}} ``` | | response code | exceptions captured | |---|---|---| | before | `INTERNAL_SERVER_ERROR` | 1 | | after | `FORBIDDEN` | 0 | Capture count measured through the console exception-handler driver, i.e. the same `captureExceptions` call site that is the Sentry driver in production. New unit spec `src/filters/__tests__/unhandled-exception.filter.spec.ts` boots a Nest + Yoga app both ways: it asserts `FORBIDDEN` with the `APP_FILTER` registration, and pins the shadowing behaviour of `app.useGlobalFilters` so the pattern cannot come back unnoticed. `granular-settings-permissions.integration-spec.ts` passes (10/10). Note it also passes *without* this fix — the harness cannot observe bootstrap-only configuration, which is the underlying reason #23104 shipped green. Closing that gap properly means sharing the post-`create` bootstrap between `main.ts` and `create-app.ts`; left as a follow-up. `file-storage-exception-filter.spec.ts` extended with a non-GraphQL host case. ## CI follow-up `failing-file-by-id-download.integration-spec.ts` snapshots were updated. That REST endpoint's 403 body changed in tests from `{}` to `{"statusCode":403,"error":"Forbidden","message":"Forbidden resource"}`. The old `{}` was an artifact of the mock: `MockedUnhandledExceptionFilter` rethrew, the exception escaped Nest's handler into Express's default error handler, and supertest saw an empty body. Production has always run the real `UnhandledExceptionFilter`, which writes `response.status(status).json(exception.response)` — the new snapshot. Production HTTP behaviour is unchanged by this PR: no other global filter matches an `HttpException` (the typed ones rethrow outside GraphQL), so the same filter handles it whether it is evaluated first or last. |
||
|
Etienne
|
a276f3277f |
feat(workflow): pin concrete model on AI agent node creation and exclude interactive tools from workflow runs (#23447)
## Context The AI Agent workflow node's model dropdown could show a model that was not the one used at run time (e.g. the node displayed "Claude Haiku 4.5" while the run log showed `openai/gpt-5.6-sol`). Root cause: workflow agents were created with `modelId: AUTO_SELECT_SMART_MODEL_ID`. The builder's model `Select` cannot represent that value — auto-select ids are filtered out of the options (`useWorkspaceAiModelAvailability`) and the pinned "default" option remaps its value to the resolved concrete model id (`useAiModelOptions`) — so `Select` silently fell back to `options[0]`, the alphabetically first enabled model. Meanwhile the runtime correctly resolved auto-select to the instance's default smart model. ## What this PR does ### 1. New workflow agents store a concrete model id `WorkflowVersionStepOperationsWorkspaceService` now reads the workspace's `fastModel` setting, expands it through `AiModelRegistryService.getEffectiveModelConfig`, validates it with `validateModelAvailability`, and stores the concrete model id — so the dropdown displays the model that will actually run, and workflow agents default to the cheaper fast tier instead of the smart one. Falls back to `AUTO_SELECT_FAST_MODEL_ID` if the lookup or validation fails (workspace missing, no AI provider configured, model disabled), so node creation never breaks. ### 2. Exclude `search_help_center` and `navigate_app` from workflow agent runs `ActionToolProvider` adds both tools unconditionally, but they only make sense in an interactive chat session (navigation targets the user's browser; help-center search is a support tool). They are now excluded via `WORKFLOW_AGENT_EXCLUDED_TOOL_NAMES` in `AgentAsyncExecutorService`, alongside the existing output-navigation exclusions. Chat agents are unaffected. ## Test coverage - Existing specs for `WorkflowVersionStepOperationsWorkspaceService` and `AgentAsyncExecutorService` updated/passing (new constructor deps mocked). - `nx typecheck twenty-server` and `lint:diff-with-main` pass. <!-- This is an auto-generated description by cubic. --> <a href="https://cubic.dev/pr/twentyhq/twenty/pull/23447?utm_source=github" target="_blank" rel="noopener noreferrer" data-no-image-dialog="true"><picture><source media="(prefers-color-scheme: dark)" srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source media="(prefers-color-scheme: light)" srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img alt="Review in cubic" src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a> <!-- End of auto-generated description by cubic. --> |
||
|
Paul Rastoin
|
4ec65ed08d |
System view tooling explicit params key naming (#23506)
# Introduction View field system always result from a field existence, the application universal identifier should be the related field one Same but for views and object <!-- This is an auto-generated description by cubic. --> <a href="https://cubic.dev/pr/twentyhq/twenty/pull/23506?utm_source=github" target="_blank" rel="noopener noreferrer" data-no-image-dialog="true"><picture><source media="(prefers-color-scheme: dark)" srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source media="(prefers-color-scheme: light)" srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img alt="Review in cubic" src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a> <!-- End of auto-generated description by cubic. --> |
||
|
Paul Rastoin
|
0c545bcdeb |
[BREAKING-CHANGE] Centralize system View viewField side effect (#23081)
# Introduction Closes https://github.com/twentyhq/core-team-issues/issues/2669 Part of the `isSystemSideEffect` engine-ownership effort. Until now, a custom object's default **INDEX** table view (`All {objectLabelPlural}`) and its view fields were built imperatively in `ObjectMetadataService` with random `v4()` identifiers, while `twenty-standard` authored its own copies with hardcoded literals. The two never converged, an object rename could drift the view, and nothing marked these rows as engine-owned. This PR makes the metadata side-effect engine the **single owner** of the INDEX view and its view fields, on name-free deterministic identifiers, for custom and standard objects alike. ## Core design - **Name-free deterministic identity.** The INDEX view identifier derives from `object identifier + ViewKey.INDEX` (`getSystemViewUniversalIdentifier`); each view-field identifier derives from `view identifier + field identifier` (`getViewFieldUniversalIdentifier`). An object rename (with a pinned object identifier) keeps the same view, losslessly. - **`isSystemSideEffect: true` is provenance.** Every INDEX view / view field the engine emits is flagged system-owned, so manifest deletion inference never drops it. The flag follows the view: a view field inherits its parent view's flag. - **The engine is the sole owner of the INDEX view.** It always emits it; a caller providing one with the same derived identifier is a genuine conflict surfaced by the engine's reserved-identifier collision, not silently deferred. ## Changes ### Shared (`twenty-shared`) - `getIndexViewUniversalIdentifier` → `getSystemViewUniversalIdentifier`, now taking a `viewKey` (generalizes to any singleton engine-owned view). - Standard field identifiers extracted into a new `STANDARD_OBJECT_FIELDS` constant, so both an object's `fields` and its INDEX view read the same field identifiers. - `buildStandardObjectIndexView` derives the standard INDEX view + view-field identifiers from `STANDARD_OBJECT_FIELDS`, replacing the hardcoded literals in `standard-object.constant.ts`. ### Metadata side-effect engine (custom objects) - **`objectSystemFieldsAndIndexViewOnCreate`** (replaces `objectSystemFieldsOnCreate`): on object creation, provisions the 7 reserved system fields **and** the INDEX view with one view field per displayable system field, all `isSystemSideEffect: true`. - **`fieldIndexViewFieldOnCreate`** (new): on field creation, provisions the field's INDEX view field. Object created in the same batch → visible, positioned before the system view fields; pre-existing object → hidden, appended (preserving the historical `createOneField` behavior). Both branches resolve the INDEX view by its derived identifier (single map access, never a scan). - **`fieldSystemViewFieldsOnDelete`** (new): on field deletion, cascade-deletes every engine-owned view field displaying it. - **`objectSystemSideEffectsOnDelete`** (extended): now also cascade-deletes the object's engine-owned views and their view fields (in addition to system fields, indexes, searchFieldMetadata). Every lookup walks a foreign-key aggregator down from the deleted object, so the work is proportional to what the object owns, never to workspace size. - Object-create and field-create positions are derived from the same caller-input field list, so the INDEX view layout is contiguous with no handler-ordering dependency. - `view` / `viewField` added to the side-effect companion metadata names for `fieldMetadata` and `objectMetadata`. ### Reserved-identifier invariant A caller can never define an entity whose identifier collides with one a system side effect produces: caller inputs are forced `isSystemSideEffect: false` at every entry point (API and app-manifest transpilers), and the engine raises `RESERVED_SYSTEM_UNIVERSAL_IDENTIFIER`, aborting the operation, when a system emission lands on a caller-claimed identifier. Covered by a new engine-level test. ### Caller-side provisioning removed The imperative INDEX view + view-field provisioning is removed from `ObjectMetadataService.createOneObject`. The record-page `FIELDS_WIDGET` view is intentionally left caller-side and deferred to the follow-up (see below). ### `twenty-standard` convergence Standard INDEX views and their view fields converge on the same derived-identifier + `isSystemSideEffect: true` scheme as the engine. `twenty-standard` syncs through the from/to migration path (which never runs the side-effect engine), so it authors this INDEX surface itself, matching what the engine produces for custom objects. ## Rollout Two `2.26.0` workspace commands, running after the `2.25` messageCampaign commands: - `upgrade:2-26:reconcile-index-view-universal-identifier` re-owns the INDEX views of the **twenty-standard and workspace-custom applications** and all their view fields to the derived identifiers with `isSystemSideEffect: true`, in a single per-workspace transaction. Each view field identifier is keyed on the application of the **displayed field** (an app or user column on a standard INDEX view converges too). Soft-deleted views and view fields are skipped: one can coexist with an active successor on the same derivation inputs and both would derive the same identifier. Children reference the view by primary key, so the re-own is lossless. - `upgrade:2-26:demote-and-backfill-application-index-view` handles **manifest-installed applications**, which never had their INDEX view auto-provisioned: every caller-authored INDEX view of another application is demoted to `key: null` (a plain additional view under its manifest identifier), then every application object gets the engine-owned INDEX view and its full view-field layout backfilled through the migration pipeline's legacy path (no side-effect expansion), views committed before view fields across applications since a view field belongs to the application owning its field. Idempotent and retry-safe: engine-owned INDEX views are neither demoted nor re-backfilled, and view creation and view-field creation are gated independently, so a retry after a partial failure still backfills the missing view fields of an already-committed view. Both support `--dry-run` and invalidate the full flat-maps closure (parents aggregate the re-owned identifiers, children resolve them as universal foreign keys, and page-layout widget universal configurations resolve view PKs at cache-build time). The `2.25` `upgrade:2-25:add-message-campaign-name-field` command is adapted to resolve the campaign INDEX view by its INDEX key on the object instead of by universal identifier: it now runs before the reconcile, on workspaces still holding legacy identifiers. ## ⚠️ Breaking change This PR **mutates 187 previously hardcoded universal identifiers** — the standard objects' INDEX views and their view fields (the literals removed from `standard-object.constant.ts`), now derived. - **Handled by the `2.26` commands above** for all existing workspaces. - **The INDEX key is now engine-reserved.** The flat view validator rejects caller-created INDEX views (API and manifest inputs are forced `isSystemSideEffect: false`) and enforces a single non-deleted INDEX view per object; `view.key` is no longer a comparable/updatable property, so no writer can promote or demote a view after creation. `ViewManifest.key` is deprecated and ignored (manifest views are always additional views, so old apps keep syncing and demoted views are not promoted back); the REST/GraphQL create path now rejects `key: INDEX`. In-repo example apps (`hello-world`, `document-generator`) no longer declare it. - **12 declared-but-never-seeded standard INDEX view field identifiers deleted** (the former `preservedViewFields` on `timelineActivity`, `workflowRun` and `workspaceMember`): after the reconcile, no workspace row references them. - **`computeFlatViewFieldsToCreate` now derives view field identifiers** instead of drawing `v4()` ones, which also changes what the committed `1-23` record-page backfill produces going forward (deliberate, documented in-code). - **Record-page views and view fields are not affected** (identifiers unchanged). - **In-repo apps: `twenty-last-contact` updated.** It was the only app declaring explicit INDEX view fields (10 columns across `allPeople` / `allCompanies` / `allOpportunities`) through manifest `viewFields`. Those target identifiers are now engine-owned and derived, so the manifest inputs no longer resolve and install failed with `View not found`. The app now declares only its fields; the engine's `fieldIndexViewFieldOnCreate` provisions the matching INDEX view field automatically. No other app under `packages/twenty-apps` references any of the 187 mutated identifiers, and apps that target standard views point at record-page views (e.g. `real-estate` → `opportunityRecordPageFields`) or their own objects (`twenty-partners`), all unchanged. ### Loss of granularity for app maintainers The engine now owns the INDEX view field of every field a caller adds to an object, so app maintainers lose direct control over those columns. Previously an app could target the engine-owned INDEX view with an explicit manifest `viewField` and set its `position` and `isVisible`. Now `fieldIndexViewFieldOnCreate` appends a **hidden** view field in caller-input order on field creation, so: - Columns an app previously showed at a **dedicated position** and **visible** (e.g. `twenty-last-contact`'s last-contact columns) become **hidden** and **appended in input order** after install. - There is currently **no manifest way to override** the engine-provisioned INDEX view field's position, visibility, or size. This is a deliberate regression accepted for the sake of single-ownership, and app maintainers should expect their INDEX columns to move/hide after upgrading. A follow-up override API will let maintainers reclaim per-field control over the engine-provisioned INDEX view field. ## Testing - Unit specs for each handler: object create (system fields + INDEX view/view fields, override, position offset), field create (same-batch vs existing-object, non-displayable noop, no-INDEX-view noop), field delete, object delete (fields/indexes/searchFieldMetadata/views/view fields cascade, reverse-relation view field on another object). - Engine-level test for the reserved-identifier collision. - `twenty-standard` guard test that its INDEX views/view fields stay on the derived scheme and stay system-owned. - Integration test: full engine provisioning of the INDEX view/view fields on object creation, same view id preserved across an object rename, and cascade delete on object deletion. ## Follow-up The full record-page stack (record-page view, its view fields, view field groups, page layout / tab / widget) is still built imperatively and moves into the engine in https://github.com/twentyhq/core-team-issues/issues/2721. |
||
|
Abdul Rahman
|
ea7863dc4e |
feat(ai-agent): lazy tool loading for the open-ended runAgent path (#23454)
## Context `AgentAsyncExecutorService.executeAgent` is shared by workflow agent nodes and the `runAgent` mutation (Slack assistant, apps). Workflow nodes run a scoped task, so pre-loading the few explicitly-granted object tools is fast and skips the `learn_tools` round trip (the behavior settled in #23400 / #23358). `runAgent` is open-ended: its role grants broad object access, so pre-loading inlines every CRUD/action schema on every step. That is what makes the Slack assistant take 3-4 minutes for a prompt Ask AI answers in seconds. Confirmed still slow with #23400 merged, so this is the payload, not object scoping. ## What Add a `toolLoadingStrategy` to `executeAgent` (default `'preload'`, so workflow nodes and evals are unchanged). `AgentRunService.run` opts into `'lazy'`, which exposes a compact tool catalog in the system prompt plus the `learn_tools` / `execute_tool` meta-tools, using composed role permissions rather than explicit grants only, so the agent keeps broad access without the full-payload latency. ## How - Split tool provisioning into two focused methods on the executor; a 3-line dispatch chooses per strategy. The pre-load path is unchanged. - `buildLazyRegistryToolset`: one reusable definition of lazy registry provisioning (catalog + meta-tools), so the chat and agent executors can share it. - Extract `buildToolCatalogSection` out of `SystemPromptBuilderService` into a `tool-provider` util so both paths format the catalog identically (no dup). - Replace the meta-tools' `excludeTools` denylist with a single `isToolAllowed` predicate: the agent path passes an allowlist closed over the shown catalog (enforced at call time), MCP passes its existing deny predicate. Workflow node and eval behavior is unchanged. <!-- This is an auto-generated description by cubic. --> <a href="https://cubic.dev/pr/twentyhq/twenty/pull/23454?utm_source=github" target="_blank" rel="noopener noreferrer" data-no-image-dialog="true"><picture><source media="(prefers-color-scheme: dark)" srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source media="(prefers-color-scheme: light)" srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img alt="Review in cubic" src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a> <!-- End of auto-generated description by cubic. --> |
||
|
Etienne
|
5ebcce0a51 |
feat(ai-tool): resolve and default icons in AI metadata tools (#23480)
## Context Objects and fields created through the AI chat / MCP metadata tools almost never get an icon, so they all render with the meaningless `123` fallback icon. Two causes: - The `icon` tool input was described only as `"Icon name"`, so the model had no idea what the value space is and mostly skipped an optional field it couldn't fill confidently. - Any invalid name is silently swapped for `Icon123` by `useIcons.getIcon` on the frontend, so near-misses were indistinguishable from unset. ## What this PR does **Guide the model** (icon names are Tabler names, which LLMs know well): - `icon` / `targetFieldIcon` schema descriptions now state the convention with examples (`IconBuildingSkyscraper`, `IconPaw`, …) and ask for one to always be set - The `metadata-building` skill gains an "Icons" section; the MCP server instructions gain a one-line reminder **Normalize server-side** (new `resolveIconName` util, used by all create/update/batch metadata tool executes incl. `relationCreationPayload.targetFieldIcon`): - Fixes shape mistakes: raw tabler slugs (`"building-skyscraper"`), separators, missing or lowercased `Icon` prefix - Deliberately does NOT validate existence against the full ~4.2k icon registry — an unknown name is harmless since the frontend falls back to its default icon, exactly as for icons stored via the API today - Unusable input (empty/garbage) resolves to nothing: creates fall back to a default, updates keep the existing icon **Fall back sensibly for fields**: - New `FIELD_TYPE_DEFAULT_ICONS` in `twenty-shared/constants` maps every `FieldMetadataType` to a sensible icon (mirroring the settings UI type illustrations), applied when the model provides no usable icon — an AI-created field always gets a meaningful icon - Lives in twenty-shared so the frontend can reuse it later (e.g. as `getIcon`'s custom default for fields) The REST/GraphQL metadata APIs are untouched — this only affects the AI tool layer. ## Test plan - `resolve-icon-name.util.spec.ts` — canonical pass-through, slug/prefix/separator fixes, unknown-name pass-through (FE fallback contract), unusable inputs, icon-key dropping on updates - `FieldTypeDefaultIcons.test.ts` — every field type mapped, all values canonically shaped (values hand-checked against the twenty-ui `ALL_ICONS` registry) - `nx typecheck` twenty-server + twenty-shared, oxlint/oxfmt clean on changed files <!-- This is an auto-generated description by cubic. --> <a href="https://cubic.dev/pr/twentyhq/twenty/pull/23480?utm_source=github" target="_blank" rel="noopener noreferrer" data-no-image-dialog="true"><picture><source media="(prefers-color-scheme: dark)" srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source media="(prefers-color-scheme: light)" srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img alt="Review in cubic" src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a> <!-- End of auto-generated description by cubic. --> |
||
|
Abdul Rahman
|
965e033f3f |
[breaking-change] fix(server): return runAgent execution failures as result errors (#23390)
## Summary
- When `executeAgent` throws, `runAgent` now returns `{ success: false,
error }` instead of a GraphQL exception
- Callers (workflows, Slack assistant, etc.) can surface the failure to
users instead of hanging or failing opaquely
Run agent exception response error format breaking-change, errors moved
from the GraphQL error channel into the response payload.
<!-- This is an auto-generated description by cubic. -->
<a
href="https://cubic.dev/pr/twentyhq/twenty/pull/23390?utm_source=github"
target="_blank" rel="noopener noreferrer"
data-no-image-dialog="true"><picture><source
media="(prefers-color-scheme: dark)"
srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source
media="(prefers-color-scheme: light)"
srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img
alt="Review in cubic"
src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a>
<!-- End of auto-generated description by cubic. -->
|
||
|
Raphaël Bosi
|
f15fabb5d9 |
Enrich workspace company via People Data Labs during onboarding (#23199)
https://github.com/user-attachments/assets/fb9001c4-195d-4735-898b-07ccbab01677 During onboarding, the workspace creator's work-email domain is enriched through People Data Labs and stored client-side. The stacked workspace-setup PR folds it into the invisible prompt that kicks off the setup chat, so the assistant knows the company from its first reply. - New `enrichWorkspaceCompany` mutation: throttled, creator-only, work domains only. Off by default: requires the `IS_WORKSPACE_COMPANY_ENRICHMENT_ENABLED` instance config variable (default false), a `PEOPLE_DATA_LABS_API_KEY`, and the `IS_ONBOARDING_AI_CHAT_ENABLED` workspace feature flag (the enrichment only feeds the AI-chat workspace setup). Every attempt past the throttle is recorded per workspace in a `keyValuePair`. - The frontend fetches once during onboarding and stores a matched result in localStorage. This PR does not deliver it to the model: the hidden-message plumbing it adds (`isHidden` on `agentMessage`, excluded from the chat UI, thread ranking and the admin transcript, included in the model conversation) is what the stacked workspace-setup PR uses to send the context and the setup prompt as one invisible first message. - The PDL wire protocol (base URL, wire types, envelope parsing, error extraction) is kept as a small self-contained copy inside the server `company-enrichment` module. The standalone people-data-labs app keeps its own copy; the two are intentionally not shared, since the app and the core-engine usage are expected to evolve independently. - `WorkspaceCompanyEnrichment` lives in `twenty-shared/workspace` so server and front share one shape. ## Flow ```mermaid flowchart LR effect[Onboarding effect] -- enrichWorkspaceCompany --> checks{creator + work domain?} checks -- no --> unavailable[unavailable] checks -- yes --> throttle{throttle 10/h/workspace} throttle -- limited --> transient[transientError] throttle -- ok --> pdl[PDL GET /company/enrich] pdl --> log[(keyValuePair attempt log)] pdl --> matched[matched] matched --> storage[(localStorage)] storage -- consumed by the stacked workspace-setup PR --> kickoff[hidden kickoff prompt] ``` 1. **Onboarding effect** — mounted app-wide, fires once per session while onboarding is in progress (before workspace activation), guarded by a sessionStorage attempt flag and the cached value. 2. **enrichWorkspaceCompany** — metadata-schema mutation returning a typed `WorkspaceCompanyEnrichmentResult` (`outcome` enum `matched`/`unavailable`/`transientError` + `enrichment` JSON). 3. **Creator + work domain checks** — only the workspace's earliest user, only non-consumer email domains, only when the config flag, API key and `IS_ONBOARDING_AI_CHAT_ENABLED` workspace flag are all on; anything else returns `unavailable` without consuming throttle quota. 4. **Throttle** — token bucket, 10 requests/hour per workspace, the sole cost bound on PDL calls; when limited the mutation returns `transientError` instead of surfacing an error. 5. **PDL call** — `GET /v5/company/enrich` with `website` + `min_likelihood` per the PDL spec; body-level statuses win over HTTP ones, 408/429/5xx map to `transientError`, other failures to `unavailable`. Every attempt past the throttle is recorded (`domain`, the pre-collapse PDL `outcome`, `httpStatus`/`message` when present, `attemptedAt`) in a workspace-scoped `keyValuePair`. 6. **matched** — the PDL payload is mapped to `WorkspaceCompanyEnrichment` through the same sanitizer as client input (all fields length-capped and control-character-stripped; summary 600 chars, 8 tags max) and returned. 7. **localStorage** — the frontend stores only a matched enrichment and never refetches it, making it the only cache; cleared on sign-out. Non-matched outcomes are not persisted; a sessionStorage flag caps retries at one attempt per browser session. 8. **Delivery** — out of scope here. The stacked workspace-setup PR reads the stored enrichment and combines it with the data-model proposal prompt into a single hidden `USER` message when the setup chat starts; it is never injected into the system prompt. Reviewer notes: sending the creator's email domain to a third party at signup is not yet disclosed in onboarding copy. <!-- This is an auto-generated description by cubic. --> <a href="https://cubic.dev/pr/twentyhq/twenty/pull/23199?utm_source=github" target="_blank" rel="noopener noreferrer" data-no-image-dialog="true"><picture><source media="(prefers-color-scheme: dark)" srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source media="(prefers-color-scheme: light)" srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img alt="Review in cubic" src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a> <!-- End of auto-generated description by cubic. --> |
||
|
Etienne
|
902bc6db63 |
fix(ai-node) - scope AI agent node database tools to explicitly granted objects (#23400)
## Context An AI agent node scoped to a single object was still loading CRUD tools for the whole workspace, inflating every run's prompt to ~200k tokens (~110k on a standard seed workspace: 146 tools across 19 objects, 18 of them system objects). Two mechanisms caused this: the roles permissions cache force-grants every system object to every role (`isSystem ? true`), and blanket role flags (`canReadAllObjectRecords`, ...) grant all remaining objects. The per-object rows written by the agent Permissions tab were additive on top of that, so scoping an agent had almost no effect on its tool payload. ## What **Backend: explicit grants only for the agent node** - New opt-in flag `requireExplicitObjectGrants` on `ToolProviderContext`, set only by the workflow agent executor. - With the flag, `DatabaseToolProvider` generates CRUD tools exclusively from the role's explicit `objectPermission` rows: no row means no tools, and each verb gate reads the row directly (`canReadObjectRecords` for find tools, `canUpdateObjectRecords` for create/update/upsert, `canSoftDeleteObjectRecords` for delete). A verb left null is not granted; composed defaults and the system force-grant can no longer leak through. Composed permissions are still used for `restrictedFields`. - Explicit rows are read from the `flatObjectPermissionMaps` workspace cache key, fetched in the same `getOrRecompute` call as `rolesPermissions`: no extra query. - Without the flag (chat, MCP, tool index, workspace stats), behavior is unchanged: composed permissions, verified live (`getToolIndex` for an Admin returns the same 245 CRUD tools as before). - Removed the `CANNOT_ADD_OBJECT_PERMISSION_ON_SYSTEM_OBJECT` guard on `upsertObjectPermissions` so system objects can be granted explicitly. **Frontend: grant system objects from the agent Permissions tab** - The objects picker in the workflow agent side panel ends with a new "System objects" submenu listing all active system objects; picking one opens the same CRUD grant flow as regular objects. - Permissions granted on system objects now resolve their labels in the existing permission list and can be deleted (both previously looked up non-system objects only, which would have hidden such grants). Result: an agent granted one object ships ~10 tools instead of 146, cutting the prompt from ~110k tokens to a few thousand and the per-run cost accordingly. ## Notes - Removing the system-object guard affects the whole upsert path: user roles can also receive explicit system object rows via the API. A `canRead: false` row on a system object now takes effect at the query layer for that role. - The agent role is resolved as the first role of the permission config, matching `getObjectsPermissionsFromRolePermissionConfig` (multi-role is not supported yet). ## Tests - `database-tool.provider.spec.ts`: three new cases for the flag (object without a row emits nothing, partial row emits only granted verbs, absent flag keeps composed behavior even with zero rows, which guards the chat regression). - `object-permission.service.spec.ts`: the system-object case now asserts a successful upsert. - Integration: dropped the failing "system object" upsert case and its snapshot, added a successful system object upsert case. Both suites pass against a live server. <!-- This is an auto-generated description by cubic. --> <a href="https://cubic.dev/pr/twentyhq/twenty/pull/23400?utm_source=github" target="_blank" rel="noopener noreferrer" data-no-image-dialog="true"><picture><source media="(prefers-color-scheme: dark)" srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source media="(prefers-color-scheme: light)" srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img alt="Review in cubic" src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a> <!-- End of auto-generated description by cubic. --> |
||
|
Etienne
|
dfea3af778 |
fix(ai-chat): enrich zero-output stream captures and keep client-error exceptions out of Sentry (#23426)
## What & why Two related fixes that clean up Sentry reporting for the AI chat flow. ### 1. Enriched zero-output stream captures The AI chat stream's rejection handler previously skipped only `AbortError` and captured everything else to Sentry as-is. Two problems: - The SDK's bare `NoOutputGeneratedError` carries no troubleshooting context, so the Sentry issues were unactionable (no model, provider, workspace, or conversation size). - Expected interruptions (user abort, `STREAM_INTERRUPTED`) still generated noise. The rejection handler now handles three cases inline: - `AbortError` and `STREAM_INTERRUPTED` are expected interruptions and are not captured. - `NoOutputGeneratedError` is replaced with a single error whose message carries the full context as plain JSON: model, provider, workspace, thread, stream, turn, message count, conversation size, elapsed time, and the underlying stream error - recorded via a new `onError` handler, which also keeps stream-level errors visible in the worker logs. - Anything else is captured unchanged. The stable message prefix and single capture site keep zero-output events grouped separately from raw provider errors in Sentry. ### 2. Keep client-error domain exceptions out of Sentry `BILLING_CREDITS_EXHAUSTED` (a 402, i.e. an expected "user out of credits" condition) was landing in Sentry. Root cause: `CustomException` carries no HTTP status, so the worker/BullMQ path hands the raw exception to `shouldCaptureException`, which can't tell a 4xx client error from a 5xx server error and captures everything. The GraphQL/REST edges convert exceptions first, but background jobs bypass those converters. Fix, mirroring how `HttpException.getStatus()` already works: - `CustomException` gains an intrinsic `statusCode`. - `shouldCaptureException` skips a `CustomException` whose `statusCode < 500`, as a branch symmetric to the existing `HttpException` check. This covers every path, including the worker. - `BillingException` populates `statusCode` from the existing `getBillingExceptionStatusCode` mapping, so credits-exhausted (402) stays out of Sentry while the 500-mapped billing codes are still captured. Exceptions that don't set `statusCode` default to undefined and are captured exactly as before, so other domains are unaffected until they opt in. ## Tests - ai-chat unit suite passes (13 suites, 76 tests). - Existing billing exception handler tests pass. - `typecheck` passes. <!-- This is an auto-generated description by cubic. --> <a href="https://cubic.dev/pr/twentyhq/twenty/pull/23426?utm_source=github" target="_blank" rel="noopener noreferrer" data-no-image-dialog="true"><picture><source media="(prefers-color-scheme: dark)" srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source media="(prefers-color-scheme: light)" srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img alt="Review in cubic" src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a> <!-- End of auto-generated description by cubic. --> |
||
|
Paul Rastoin
|
ceb699c43f |
Message campaign backfill search field metadata (#23428)
<!-- This is an auto-generated description by cubic. --> <a href="https://cubic.dev/pr/twentyhq/twenty/pull/23428?utm_source=github" target="_blank" rel="noopener noreferrer" data-no-image-dialog="true"><picture><source media="(prefers-color-scheme: dark)" srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source media="(prefers-color-scheme: light)" srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img alt="Review in cubic" src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a> <!-- End of auto-generated description by cubic. --> |
||
|
Abdul Rahman
|
6c66d4c862 |
fix(ai-agent): use a non-workflow base system prompt for programmatic agent runs (#23394)
`AgentAsyncExecutorService` hardcoded `WORKFLOW_SYSTEM_PROMPTS.BASE`, so every caller was told "You are executing as part of a workflow automation" and "your output may be used by downstream workflow nodes". That is only true for the workflow AI-agent action. The `runAgent` API (used by apps such as the call recorder) and agent evaluations got the same framing, which does not describe how they run or where their output goes. The executor no longer asserts its own execution context: `executeAgent` now takes a required `baseSystemPrompt` and each caller supplies its own. - Workflow AI-agent action passes `WORKFLOW_SYSTEM_PROMPTS.BASE` (unchanged behavior) - `runAgent` and evaluations pass the new `AGENT_RUN_BASE_SYSTEM_PROMPT` The param is required rather than defaulted so every call site states its context and no future caller silently inherits the wrong one. Prompt constants are also split one export per file, with the shared tool-usage guidance extracted into `TOOL_USAGE_STRATEGY` so both bases compose it. No GraphQL schema, SDK, or database changes. <!-- This is an auto-generated description by cubic. --> <a href="https://cubic.dev/pr/twentyhq/twenty/pull/23394?utm_source=github" target="_blank" rel="noopener noreferrer" data-no-image-dialog="true"><picture><source media="(prefers-color-scheme: dark)" srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source media="(prefers-color-scheme: light)" srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img alt="Review in cubic" src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a> <!-- End of auto-generated description by cubic. --> |
||
|
neo773
|
1e58c3073c |
Feat/email composer improvements (#23188)
- Move composer to dedicated page - Add test email option - Auto saved as draft can be revisited from `objects/messageCampaigns` later - Campaign stats component https://github.com/user-attachments/assets/9e523116-e79b-496d-9c9d-3887e0c9213f <!-- This is an auto-generated description by cubic. --> <a href="https://cubic.dev/pr/twentyhq/twenty/pull/23188?utm_source=github" target="_blank" rel="noopener noreferrer" data-no-image-dialog="true"><picture><source media="(prefers-color-scheme: dark)" srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source media="(prefers-color-scheme: light)" srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img alt="Review in cubic" src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a> <!-- End of auto-generated description by cubic. --> --------- Co-authored-by: Félix Malfait <felix.malfait@gmail.com> |
||
|
Etienne
|
74260a161d |
fix(ai): stop double-counting cache-creation tokens in reported token totals (#23405)
## Context Under AI SDK v6 usage normalization, `usage.inputTokens` is the **full prompt**: fresh (noCache) + cache-read + cache-creation tokens. Our `totalTokens` formulas still added `cacheCreationTokens` (extracted from provider metadata) on top of `inputTokens` — a leftover from the pre-v6 SDK generation, where flat `inputTokens` excluded cache tokens. The v6 upgrade changed the semantics under the formula's feet, so every Claude run using prompt caching reported a `totalTokens` inflated by exactly `cacheCreationTokens`. ## Evidence, traced through AI SDK source **1. The Anthropic provider folds cache tokens into `inputTokens`.** The raw Anthropic API reports `input_tokens` *excluding* cache tokens; the provider sums all three components — [`convertAnthropicMessagesUsage`, `@ai-sdk/anthropic@3.0.84`](https://github.com/vercel/ai/blob/%40ai-sdk/anthropic%403.0.84/packages/anthropic/src/convert-anthropic-messages-usage.ts): ```ts inputTokens: { total: inputTokens + cacheCreationTokens + cacheReadTokens, noCache: inputTokens, cacheRead: cacheReadTokens, cacheWrite: cacheCreationTokens, } ``` **2. ai core surfaces that total as the app-visible `usage.inputTokens`** — [`asLanguageModelUsage`, `ai@6.0.97`](https://github.com/vercel/ai/blob/ai%406.0.97/packages/ai/src/types/usage.ts): ```ts inputTokens: usage.inputTokens.total, ... totalTokens: addTokenCounts(usage.inputTokens.total, usage.outputTokens.total), ``` So the SDK's own `totalTokens` is already "full prompt (incl. cache read + creation) + output". **3. The value we were adding on top is the same one already inside `inputTokens`.** The provider also exposes the raw API field in metadata (`@ai-sdk/anthropic` dist): ```ts const anthropicMetadata = { usage: response.usage, cacheCreationInputTokens: response.usage.cache_creation_input_tokens ?? null, ... ``` `extract-cache-creation-tokens.util.ts` reads exactly `providerMetadata.anthropic.cacheCreationInputTokens` — the same `cache_creation_input_tokens` that step 1 already folded into `inputTokens.total`. Adding it again counts it twice. **Worked example** (matches the new pinning test): API returns `input_tokens: 400, cache_read_input_tokens: 600, cache_creation_input_tokens: 200, output_tokens: 500` → app sees `usage.inputTokens = 1200`, `providerMetadata.anthropic.cacheCreationInputTokens = 200` → old formula reported `1200 + 500 + 200 = 1900`; actual tokens processed: `1700`. All snippets are verbatim from the version tags in `vercel/ai` and match the installed `node_modules` dists. ## Provider independence `inputTokens + outputTokens` is correct for every provider Twenty routes through, not just Anthropic: - The v3 provider spec (`@ai-sdk/provider`) defines `inputTokens.total` as "the total number of input (prompt) tokens used", with `noCache`/`cacheRead`/`cacheWrite` as its components — and all 8 installed provider packages comply (verified in dists): `anthropic` and `amazon-bedrock` sum the components explicitly ([`convertBedrockUsage`, `@ai-sdk/amazon-bedrock@4.0.117`](https://github.com/vercel/ai/blob/%40ai-sdk/amazon-bedrock%404.0.117/packages/amazon-bedrock/src/convert-bedrock-usage.ts): `total: inputTokens + cacheReadTokens + cacheWriteTokens`); `openai`, `azure`, `google`, `mistral`, and `openai-compatible` pass through wire values that already include cached tokens; `xai` even detects which wire convention the API used and normalizes either way. - The removed `cacheCreationTokens` term was already 0 for every provider except Anthropic/Bedrock (`extract-cache-creation-tokens.util.ts` only reads those two metadata namespaces), so this PR is a strict no-op for OpenAI-style providers and only removes the double-count where it existed. Caveat: a custom `AI_PROVIDERS` entry pointing at a legacy V2-spec provider package bypasses this normalization (ai core's shim passes flat usage through verbatim); that path could misreport under any formula, and none of the built-in providers use it. ## What changed Four sites computed the inflated total: - `ai-billing.service.ts` — `quantity` on the emitted AI token usage event - `chat-execution.service.ts` — chat-turn usage event - `agent-async-executor.service.ts` — workflow-agent usage event - `build-ai-agent-step-log.util.ts` — workflow step log (display) The first three now compute `totalTokens = inputTokens + outputTokens`; the step-log util uses the SDK's `usage.totalTokens` directly (it receives the `generateText` usage object, where the field is guaranteed). The explicit sum is used where usage objects are hand-assembled or merged — e.g. the streaming path in `stream-agent-chat.job.ts` builds usage literals with no `totalTokens` field at all, so `usage.totalTokens ?? 0` would silently emit 0. Both forms are definitionally identical where the SDK object exists, since ai core computes `totalTokens` as `input + output` (see evidence above). **Impact: reported/analytics quantities only.** Billed credits (`creditsUsedMicro`) come from `computeCostBreakdown`, which already handles the cache-inclusive convention correctly and is unchanged. **Ops note:** `usageEvent.quantity` for cache-heavy workspaces steps down on deploy — dashboards trending this metric may want an annotation. Historical rows are not backfilled (per-row component fields aren't stored, so mixed-era rows can't be reliably corrected). ## How tested - Updated `build-ai-agent-step-log.util.spec.ts` expectation (155 → 150 with `cacheCreationTokens: 5` still present) - New pinning test in `ai-billing.service.spec.ts`: emitted `quantity` is 1700 (not 1900) for inclusive Anthropic usage with `cacheCreationTokens: 200` - New pinning test in `agent-async-executor.service.spec.ts`: emitted total is 150 (not 180) when steps carry `providerMetadata.anthropic.cacheCreationInputTokens` - 3 suites / 20 tests pass; oxlint, oxfmt, and `nx typecheck twenty-server` clean <!-- This is an auto-generated description by cubic. --> <a href="https://cubic.dev/pr/twentyhq/twenty/pull/23405?utm_source=github" target="_blank" rel="noopener noreferrer" data-no-image-dialog="true"><picture><source media="(prefers-color-scheme: dark)" srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source media="(prefers-color-scheme: light)" srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img alt="Review in cubic" src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a> <!-- End of auto-generated description by cubic. --> |
||
|
Weiko
|
ae0ffb1373 |
perf: use cache for view entity lookups (#23384)
## Context View child mutation guards resolve a parent view before checking access. The lookup service queried PostgreSQL for a single `viewId`, even though the same relationship already exists in the workspace flat-map cache. With 15 guards using this service, each guarded mutation could add an unnecessary database round trip. ## What changed - Replace the five workspace-scoped repositories with `WorkspaceManyOrAllFlatEntityMapsCacheService` - Load only the flat map matching the requested child kind - Resolve view fields, filters, filter groups, groups, and sorts by ID - Preserve the existing `null` behavior for missing entities Each lookup is keyed by entity ID, no workspace-wide filtering is introduced. ## Expected impact On a warm workspace cache, permission guards resolve the parent `viewId` without querying PostgreSQL. Cold caches retain the normal workspace cache recomputation behavior. ## Validation - Typecheck reports no errors in the changed file - Existing lookup semantics are preserved for all supported entity kinds and missing IDs <!-- This is an auto-generated description by cubic. --> <a href="https://cubic.dev/pr/twentyhq/twenty/pull/23384?utm_source=github" target="_blank" rel="noopener noreferrer" data-no-image-dialog="true"><picture><source media="(prefers-color-scheme: dark)" srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source media="(prefers-color-scheme: light)" srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img alt="Review in cubic" src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a> <!-- End of auto-generated description by cubic. --> |
||
|
Weiko
|
19903f89e5 |
Add indexes for channel webhook subscription external IDs (#23386)
## Context Incoming Microsoft messaging, Microsoft calendar, and Google calendar webhook notifications resolve their channel through `webhookSubscriptionExternalId`. The column was added without an index, so PostgreSQL has to scan the corresponding channel table for every notification. Under sustained webhook traffic, these repeated scans add unnecessary database work and keep core database connections occupied longer. ## What changed - Add a partial B-tree index on `messageChannel.webhookSubscriptionExternalId` - Add the equivalent index on `calendarChannel.webhookSubscriptionExternalId` - Register both indexes in the TypeORM entity metadata - Add an idempotent 2.25 fast instance upgrade command to create and remove them The webhook handlers and their queries remain unchanged. ## Why this design - The indexes contain only non-null subscription IDs, channels without an active subscription do not add index entries - A single-column index supports both the equality lookup used by Google and the `IN` lookup used by Microsoft - The indexes are intentionally non-unique, this preserves existing behavior and avoids making the upgrade fail if historical duplicate values exist - Subscription IDs are read much more often than they are updated, so index maintenance overhead should be negligible ## Expected impact Webhook channel resolution should require a targeted index lookup instead of a table scan. This reduces database work, shortens connection occupancy, and improves latency on webhook notification paths. This is a targeted database optimization. It complements the database pool changes, but is not expected to resolve every source of API tail latency by itself. ## Validation - Server typecheck passes - Oxlint and formatting checks pass - Upgrade command uses idempotent `CREATE INDEX IF NOT EXISTS` and `DROP INDEX IF EXISTS` statements <!-- This is an auto-generated description by cubic. --> <a href="https://cubic.dev/pr/twentyhq/twenty/pull/23386?utm_source=github" target="_blank" rel="noopener noreferrer" data-no-image-dialog="true"><picture><source media="(prefers-color-scheme: dark)" srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source media="(prefers-color-scheme: light)" srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img alt="Review in cubic" src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a> <!-- End of auto-generated description by cubic. --> |
||
|
Weiko
|
cc5ff4869d |
perf: use cache for role validation (#23383)
## Context Role assignment validation queried PostgreSQL only to check whether a role exists and whether `canBeAssignedToUsers` is enabled. Both values already exist in `flatRoleMaps`. This validation runs when inviting users and assigning a role to a user workspace. ## What changed - Replace the role repository lookup with `flatRoleMaps` - Resolve the role through the existing keyed flat-map helper - Preserve the existing role-not-found and role-not-assignable errors - Replace unused TypeORM module wiring with the flat entity cache module ## Expected impact On a warm workspace cache, role assignment validation uses an O(1) map lookup and avoids a PostgreSQL round trip. Cold caches retain the normal workspace cache recomputation behavior. ## Validation - Typecheck reports no errors in the changed files - Existing validation outcomes and exception codes are preserved <!-- This is an auto-generated description by cubic. --> <a href="https://cubic.dev/pr/twentyhq/twenty/pull/23383?utm_source=github" target="_blank" rel="noopener noreferrer" data-no-image-dialog="true"><picture><source media="(prefers-color-scheme: dark)" srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source media="(prefers-color-scheme: light)" srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img alt="Review in cubic" src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a> <!-- End of auto-generated description by cubic. --> |
||
|
Weiko
|
8481c76bfb |
perf: use cache for webhook reads (#23382)
## Context Webhook reads queried both the webhook and application tables, then rebuilt the same flat webhook representation already maintained by the workspace cache. This affected REST, GraphQL, and the webhook listing tool. ## What changed - Read `findAll` and `findById` from `flatWebhookMaps` - Keep `findById` as a keyed ID lookup - Preserve `findAll` ordering by `createdAt` - Remove unused webhook and application repository wiring The flat webhook cache contains only active webhooks and already includes the application universal identifier needed by the DTO conversion. ## Expected impact On a warm workspace cache, webhook reads avoid queries to both the webhook and application tables. `findAll` still iterates over every returned webhook, matching the original query's result cardinality, while `findById` uses a keyed lookup. ## Validation - Typecheck reports no errors in the changed files - `findAll` ordering and `findById` missing-record behavior are preserved <!-- This is an auto-generated description by cubic. --> <a href="https://cubic.dev/pr/twentyhq/twenty/pull/23382?utm_source=github" target="_blank" rel="noopener noreferrer" data-no-image-dialog="true"><picture><source media="(prefers-color-scheme: dark)" srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source media="(prefers-color-scheme: light)" srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img alt="Review in cubic" src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a> <!-- End of auto-generated description by cubic. --> |
||
|
Paul Rastoin
|
49e2272fb9 |
fix(workspace-migration): stop leaking workspace ids in delete action payloads (#23377)
Closes https://github.com/twentyhq/core-team-issues/issues/2732 ## Problem Workspace migration delete actions embedded the raw workspace-cache flat entity as their `flatEntity` payload, leaking: - `id`, `workspaceId`, `applicationId` - raw many-to-one join columns (`objectMetadataId`, `relationTargetFieldMetadataId`, ...) - raw FK aggregators (`viewFieldIds`, ...) - raw jsonb properties containing serialized relations (`settings`, `overrides`, `configuration`) Create actions already expose universal identifiers only. The asymmetry made identical migrations non-portable across workspaces (payloads embed random workspace primary keys) and caused snapshot flakiness in integration suites. ## Fix - Add `deleteFlatEntityForeignKeyAggregators` (raw-side counterpart of `deleteUniversalFlatEntityForeignKeyAggregators`, following the `flatEntityForeignKeyAggregator` / `universalFlatEntityForeignKeyAggregator` naming of `ALL_ONE_TO_MANY_METADATA_RELATIONS`). It strips base workspace-scoped properties, every property registered with a `universalProperty` counterpart in `ALL_ENTITY_PROPERTIES_CONFIGURATION_BY_METADATA_NAME` (covers raw join columns and serialized jsonb, including cases not modeled as many-to-one relations like `labelIdentifierFieldMetadataId`), and raw one-to-many `...Ids` aggregators. Its scope is disjoint from the universal-side util. - Apply it in the delete branch of `WorkspaceEntityMigrationBuilderService` — the single point where delete-action `flatEntity` is attached — so the payload matches its `MetadataUniversalFlatEntity<T>` type at runtime. Universal `...UniversalIdentifiers` aggregators are kept (they are portable), so `BaseUniversalDeleteWorkspaceMigrationAction` needs no type change. - Regenerate the affected `successful-sync-application-workspace-migration` snapshot: the delete payload now only carries universal identifiers. Safe downstream: the runner resolves delete targets via `universalIdentifier` lookups in current maps and metadata events fetch the deleted entity from maps by `entityId`; no consumer reads the stripped properties (only create handlers consume `action.flatEntity`). Note: the `normalizeIdCollections` mitigation flag mentioned in the issue does not exist on `main`, so there was nothing to remove. ## Tests - New snapshot-based unit spec for the strip util (objectMetadata and fieldMetadata shapes, plus input immutability). - Full twenty-server unit suite: 6922 passed. - Integration with live DB: full `metadata/suites/application` (50 suites), object/field/index/agent metadata suites, all 26 `graphql/suites/view` suites, `failing-agent-deletion`, `object-identifier-update-side-effect-on-view-field` — all green, no other snapshot changes. |
||
|
Weiko
|
a66aacfb82 |
perf: deduplicate tool permission role loads (#23366)
## Context Building the tool catalog asks every provider whether it is available for the current role configuration. Several providers perform multiple permission checks, so one catalog build can evaluate the same roles repeatedly. Previously, each `checkRolesPermissions` or `hasToolPermission` call loaded the configured roles and their permission flags from PostgreSQL. These queries returned data that already exists in the workspace cache: - `flatRoleMaps` contains role settings and the IDs of assigned permission flags - `flatRolePermissionFlagMaps` links those assignments to permission flag universal identifiers This created redundant database round trips on the latency-sensitive tool discovery path. ## What changed Permission checks now evaluate roles from the existing workspace cache instead of loading `RoleEntity` records and relations from PostgreSQL. The new flow: 1. Load `flatRoleMaps` and `flatRolePermissionFlagMaps` through `WorkspaceCacheService` 2. Resolve every role ID from `flatRoleMaps` 3. Check `canAccessAllTools` or `canUpdateAllSettings` 4. If needed, check explicit permission flags with `flatRoleHasPermissionFlag` 5. Apply the existing union or intersection rule This also benefits callers outside the tool registry, without adding provider parameters or request-scoped context plumbing. The direct agent-only role deletion path now invalidates and recomputes the two consumed cache maps after deleting a role. This prevents that path from leaving stale permission data behind. ## Why this is safe The authorization behavior remains unchanged: - `shouldBypassPermissionChecks` still grants access without loading permission data - A union grants access when at least one role grants it - An intersection grants access only when every role grants it - Base role permissions and explicitly assigned permission flags are both supported - Empty, duplicate, or missing role IDs fail closed - Cache failures fail closed This PR does not introduce a separate permission cache. It reuses the existing workspace metadata cache and its invalidation model. ## Expected impact On a warm workspace cache, these permission checks no longer query the role tables. Repeated checks during catalog and schema construction become in-memory cache lookups, reducing database pressure and avoiding repeated network round trips. A cold cache can still require its normal database recomputation. Subsequent permission checks reuse the populated workspace cache. ## Test coverage The permission service tests cover: - Union and intersection behavior - Base role grants - Explicit permission flag grants - Unrelated permission flags - Permission bypass - Empty, duplicate, and missing roles - Cache failures - No role repository query during cached evaluation The agent-role tests also verify that deleting an unused agent-only role refreshes the relevant cache maps, while a role that remains assigned does not trigger deletion or invalidation. |
||
|
Raphaël Bosi
|
2899058b5f |
Warn users before front components navigate to an external site (#23270)
https://github.com/user-attachments/assets/af3fb042-d066-4e0c-9348-f86ea92a6fcd Front component anchors render a real host `<a>`, so clicking a link to another domain performed an uncontrolled full-page navigation. This adds a phishing-resistant "you're leaving Twenty" confirmation modal before navigating to an external origin (Fixes [#23260](https://github.com/twentyhq/twenty/issues/23260)). The renderer intercepts external anchor clicks in `createHtmlHostWrapper` and hands the destination to a host callback via context; twenty-front owns the modal (reuses `ConfirmationModal`) and a per-application list of trusted origins persisted in localStorage. A "Don't ask again for this site" checkbox (checked by default) skips the modal next time for that app. Scope is external cross-origin http(s) links only; same-origin links keep native behavior. External links always open in a new tab, so a component can never navigate the Twenty tab away, even once its origin is trusted. The modal is rendered by the trusted host, so components cannot style or suppress it. <!-- This is an auto-generated description by cubic. --> <a href="https://cubic.dev/pr/twentyhq/twenty/pull/23270?utm_source=github" target="_blank" rel="noopener noreferrer" data-no-image-dialog="true"><picture><source media="(prefers-color-scheme: dark)" srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source media="(prefers-color-scheme: light)" srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img alt="Review in cubic" src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a> <!-- End of auto-generated description by cubic. --> |
||
|
neo773
|
3fb29db28a |
Feat/email settings v2 (#23180)
Settings pages changes - Add `displayName` - Unsubscribers Page <img width="1496" height="844" alt="Screenshot 2026-07-22 at 8 52 15 PM" src="https://github.com/user-attachments/assets/69bc1993-4547-4a64-83a6-b47fef1a4e40" /> <!-- This is an auto-generated description by cubic. --> <a href="https://cubic.dev/pr/twentyhq/twenty/pull/23180?utm_source=github" target="_blank" rel="noopener noreferrer" data-no-image-dialog="true"><picture><source media="(prefers-color-scheme: dark)" srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source media="(prefers-color-scheme: light)" srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img alt="Review in cubic" src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a> <!-- End of auto-generated description by cubic. --> --------- Co-authored-by: Félix Malfait <felix.malfait@gmail.com> |
||
|
github-actions[bot]
|
763d31a859 |
chore: sync AI model catalog from models.dev (#23298)
Automated daily sync of `ai-providers.json` from [models.dev](https://models.dev). This PR updates pricing, context windows, and model availability based on the latest data. New models meeting inclusion criteria (tool calling, pricing data, context limits) are added automatically. Deprecated models are detected based on cost-efficiency within the same model family. **Please review before merging** — verify no critical models were incorrectly deprecated. <!-- This is an auto-generated description by cubic. --> <a href="https://cubic.dev/pr/twentyhq/twenty/pull/23298?utm_source=github" target="_blank" rel="noopener noreferrer" data-no-image-dialog="true"><picture><source media="(prefers-color-scheme: dark)" srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source media="(prefers-color-scheme: light)" srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img alt="Review in cubic" src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a> <!-- End of auto-generated description by cubic. --> Co-authored-by: FelixMalfait <6399865+FelixMalfait@users.noreply.github.com> |
||
|
Etienne
|
4851489ebc |
fix(ai-chat) fix AI chat tool-output spill leaks: spill learn_tools, cap navigation tools, truncate on spill failure (#23286)
## Context
AI chat spills tool outputs larger than `MAX_INLINE_TOOL_OUTPUT_BYTES`
(16 kB) to a file and lets
the model page them back with `search_output` / `extract_json_paths`.
Three paths bypass this and
let unbounded payloads into conversation history:
1. **`learn_tools` is never spilled.** Tool schemas go inline whatever
their size.
2. **Navigation tools have no inline cap.** They are exempt from
spilling by design (they page
spilled files), but nothing bounds their own output.
3. **Spill failure falls back to full inline.** On any spill error the
service returns the complete
payload with only a warning appended.
## What changed
- **`learn_tools` now spills.** `createLearnToolsTool` takes `{
excludeTools?, spillLargeOutput? }`
(same shape as `createExecuteToolTool`); chat execution enables it. Only
the bulky `tools`
schemas are spilled; `message` / `notFound` / `suggestions` stay inline
and the response carries
a `spilledTools` envelope (fileId, preview, hint) pageable via
`extract_json_paths`. MCP is
unchanged.
- **Navigation tools get a hard inline cap.** Still never spilled, but
output above 16 kB is
head+tail truncated with a marker telling the model to narrow the query
or page with `offset`.
- **Spill failure truncates instead of inlining.** The fallback returns
head+tail within the 16 kB
budget with the original byte size in the marker, keeping the warning.
- New `truncateHeadTail` util: byte-budgeted, marker-aware, UTF-8
codepoint-safe.
## Test plan
- `tool-output-spill.service.spec.ts`: spill envelope unchanged,
under-budget passthrough,
navigation cap for both tools (budget respected, marker mentions
`offset`, no file written),
truncated fallback on spill failure with warnings preserved.
- `learn-tools.tool.spec.ts`: no spill without the option, inline under
budget,
`message`/`notFound`/`suggestions` intact when spilled, spill-failure
warnings surfaced.
- `truncate-head-tail.util.spec.ts`: budget, head+tail+marker,
multibyte-safe cuts.
- 63 tests across 6 suites; `lint:diff-with-main` and `typecheck` green.
## Post-deploy
Watch the `AiChatToolOutputTokens` histogram (p95 should collapse to ~4k
tokens) and the
`AiChatInputTokens` / `AiChatCacheReadTokens` ratio on GPT-5-class
models.
<!-- This is an auto-generated description by cubic. -->
<a
href="https://cubic.dev/pr/twentyhq/twenty/pull/23286?utm_source=github"
target="_blank" rel="noopener noreferrer"
data-no-image-dialog="true"><picture><source
media="(prefers-color-scheme: dark)"
srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source
media="(prefers-color-scheme: light)"
srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img
alt="Review in cubic"
src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a>
<!-- End of auto-generated description by cubic. -->
|
||
|
Etienne
|
3a1067ec6d |
fix(server): index page-layout FKs to fix workspace cleanup timeout (#23289)
The cleanSuspendedWorkspacesJob cron timed out every run (Sentry monitor
"a timeout check-in was detected"): hard-deleting soft-deleted
workspaces hung on `DELETE FROM core.pageLayout`, hit the 10s query
timeout, rolled back, so those workspaces were never destroyed and got
retried hourly.
Root cause: the FKs in the pageLayout -> pageLayoutTab ->
pageLayoutWidget tree had no usable index on the referencing column. The
existing indexes lead with workspaceId and are partial ("deletedAt" IS
NULL), so ON DELETE CASCADE / SET NULL fell back to full sequential
scans of the shared core tables per deleted row; on layout-heavy
workspaces this exceeded 10s.
- Add non-partial FK-column indexes on pageLayoutTab(pageLayoutId) and
pageLayoutWidget(pageLayoutTabId)
<!-- This is an auto-generated description by cubic. -->
<a
href="https://cubic.dev/pr/twentyhq/twenty/pull/23289?utm_source=github"
target="_blank" rel="noopener noreferrer"
data-no-image-dialog="true"><picture><source
media="(prefers-color-scheme: dark)"
srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source
media="(prefers-color-scheme: light)"
srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img
alt="Review in cubic"
src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a>
<!-- End of auto-generated description by cubic. -->
|
||
|
Thomas Trompette
|
d6c186a71e |
Fix system objects bypassing role object permission overrides (#23280)
## Bug Fixes #23062 (security). A workspace member whose role denies all object access could still read/mutate system-object records (messages, calendar events, and related system objects). System objects bypassed explicit role-level object permissions. ## Root cause In `workspace-roles-permissions-cache.service.ts`, the per-object permission helper resolved values as: ```ts (isSystem ? true : (overrideValue ?? defaultValue)) ``` For every non-workflow, non-workspace-member system object this forced `read`/`update`/`softDelete`/`destroy` to `true`, so an explicit deny override on the role was never consulted. ## Fix Flip the precedence so an explicit role override wins, and the `isSystem` default only applies when the role provides no override: ```ts overrideValue ?? (isSystem ? true : defaultValue) ``` Because the override fields are `boolean | undefined`, `??` correctly honors an explicit `false` while still falling back to the system default (`true`) when the role has no override row for that object. Workflow objects (settings-gated via the `WORKFLOWS` flag) and workspace-member objects (settings-gated, always readable) are handled in separate branches and are unchanged, so their intended defaults do not regress. ## Testing - `nx lint:diff-with-main twenty-server` passes. - Typecheck: no new errors from this change (pre-existing unrelated failures in `twenty-shared` date-filter utils only). - Manually verified on a local instance that a deny-all role no longer has read access to system objects. <!-- This is an auto-generated description by cubic. --> <a href="https://cubic.dev/pr/twentyhq/twenty/pull/23280?utm_source=github" target="_blank" rel="noopener noreferrer" data-no-image-dialog="true"><picture><source media="(prefers-color-scheme: dark)" srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source media="(prefers-color-scheme: light)" srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img alt="Review in cubic" src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a> <!-- End of auto-generated description by cubic. --> |
||
|
Abdul Rahman
|
7067f6ef88 |
fix: honor agent rolePermissionConfig in record CRUD (#23248)
## Summary - Agent tools were built with the agent’s `rolePermissionConfig`, but record CRUD ignored it and re-resolved permissions from `authContext` (app `defaultRoleId`) - CRUD services now pass `rolePermissionConfig` through `CommonApiContextBuilder` and the common query runner, so repository access matches the agent role - Workflow/chat paths already use the same role for auth and `rolePermissionConfig`, so their behavior should be unchanged <!-- This is an auto-generated description by cubic. --> <a href="https://cubic.dev/pr/twentyhq/twenty/pull/23248?utm_source=github" target="_blank" rel="noopener noreferrer" data-no-image-dialog="true"><picture><source media="(prefers-color-scheme: dark)" srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source media="(prefers-color-scheme: light)" srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img alt="Review in cubic" src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a> <!-- End of auto-generated description by cubic. --> |
||
|
Etienne
|
0bba75dd18 |
fix(ai-chat): stop duplicate tool_use ids from bricking threads (#23277)
## Issue Some AI chat threads become permanently broken. Every turn fails with an Anthropic 400: messages.5.content.1: tool_use ids must be unique The error is on the message *history*, so once a thread is in this state every subsequent turn fails too, not just the one that triggered it. It surfaces most visibly when aborting a thread and continuing it, but the abort is incidental: it just replays the already-corrupted history. ## Root cause A single tool call gets persisted as **two message parts sharing one `toolCallId`**. Confirmed in the DB for the affected thread, one assistant message held: - `tool-extract_json_paths` and `dynamic-tool`, both `toolu_01FXxxfBYP27NZEvA6AZ3AJc` - `tool-search_output` and `dynamic-tool`, both `toolu_01VqnFrYdGCERAc2dbG3zAyx` On the next turn `convertToModelMessages` turns each pair into two `tool_use` blocks with the same id, which Anthropic rejects. ## Why it happens It is not two concurrent calls. It is one call the AI SDK classifies inconsistently across its own stream chunks. The chat only exposes a small set of directly-callable tools (`execute_tool`, `learn_tools`, `load_skills`, `ask_questions`, plus native/preloaded ones). Registry tools like `extract_json_paths` and `search_output` are reachable only through `execute_tool`. When the model shortcuts that and calls one directly, the name is not in the active `ToolSet`, and the SDK does this: 1. On `tool-input-start`, `dynamic` is derived from the tool set: `tools[name]?.type === "dynamic"`. The tool is absent, so `dynamic: false`, and a **static** `tool-<name>` part is created. 2. On finalization the unknown tool throws `NoSuchToolError`. `repairToolCall` intentionally skips name errors (`return null`), so the SDK re-emits the call with a hardcoded `dynamic: true`. That error routes to the **dynamic** path and creates a second `dynamic-tool` part with the same id. The UI-message builder keeps static and dynamic tool parts in separate buckets, each searched by `toolCallId` independently, so the mid-call static-to-dynamic flip produces two parts for one call. Both persist and break the next turn. ## Fix Two independent read/convert-path passes, both in `sanitizeMessagePartsForModel` in `chat-execution.service.ts`, running before `convertToModelMessages`: 1. **`finalizeDanglingToolParts`** now dedupes tool parts by `toolCallId` (first-wins), keeping the `input-streaming` filter ahead of the dedup so a leading streaming duplicate can't strand the call. Because it runs before conversion and on the write paths too, already-corrupted threads are un-bricked on their next turn with no migration. 2. **`guideUncallableToolCallsToMetaTool`** addresses the behavior that caused it: when the model calls a tool that is not directly callable, it appends the `learn_tools` -> `execute_tool` flow to that failed tool result, so the model reads how to reach the tool. Detection is structural (a failed tool part whose name is not in the active tool set), not string-matched against the SDK's error wording. Unit tests added for both. Typecheck, lint, and the suite pass. Note: the stale duplicate rows already in the DB are harmless (deduped on every read); no migration is required. <!-- This is an auto-generated description by cubic. --> <a href="https://cubic.dev/pr/twentyhq/twenty/pull/23277?utm_source=github" target="_blank" rel="noopener noreferrer" data-no-image-dialog="true"><picture><source media="(prefers-color-scheme: dark)" srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source media="(prefers-color-scheme: light)" srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img alt="Review in cubic" src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a> <!-- End of auto-generated description by cubic. --> |
||
|
Félix Malfait
|
1fdb5605f1 |
feat: kanban, calendar and grouped-table layouts for relation field widgets (#23112)
## Context A relation field widget on a record page can already embed a record-scoped view rendered as a **table** (`FieldDisplayMode.TABLE`) — e.g. a Company's Opportunities. This brings **kanban, calendar and grouped-table** to that same embedded view, so the board/calendar stays scoped to *this* record's related records (not a standalone all-records widget — that was the earlier #23003 approach, closed). Builds directly on the merged dashboard widget layouts (#22963), reusing its renderer, draft/save pipeline, and settings dropdowns. ## Approach — extend the existing "Table" display mode The relation field widget already stores a `viewId` and renders it through the layout-agnostic `RecordTableWidgetRendererContent` (which branches on the embedded view's `type`), scoped to the current record via `RecordFilterValueDependenciesContext`. So rendering + persistence already work for any widget view type — only the authoring UI and one server gate were missing. **No new `FieldDisplayMode`, no data migration.** ## Server - `view-widget-upsert.service.ts`: a field widget in table display mode (`isFieldTableWidget`) could already persist viewFields/filters/sorts through this path, but was **blocked from updating view settings** (`type` / group-by / calendar), pinning its embedded view to a table. The widget-type guard earlier in the method already rejects every widget kind other than record-table and field-table, so the now-redundant record-table-only guard on the view-settings branch is dropped. The allowed-widget-view-types check and the downstream group-by / calendar-field validations still apply equally. ## Frontend - **One merged Layout picker.** The field widget's Layout dropdown lists **Field / Card / Table / Kanban / Calendar** in a single flat list — you pick Kanban directly, instead of "Display as: Table" first and a separate embedded-view layout second. Picking a view layout selects the `TABLE` display mode under the hood, seeds the record-scoped embedded view on first use (with a default group-by / date field), and applies the layout in the same click. Kanban/Calendar are disabled with a hint ("Needs a Select field" / "Needs a Date field") when the relation target can't support them — same gating as the dashboard picker. The row's icon and description reflect the effective selection (e.g. Kanban), and the dropdown mounts the draft-init effect so switching straight from Field/Card to Kanban works before the table renderer has ever mounted. - **Contextual rows** (Group by / Date field / Calendar view / Hide empty groups) extracted from the dashboard panel into a reusable `WidgetViewLayoutSettingsRows` (source object passed in — fixed to the relation target; no Source / Limit rows) and surfaced under the picker while a view layout is active. Its standalone layout row is hidden here (`isLayoutRowHidden`) since layout lives in the merged picker. - Reuses the dashboard draft snapshot + `upsertViewWidget` save pipeline and the group-by/calendar dropdown components unchanged. ## Scope - **One-to-many relations only** (matches the existing `getFieldWidgetAvailableDisplayModes` gate; junction / many-to-many stay table-only — a pre-existing inconsistency left untouched here). - Field-widget **calendars inherit the dashboard's behavior** (month read-only by default; day/week + drag-to-reschedule only behind `IS_CALENDAR_WEEK_VIEW_ENABLED`), since it's literally the same renderer. ## Tests - Server integration (`upsert-view-widget-view-settings.integration-spec.ts`): a FIELD + TABLE widget can switch its embedded view to `KANBAN_WIDGET` (with group-by) and `CALENDAR_WIDGET` (with date field), and the kanban group-by validation still applies through the newly-opened path. - Front unit: `getWidgetViewLayoutSettingsItemIds` (keyboard-nav row ids per layout/flag/group state). ## Follow-ups (intentionally not in this PR) - Migrate the dashboard settings panel onto the shared `WidgetViewLayoutSettingsRows` (kept out to avoid churning the just-merged #22963 file; behavior-preserving refactor). https://claude.ai/code/session_01E5N87kwwZWhDtEQaP72cMf |
||
|
github-actions[bot]
|
5bc97f8591 |
chore: sync AI model catalog from models.dev (#23242)
Automated daily sync of `ai-providers.json` from [models.dev](https://models.dev). This PR updates pricing, context windows, and model availability based on the latest data. New models meeting inclusion criteria (tool calling, pricing data, context limits) are added automatically. Deprecated models are detected based on cost-efficiency within the same model family. **Please review before merging** — verify no critical models were incorrectly deprecated. <!-- This is an auto-generated description by cubic. --> <a href="https://cubic.dev/pr/twentyhq/twenty/pull/23242?utm_source=github" target="_blank" rel="noopener noreferrer" data-no-image-dialog="true"><picture><source media="(prefers-color-scheme: dark)" srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source media="(prefers-color-scheme: light)" srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img alt="Review in cubic" src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a> <!-- End of auto-generated description by cubic. --> Co-authored-by: FelixMalfait <6399865+FelixMalfait@users.noreply.github.com> |
||
|
Abdul Rahman
|
2c79093b74 |
feat: agent roleUniversalIdentifier for manifest-driven role assignment (#23206)
## Summary - Adds optional `roleUniversalIdentifier` on `AgentManifest` / `defineAgent` so apps can declaratively assign a role to an agent (same config shape as `defaultRoleUniversalIdentifier`). - Wires `agentUniversalIdentifier` as a sync many-to-one FK on `roleTarget`, and emits a deterministic `roleTarget` from the agent during app sync (create / update / delete). - Enables app agents (e.g. Slack assistant) to get a role on install without postInstall hooks or manual admin assignment. <!-- This is an auto-generated description by cubic. --> <a href="https://cubic.dev/pr/twentyhq/twenty/pull/23206?utm_source=github" target="_blank" rel="noopener noreferrer" data-no-image-dialog="true"><picture><source media="(prefers-color-scheme: dark)" srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source media="(prefers-color-scheme: light)" srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img alt="Review in cubic" src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a> <!-- End of auto-generated description by cubic. --> |
||
|
Weiko
|
66df0ac47c |
Switch application stop/start commands to Redis-backed global kill switch (#23202)
## Context [#23183](https://github.com/twentyhq/twenty/pull/23183) introduced the right enforcement point: every logic-function execution is rejected centrally before consuming the shared workspace throttle when its application is stopped. However, its server-wide path reads PostgreSQL for every execution attempt. A kill switch is most useful while an application is producing abnormal load, potentially while PostgreSQL is already under pressure. The enforcement mechanism should not add more database traffic in that situation. This state is also operational and temporary. It is used to troubleshoot an application, not as durable application configuration. ## What this PR changes - Uses one global Redis key per application universal identifier: ```text module:applications:kill-switch:{applicationUniversalIdentifier} ``` - Keeps the check in `LogicFunctionExecutorService`, before the workspace execution throttle. - Adds a 60-second process-local cache for both present and absent keys. - Deduplicates concurrent cache refreshes, so an execution burst causes at most one Redis read per application and process. - Fails open when Redis cannot be read and caches that result for the same minute, avoiding a Redis retry storm. - Removes the database columns, upgrade command, workspace-cache recomputation, registration lookup, and stop/start CLI commands introduced by #23183. - Keeps disabled queued executions non-retriable, without emitting one warning for every skipped payload. The switch is operated directly in Redis. For example: ```redis SET module:applications:kill-switch:{applicationUniversalIdentifier} 1 EX 3600 DEL module:applications:kill-switch:{applicationUniversalIdentifier} ``` Any value means stopped; deleting or expiring the key means enabled. ## Why this is a better fit | | #23183 | This PR | |---|---|---| | State | Durable PostgreSQL fields | Ephemeral Redis key | | Server-wide hot path | PostgreSQL lookup per execution | At most one Redis lookup per app/process/minute | | Scope | Workspace and application registration | Application universal identifier across all workspaces | | Operational cleanup | Explicit start command | `DEL`, eviction, restart, or operator-selected TTL | | Database dependency during an incident | Required | None | The trade-off is deliberate: a Redis change can take up to 60 seconds to reach every process, and the switch is lost when the cache key disappears. That is acceptable for a temporary troubleshooting control and keeps the normal execution path inexpensive. Existing in-flight functions are not interrupted. New direct or queued executions are rejected when they reach the executor. |
||
|
Etienne
|
5c825e8712 |
fix(ai-chat): write the stream heartbeat before the DB claim (#23198)
## Problem Answering an `ask_questions` (select) prompt sometimes killed the turn with "Failed to get response. The response was interrupted before it could finish." The answer was swallowed and Retry rewound the whole turn. It was intermittent, worse on long threads and when coming back from another tab. Reported in [discord quality issue](https://discord.com/channels/1130383047699738754/1526875783170097172). Confirmed in prod: ~28 `ai_chat_turn_failed_total{failure_phase="interrupted"}` over the last 7 days (the only failure phase firing), plus matching `the thread no longer holds this claim` worker logs around the report time. ## Root cause A stream is tracked by two records: the claim (`activeStreamId` in Postgres) and the heartbeat (a Redis key refreshed while the worker runs). `reapDeadStream` treats "claim set but no heartbeat" as a crashed worker and kills the turn. On the answer path the ordering left a window where that was falsely true: 1. `resolvePendingQuestion` writes `activeStreamId` to Postgres (claim set) 2. `enqueueResumeStream` reloads the thread and runs `loadMessagesFromDB` (reads every message and part, signs a URL per file, hundreds of ms on long threads) 3. only then `markClaimed` writes the heartbeat Between 1 and 3 the thread looks dead to the reaper. Worse, `question-answered` was published inside that window, so the client refetched, and the refetch's `chatStreamCatchupChunks` query runs the reaper, racing the server into its own setup window. The keepalive reap tick could land there too. ## Fix Enforce one invariant everywhere: the heartbeat exists before any DB row carries the `activeStreamId`, so "claim without heartbeat" can only ever mean a genuinely dead worker. - New `answerPendingQuestionAndResumeStream` owns the answer flow: `markClaimed` first, then the DB claim, then enqueue, then publish `question-answered` (moved after the enqueue so client refetches can't race the setup, and so we don't tell the client "answered" when the enqueue failed and rolled back). - Both failure paths clean up: clear the heartbeat if resolving fails; restore the pending question and clear the heartbeat if enqueueing fails. - `tryClaimStream` (send / retry / queue-flush) reordered the same way: heartbeat before the claim, cleared if the claim is lost. - `releaseStreamClaim` now also clears the heartbeat so failed claims leave no orphan key. No grace period or schema change needed: the ordering closes the race structurally. The Retry-rewinds-the-turn behavior is unrelated and left as a separate follow-up. ## Testing - New `agent-chat-streaming.service.answer.spec.ts`: heartbeat marked before the claim, publish only after enqueue, both failure paths restore state and clear the key. - Extended `agent-chat-streaming.service.claim.spec.ts`: heartbeat-before-claim ordering and key cleanup on lost claim / failed enqueue. - Full ai-chat suite green (74 tests), lint and typecheck clean. After deploy, `sum(increase(ai_chat_turn_failed_total{failure_phase="interrupted"}[1d]))` trending to zero confirms the fix. <!-- This is an auto-generated description by cubic. --> <a href="https://cubic.dev/pr/twentyhq/twenty/pull/23198?utm_source=github" target="_blank" rel="noopener noreferrer" data-no-image-dialog="true"><picture><source media="(prefers-color-scheme: dark)" srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source media="(prefers-color-scheme: light)" srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img alt="Review in cubic" src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a> <!-- End of auto-generated description by cubic. --> |
||
|
Scarab Systems
|
d1c70ab0bf |
Fix dashboard record table widget aggregate persistence (#23008)
## Summary - Update dashboard record-table widget aggregate changes to write into the widget draft while page layout edit mode is active. - Include `aggregateOperation` when saving record-table widget view fields through `upsertViewWidget`. - Persist aggregate operations server-side for widget view-field create, update, and clear flows. - Add frontend utility tests and backend integration coverage for widget aggregate create/update/clear behavior. Fixes #22934. ## Why Dashboard record-table widgets use their own draft view state while a page layout is being edited. The aggregate footer path was resolving fields through the normal current-view flow and then trying to persist immediately, which can miss widget draft fields and fail before the save flow runs. This change keeps aggregate edits in the widget draft during page layout editing, then saves the aggregate operation with the rest of the widget view configuration. ## Validation - `npx nx lint twenty-front` - `npx nx typecheck twenty-front` - `npx nx test twenty-front --configuration=ci` - `npx nx build twenty-front` - `npx nx build twenty-server` - `npx nx lint twenty-server --configuration=ci` - `npx nx typecheck twenty-server` - `npx nx test twenty-server --configuration=ci` - `npx nx jest --config ./jest-integration.config.ts --logHeapUsage --runTestsByPath test/integration/metadata/suites/view/upsert-view-widget.integration-spec.ts` - `git diff --check` Disclosure: I used AI-assisted coding tools while preparing this PR. I reviewed the changes myself, tested them, and take responsibility for the implementation and any follow-up revisions needed. <!-- This is an auto-generated description by cubic. --> <a href="https://cubic.dev/pr/twentyhq/twenty/pull/23008?utm_source=github" target="_blank" rel="noopener noreferrer" data-no-image-dialog="true"><picture><source media="(prefers-color-scheme: dark)" srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source media="(prefers-color-scheme: light)" srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img alt="Review in cubic" src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a> <!-- End of auto-generated description by cubic. --> |
||
|
martmull
|
a3f4acadb1 |
Add workspace and server level stop commands for applications (#23183)
## Context When an installed application misbehaves (e.g. a logic function loop DDoSing the server or the database), we currently have no targeted way to shut it down in production: the only kill switch is `LOGIC_FUNCTION_TYPE=DISABLED`, which disables logic functions for the whole instance. This PR adds an emergency stop mechanism at two levels: - **Workspace level**: stop one installed application in its workspace. - **Server level**: stop every application installed from an `applicationRegistration`, across all workspaces. ## How it works **New nullable `stoppedAt` columns** on `core.application` and `core.applicationRegistration` (fast instance command `2.24.0`, with `up`/`down` and `@WasIntroducedInUpgrade` decorators on the entities). **Enforcement in a single choke point**: `LogicFunctionExecutorService.execute()` is the funnel behind every execution path (public route triggers, server route triggers, cron triggers, database event triggers, workflow actions, agent tool calls, manual GraphQL execution, install hooks). A new `assertApplicationNotStopped` guard runs right after the flat entities are resolved and throws `LOGIC_FUNCTION_DISABLED` (already mapped to a 403 on route triggers and handled by the GraphQL exception handler) when: - `flatApplication.stoppedAt` is set (workspace-level stop, read from the cached flat application maps: zero extra runtime cost), or - the linked registration is stopped (one indexed PK lookup, same pattern as the existing per-execution server-variable query). **Propagation**: the workspace-level stop invalidates and recomputes `flatApplicationMaps` for the workspace, so all server instances pick the flag up within the local cache TTL (100ms). The registration-level flag is read live, so it is effective immediately. ## Ops commands ```bash # Workspace level yarn command:prod application:stop -a <application-id> yarn command:prod application:start -a <application-id> # Server level (all applications of the registration, all workspaces) yarn command:prod application-registration:stop -r <application-registration-id> yarn command:prod application-registration:start -r <application-registration-id> ``` Each command logs what was stopped/started and, for registrations, how many installed applications are affected. ## Notes - Stopped executions fail fast at the guard, so queued trigger jobs (cron/db-event) burn a negligible amount of work while stopped. - The two flags are independent: lifting a registration-level stop does not clear workspace-level stops that were set individually, and vice versa. - Unit tests added for `ApplicationStopService`. --- _Generated by [Claude Code](https://claude.ai/code/session_01CjEnKUACn89aSgK1wEMH2d)_ <!-- This is an auto-generated description by cubic. --> <a href="https://cubic.dev/pr/twentyhq/twenty/pull/23183?utm_source=github" target="_blank" rel="noopener noreferrer" data-no-image-dialog="true"><picture><source media="(prefers-color-scheme: dark)" srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source media="(prefers-color-scheme: light)" srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img alt="Review in cubic" src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a> <!-- End of auto-generated description by cubic. --> |
||
|
Weiko
|
0108a34765 |
Rekey metadata caches when flat map hashes change (#23164)
## Context
The `/metadata` GraphQL response cache (`ObjectMetadataItems`,
`FindAllViews`) and the workspace SDL cache were keyed on
`workspace.metadataVersion`, an integer bumped on every object/field
migration. That mechanism is legacy (the migration runner literally
calls it `getLegacyCacheInvalidationPromises`): the data plane already
moved to `WorkspaceCacheService`, which versions each flat entity map
with its own hash minted on invalidation.
Version keying had two concrete costs: the version bump was the only
proactive invalidation for `ObjectMetadataItems`, and keeping
`FindAllViews` fresh required `flushGraphQLOperation`, a full Redis
keyspace SCAN on every relevant migration. It is also the main blocker
for deprecating `metadataVersion` entirely.
This PR re-keys both caches on the flat-map hashes instead.
## What changed
**Response cache** (`use-cached-metadata.ts`): the key is now
`{operation}:{workspaceId}:{combinedDependencyHash}[:{userWorkspaceId}]:{locale}:{queryHash}`.
Each cached operation declares which flat maps its resolvers read
(`metadata-graphql-operations-to-cache.constant.ts`) plus a scope:
`ObjectMetadataItems` stays workspace-shared, `FindAllViews` is per-user
because unlisted-view visibility depends on the caller. When any
declared map changes, its hash rotates and the key rotates with it; no
flush needed. The key is resolved once per request and reused in
`onResponse`, so a rotation mid-request can never cache a response under
a fresher key than the data it was built from. If hash resolution fails,
the request is served uncached (Sentry-captured).
This also fixes three pre-existing key soundness gaps: `FindAllViews`
ignored locale although view names are translated server-side, the query
hash ignored GraphQL variables (`$viewTypes`), and mid-request version
rotation could re-key between request and response.
**SDL cache** (`workspace-graphql-schema-sdl.service.ts`): keyed on the
combined hash of the four maps the schema is generated from, taken from
the same `getOrRecomputeWithHashes` call that returns the data, so key
and content cannot skew. The `metadataVersion` read/seed block there is
gone; the Redis seed moved to `middleware.service.ts` so the
`X-Schema-Version` "refresh the page" check keeps working after the
Redis key's TTL expires.
**`WorkspaceCacheService`**: the internal pipeline now threads `{data,
hashes}` through every stage (local hit, hash validation, Redis fetch,
recompute) and the memoizer stores the pair, so returned hashes are
always consistent with returned data. New public
`getOrRecomputeWithHashes` and `getOrRecomputeCombinedHash`
(hashes-first: one MGET of the small `:hash` keys, full pipeline only
for missing ones, so cold pods never pull map payloads just to build a
key).
**Atomic pair writes** (`cache-storage.service.ts`): `mset` on the Redis
driver now delegates to the store's own `mset` (a MULTI of `SET ... PX`,
or native `MSET`), grouped by TTL. Previously it was a `Promise.all` of
independent SETs, so two concurrent recomputes could interleave and
leave one recompute's `:data` next to the other's `:hash`; with
hash-keyed caches that torn pair could persist a stale response under a
live key. `CoreEntityCacheService` writes through the same method and is
fixed for free.
**Cleanup**: `flush()` lost its `metadataVersion` parameter (always
pattern-flush per key on workspace deletion),
`METADATA_VERSIONED_WORKSPACE_CACHE_KEY` became
`HASH_KEYED_WORKSPACE_CACHE_KEYS` with the `MetadataVersion` key
relocated to `WORKSPACE_CACHE_KEYS` and the dead `ORMEntitySchemas`
entry removed.
## Deliberately unchanged
- `incrementMetadataVersion` and all its callers stay: the version still
feeds the `X-Schema-Version` check and the pinned upgrade commands.
Deprecating the column is a later stage.
- The runner's `FindAllViews` pattern-flush is kept for exactly one
release: view-only migrations never bump `metadataVersion`, so old pods
in a rolling deploy have no other invalidation signal for their
version-keyed entries. It gets deleted next release, which removes the
SCAN entirely.
- Old-shape cache entries are not migrated; they expire via the 7-day
TTL.
## Known limitations (follow-ups, not regressions)
- The plugin reads dependency hashes Redis-fresh while resolvers can
serve up to 10s-old memoized data, so a request landing right after a
migration can cache a pre-rotation response under the new key. Same
shape existed under `metadataVersion`; closing it needs request-scoped
snapshot plumbing.
- Concurrent recomputes are last-writer-wins (lost update). Fencing with
a conditional write is a follow-up.
## Validation
- Unit: response-cache plugin behavior (scope, key stash, serve-uncached
on failure, prototype-name guard), atomic `mset` batching, existing
`WorkspaceCacheService` spec passing unchanged.
- Integration: a new drift-guard spec runs the real
`ObjectMetadataItems`/`FindAllViews` operations with full frontend
selection sets against the in-process app, spies on
`WorkspaceCacheService`, and fails if resolvers read a flat map missing
from the declared dependency lists, so the constant cannot silently
drift.
- Manual against a live server: creating a field rotates the field-map
hash and the very next `ObjectMetadataItems` response contains it (hash
rotation is now its only invalidation path); warm hits are ~5ms; SDL
entries appear under hash-shaped keys via introspection.
## Suggested reading order
1. `workspace-cache.service.ts`, `workspace-cache-key.type.ts`,
`combine-cache-hashes.util.ts` (the `{data, hashes}` pipeline)
2. `use-cached-metadata.ts`,
`metadata-graphql-operations-to-cache.constant.ts`,
`metadata.module-factory.ts` (response cache)
3. `workspace-graphql-schema-sdl.service.ts`,
`workspace-cache-storage.service.ts` (SDL cache and renames)
4. `middleware.service.ts` (metadata version seed relocation)
5. `cache-storage.service.ts` (atomic writes)
6. Tests
|
||
|
Etienne
|
f67e9c6b05 |
fix(ai-chat): disable Responses API storage for Azure models to stop "Item with id ... not found" stream failures (#23182)
## Problem AI chat and workflow-agent turns on Azure-routed reasoning models (e.g. `azure-foundry-us/gpt-5.5`) intermittently die mid-answer with: > Failed to get response — `400 Item with id 'rs_...' not found` ## Root cause The OpenAI Responses API is stateful: each output item gets a server-side id (`rs_...` reasoning, `fc_...` function call). With `store: true` (the API default), the Vercel AI SDK replays prior assistant reasoning as `item_reference` entries that the provider must resolve from its own storage. For reasoning models the chain-of-thought is never returned in plaintext — it lives only as a stored referenced item, or as `encrypted_content` requested via `include: ['reasoning.encrypted_content']` (which the SDK only auto-adds when `store: false`). PR #20888 (June 22) switched to the safe `store: false` mode, but only for `AI_SDK_OPENAI`. `AI_SDK_AZURE` fell through to `default` and got no options, so every Azure request ran in stored-reference mode. Azure's item storage resolves those references unreliably (a server-side race — the July 21 failure could not find the exact `rs_...` id Azure itself had streamed seconds earlier in the same turn), and the failure is fatal to the stream. Retries replay the same persisted references, so they can fail again. ## Fix Add an `AI_SDK_AZURE` case in `getCallLevelProviderOptions` that passes `azure: { store: false }`. Both AI chat and workflow agents go through this helper, so both paths are covered. The provider-options key for `@ai-sdk/azure` is `azure` (its responses model is registered as `azure.responses`); I verified the `@ai-sdk/openai@3.0.54` copy nested inside `@ai-sdk/azure` has the same guards as the root `3.0.71`. With `store: false` the SDK: - auto-requests `include: ['reasoning.encrypted_content']` for reasoning models (the `gpt-5.5` deployment name passes reasoning detection), - replays reasoning as self-contained encrypted items instead of server-side references, - drops any stale unencrypted reasoning parts instead of referencing them. ### Transition safety Existing threads whose persisted reasoning has `reasoningEncryptedContent: null` simply have those parts skipped on replay. I checked prod logs for the related pairing-validation error ("was provided without its required...") and found zero occurrences since OpenAI-direct made this same switch a month ago, so the transition is safe. ## Testing - Added two Azure cases to `provider-options.util.spec.ts` — all 9 pass. - oxlint (type-aware) and oxfmt clean on the changed files. ## Post-deploy validation Inverse of the evidence: new Azure reasoning parts should persist with non-null `reasoningEncryptedContent`, and the Loki query below should go quiet. fixes : https://discord.com/channels/1130383047699738754/1526873169124659230 |
||
|
Etienne
|
c54ad3c0b9 |
feat(ai-instrumentation): fix AI histogram buckets & add tool-call duration instrumentation (#23173)
## What & why
Two related observability fixes for AI chat / agent / MCP metrics.
### 1. Widen histogram buckets (`widden-ai-histo`)
Latency percentiles for AI chat pinned at exactly **10s** on the
dashboard — not because anything times out, but because the histogram
buckets top out at 10,000.
`recordHistogram` created histograms without explicit bucket boundaries,
so the OTel SDK applied its defaults: `[0, 5, 10, 25, 50, 75, 100, 250,
500, 750, 1000, 2500, 5000, 7500, 10000]`. These were designed for small
generic values; interpreted as **ms**, the last finite bucket is exactly
10s. Every observation above 10s falls into the `+Inf` overflow bucket,
and quantile estimation clamps to the highest finite bound — so any
percentile that lands in the overflow draws a flat line at 10s.
Reality (gpt-5.5, last 30 days): 64% of turns exceed 10s (so even p50
pins at 10s), mean turn latency ~38s, max ~29min. The same 10k cap
affects the token-unit `tool-output-tokens` histograms.
**Changes:**
- `recordHistogram` now accepts an optional `bucketBoundaries`, applied
per-instrument via `advice.explicitBucketBoundaries` (supported in
`@opentelemetry/api@1.9.1`). Colocated with the metric, no
instrument-setup changes.
- Added bucket-boundary constants:
- `AI_LATENCY_MS_BUCKET_BOUNDARIES` — `[250, 500, 1000, 2500, 5000,
10000, 20000, 30000, 60000, 120000, 300000, 600000]` (250ms → 10min)
- `TOOL_OUTPUT_TOKENS_BUCKET_BOUNDARIES` — `[100, 250, 500, 1000, 2500,
5000, 10000, 25000, 50000, 100000, 250000, 500000]` (100 → 500k tokens)
- Wired boundaries into `ai-chat/turn-latency-ms`,
`ai-chat/step-latency-ms`, `ai-chat/ttft-ms` (latency) and
`ai-chat/tool-output-tokens`, `workflow-agent/tool-output-tokens`,
`mcp/tool-output-tokens` (tokens).
### 2. Instrument tool-call duration (`add-tool-duration-monitoring`)
Previously we tracked tool success/failure counts and output tokens, but
not how long each tool call took. Added a `*/tool-execution-duration-ms`
histogram for each execution path.
**Changes:**
- New metric keys: `ai-chat/tool-execution-duration-ms`,
`workflow-agent/tool-execution-duration-ms`,
`mcp/tool-execution-duration-ms`.
- New constant `TOOL_EXECUTION_DURATION_MS_BUCKET_BOUNDARIES` — `[25,
50, 100, 250, 500, 1000, 2500, 5000, 10000, 30000, 60000, 120000]` (25ms
→ 2min).
- **AI chat / agent node**: use the AI SDK's
`experimental_onToolCallFinish` callback (`ai@6.0.97`), which reports an
exact `durationMs` measured around the tool's `execute()`, plus
`toolCall.toolName` and `success`. Attributes `{ model, tool }`.
- **MCP**: measured directly around `tool.execute` with
`performance.now()`, recorded on both success and failure paths.
Attributes `{ tool }`.
## Notes
- Backward-compatible: metrics without `bucketBoundaries` (e.g.
`job/latency-ms`, `sdk-client-generation/duration-ms`) keep OTel
defaults.
- **Historical data stays clamped** — only writes after deploy use the
new buckets, so percentile panels will show a step change at deploy
time. For truth on existing data, use a mean panel (Sum/Count is exact)
or Sentry trace span durations.
- Provider-executed tools (e.g. native web search) run inside the
provider, so they don't emit a local duration — same limitation as the
existing tool counters.
<!-- This is an auto-generated description by cubic. -->
<a
href="https://cubic.dev/pr/twentyhq/twenty/pull/23173?utm_source=github"
target="_blank" rel="noopener noreferrer"
data-no-image-dialog="true"><picture><source
media="(prefers-color-scheme: dark)"
srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source
media="(prefers-color-scheme: light)"
srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img
alt="Review in cubic"
src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a>
<!-- End of auto-generated description by cubic. -->
|
||
|
Abdul Rahman
|
04d1c2035c |
feat(connections): run a logic function on connection provider connect (#23167)
## What Adds an optional `onConnectLogicFunctionUniversalIdentifier` field to the connection provider manifest. When set, the referenced logic function is dispatched right after an OAuth connection is successfully established for that provider. This gives apps a first-class "on connect" hook — e.g. the Slack app can resolve the workspace's `team_id` via `auth.test` and claim the `team_id -> workspaceId` mapping in the SERVER key-value store immediately on connect, instead of racing against later events. Follow-up to the app key-value store PR (#23089). ## How - **twenty-shared**: add `onConnectLogicFunctionUniversalIdentifier` to `ConnectionProviderManifest`. - **twenty-sdk**: expose the field in `defineConnectionProvider` and validate it is a UUID `universalIdentifier`. - **twenty-server**: - add a nullable `onConnectLogicFunctionUniversalIdentifier` column to `ConnectionProviderEntity` (+ fast instance command / migration). - map the field through the <!-- This is an auto-generated description by cubic. --> <a href="https://cubic.dev/pr/twentyhq/twenty/pull/23167?utm_source=github" target="_blank" rel="noopener noreferrer" data-no-image-dialog="true"><picture><source media="(prefers-color-scheme: dark)" srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source media="(prefers-color-scheme: light)" srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img alt="Review in cubic" src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a> <!-- End of auto-generated description by cubic. --> |
||
|
Thomas Trompette
|
25f28ee299 |
Fix TWENTY-SERVER-60Y: register permissions exception filter globally (#23104)
## Context Sentry [TWENTY-SERVER-60Y](https://twenty-v7.sentry.io/issues/6633503406) ("Permission Denied: Entity performing the request does not have permission") has ~12.9k occurrences / 151 users. It groups by the shared throw site `settings-permission.guard.ts:57`, so it's a **catch-all bucket** for settings-permission denials across many resolvers, not a single operation. Sampled events include `findOneApplication` (app runtimes reading their own `applicationVariables`), `uploadFilesFieldFileByUniversalIdentifier`, `UpdatePageLayoutWithTabsAndWidgets`, `CreateFileUpload`, etc. ## Root cause Settings-permission denials are only converted to a client-appropriate `FORBIDDEN` when a resolver manually attaches `PermissionsGraphqlApiExceptionFilter`. **28** GraphQL resolvers do; **~35** guarded resolvers do not. On those, the guard-thrown `PermissionsException` (a `CustomException`, not a `BaseGraphQLError`) falls through to the Yoga error hook, is serialized as `INTERNAL_SERVER_ERROR`, and `shouldCaptureException` reports it to Sentry as a 500-class error. REST is not affected: every `SettingsPermissionGuard` controller already carries `PermissionsRestApiExceptionFilter` (15/15). ## Fix Register `PermissionsGraphqlApiExceptionFilter` globally via `APP_FILTER` in the core and metadata engine modules, mirroring the existing global GraphQL filters `BillingGraphqlApiExceptionFilter` and `FlatEntityMapsGraphqlApiExceptionFilter`. The filter is guarded on the GraphQL context (`host.getType()`), so REST keeps its existing per-controller filter untouched and no request-scoped dependency is pulled into a global provider. Every settings-guarded resolver now returns `FORBIDDEN` for denials, which is both the correct client error code and excluded from Sentry. Existing per-resolver `@UseFilters(PermissionsGraphqlApiExceptionFilter)` entries remain valid (handler-scoped takes precedence, identical result); collapsing them into the global registration is a possible follow-up. ## Test Added a case to `granular-settings-permissions.integration-spec.ts`: a member without the `APPLICATIONS` flag calling `findOneApplication{applicationVariables{key value}}` (the exact Sentry query, and a resolver with **no** resolver-scoped filter) must receive `FORBIDDEN`, exercising the global filter. Verified against the local test DB: - With the global filter: passes (`FORBIDDEN`). - Without it: fails with `INTERNAL_SERVER_ERROR`, reproducing the leak. - The existing roles / workspace-members / api-keys denial tests (per-resolver filters) still pass, confirming no precedence conflict and no boot issue from dual `APP_FILTER` registration. Fixes TWENTY-SERVER-60Y |
||
|
Abdul Rahman
|
4c4a154d31 |
key-value storage for applications (#23089)
## What
Key-value storage for applications, as proposed in
twentyhq/core-team-issues#2391 — built on the existing `keyValuePair`
entity:
- Nullable `applicationId` relation on `keyValuePair` + a new
`APPLICATION_VARIABLE` type (fast instance command included)
- GraphQL CRUD on the metadata schema (`appKeyValue`, `setAppKeyValue`,
`deleteAppKeyValue`), requiring an `APPLICATION_ACCESS` token —
`applicationId` always comes from the token, never from arguments, so
apps can't touch each other's entries
- `kv.get` / `kv.set` / `kv.delete` helpers in
`twenty-sdk/logic-function`
## Scopes
- **`INSTALL`** (default): entries are private to one workspace install;
arbitrary JSON values
- **`GLOBAL`**: entries are shared across every install of the app, with
claim semantics — the value is always the claiming `workspaceId` and
only that workspace can overwrite or delete the key (guarded writes,
race-safe via insert-if-absent)
Since `applicationId` identifies an install (one row per workspace),
GLOBAL entries are stored under the registration owner workspace's
install so all installs of the same app share one namespace.
The GLOBAL scope is what enables cross-workspace webhook routing: e.g.
the Slack app's `serverRoute` resolver (running in the owner workspace)
can resolve `kv.get('slack:team:' + team_id, { scope: 'GLOBAL' })` to
find the workspace that connected that Slack team — without a workspace
being able to hijack another's mapping.
## Follow-ups
- Wire the Slack assistant PR (#22984) to write the claim at connect
time and read it in the events resolver
- `kv.*` access from front components
<!-- This is an auto-generated description by cubic. -->
<a
href="https://cubic.dev/pr/twentyhq/twenty/pull/23089?utm_source=github"
target="_blank" rel="noopener noreferrer"
data-no-image-dialog="true"><picture><source
media="(prefers-color-scheme: dark)"
srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source
media="(prefers-color-scheme: light)"
srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img
alt="Review in cubic"
src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a>
<!-- End of auto-generated description by cubic. -->
|
||
|
github-actions[bot]
|
7fa3c5c77b |
chore: sync AI model catalog from models.dev (#23143)
Automated daily sync of `ai-providers.json` from [models.dev](https://models.dev). This PR updates pricing, context windows, and model availability based on the latest data. New models meeting inclusion criteria (tool calling, pricing data, context limits) are added automatically. Deprecated models are detected based on cost-efficiency within the same model family. **Please review before merging** — verify no critical models were incorrectly deprecated. Co-authored-by: FelixMalfait <6399865+FelixMalfait@users.noreply.github.com> |
||
|
Weiko
|
28adcbffb9 |
Remove dead code from legacy metadataVersion cache mechanism (#23122)
- Drop unused setORMEntitySchema/getORMEntitySchema from WorkspaceCacheStorageService - Drop MetadataObjectMetadataMaps cache key, only referenced by the flush loop - Drop never-populated workspaceMetadataVersion field from workspace auth context type and builders - Drop unthrown TwentyORMExceptionCode.METADATA_VERSION_MISMATCH - Drop unused WorkspaceMetadataVersionModule imports in field-metadata and object-metadata modules <!-- This is an auto-generated description by cubic. --> <a href="https://cubic.dev/pr/twentyhq/twenty/pull/23122?utm_source=github" target="_blank" rel="noopener noreferrer" data-no-image-dialog="true"><picture><source media="(prefers-color-scheme: dark)" srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source media="(prefers-color-scheme: light)" srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img alt="Review in cubic" src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a> <!-- End of auto-generated description by cubic. --> |
||
|
Paul Rastoin
|
71a1ff7ac8 |
Cache twenty-client-sdk modules host-side via content-addressed URLs (#22981)
## Context Front component sources are fetched host-side and integrity-verified by the SHA-256 checksum embedded in their URL, cached in Cache Storage — a layer that exists specifically because their download URLs are presigned and rotate. The `twenty-client-sdk` modules (`core` and `metadata`) were re-fetched on every render and could not be cached safely: their URLs carried no checksum and the server exposed no freshness signal. This PR makes the SDK module URLs **content-addressed** and relies on the **browser HTTP cache** for immutability, and it keys the checksums on their real owners: the **application** for `core`, the **instance** for `metadata`. The checksum does double duty: cache invalidation (regeneration changes the checksum → the URL changes → guaranteed cache miss) and a server-side cacheability guard (the server only grants `immutable` when the checksum in the URL matches the authoritative checksum it knows for that module — persisted at generation time for `core`, hashed once at bootstrap for `metadata` — so no per-request hashing of the served bytes). Note this is **not** an end-to-end integrity guarantee: there is no client-side hash verification, and on a fingerprint mismatch the server still serves the current bytes with `no-store` (self-healing for stale URLs) rather than failing. <img width="2412" height="926" alt="image" src="https://github.com/user-attachments/assets/d97935d2-0fdb-4c44-89ac-596b7ca8ca64" /> Closes twentyhq/core-team-issues#2688. ## Routes | Module | URL | Scope | | --- | --- | --- | | `core` | `/rest/sdk-client/{applicationId}/core[/{checksum}]` | Per application (generated bundle) | | `metadata` | `/rest/sdk-client/metadata[/{checksum}]` | **Instance-wide**: no application segment, so every application converges on one URL and the browser downloads the module once per release instead of once per application | The previous application-scoped metadata path (`/rest/sdk-client/{applicationId}/metadata[/{checksum}]`) is **kept for backward compatibility**, new clients just stop generating those URLs. The instance-wide route is declared before the parameterized route so `metadata/{checksum}` is not swallowed as `:applicationId/:moduleName`. ## Caching model | Request | `Cache-Control` | Effect | | --- | --- | --- | | Fingerprinted URL, checksum matches the known module checksum | `immutable` | Cached indefinitely by the browser HTTP cache; a new checksum is a new URL | | Fingerprinted URL, checksum does not match | `no-store` | Current bytes served uncached (self-healing for stale URLs) | | Bare URL (pre-generation fallback, `core` only in practice) | `no-store` | Never cached | - Both responses also set `X-Content-Type-Options: nosniff` and `Content-Type: application/javascript`. - SDK modules are intentionally **not** placed in Cache Storage. That layer stays reserved for the presigned/rotating component-source URLs; SDK modules are served directly and authenticated, so the browser HTTP cache (keyed by the content-addressed URL) is their single cache layer. ## Checksum provenance - **core** — per **application**, persisted on `application.sdkClientCoreChecksum` at generation time and read back from `flatApplicationMaps` (never re-hashed per request). - **metadata** — **instance-wide**, hashed once from the installed `twenty-client-sdk/dist/metadata.mjs` package (warmed at bootstrap, memoized per process) and served straight from that package, so it is fresh from the first request after a release with no archive dependency. ## Server (twenty-server) - Hash `dist/core.mjs` at SDK generation and persist `sdkClientCoreChecksum` via `applicationRepository.update`. Adds the nullable text column to `application.entity.ts` (mirroring `packageJsonChecksum`) plus a fast instance command with up/down; `FlatApplication` picks it up automatically. - New **application-scoped** query `applicationSdkClientChecksums(applicationId: UUID!): SdkClientChecksums` on `ApplicationResolver` (metadata schema, `WorkspaceAuthGuard` + `NoPermissionGuard`). `SdkClientChecksums.core` is **nullable** and stays `null` until the SDK has been generated at least once; `metadata` is **always present** (bootstrap-warmed), so the metadata module is cacheable from the very first render of any app. The query itself returns `null` only for unknown applications. - `SdkClientChecksumsDTO` now lives in the shared `core-modules/sdk-client/dtos/`. `FrontComponentDTO` and the `frontComponent` resolver no longer carry checksums (decoupled from the front-component row). - `sdk-client` controller: instance-wide `metadata[/:checksum]` route (no workspace-cache or application lookup, serves the memoized installed module) + application-scoped `:applicationId/:moduleName[/:checksum]` route (serves `core` from the per-application archive, `metadata` kept for back-compat). Cacheability compares the URL checksum against the **known** checksum — persisted `sdkClientCoreChecksum` for `core`, memoized package hash for `metadata` — instead of hashing the served bytes on every request: `immutable` on match, `no-store` otherwise (bare URL or stale fingerprint), plus `nosniff`. A persisted checksum out of sync with the archive only downgrades to `no-store` until the next regeneration. ## Front (twenty-front) - New metadata query `GetApplicationSdkClientChecksums`, keyed by `applicationId`; removed the `sdkClientChecksums` selection from `FindOneFrontComponent`. - `getSdkClientUrls` builds the two module URLs independently: `/sdk-client/{applicationId}/core/{checksum}` and the **instance-wide** `/sdk-client/metadata/{checksum}` (no application segment → one shared browser cache entry per release across all applications). Each falls back to its bare URL when its checksum is absent — since `core` is nullable, a never-generated app still gets a content-addressed metadata URL and only `core` falls back. The checksum type is sourced from the codegen `SdkClientChecksums` type rather than a hand-maintained duplicate. - `FrontComponentRenderer` is split into a gating outer component (runs `FindOneFrontComponent`, renders nothing while loading) and a content component that receives a guaranteed-non-null `frontComponent`. Following project conventions, the side effects live in dedicated effect components: `FrontComponentLoadErrorSnackBarEffect` (query error → snackbar) and `FrontComponentApplicationTokenPairEffect` (mirrors the query-derived token pair into component state unconditionally, `null` included, so revoked credentials can never be retained or refreshed). The content component fetches checksums via the application-keyed query and **gates the mount of SDK-using components on that query**, so the very first module fetch is always the content-addressed (`immutable`) URL instead of the bare `no-store` one. Non-SDK components skip the query and are never blocked. - **Live invalidation without reload:** SDK regeneration updates the application row, and the server broadcasts an `application` metadata event carrying the new core checksum. `useOnApplicationSdkClientChecksumsUpdated` / `useUpdateSdkClientChecksumsApolloCache` patch the application-keyed checksum query cache (core only; the instance-wide metadata is preserved), so every mounted component of that application picks up the new URL at once. This replaces the previous frontComponent-derived field and closes the earlier "known gap" (a mounted component staying on a session-old checksum until a full reload). The cache-patching callback is memoized (`useCallback`) so the window listener is registered once per application, and the listener is **skipped entirely** for non-SDK components (`useListenToMetadataOperationBrowserEvent` gained a `skip` option) — they register no listener and never refetch a query they don't consume. ## Renderer (twenty-front-component-renderer) - SDK sources are fetched through a dedicated plain authenticated fetch, `fetchJavaScriptModuleSourceText` (Bearer header, `credentials: 'omit'`), instead of the Cache Storage `fetchComponentSource` path; `fetchSdkClientSources` uses it. Execution stays exclusively in the opaque-origin worker via blob URLs; the host only fetches and forwards source strings (no hashing host-side). Staleness self-resolves through the checksum: new checksum → new URL → cache miss. <!-- This is an auto-generated description by cubic. --> <a href="https://cubic.dev/pr/twentyhq/twenty/pull/22981?utm_source=github" target="_blank" rel="noopener noreferrer" data-no-image-dialog="true"><picture><source media="(prefers-color-scheme: dark)" srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source media="(prefers-color-scheme: light)" srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img alt="Review in cubic" src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a> <!-- End of auto-generated description by cubic. --> |
||
|
Etienne
|
8f704e87e3 |
fix(ai-chat) - fix AI record references when display names contain markdown characters (#23113)
<img width="516" height="86" alt="Screenshot 2026-07-21 at 16 41 16" src="https://github.com/user-attachments/assets/6e14b933-b48e-49ab-856b-400efdeba53a" /> ## Summary - Switch record references from `[[record:object:id:label]]` to `[[record:object:id:label[[/record]]` so labels can include `]`, backticks, brackets, and other markdown-significant characters - Parse references with an explicit close tag (still accepting legacy `]]`), escape labels before markdown lexing, and serialize mentions through a shared formatter - Update the AI chat system prompt so the model emits the new format ## Test plan - [ ] Ask AI about a record whose name contains `` ` ``, `[`, `]`, or `]]` and confirm it renders as a chip, not broken markdown - [ ] Confirm legacy `[[record:...]]` references still chip correctly - [ ] Mention a record in the chat editor and verify serialized text uses `[[/record]]` - [ ] Run: - `npx jest src/modules/ai/utils/__tests__/findRecordReferences.test.ts src/modules/ai/utils/__tests__/formatRecordReference.test.ts src/modules/ai/utils/__tests__/protectRecordReferencesForMarkdown.test.ts src/modules/ai/components/__tests__/TextWithRecordLinks.test.tsx --config=packages/twenty-front/jest.config.mjs` - mention extension tests for `MentionTag` / `MentionSuggestion` <!-- This is an auto-generated description by cubic. --> <a href="https://cubic.dev/pr/twentyhq/twenty/pull/23113?utm_source=github" target="_blank" rel="noopener noreferrer" data-no-image-dialog="true"><picture><source media="(prefers-color-scheme: dark)" srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source media="(prefers-color-scheme: light)" srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img alt="Review in cubic" src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a> <!-- End of auto-generated description by cubic. --> |
