1362 Commits

Author	SHA1	Message	Date
Paul Rastoin	60fd322b49	Centralize system field side effects + search field metadata (#22594) ... ## Introduction Closes twentyhq/core-team-issues#2635 and twentyhq/core-team-issues#2642 and twentyhq/core-team-issues#2589 Object system fields (`searchVector` + its GIN index + `searchFieldMetadata`, the reserved system fields, default relations) were provisioned through several scattered, path-specific code paths. As a result the **app-manifest sync path** authored objects with an empty/`NULL` `searchVector` and **zero `searchFieldMetadata`**, so app-owned objects shipped a broken generated search column (see #22657). The generation logic also lived partly in imperative services rather than in the metadata side-effect engine, and relied on non-deterministic (`v4`) universal identifiers that `twenty apply` could not converge, destroying manually backfilled rows. This PR centralizes every object-creation system side effect into the **metadata side-effect engine**, extends the engine to keep search metadata consistent on field delete and object relabel, makes the standard app's search identifiers deterministic, and ships upgrade commands to reconcile existing workspaces. ## What changed ### Side effects moved into the metadata side-effect engine New dedicated, self-contained handlers — so every write path (API and app manifest) gets identical results, and side effects never trigger other side effects. **Object create / delete** (`handlers/object-metadata`) * **`objectSystemFieldsOnCreate`** — generates the 7 reserved system fields (`id`, `createdAt`, `updatedAt`, `deletedAt`, `createdBy`, `updatedBy`, `position`). * **`objectSearchVectorOnCreate`** — provisions the full-text search surface as one unit: the `searchVector` `TS_VECTOR` field, its backing GIN index, and the `searchFieldMetadata` row (for searchable objects whose label identifier is a searchable field) that keeps `searchVector` populated instead of `NULL`. * **`objectSystemSideEffectsOnDelete`** — tears the above down on object deletion. **Search-metadata consistency on relabel / field delete** (new — these are what close the manifest-path gaps) * **`objectSearchVectorOnUpdate`** (`handlers/object-metadata`) — when a searchable object is relabeled onto a new searchable field, provisions the `searchFieldMetadata` row that indexes it. Relabeling is **additive**: existing rows (e.g. the provisioned `name` row) are preserved, so the previous label identifier stays searchable. Mirrors the API update path so a manifest re-sync that changes the label identifier reaches search parity. No-ops for junction objects (`id` label identifier) and non-searchable field types. * **`fieldSearchFieldMetadataOnDelete`** (`handlers/field-metadata`) — when a field is deleted, cascade-deletes every `searchFieldMetadata` row that indexes it. `searchFieldMetadata` is excluded from manifest deletion inference, so this explicit cascade is what covers **both the API and manifest paths** (the object-scoped DB cascade only fires on object deletion). Uses the `searchFieldMetadataUniversalIdentifiers` aggregator on the flat field for an O(k) lookup instead of scanning all rows. The **default `name` field and default relations are now caller-provided default fields** (SDK autocomplete on the manifest path, input transpiler on the API path) rather than system side effects — removing duplicate name generation, the imperative `build-default-*-for-custom-object` utilities, and the ad-hoc system-field integrity validator. ### Deterministic identifiers for the standard app The twenty-standard search GIN index and `searchFieldMetadata` now derive deterministic universal identifiers (`getIndexUniversalIdentifier` / `getSearchFieldUniversalIdentifier`) instead of `v4`, so `twenty apply` converges instead of recreating. ### Upgrade commands (`2-20`) to reconcile existing workspaces **Instance commands** (run once per instance; ordered fast → slow → workspace): 1. **`AddIsSystemSideEffectToSearchFieldMetadata`** (fast) — adds the `isSystemSideEffect` column to `core.searchFieldMetadata`. Defaults to `true`, which also correctly backfills every existing row since `searchFieldMetadata` is always system-derived (never user-authored). 2. **`BackfillNameFieldIsSystemSideEffect`** (slow) — re-flags existing `name` fields from `isSystemSideEffect: true` → `false`, since the default `name` field is now a caller-provided default like any other user-owned field (it was provisioned as `true` in 2.15 → 2.19). This is a pure data backfill, so the bulk `UPDATE` lives in `runDataMigration()` rather than `up()` — keeping it out of the fast schema transaction avoids holding an `ACCESS EXCLUSIVE` lock that could stall reads during the deploy. Slow instance commands still run before every workspace command of the version, so the fresh value is in place before the search-reconcile workspace commands recompute the `fieldMetadata` flat-entity cache. Scoping by name alone is safe (no engine-owned field is named `name`); `down()` is best-effort (pre-2.15 `false` rows are indistinguishable from flipped ones). **Workspace commands** (idempotent, dry-run supported): 1. **`reconcile-search-vector-gin-index-universal-identifier`** — re-owns every searchVector GIN index UID to its deterministic value (all applications), then backfills the missing GIN index for installed-app objects. 2. **`reconcile-search-field-metadata`** — re-owns every `searchFieldMetadata` UID (all applications), then backfills the missing rows for installed-app searchable objects. 3. **`rebuild-installed-app-search-vectors`** — rebuilds the `searchVector` column of every installed-app `TS_VECTOR` field, once the index and rows exist. Design notes: * **Re-own is global** (twenty-standard, workspace-custom, installed) — a UID convergence keyed on each row's own application. * **Backfill is installed-app only** — standard/custom objects already have these rows via the manifest funnel. * Re-own runs **before** backfill and is transaction-guarded; a failure aborts that workspace to avoid a unique-identifier collision. ## Tests * Integration: app manifest sync now asserts system fields + searchable objects (searchVector, GIN index, searchFieldMetadata) are created; a new relabel suite drives three manifest syncs and asserts records stay searchable through the old + new label identifiers and lose searchability when a field is removed; removed the obsolete system-fields-integrity suite/snapshots. * Unit: per-handler side-effect specs (including the new `objectSearchVectorOnUpdate` and `fieldSearchFieldMetadataOnDelete` handlers), and per-util specs for the re-own / backfill operation builders and the GIN-index classifier. ## Upgrade / migration notes * Existing workspaces converge on the next upgrade run via the `2-20` instance + workspace commands (idempotent, dry-run supported). * Backfill and rebuild go through the workspace-migration runner (automatic cache invalidation); the re-own step invalidates only the affected flat-entity maps directly. * The cross-version upgrade CI now flushes the cache before running the upgrade, so the new version recomputes every flat-entity map from the database instead of reading blobs the old version serialized in an older shape. ## Follow-up * `object-metadata.service.ts` still carries a `TODO: remove once default view fields move to the metadata side effect engine` — default view fields are the next candidate to move into the engine. * A single manifest sync cannot yet both create a field and relabel the object onto it, because `objectMetadata.update` is ordered before `fieldMetadata.create` in the migration runner. Tracked in twentyhq/core-team-issues#2655; to be fixed in a follow-up.	2026-07-09 16:59:54 +02:00
Etienne	a51c37dae5	feat(ai) - add light AI chat turn instrumentation (metrics + Sentry correlation) (#22692) ... ## Summary Adds minimal server-side observability for AI chat turns: lifecycle counters to measure success/failure rates, Sentry scope tags to correlate API and worker traces, and per-LLM-call telemetry metadata for turn/stream correlation. - Add turn lifecycle metrics: `ai-chat/turn-started`, `ai-chat/turn-completed`, `ai-chat/turn-failed` (with `failure_phase` and `error_code` attributes) - Emit counters at key points: job start, clean completion, execution failures, enqueue failures, interrupted streams, and empty completions - Tag Sentry scope with `streamId`, `turnId`, `threadId`, and `workspaceId` at API entry points (`sendChatMessage`, `retryChatMessage`, `answerAgentChatQuestion`) and worker entry (`StreamAgentChatJob`) - Enrich LLM `experimental_telemetry` metadata with stream/turn/thread/workspace IDs - Return `turnId` from streaming service methods so resolvers can tag the scope - Remove granular tool-learned/skill-loaded metrics in favor of the turn-level counters ## Test plan - [ ] Send a chat message and verify `ai-chat/turn-started` and `ai-chat/turn-completed` increment - [ ] Trigger a stream failure (e.g. interrupted/dead stream) and verify `ai-chat/turn-failed` with correct `failure_phase` - [ ] Retry a failed turn and confirm a new `turn-started` is emitted for the retry attempt - [ ] Answer an `ask_questions` prompt and confirm Sentry tags include `streamId` and `turnId` - [ ] Check Sentry spans for LLM calls include `streamId`, `turnId`, `threadId`, `workspaceId` in telemetry metadata - [ ] Run unit tests: - `npx jest packages/twenty-server/src/engine/metadata-modules/ai/ai-chat/jobs/__tests__/stream-agent-chat.job.spec.ts` - `npx jest packages/twenty-server/src/engine/metadata-modules/ai/ai-chat/services/__tests__/agent-chat-streaming.service.claim.spec.ts` - `npx jest packages/twenty-server/src/engine/metadata-modules/ai/ai-chat/services/__tests__/agent-chat-streaming.service.retry.spec.ts` <!-- This is an auto-generated description by cubic. --> <a href="https://cubic.dev/pr/twentyhq/twenty/pull/22692?utm_source=github" target="_blank" rel="noopener noreferrer" data-no-image-dialog="true"><picture><source media="(prefers-color-scheme: dark)" srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source media="(prefers-color-scheme: light)" srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img alt="Review in cubic" src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a> <!-- End of auto-generated description by cubic. -->	2026-07-09 08:24:27 +00:00
github-actions[bot]	1129cc7ae0	chore: sync AI model catalog from models.dev (#22710) ... Automated daily sync of `ai-providers.json` from [models.dev](https://models.dev). This PR updates pricing, context windows, and model availability based on the latest data. New models meeting inclusion criteria (tool calling, pricing data, context limits) are added automatically. Deprecated models are detected based on cost-efficiency within the same model family. **Please review before merging** — verify no critical models were incorrectly deprecated. <!-- This is an auto-generated description by cubic. --> <a href="https://cubic.dev/pr/twentyhq/twenty/pull/22710?utm_source=github" target="_blank" rel="noopener noreferrer" data-no-image-dialog="true"><picture><source media="(prefers-color-scheme: dark)" srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source media="(prefers-color-scheme: light)" srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img alt="Review in cubic" src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a> <!-- End of auto-generated description by cubic. --> Co-authored-by: FelixMalfait <6399865+FelixMalfait@users.noreply.github.com>	2026-07-09 09:08:22 +02:00
Etienne	cc7b41db0e	feat(ai-chat): inject current date & per-message timestamps into agent context (#22632) ... ## Summary Gives the AI chat agent temporal awareness by injecting the current date into the system prompt and a per-message "sent at" timestamp into each user message, formatted in the member's timezone. Also hardens all timezone formatting against the `"system"` sentinel value, which was crashing the stream job. ## What changed **Message timestamps (new)** - Added `injectMessageTimestamps` util: prepends a `<message_timestamp>Sent: …</message_timestamp>` text part to each user message before it's sent to the model, so the agent can reason about "yesterday", "last week", etc. - `loadMessagesFromDB` now stores the message time in the canonical `metadata.createdAt` slot (ISO string, JSON-serializable for the BullMQ job payload) instead of a non-typed top-level `createdAt` field that nothing read. - Migrated the AI chat message pipeline from the generic `UIMessage` to the typed `ExtendedUIMessage` (`chat-execution.service`, `extract-code-interpreter-files`, `replace-unsupported-file-parts`, and related types), since `metadata.createdAt` is declared on `ExtendedUIMessage`. **Current date in context** - System prompt now includes `Current date: …` formatted in the member's timezone (`system-prompt-builder.service`). - Settings › AI prompt preview mirrors the same `Current date` line. **Timezone safety (bug fix)** - Workspace members default `timeZone` to the `"system"` sentinel, which is only resolvable client-side. Passing it (or any invalid IANA zone) to `Intl.DateTimeFormat` throws `RangeError: Invalid time zone specified: system`, which was failing the stream job. - Added `getValidTimeZoneOrUndefined`, which returns a valid IANA zone or `undefined` (letting the runtime fall back to its default). Used in both `injectMessageTimestamps` and `formatCurrentDate`. This mirrors the existing `isValidTimeZone` convention in the calendar module. ## Notes / follow-ups - For members who never changed `timeZone` from `"system"`, timestamps fall back to the server's default zone (UTC). To honor their real local time, the frontend would need to send the browser-detected zone with the chat request (the same way calendar/charts already pass a resolved zone). Not included here. ## Test plan - [x] `inject-message-timestamps.util.spec.ts` — covers timestamp injection, assistant messages untouched, invalid `createdAt`, and the `"system"` timezone no longer throwing. - [ ] Send a chat message and confirm the agent sees the correct date/time. - [ ] Verify a member with `timeZone = "system"` no longer crashes the stream job. <!-- This is an auto-generated description by cubic. --> <a href="https://cubic.dev/pr/twentyhq/twenty/pull/22632?utm_source=github" target="_blank" rel="noopener noreferrer" data-no-image-dialog="true"><picture><source media="(prefers-color-scheme: dark)" srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source media="(prefers-color-scheme: light)" srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img alt="Review in cubic" src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a> <!-- End of auto-generated description by cubic. -->	2026-07-09 07:02:56 +00:00
Pratik Mahajan	72d728ea8e	fix(messaging): add sender name to IMAP/SMTP From headers (#22603) ... Manual IMAP/SMTP outbound emails were being composed with a bare email address in the From header, so recipients did not see the sender's display name. Gmail already built a proper sender header from Google profile data, but manually configured IMAP/SMTP accounts had no equivalent path and fell back to the raw address only. Fix this by storing the optional sender display name in the IMAP/SMTP/CALDAV connection parameters, exposing it through the settings flow and metadata API, and reusing a shared From-header formatter when composing outbound messages. The formatter now builds a properly encoded sender header when a name is available and falls back to the bare email address when it is not, keeping the behavior safe for blank or missing names. Gmail keeps using its existing Google-derived display name source; this change only brings manual accounts up to the same header formatting standard and removes duplicated formatting logic between outbound drivers. After this change, manual SMTP sends and IMAP draft creation include the configured sender name in the From header, while blank names are normalized away instead of producing malformed headers. Existing manual account names are preserved when updates omit the field, and edited accounts can still explicitly clear it through the settings flow. Add focused utility coverage for the shared From-header formatter. Fixes: #22608 <!-- This is an auto-generated description by cubic. --> <a href="https://cubic.dev/pr/twentyhq/twenty/pull/22603?utm_source=github" target="_blank" rel="noopener noreferrer" data-no-image-dialog="true"><picture><source media="(prefers-color-scheme: dark)" srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source media="(prefers-color-scheme: light)" srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img alt="Review in cubic" src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a> <!-- End of auto-generated description by cubic. --> --------- Co-authored-by: neo773 <neo773@protonmail.com>	2026-07-08 20:29:25 +05:30
martmull	b733a79821	feat(server): support server-scoped files via nullable workspaceId on file table (#22587) ... Part of the app settings architecture cleanup (twentyhq/core-team-issues#2456) — PR 1 of the server-level documents plan, reworked after the revert of #22560 (#22579). Same capability, different shape: **no new entity** — server-level documents live in the existing `file` table with a nullable `workspaceId`. ## Problem All file storage is workspace-scoped (`FileEntity.workspaceId NOT NULL`, `{workspaceId}/{app}/…` storage keys). Server-level data like application-registration manifests and tarballs for ownerless catalog registrations has no first-class home, forcing raw-driver bypasses (`DefaultAiCatalogService`, prototype #22556). ## Changes (core storage layer only — no HTTP serving, no GraphQL exposure) **`FileEntity` gains server scope** (mirrors `KeyValuePairEntity`, which already supports both instance-level and per-workspace rows): - `workspaceId` uuid becomes **nullable** — NULL means server-scoped; the entity no longer extends `WorkspaceRelatedEntity` and declares its columns directly - `applicationRegistrationId` nullable FK (`onDelete: CASCADE`) — registration-owned documents follow their registration - ownership checks: `workspaceId IS NOT NULL OR applicationRegistrationId IS NOT NULL` and `workspaceId IS NULL OR applicationRegistrationId IS NULL` — every row has exactly one owner - `IDX_FILE_APPLICATION_REGISTRATION_ID_PATH_UNIQUE` UNIQUE (`applicationRegistrationId`, `path`) — mirrors the workspace unique-constraint pattern; workspace rows are exempt via their NULL `applicationRegistrationId` **New `ServerFileStorageService`** (`file-storage/services/`, exported from the global `FileStorageModule`; `FileStorageService` moved alongside it): - storage keys `server/{fileFolder}/{applicationRegistrationId}/{resourcePath}` — the registration segment is injected by the service itself, so paths cannot collide across registrations; scope-validation util mirroring `validateStoragePathIsWithinWorkspaceOrThrow`; new `ServerFileFolder` enum in twenty-shared - `writeServerFile` (upsert on (`applicationRegistrationId`, `path`) + driver write; throws on failure), `readServerFile`/`readServerFileById` (missing row or bytes surfaces `FILE_NOT_FOUND`), `checkServerFileExists`, `deleteServerFile`/`deleteByServerFileId` (bytes best-effort, row authoritative), `deleteByApplicationRegistrationId` - rows are accessed through a plain repository pinned to `workspaceId: IsNull()` on every query; workspace-file code paths still go through `WorkspaceScopedRepository`, which never sees NULL rows **Null-safety ripples** (workspaceId is now `string | null`): - `WorkspaceScopedEntity` bound widened to `workspaceId: string | null` (the wrapper always filters with a concrete id) - `list-and-delete-orphaned-workspace-entities` now skips `workspaceId IS NULL` rows — previously `NOT EXISTS` would have flagged server rows as orphans and deleted them - `PendingFileCleanupService` sweeps only `workspaceId IS NOT NULL` rows; `application-package-fetcher` pins its tarball lookup to workspace rows (tarball migration to server scope is a follow-up PR) **Migration**: `allow-server-scoped-file` ships as a **2-20 fast instance command** (2.20.0 is current since #22639; re-slotted from 2-19 per review). Command runs are tracked by name, so instances that already executed the 2-20 `standardOverrides` drop command still pick this one up. Its realistic timestamp sorts before that drop command's fabricated `1825000000000`, which the `ci:allow-upgrade-command-timestamp-exception` label covers. ## Next PRs in the plan - PR 2: HTTP serving + token type for server files - PR 3: application-registration manifests stored as versioned server files (rework of draft #22556) - PR 4 (optional): registration tarballs migrate to server scope ## Verification - New spec `server-file-storage.service.spec.ts` (traversal table, upsert conflict semantics, row-before-bytes reads, best-effort byte deletion, registration cascade) + scope-validation util spec; affected suites all green - Typecheck (server + shared), `lint:diff-with-main`, full `oxfmt --check src/` on both packages clean - Fresh `database:reset` on the re-slotted branch: the 2-20 command executes, generator then reports **no schema drift**; both ownership checks and the composite unique verified live (dual-owner insert and duplicate registration+path both rejected)	2026-07-08 11:56:24 +02:00
Paul Rastoin	07b9d855b7	2.20 fieldMetadata and objectMetadata standardOverrides deprecation (#22650) ... <!-- This is an auto-generated description by cubic. --> <a href="https://cubic.dev/pr/twentyhq/twenty/pull/22650?utm_source=github" target="_blank" rel="noopener noreferrer" data-no-image-dialog="true"><picture><source media="(prefers-color-scheme: dark)" srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source media="(prefers-color-scheme: light)" srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img alt="Review in cubic" src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a> <!-- End of auto-generated description by cubic. -->	2026-07-08 10:33:03 +02:00
Paul Rastoin	5dc9d7ab36	fix(server): make all view children reparentable across a workspace migration sync (#22600) ... ## Summary Uniformizes the workspace migration engine so **every** view child entity — `viewField`, `viewFieldGroup`, `viewGroup`, `viewFilter`, `viewSort`, `viewFilterGroup` — can be reparented from one view to another within a single manifest sync, including when the previous parent view is deleted in the same sync. ### Context When an app manifest deletes a view and reparents its children onto another view in the same sync (e.g. replacing a custom `FIELDS_WIDGET` view with a standard one), the sync failed with a builder validation error `View field to update parent view not found`. Root causes: 1. `viewField`, `viewFieldGroup` and `viewGroup` had `viewId.toCompare: false`, so the diff never detected the parent-view change and never emitted a reparent update (the already-reparentable siblings `viewFilter`/`viewSort`/`viewFilterGroup` had `toCompare: true`). 2. `validateFlatViewFieldGroupUpdate` resolved the *old* parent view (it ignored the update patch), inconsistent with the other view-child validators. 3. Once the builder no longer errors, the runner would fail silently: `view.delete` ran **before** the child reparent updates, and `viewId` is `onDelete: CASCADE`, so the old view's deletion cascade-deleted the children before they could be reparented (silent data loss, since `repository.update` on a missing row is a no-op). ### Changes - **`all-entity-properties-configuration-by-metadata-name.constant.ts`**: set `viewId.toCompare: true` for `viewField`, `viewFieldGroup`, `viewGroup`. Because `viewId` maps to `universalProperty: 'viewUniversalIdentifier'`, the diff compares **only** `viewUniversalIdentifier` (never the raw FK). Snapshot updated accordingly. - **`flat-view-field-group-validator.service.ts`**: merge `flatEntityUpdate` and resolve the **new** parent view, matching the `viewField`/`viewGroup`/`viewSort` validators. - **`compute-ordered-migration-actions.util.ts`**: move `view.delete` to run **after** all view-child create/update actions so a child can be reparented off a view that is being deleted in the same sync. Child `delete → create → update` order is preserved (needed for `viewField`'s partial-unique `(fieldMetadataId, viewId)`). - **New integration test** `successful-manifest-reparent-view-children.integration-spec.ts` covering reparenting of every view child (a) between two persisting views and (b) when the source view is deleted in the same sync. ## Test plan - [x] `nx typecheck twenty-server` - [x] oxlint + oxfmt on changed files - [x] Unit snapshot regenerated: `all-universal-flat-entity-properties-to-compare-and-stringify.constant.spec` - [x] New integration test passes (both scenarios) - [x] Verified the delete-source scenario **fails** on the old action ordering (children cascade-deleted, `Received length: 0`) and **passes** after the reorder — confirming it's a genuine regression guard Made with [Cursor](https://cursor.com) <!-- This is an auto-generated description by cubic. --> <a href="https://cubic.dev/pr/twentyhq/twenty/pull/22600?utm_source=github" target="_blank" rel="noopener noreferrer" data-no-image-dialog="true"><picture><source media="(prefers-color-scheme: dark)" srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source media="(prefers-color-scheme: light)" srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img alt="Review in cubic" src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a> <!-- End of auto-generated description by cubic. -->	2026-07-07 10:52:34 +02:00
Marie	c8f7315c67	fix(server): converge app-sync isUnique diff for single-column unique constraints (#22592) ... ## Context Fixes #22550. On app sync (`twenty dev`), a field backed by a single-column unique constraint produces a **permanent, non-converging `[isUnique] changed` field-metadata diff**: the change is reported, the apply succeeds, and the identical change is reported again on the very next sync. ## Root cause Since the uniqueness-source-of-truth moved from a `FieldMetadata.isUnique` column to `IndexMetadata` (@FelixMalfait's #20846 / #20883), `field.isUnique` is a **derived** property. The two sides of the app-sync diff derive it differently: - **"from" side** (workspace cache, `WorkspaceFlatFieldMetadataMapCacheService`) derives `isUnique` from indexes via `computeUniqueFieldMetadataIdsFromIndexes`, which counted **any** single-column `UNIQUE` index. - **"to" side** (`from-field-manifest-to-universal-flat-field-metadata.util.ts`) sets `isUnique` from the field-level manifest flag (`fieldManifest.isUnique ?? false`). For a field whose uniqueness is declared with `defineIndex({ isUnique: true, fields: [oneField] })` (no field-level flag): - "from" derives `true` (the custom unique index exists), - "to" is `false` (no field-level flag), so the diff emits a `fieldMetadata … [isUnique] changed` update forever. The field-update runner drops `isUnique` before the SQL `UPDATE` (it has no column), and the custom index persists, so the derived value never changes — the loop cannot converge. ## Fix Restrict the derivation in `computeUniqueFieldMetadataIdsFromIndexes` to the field's **engine-owned backing constraint** — a `UNIQUE` index with `isSystemSideEffect: true` — rather than any single-column unique index. This makes `field.isUnique` mean the same thing on both sides: | declaration | backing index (`isSystemSideEffect`) | "from" derived | "to" flag | converges | |---|---|---|---|---| | field-level `isUnique: true` | side-effect handler generates it → `true` | `true` | `true` | ✅ | | `defineIndex({ isUnique: true, fields:[x] })` | custom index → `false` | `false` | `false` | ✅ (index converges on its own) | Standard objects and the create/update side-effect backing indexes are all `isSystemSideEffect: true` (`create-standard-index-flat-metadata.util.ts`, `generate-deterministic-index-for-flat-field-metadata-or-throw.util.ts`), so their fields keep `isUnique = true`. Only a user-declared custom `defineIndex` unique index (`isSystemSideEffect: false`) is now excluded — which is also what stops the create/update side-effect from generating a **second, duplicate** backing index for a field the custom index already covers (which would otherwise trip `DUPLICATE_UNIQUE_INDEX` on first apply). ## Why this location (and not the manifest "to" side) An earlier attempt derived `isUnique` on the manifest "to" side from the built indexes. That breaks after #22295: `field.isUnique === true` is the **trigger** for the `fieldUniqueBackingIndexOnCreate/Update` side-effect handlers, so forcing it to derive from the compute-service maps (which don't yet contain the not-yet-generated backing index) would suppress the backing index for field-level unique fields. Narrowing the shared "from" derivation keeps the side-effect trigger intact and makes both sides symmetric in one place. ## For review — @FelixMalfait This touches the uniqueness model you own in #20883 ("make IndexMetadata the source of truth for uniqueness"), and interacts with the side-effect engine from #22295. The semantic change is: **`field.isUnique` now reflects only the field's backing constraint, not an arbitrary user-declared single-column unique index.** A `defineIndex`-declared single-column unique field now surfaces `isUnique: false` on the field (the constraint is still enforced by the index). If instead you'd want `defineIndex` single-column uniqueness to surface as `field.isUnique: true`, the fix would need to live in the side-effect engine (dedupe the backing index against the declared one) rather than the derivation — happy to take it that direction. Flagging for your call before this leaves draft. ## Related - Issue #22550 - @FelixMalfait #20883, #20846 (IndexMetadata as source of truth for uniqueness) - #22295 (centralized side-effect engine — unique field backing index) - #21383 (adjacent field-`isUnique` handling) https://claude.ai/code/session_01T1Cqvt5tHS6tZ1FeQQWyRo	2026-07-07 09:34:47 +02:00
Paul Rastoin	6c40c7b91a	Deterministic system field universal identifier (#22565) ... # Introduction Close twentyhq/core-team-issues#2641 Auto-provisioned field metadata used to get its `universalIdentifier` from three unrelated sources: random `v4()` on the server when creating custom objects, hardcoded values in `STANDARD_OBJECTS`, and an ad-hoc `v5` derivation in the SDK manifest build. This PR unifies all of them behind the shared `getFieldUniversalIdentifier` derivation: ``` universalIdentifier = f(applicationUniversalIdentifier, objectUniversalIdentifier, fieldName) ``` ## Ownership model The rollout is built on an explicit split of who owns a field's universal identifier: - **The 8 system fields** (`id`, `createdAt`, `updatedAt`, `deletedAt`, `createdBy`, `updatedBy`, `position`, `searchVector`) are **server-owned**. Their universal identifiers are always the deterministic derivation, on **every** application (standard, workspace-custom, installed). Clients cannot provide custom values: a temporary check in `validateObjectMetadataSystemFieldsIntegrity` rejects any non-derived system field identifier at migration build time. This check stands in until system fields are generated exclusively server side by the metadata side-effect engine and stripped from client inputs — at which point it becomes structurally impossible to send one. - **`name` is a default field, not a system field**: it is auto-provisioned when absent (server side for custom objects, SDK side for application objects) but authors can define their own. It is only derived where it is guaranteed to be auto-provisioned. In particular, standard objects keep their **historical hardcoded** `name` identifiers: the standard app authors its `name` fields like any installed app would, and moving those identifiers would break every installed application referencing them (e.g. views on `opportunity.name`). - **User-created and author-provided fields** keep random / explicit identifiers, untouched. ## Server - `validateObjectMetadataSystemFieldsIntegrity` now validates, on top of the existing type/`isSystem` checks, that each system field's `universalIdentifier` equals the deterministic derivation. Runs for every object creation going through the migration orchestrator: app sync, custom object creation, standard provisioning - `build-default-flat-field-metadatas-for-custom-object.util.ts` derives the system field identifiers (and the auto-provisioned `name`) with `getFieldUniversalIdentifier` instead of `v4()` - `build-default-relation-flat-field-metadatas-for-custom-object.util.ts` derives both the forward and the reverse default relation field identifiers deterministically - `generateMorphOrRelationFlatFieldMetadataPair` accepts optional `sourceFieldUniversalIdentifier` / `targetFieldUniversalIdentifier` so callers can inject deterministic values; user-created relations still default to `v4()` ## twenty-shared - `STANDARD_OBJECTS` system field identifiers (the 8) are now computed at module load via `buildStandardObjectSystemFields`; `name` and every other identifier keep their hardcoded values - New snapshot test pinning **every** universal identifier of `STANDARD_OBJECTS`: any identifier change now requires an explicit snapshot update and should ship with a coordinated backfill ## SDK (breaking, pre-GA) - `generateDefaultFieldUniversalIdentifier` delegates to `getFieldUniversalIdentifier` and now requires `applicationUniversalIdentifier` - Reverse default relation field identifiers are derived from the field's real coordinates (standard object UID + actual field name, e.g. `targetRocket` on `attachment`) instead of the legacy custom-object UID + synthetic `${fieldName}Inverse` hash input. Field *names* are unchanged - The manifest build threads the application universal identifier through default field injection (two-pass over object configs) - `twenty dev:add` now resolves the application universal identifier upfront and refuses to scaffold anything until `defineApplication` declares one — no more `fill-later` placeholder for the app UID in generated files ## Upgrade A 2.19 **workspace command** backfills existing `fieldMetadata.universalIdentifier` rows to the deterministic derivation. Coverage follows the ownership model: - **The 8 system fields**: taken over for **every application**, whatever value they currently hold. This is both safe and required now that sync rejects non-derived values — leaving a row unconverged would make its application unsyncable - **`name`**: workspace-custom app → always taken over (server-generated, no author to clobber); installed applications → only rows still carrying the legacy SDK derivation are recomputed, author-provided identifiers are never touched; standard app → never touched (hardcoded in `STANDARD_OBJECTS`) - **Default relation fields**: workspace-custom app → forward fields on custom objects and reverse fields on the standard relation objects; installed applications → legacy-derivation probe only All identifiers of a workspace are updated inside a single transaction, then the command flushes the field-metadata-related workspace caches and bumps the metadata version. Stored `applicationRegistration.manifest` snapshots are intentionally **not** rewritten: installs and upgrades always sync from the `manifest.json` inside the resolved package (npm/tarball), the stored column is only used for display/marketplace purposes. ## Breaking behavior for old packages (fail closed) Packages built with an older SDK carry legacy system field identifiers in their tarball `manifest.json`. Installing or upgrading such a package now fails with an explicit `INVALID_SYSTEM_FIELD` validation error ("universal identifier is not deterministic") instead of silently mismatching against the backfilled rows and triggering a destructive delete+create. The remediation is to rebuild the package with the new SDK; the backfill has already converged the installed rows, so the rebuilt manifest syncs cleanly. ## Test plan - [x] `twenty-sdk` unit tests (526 tests) and typecheck - [x] `twenty-shared` unit tests (1635 tests) including the `STANDARD_OBJECTS` snapshot; `name` identifiers verified byte-for-byte identical to `main` - [x] Lint and typecheck clean on all touched packages - [x] Integration: create a custom object and verify system + default relation field identifiers match the deterministic derivation (`create-one-object-metadata-deterministic-field-universal-identifiers`, 13 assertions passing) - [x] Integration: `failing-sync-application-object-system-fields` extended with a non-derived system field identifier case; all identifiers in the spec pinned deterministically so snapshots embedding expected/actual values are stable across runs (verified with a double run) - [x] Integration: all application sync suites pass with the derived system field identifiers now required by the `buildDefaultObjectManifest` test helper (9 suites, 20 tests) - [x] Full test-database reset: standard app provisioning and seeded workspaces pass the new validation - [x] SDK manifest build verified on the postcard example app: all auto-generated default field identifiers match the derivation - [ ] Run `upgrade:2-19:backfill-deterministic-field-universal-identifiers` (dry-run then real) on a seeded workspace and verify identifier convergence with a rebuilt app manifest	2026-07-06 13:34:33 +00:00
Paul Rastoin	22b49a502c	chore(server): remove unused flat-field-metadata per-object mocks (#22581) ... ## Context The `flat-field-metadata/__mocks__/` directory contained 11 large per-object `as const` mock catalogs (`OPPORTUNITY_FLAT_FIELDS_MOCK`, `PERSON_FLAT_FIELDS_MOCK`, `PET_FLAT_FIELDS_MOCK`, ...) plus a `getRelationTargetFlatFieldMetadataMock` helper. An audit of the whole server package (searching both the constant names and any import of the directory) showed almost none of them are consumed anymore — tests have moved to building exactly the fields they need with the `getFlatFieldMetadataMock` factory. Usage found: - `getFlatFieldMetadataMock` (factory): ~25 spec files + 2 core-modules mocks — **kept** - `COMPANY_FLAT_FIELDS_MOCK`: 1 spec (`object-record-event-publisher.spec.ts`), which only used the `name` field - The other 10 `*_FLAT_FIELDS_MOCK` catalogs and `getRelationTargetFlatFieldMetadataMock`: **zero consumers** ## Changes - Delete the 11 unused `*-flat-fields.mock.ts` catalogs and `get-morph-or-relation-target-flat-field-metadata-mock.ts` (~4,900 lines). Only `get-flat-field-metadata.mock.ts` remains. - In `object-record-event-publisher.spec.ts`, build the company `name` field inline with `getFlatFieldMetadataMock` (wired to `COMPANY_FLAT_OBJECT_MOCK.id`/`workspaceId`) and replace the three `COMPANY_FLAT_FIELDS_MOCK.name.type` references with `FieldMetadataType.TEXT`. The sibling `flat-object-metadata/__mocks__/` catalogs are untouched — several of those are still consumed by the morph/relation specs. ## Verification - `object-record-event-publisher.spec.ts`: 27/27 passing - `npx nx lint:diff-with-main twenty-server`: green - `npx nx typecheck twenty-server`: green https://claude.ai/code/session_01XcGEtwdXQo9uJibGRPexuG --- _Generated by [Claude Code](https://claude.ai/code/session_01XcGEtwdXQo9uJibGRPexuG)_ <!-- This is an auto-generated description by cubic. --> <a href="https://cubic.dev/pr/twentyhq/twenty/pull/22581?utm_source=github" target="_blank" rel="noopener noreferrer" data-no-image-dialog="true"><picture><source media="(prefers-color-scheme: dark)" srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source media="(prefers-color-scheme: light)" srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img alt="Review in cubic" src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a> <!-- End of auto-generated description by cubic. -->	2026-07-06 13:47:17 +02:00
github-actions[bot]	e23f82f700	chore: sync AI model catalog from models.dev (#22568) ... Automated daily sync of `ai-providers.json` from [models.dev](https://models.dev). This PR updates pricing, context windows, and model availability based on the latest data. New models meeting inclusion criteria (tool calling, pricing data, context limits) are added automatically. Deprecated models are detected based on cost-efficiency within the same model family. **Please review before merging** — verify no critical models were incorrectly deprecated. <!-- This is an auto-generated description by cubic. --> <a href="https://cubic.dev/pr/twentyhq/twenty/pull/22568?utm_source=github" target="_blank" rel="noopener noreferrer" data-no-image-dialog="true"><picture><source media="(prefers-color-scheme: dark)" srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source media="(prefers-color-scheme: light)" srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img alt="Review in cubic" src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a> <!-- End of auto-generated description by cubic. --> Co-authored-by: FelixMalfait <6399865+FelixMalfait@users.noreply.github.com>	2026-07-06 09:25:31 +02:00
Félix Malfait	3a21089e0a	feat(server): abort ai stream jobs that outlive the shutdown drain budget (#22517) ... ## Why #22514 makes SIGTERM drain workers, but `worker.close()` waits for active jobs with **no upper bound** (BullMQ semantics). An AI stream job can run for 10 minutes; a deploy would either hang the rollout or hit the pod's termination grace deadline and get SIGKILLed anyway — back to the frozen-stream + stalled-rerun failure this series eliminates. ## What On shutdown, `aiStreamQueue` gets a bounded drain: active stream jobs have `AI_STREAM_SHUTDOWN_DRAIN_MS` (60s) to finish naturally; stragglers are then aborted and terminate exactly like a stream failure — `lastStreamError` persisted with the new typed `AiExceptionCode.STREAM_INTERRUPTED`, the pinned `stream-error` → `queue-updated` terminal sequence published, claim released. The client shows the interrupted state with Retry (#22434) within seconds instead of a stream frozen mid-sentence. This is deliberately **not** the user-cancel path, which resolves cleanly and persists no error. Mechanism — evaluated BullMQ 5.78's native cancellation vs a parallel in-process registry, and picked native: - The driver's processor now declares the 3-arg signature, which makes BullMQ create a per-job `AbortController` (`processor.length >= 3` is the trigger), and the signal is handed to job handlers as an optional `MessageQueueJobContext`. - `worker.cancelAllJobs()` is purely cooperative: it aborts the signal and nothing else, so the job's own persist/publish/cleanup still runs to completion and `worker.close()` still waits for it — no force-fail race, no second signaling channel to maintain, and the timer lives inside the same `closeWorker()` call so there is no dependence on Nest module-destroy ordering. - The stream job maps the shutdown signal onto its **existing** AbortController (the one already wired through the AI SDK for user cancel), with an `AiException(STREAM_INTERRUPTED)` reason to tell the two apart. One abort path end to end, no new infrastructure. Error-type choice: the job throws a plain `AiException`, not BullMQ's `UnrecoverableError`. Stream jobs are enqueued with `attempts: 1` (no `retryLimit`), so there is no BullMQ retry to suppress — retryability for this queue lives at the app layer (`lastStreamError` + client Retry), and an `UnrecoverableError` would only obscure the typed exception. `STREAM_INTERRUPTED` also replaces the string constant introduced on the base branch (#22482's reap now uses the same enum member) — one code, two producers (reap for dead workers, abort for live shutdowns), identical client behavior. Notes: - 60s is a static constant mirroring `AI_STREAM_LOCK_DURATION_MS` rather than an env var — it has to move in lockstep with the worker's `terminationGracePeriodSeconds` (120s, twenty-infra PR) anyway, and we ship multiple releases a day. Happy to lift it into a config variable if you want runtime tunability. - The `onModuleDestroy` scaffolding (drain logs, workers-close-before-queues) deliberately matches #22514; whichever lands second rebases clean. Stacked on #22482 (needs the heartbeat/reap base). Merge order: #22482 → this. Depends on #22514 for SIGTERM to reach the driver at all. ## Validation - `stream-agent-chat.job.spec.ts`: shutdown-abort persists `STREAM_INTERRUPTED`, publishes `stream-error` before `queue-updated`, releases the claim, skips the queued-message flush; user-cancel semantics unchanged with a wired-but-idle shutdown signal (60/60 ai-chat tests green). - Local end-to-end: real AI stream mid-flight, SIGTERM the worker → drain window → abort → interrupted state persisted, process exits on its own. (Transcript in the PR conversation.) <!-- This is an auto-generated description by cubic. --> <a href="https://cubic.dev/pr/twentyhq/twenty/pull/22517?utm_source=github" target="_blank" rel="noopener noreferrer" data-no-image-dialog="true"><picture><source media="(prefers-color-scheme: dark)" srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source media="(prefers-color-scheme: light)" srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img alt="Review in cubic" src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a> <!-- End of auto-generated description by cubic. -->	2026-07-05 14:15:24 +02:00
github-actions[bot]	e90ab56c7a	chore: sync AI model catalog from models.dev (#22558) ... Automated daily sync of `ai-providers.json` from [models.dev](https://models.dev). This PR updates pricing, context windows, and model availability based on the latest data. New models meeting inclusion criteria (tool calling, pricing data, context limits) are added automatically. Deprecated models are detected based on cost-efficiency within the same model family. **Please review before merging** — verify no critical models were incorrectly deprecated. <!-- This is an auto-generated description by cubic. --> <a href="https://cubic.dev/pr/twentyhq/twenty/pull/22558?utm_source=github" target="_blank" rel="noopener noreferrer" data-no-image-dialog="true"><picture><source media="(prefers-color-scheme: dark)" srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source media="(prefers-color-scheme: light)" srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img alt="Review in cubic" src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a> <!-- End of auto-generated description by cubic. --> Co-authored-by: FelixMalfait <6399865+FelixMalfait@users.noreply.github.com>	2026-07-05 09:03:02 +02:00
martmull	4e43a0fb4e	refactor(server): regroup application resolvers by resource and unify install permission flag (#22532) ... Part of the app settings architecture cleanup (twentyhq/core-team-issues#2456) — implements the API-surface regroup Charles asked for in #20825 ("in application resolvers we have uninstall, upgrade, findMany, etc. and for some reason install is part of the marketplace, and they are not protected by same guards"). ## Changes **Resolver regroup by resource** (GraphQL operation names and signatures unchanged): - `installApplication` + `installMarketplaceApp` (deprecation preserved) move from the marketplace resolver into `application-install.resolver.ts`, next to `findManyApplications`/`findOneApplication`/`uninstallApplication`. - `uninstallApplication` moves from the manifest resolver into `application-install.resolver.ts`. - `runWorkspaceMigration` is deleted outright (unused — no consumer anywhere in the repo, front/SDK/e2e/docs); the now-empty manifest resolver is deleted. Its `AllMetadataName` GraphQL enum registration moves to `collection-hash.dto.ts` (its remaining consumer). - `generateApplicationToken` moves from the development resolver into `application-oauth.resolver.ts` next to `renewApplicationToken`, keeping its effective guards (`WorkspaceAuthGuard` + `SettingsPermissionGuard(APPLICATIONS)`) and its token-bucket throttle verbatim. - `upgradeApplication` stays in the upgrade resolver (moving it into the install resolver would create a module cycle — the upgrade module imports the install module). - Marketplace resolver now only holds catalog concerns: `findManyMarketplaceApps`, `findMarketplaceAppDetail`, `syncMarketplaceCatalog`. **Permission unification** (the only behavior change): `installApplication`, `installMarketplaceApp` and `upgradeApplication` move from `MARKETPLACE_APPS` to `APPLICATIONS`, matching uninstall and the find queries. Front-end install/upgrade button gating updated accordingly (`SettingsApplicationDetails` / `SettingsAvailableApplicationDetails`). **Module wiring**: `MarketplaceModule` no longer imports `ApplicationInstallModule` (only the moved resolver needed it); `ApplicationInstallModule` now imports `MarketplaceModule` — no cycle. Exception filters follow the moved operations (`ApplicationRegistrationExceptionFilter` on the install resolver; `ApplicationExceptionFilter` on the oauth resolver, which also fixes `renewApplicationToken`'s previously unmapped FORBIDDEN). **Codegen**: `twenty-client-sdk` metadata client regenerated for the new schema ordering (pure reordering — no field changes); all front `graphql:generate` configurations produced zero diffs. ## Explicitly kept (per review discussion) `installMarketplaceApp` (deprecated) and `generateApplicationToken` are kept for SDK back-compat despite having no current consumers. `runWorkspaceMigration` was also consumer-less but, unlike those two, had no back-compat rationale (not a deprecated alias, not a token primitive), so it is removed rather than relocated. ## Deferred follow-ups (guard inconsistencies found in the audit, intentionally NOT changed here) - `findApplicationRegistrationByUniversalIdentifier` uses `NoPermissionGuard` and returns the full registration entity, bypassing the `API_KEYS_AND_WEBHOOKS` gate that `findOneApplicationRegistration` enforces on the same data (SDK CLI `ensure-app-registration` depends on it today). - `upgradeApplication` alone requires `UserAuthGuard` — an API key can install but not upgrade. - `uploadAppTarball` (`MARKETPLACE_APPS`) and `transferApplicationRegistrationOwnership` (`APPLICATIONS`) are flag-inconsistent with the rest of registration CRUD (`API_KEYS_AND_WEBHOOKS`). - `syncMarketplaceCatalog` triggers an instance-wide job but is gated only by a per-workspace settings flag. ## Verification - `npx nx typecheck twenty-server` / `twenty-front` ✓; `lint:diff-with-main` clean for both - `npx jest "application"` in twenty-server: 30 suites / 154 tests passed - Server boots with the new module graph (DI verified at runtime); codegen run against the live server - Repo-wide grep: no remaining imports of the deleted manifest resolver <a href="https://cubic.dev/pr/twentyhq/twenty/pull/22532?utm_source=github" rel="nofollow noreferrer noopener" target="_blank">``&lt;img alt="Review in cubic" src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"&gt;``</a> --------- Co-authored-by: Charles Bochet <charles@twenty.com>	2026-07-05 00:24:12 +02:00
Thomas des Francs	1a60d4eaa3	Add MCP setup screen (#22468) ... ## Summary - Add a first-tab MCP setup experience under MCP & APIs with quick install cards, manual configuration, client logos, and HTTPS gating for Claude install links. - Rename API/Webhooks settings surfaces to MCP & APIs and update related icons, permissions, breadcrumbs, and command menu entries. - Add the Tabler sparkle-2 icon wrapper and MCP setup visual assets. ## Screenshots | Before | After | | --- | --- | |  | <img alt="image" src="https://github.com/user-attachments/assets/a6ae2ae6-322b-4370-b9b6-0a3d73ff7fa7" /> | <!-- This is an auto-generated description by cubic. --> <a href="https://cubic.dev/pr/twentyhq/twenty/pull/22468?utm_source=github" target="_blank" rel="noopener noreferrer" data-no-image-dialog="true"><picture><source media="(prefers-color-scheme: dark)" srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source media="(prefers-color-scheme: light)" srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img alt="Review in cubic" src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a> <!-- End of auto-generated description by cubic. --> --------- Co-authored-by: Charles Bochet <charles@twenty.com>	2026-07-04 16:38:59 +02:00
Félix Malfait	9be2b21e51	fix(ai): gate chat thread totals on stream ownership to prevent usage under-counting (#22534) ... Fixes a usage under-counting bug introduced by #22524 (progressive assistant-message persistence), flagged by @cubic-dev-ai and confirmed by @FelixMalfait. ## Root cause #22524 gated the thread-totals update (tokens, credits, `conversationSize`) on `assistantMessageExistedAtStart` — an existence check on the deterministic `uuidv5(streamId)` message id captured at job start. That was a sound idempotency signal *before* #22524, when the message only ever existed if a prior run had completed and applied totals. Progressive checkpoints broke that assumption: a checkpoint creates the message row ~2s into the stream, without applying totals. So if the worker is SIGKILLed after a checkpoint but before `handleStreamFinish`, and the job is re-delivered (BullMQ's stalled re-run — `aiStreamQueue` has no `maxStalledCount: 0` yet, that's #22518 — or an admin `retryJobs`), the re-run sees `assistantMessageExistedAtStart === true` and returns before the totals update. The turn's usage is lost permanently. cubic's P2 (the non-transactional `delete`+`insert` in `upsertAssistantMessage`) is the same root cause: its partless window only mattered because it tripped the same existence-based gate. ## Fix Stop inferring "totals already applied" from message existence. Gate the totals update on **still owning the stream** — a conditional `UPDATE ... WHERE id = :threadId AND activeStreamId = :streamId`, and only `notifyThreadUsageUpdated` when it affects a row. This is the same claim pattern the stream already uses (#22481), and it's idempotent by construction: - The run that completes while holding the claim → `affected = 1` → totals applied exactly once. This holds **even when a checkpoint already created the message**, which is precisely the bug. - A duplicate/zombie run after another run completed (and its `finally` cleared `activeStreamId`) → `affected = 0` → skipped, no double-count. - A superseded run whose thread has moved to a newer stream → `affected = 0` → skipped (defense-in-depth, aligns with #22518's ownership pre-check). The `assistantMessageExistedAtStart` flag and its start-of-stream `hasMessageById` query are removed entirely — the message write is already idempotent via the deterministic id + `upsert`, so it needs no gate. This subsumes cubic's P2: the totals are no longer lost regardless of the `delete`+`insert` window, so no transaction is required for correctness (the residual window is a benign sub-millisecond transient for an actively-streaming message; happy to add a workspace-datasource transaction as separate hardening if you'd prefer). ## Validation `stream-agent-chat.job.spec.ts` (9 green): - New: totals update returns `affected: 0` → `notifyThreadUsageUpdated` **not** called (prior completion not double-counted), message still upserted. - New: message already exists from a checkpoint but claim still held (`affected: 1`) → totals **are** applied — the exact regression #22524 caused. - Existing success/error/cancel/abort flows updated for the conditional criteria and still green. <!-- This is an auto-generated description by cubic. --> <a href="https://cubic.dev/pr/twentyhq/twenty/pull/22534?utm_source=github" target="_blank" rel="noopener noreferrer" data-no-image-dialog="true"><picture><source media="(prefers-color-scheme: dark)" srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source media="(prefers-color-scheme: light)" srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img alt="Review in cubic" src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a> <!-- End of auto-generated description by cubic. -->	2026-07-03 18:50:10 +02:00
Félix Malfait	9496a98aa3	feat(ai): persist assistant chat messages progressively during streaming (#22524) ... ## Why Today the assistant chat message is written to the DB exactly **once**, at `onFinish` (`handleStreamFinish`). The per-step hook (`onStepFinish` in `chat-execution.service.ts`) only does billing/metrics — no persistence. Content streams to the client live over Redis, but the durable record only lands at the end. The graceful paths already cover partials: normal completion, user cancel, and the shutdown-abort from #22517 all fire `onFinish` with `isAborted` and persist whatever parts exist. The gap is a **true SIGKILL** — OOM, node loss, or a grace-period overrun — where `onFinish` never runs. When that happens mid-turn, the assistant message vanishes from the thread even though its tool calls already executed real CRM mutations. That's the worst failure shape: side effects persisted, the record of them didn't. This closes that gap by materializing the assistant message progressively, so a hard kill leaves the tools-already-run on the thread. It's the app-level piece behind the earlier discussion on #22518 — with this, a retried/interrupted turn also resumes from its own partial (the model reloads history and continues) instead of re-doing completed steps. Scope is **chat only** — the workflow agent path (`AgentAsyncExecutorService`, blocking `generateText`) is a different model with its own step-log persistence and workflow-engine resumption, and is deliberately out of scope here. ## What - `AgentChatService.upsertAssistantMessage`: idempotent message+parts write keyed on the deterministic `uuidv5(streamId)` id (upsert the row, replace its parts), reusing the existing `mapUIMessagePartsToDBParts` / `finalizeDanglingToolParts`. - `stream-agent-chat.job.ts`: tee the assembled UI stream — one branch keeps publishing chunks unchanged; the other drives the SDK's own `readUIMessageStream` and, throttled to `AGENT_CHAT_CHECKPOINT_INTERVAL_MS` (2s), fires a serialized fire-and-forget `upsertAssistantMessage`. No chunk re-assembly — the parts come straight from the SDK assembler, identical to what `onFinish` produces. - `handleStreamFinish` now upserts (authoritative) instead of insert-then-skip. Two ordering/idempotency guards: - Checkpoints are serialized through one promise chain and gated off (`isFinalizingPersist`) before the final write, which drains the chain first — so the authoritative write always lands last and never races a checkpoint on the parts table. - The old `hasMessageById`→skip protected the thread-totals accumulation from double-counting on a re-executed job. Since checkpoints now make the row exist mid-stream, that signal is captured **once at stream start** (`assistantMessageExistedAtStart`) and used to gate the totals update — preserving the exact prior idempotency while allowing progressive writes. `readUIMessageStream` runs with `terminateOnError: false` and the checkpoint consumer swallows errors: checkpoints are best-effort and must never affect the stream or the authoritative persist. ## User impact A worker that dies hard mid-turn no longer erases the assistant message. Combined with #22434's Retry, the user sees the partial turn (including executed tools) and can continue, rather than a turn that silently disappeared while its side effects stuck. Note: this is insurance against true SIGKILL specifically — graceful shutdown (#22514/#22517) already persists partials — so it's most valuable for OOM/node-loss/grace-overrun. Framed that way deliberately; happy to drop it if you'd rather not touch this path for that scope. ## Validation - Unit (`stream-agent-chat.job.spec.ts`, 59 green): the success path persists via `upsertAssistantMessage` with the assembled parts + turnId; a re-executed job whose message existed at start still upserts but does **not** re-apply thread totals; all existing flows (mid-stream error, user cancel, shutdown-abort, missing workspace) unchanged. - Local runtime (isolated instance, real OpenAI stream, checkpoint interval shortened for the test): - **SIGKILL mid-stream** (no graceful onFinish) → the assistant message row (deterministic id) is present afterward with a partial text part (~381 chars) that would otherwise have been lost. - **Normal completion** → the final upsert converges to the full message ("Hello, Tim."), `activeStreamId` cleared, `totalOutputTokens` applied exactly once. <!-- This is an auto-generated description by cubic. --> <a href="https://cubic.dev/pr/twentyhq/twenty/pull/22524?utm_source=github" target="_blank" rel="noopener noreferrer" data-no-image-dialog="true"><picture><source media="(prefers-color-scheme: dark)" srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source media="(prefers-color-scheme: light)" srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img alt="Review in cubic" src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a> <!-- End of auto-generated description by cubic. -->	2026-07-03 18:14:56 +02:00
Paul Rastoin	43730d7748	Centralized side effects devxp basis (#22295) ... # Introduction This PR introduces a centralized, strictly-typed **metadata side-effect engine** that unifies how system metadata side effects are derived and applied across both metadata entry points — the **metadata GraphQL API** and the **application sync / manifest** flow — and migrates the first side effect end-to-end: **a unique scalar field owns its backing single-field `UNIQUE` index** (full create / update / delete lifecycle). ## New conventions - **Engine-owned companions**: metadata flagged `isSystemSideEffect: true` is owned by the engine. Its deletion is never inferred from absence in a manifest — it results from PG-level cascade or from a delete side effect (a side effect always has a cause, its parent metadata). - **Reserved deterministic identifiers**: apps cannot declare metadata reusing an engine-owned deterministic `universalIdentifier`. Doing so fails validation with `RESERVED_SYSTEM_UNIVERSAL_IDENTIFIER` (until an explicit override API exists). - **Record-native operation matrix**: the operation matrix is keyed by `universalIdentifier` (`AllFlatEntityOperationRecordByMetadataName`) instead of arrays, making parent resolution and deduplication O(1). Array-based API callers are transpiled to records at the validate-build-and-run boundary. - Twenty-sdk user-facing experience with system fields will only be related to overrides. # What this PR does ## 1. Side-effect engine (foundation) - `MetadataSideEffectEngineService.expandWithSideEffects(...)` takes the intention-carrying record matrix and returns it expanded with derived side effects, or a structured failure. - Handlers are registered via a typed **decorator + registry** pattern (`MetadataSideEffectHandler({ operation, metadataName, name, description })`), with runtime duplicate-name detection. Multiple handlers per (operation, metadataName) are supported. - Handler contract mirrors the validator pattern: - receives the trigger flat entity, the live record matrix, and **strictly-typed related flat entity maps** (`MetadataFlatEntityAndRelatedFlatEntityMapsForSideEffect<P>`, derived from declared companion metadata names — no loose `Partial<AllFlatEntityMaps>` context) - returns `MetadataSideEffectResult`: `success` (operations record) | `noop` | `fail` (structured failure) - **Non-recursion is structural**: triggers are read from the original caller input, never from the expanded matrix, so a side effect can never trigger another side effect. - **Deduplication + collision detection**: side effects are deduped by `universalIdentifier` per operation; a caller-declared entity colliding with an engine-owned deterministic identifier is recorded as a collision. - **Unified failure channel**: handler failures and reserved-identifier collisions are merged into the same `OrchestratorFailureReport` contract as builder validation errors, and the run short-circuits (fail-closed, nothing is applied). ## 2. First migrated side effect — unique field → backing unique index Three handlers own the complete lifecycle of the deterministic single-field `UNIQUE` index backing a unique scalar field: - **create**: unique scalar field → generate the deterministic backing index (`fieldUniqueBackingIndexOnCreate`) - **update**: `isUnique` flips and renames of still-unique fields (the index name — and therefore its deterministic identifier — derives from the field name, so a rename drops the stale index and recreates the deterministic one) (`fieldUniqueBackingIndexOnUpdate`) - **delete**: cascade-delete the backing index (`fieldUniqueBackingIndexOnDelete`) Supporting rules: - The primary key `id` field never spawns a backing index (uniqueness comes from the PK constraint) — explicit `isPrimaryKeyFlatFieldMetadata` guard. - Parent object resolution is **optimistic-first**: an object created or updated in the same batch wins over the workspace cache (so e.g. renaming an object while flipping a field to unique builds the index from the post-rename object), resolved in O(1) via the record matrix. - A missing parent object is reported as a structured side-effect failure, never silently skipped. ## 3. Path convergence — manifest and API share one flow - The manifest sync now derives a from→to **record matrix** from the cache and feeds `validateBuildAndRunWorkspaceMigrationFromRecord`, the same flow the API uses — both paths converge on the engine. - Manifest-side unique-index generation and API transpiler system-unique-index handling were removed (declared/composite/relation indexes stay untouched). - New `WorkspaceMigrationFlatEntityMapsService` mutualizes flat-entity-maps computation between the side-effect engine and the builder: cache keys are derived from the caller metadata names (+ validation- and side-effect-related closures) instead of hardcoded loads. - App-scoping and pruning are folded into one shared primitive (`getSubAllFlatEntityMapsByApplicationIdsOrThrow`): slicing dependency maps to the involved applications always prunes dangling one-to-many aggregators — callers can no longer forget it. - **Behavior change**: an app extending another app's view with a view field now syncs successfully (cross-app view-field extension), covered by a dedicated integration test. ## 4. Backfill upgrade command (2.19) `upgrade:2-19:backfill-system-unique-index-universal-identifier` rewrites legacy system unique-index `universalIdentifier`s to their deterministic value so the engine can own pre-existing indexes. The backfill is **driven from `isUnique: true` fields** (mirroring the engine ownership predicate — excludes PK / morph / relation fields) and resolves each field's backing index in O(1). # Bugs fixed along the way - `database:reset` seeding failed with `INDEX_FIELD_INVALID_DEFAULT_VALUE`: the engine derived a backing `UNIQUE` index for the default `id` primary key. Fixed with the explicit primary-key guard. - `isUnique` updates on system-flagged standard fields (e.g. auto-created `name`) did not trigger the backing-index side effect. - Manifest sync crashed with "Could not find flat entity with universal identifier ..." when app-scoped slices left dangling aggregator references — fixed by centralizing pruning in the shared slice primitive	2026-07-03 18:13:20 +02:00
Félix Malfait	13f380b80d	perf(front-component): fingerprint built-JS URLs by path for CDN caching (#22530) ... **Stacked on #22523** — base is that branch, so the diff shows only this commit. GitHub will retarget it to `main` automatically once #22523 merges. Follows up on @FelixMalfait's question on #22523: move the BuiltFrontComponent cache key from a query string into the path so it plays well with Cloudflare cache rules. ## What - URL: `/rest/front-components/:id?checksum=<c>` → `/rest/front-components/:id/<c>.js` (`getFrontComponentUrl`). - Route: the controller now accepts `[':frontComponentId', ':frontComponentId/:cacheKey']`. `:cacheKey` is a pure cache-buster the server **ignores** — it still resolves by `:frontComponentId`, exactly as the query param did. ## Why a path segment (not `:id-<checksum>.js`) A path-based, extension-bearing URL is matched by Cloudflare's **default** static-asset caching and by trivial `*.js` path cache rules, and it's immune to any "ignore query string" cache setting that would otherwise collapse `?checksum=` to one entry and serve stale JS. I used a path **segment** (`/:id/:checksum.js`) rather than the literal `:id-<checksum>.js` you sketched because the id is a **UUID — which itself contains hyphens** — so a `-` separator is ambiguous to parse. A segment is unambiguous and equally CDN-friendly (still ends in `.js`). ## Backward compatibility The bare `:frontComponentId` route is kept, so URLs minted before this deploys (query-string form, or in-flight pages) still resolve. It can be dropped in a later release once no client mints the old form. No data migration — the URL is computed at render time from `frontComponentId` + `builtComponentChecksum`. ## ⚠️ Decision for you: this alone does not edge-cache — `private` vs `public` BFC is served behind `WorkspaceAuthGuard` and #22523 set its header to **`private`**, max-age, immutable. `private` means shared caches (Cloudflare) **won't** store it — so today this is browser-cache only, and the path change just makes it *ready* for edge caching + clean cache rules. To actually get **edge** caching you'd additionally either flip BFC to `public` or add a Cloudflare rule that overrides cache-control — which means **accepting that the `id`+`checksum` URL becomes the access capability** (a cache hit is served without re-checking origin auth). The cache key is unique per component+build so there's no cross-workspace mixup, but the built JS effectively becomes public-by-URL (same posture PublicAsset already has). I've **left it `private`** here; flipping to `public` is your call and can be a one-line follow-up. ## Tests - `getFrontComponentUrl` unit test: fingerprinted path when a checksum is present, bare fallback otherwise. - Integration test: the `/front-components/:id/:checksum.js` path serves the built JS with `Content-Type: application/javascript` and `Cache-Control: private, max-age=86400, immutable`. Existing bare-route tests remain and still pass. https://claude.ai/code/session_01AKwhTxYFDhWhCZ4b7sf35W --- _Generated by [Claude Code](https://claude.ai/code/session_01AKwhTxYFDhWhCZ4b7sf35W)_ <!-- This is an auto-generated description by cubic. --> <a href="https://cubic.dev/pr/twentyhq/twenty/pull/22530?utm_source=github" target="_blank" rel="noopener noreferrer" data-no-image-dialog="true"><picture><source media="(prefers-color-scheme: dark)" srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source media="(prefers-color-scheme: light)" srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img alt="Review in cubic" src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a> <!-- End of auto-generated description by cubic. -->	2026-07-03 17:17:14 +02:00
Félix Malfait	41ef601a7a	perf(twenty-server): cache BuiltFrontComponent and PublicAsset responses (#22523) ... Follow-up to #22510. Closes #22515 — extends `Cache-Control` to the two remaining app-asset folders that #22510 left `immutable: false` because they're path-addressed. Each now gets the directive that matches **how it is addressed**. ## BuiltFrontComponent → immutable I was wrong in the #22515 write-up to call this "stable URL, mutable bytes." The browser **already content-addresses it**: `FrontComponentRenderer` fetches `/rest/front-components/:id?checksum=${builtComponentChecksum}` (`getFrontComponentUrl`), so a rebuild changes the checksum → changes the URL → busts the cache. That makes `immutable` safe — no stale-code window — and needs no new versioning machinery. Wired the header into `FrontComponentController.getBuiltJs` (which passed no folder) and the front-component presign path. ## PublicAsset → bounded public cache Genuinely path-addressed and overwritten in place on every app (re)install/redeploy (upsert on `['path','workspaceId','applicationId']`), so it **cannot** be `immutable`. Instead: - **`public`** — the `/public-assets/...` endpoint is unauthenticated (`PublicEndpointGuard`), so the bytes are already world-readable; marking the response `public` lets a CDN (e.g. Cloudflare in front of the server) serve app/marketplace logos from the edge instead of hitting the origin on every render. Today these responses carry no `Cache-Control` at all. - **`max-age=3600`, not `immutable`** — a bounded window so an asset overwrite recovers within an hour. This one hour is the single judgement call here; tune it (or add `stale-while-revalidate`) to taste. ## Mechanism Generalized `FileFolderConfig.immutable` (boolean) into `cacheControl` (`string | null`) so a folder can carry its own directive instead of only opting into one hardcoded string. `setFileResponseHeaders` and the presign paths now read `cacheControl` directly. The immutable-folder set is unchanged; only BuiltFrontComponent (→ immutable) and PublicAsset (→ bounded public) move. ## Tests `setFileResponseHeaders` spec updated: BuiltFrontComponent now asserts immutable, PublicAsset asserts `public, max-age=3600`, and the remaining path-addressed folders (`AppTarball`, `Source`, `BuiltLogicFunction`, `Dependencies`) assert no `Cache-Control`. _Note: I bundled both folders into one PR since they share the config generalization — happy to split BuiltFrontComponent (safe/immutable) from PublicAsset (the `max-age` judgement call) if you'd rather review them separately._ https://claude.ai/code/session_01AKwhTxYFDhWhCZ4b7sf35W --- _Generated by [Claude Code](https://claude.ai/code/session_01AKwhTxYFDhWhCZ4b7sf35W)_ <!-- This is an auto-generated description by cubic. --> <a href="https://cubic.dev/pr/twentyhq/twenty/pull/22523?utm_source=github" target="_blank" rel="noopener noreferrer" data-no-image-dialog="true"><picture><source media="(prefers-color-scheme: dark)" srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source media="(prefers-color-scheme: light)" srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img alt="Review in cubic" src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a> <!-- End of auto-generated description by cubic. -->	2026-07-03 17:14:12 +02:00
Etienne	1270054d35	feat(ai): dashboard & view building (#22411) ... ## Why Building a dashboard through AI chat used to cost ~9 sequential LLM round-trips (~160K input tokens for a single request): the agent had to resolve object/field UUIDs and assemble views through many granular tool calls, each step replaying the full cached context. ## What changed ### 1. Reference objects & fields by name (fewer round-trips) The agent no longer needs to resolve UUIDs before acting. - `get_object_metadata`: filter by `objectName` (singular/plural) and a new `includeFields` flag returning each object's fields (`{id, name, type, label}`) inline — object + field IDs in one call. - `get_field_metadata`: accepts `objectName` as an alternative to `objectMetadataId`. - All three dashboard write tools (`create_complete_dashboard`, `add_dashboard_widget`, `update_dashboard_widget`): accept `objectName` and `*FieldName` variants (`aggregateFieldName`, `primaryAxisGroupByFieldName`, `secondaryAxisGroupByFieldName`, `groupByFieldName`, ratio `fieldName`), resolved to UUIDs server-side by `resolveWidgetFieldNamesToIds`. UUID variants still win when both are given. ### 2. `upsert_complete_view` — one atomic call to build/reconfigure a view - New `upsert_complete_view` tool + `ViewService.upsertCompleteView`: create or update a view together with its fields, filters, and sorts. - Children are **declarative**: a provided array replaces all existing entries of that kind, `[]` clears them, omitting leaves them untouched. Fields are referenced by name or UUID; no child-row IDs needed. - Runs as a **single workspace migration** (`view` + `viewField` + `viewFilter` + `viewSort` in one `validateBuildAndRunWorkspaceMigration` matrice) instead of chained per-entity service calls. New `buildCompleteViewChildrenFlatOperations` util assembles the child create/delete operations. - Granular tools (`create_view_filter`, `update_view_sort`, …) are retained for surgical single-entry edits. ### 3. Chart filters on dashboard widgets (end-to-end) - Added `chartFilterSchema` (`recordFilters` + optional `recordFilterGroups` for AND/OR logic) to the four chart configs, with field-by-name or -UUID references and documented operands/value formats. - **Relative dates supported** — e.g. `PAST_7_DAY`, `THIS_1_MONTH`, `NEXT_3_WEEK`, plus open-ended `IS_IN_PAST` / `IS_IN_FUTURE` / `IS_TODAY`. Filters route through the same read pipeline (`computeRecordGqlOperationFilter`) as view filters, so they resolve and apply correctly. - `resolveChartFilterFieldNamesToIds` resolves filter `fieldName` → id against the widget object. ### 4. Re-enable AI-assisted dashboards - Removed the "coming soon" gating (`isActive: false` on the dashboard skill and the "not available yet" copy in the MCP server + chat prompts) and registered `DashboardToolProvider`. - Rewrote the dashboard skill prompt: confirmation gate (present a plan, wait for confirmation), completion guard (once confirmed, emit the create tool in-turn — no "now let me…" preambles), default-and-proceed (pick sensible defaults for missing fields instead of stalling), and an intent gate so informational dashboard questions are answered directly without loading skills. ### 5. Frontend: clearer advanced-filter labels - `useRecordFilterField` now derives the filter label from field metadata and appends the relation target field (e.g. `Company → Name`), so relation/target filters — including those set by the AI — display correctly instead of showing a stale/blank stored label. ## Fixes - **`get_object_metadata({ objectName })` crash.** `ObjectMetadataService.findManyWithinWorkspace` spread an array-form (`OR`) `where` into a plain object, producing `{ "0": {...}, "1": {...}, workspaceId }` → `Property "0" was not found in "ObjectMetadataEntity"`. Now injects `workspaceId` into each OR clause, so name lookups work. - **Invalid SELECT/MULTI_SELECT filter options silently produced broken charts/views.** Chart-configuration validation and the migration-layer `FlatViewFilterValidator` now reject filters that reference options that don't exist, with a clear `Allowed values: …` message at creation time (shared `getInvalidSelectFilterOptionValues` util + tests). - **Non-atomic view assembly.** The previous multi-call view build could leave a half-built view on failure; `upsert_complete_view` now runs as a single transaction (one validation pass, one cache recompute, rollback on error). - **Blank RECORD_TABLE widgets from UNLISTED views.** Guidance + the upsert ownership check steer widget-backing views to `WORKSPACE` visibility; an UNLISTED view created without an owner renders a blank widget. - **Extra discovery round-trip removed.** Deleted the skill→tool bundle mechanism (`SKILL_TOOL_BUNDLES`, `getBundledToolNamesForSkills`, and the `load_skills` schema-loading path) that forced a second `learn_tools` call. - **Type-safety of widget resolution.** Reworked the widget resolver to build a properly typed `WidgetWithMetadataIds` (dedicated input/output types) instead of returning an untyped, cast-heavy object. ## Notes - Backend changes are in `twenty-server`; one small `twenty-front` change to the advanced-filter label hook. No entity/schema changes, so no migration. - Tests added: `getInvalidSelectFilterOptionValues`, `resolveWidgetFieldNamesToIds` (incl. filter/relative-date resolution), `update_dashboard_widget`, and expanded view-tools factory specs. - Design decisions: dedicated composite tool over code-interpreter orchestration (atomicity + validation + consistency with `create_complete_dashboard` / `create_complete_workflow`); name-or-UUID but no child-row IDs on `upsert_complete_view`; name→id resolution kept as stateless utils, not services. ## Test plan - [ ] `npx nx run twenty-server:typecheck` - [ ] `npx nx lint:diff-with-main twenty-server` and `twenty-front` - [ ] `npx nx test twenty-server` (view tools factory, `getInvalidSelectFilterOptionValues`, `resolveWidgetFieldNamesToIds`, `update_dashboard_widget`) - [ ] AI chat: "Create a dashboard with a chart of deal value by pipeline stage and a table of the top 10 open opportunities" → plans, waits for confirmation, then builds with fewer round-trips - [ ] AI chat: add a chart widget filtered by a relative date (e.g. deals created in `PAST_7_DAY`) and confirm the chart is actually filtered - [ ] Filter on a non-existent SELECT option is rejected with a clear error <!-- This is an auto-generated description by cubic. --> <a href="https://cubic.dev/pr/twentyhq/twenty/pull/22411?utm_source=github" target="_blank" rel="noopener noreferrer" data-no-image-dialog="true"><picture><source media="(prefers-color-scheme: dark)" srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source media="(prefers-color-scheme: light)" srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img alt="Review in cubic" src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a> <!-- End of auto-generated description by cubic. -->	2026-07-03 14:15:27 +00:00
Félix Malfait	5d3b6d05b3	feat(ai): add a stream heartbeat and reap dead claims so a worker crash cannot brick a thread (#22482) ... ## Rationale If the worker process dies mid-stream (OOM, deploy, crash), nothing ever clears `activeStreamId`: `aiStreamQueue` runs with `attempts: 1`, the job's `finally` never executes, and the SSE keepalive comes from the API server — so it actively masks worker death. The thread is bricked: every send queues behind a dead claim until someone intervenes manually. This is a CONFIRMED-high from the chat-stack audit, and worker death is not hypothetical: Sentry shows an unhandled promise rejection inside the AI SDK in the worker ([TWENTY-SERVER-H7Y](https://twenty-v7.sentry.io/issues/TWENTY-SERVER-H7Y)) — unhandled rejections terminate Node by default. ## Design - **Claim-time mark**: every enqueue site marks `agent-chat-stream-alive:<streamId>` with a TTL matching the job lock horizon (600s) — covering the enqueue→pickup window where a waiting job holds no lock. - **Running refresh**: the job tightens it to **30s, refreshed every 5s**; if the process dies, the interval dies with it and the key expires. The expiry *is* the death signal. (Was 60s/15s — tightened after review: detection latency is bounded by the TTL, robustness by TTL−interval and the missed-beat tolerance; 30s/5s halves detection while tolerating *more* missed beats, 5 vs 3.) - **Read-path reap**: the send gate and the catchup query convert a heartbeat-less claim into a normal retryable `STREAM_INTERRUPTED` failed-turn state (conditional UPDATE guarded on the observed streamId, so a newer stream's claim is never touched), reset the Redis chunk state, and publish the terminal error. `isAlive` fails open on Redis errors — a liveness probe must not turn a Redis blip into a broken send path. ## Why this is the root cause, not a symptom patch The strongest alternative — BullMQ's own stalled-job detection — fails on four concrete grounds: detection latency is bounded by the deliberate 10-minute `AI_STREAM_LOCK_DURATION_MS` (long silent tool runs must not spuriously stall); the stalled checker needs a *surviving* worker in the pool; the signal fires in the worker process while the thing needing repair is a DB claim read by API-server resolvers; and a `waiting` job holds no lock at all. Reaping at the read path means recovery happens exactly when a user is looking — the moment it matters — with zero background machinery. **Relationship to the graceful-shutdown work (planned follow-ups)**: shutdown hooks + drain-then-abort will make *deploys* (cooperative SIGTERM) end streams cleanly, and disabling stalled re-runs will stop hard-killed jobs from zombie re-executing tools. This PR remains the only recovery layer for non-cooperative deaths — OOMKill is a straight SIGKILL, crashes and unhandled rejections never run shutdown hooks — and the backstop when the drain path itself fails. The two are complements, not alternatives. ## User impact Today a worker crash mid-answer bricks the thread until manual intervention; users see sends silently queue forever. With this, the next interaction (send, reload) converts it into a visible "response was interrupted" error with a working Retry, within ~30s of actual death. ## Test plan - [x] Claim spec: live stream untouched; heartbeat-less claim reaped into retryable `STREAM_INTERRUPTED` + chunk-state reset + published terminal event; no-op when the claim moved to a newer stream mid-check - [x] CI green https://claude.ai/code/session_01Lyi6zTema2FMVVh8MD6c38	2026-07-03 13:29:02 +02:00
martmull	af1b89c788	feat(server): import application logo into file storage at install (#22437) ... Installed apps stored the logo as the manifest's relative path but never imported the file, so the public-assets URL 404'd and logos went missing in the UI for npm/tarball sources. Import the logo (best-effort — a declared but unshipped logo is skipped, not fatal) and record it as a first-class logoFileId on the application so it can be served reliably. <!-- This is an auto-generated description by cubic. --> <a href="https://cubic.dev/pr/twentyhq/twenty/pull/22437?utm_source=github" target="_blank" rel="noopener noreferrer" data-no-image-dialog="true"><picture><source media="(prefers-color-scheme: dark)" srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source media="(prefers-color-scheme: light)" srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img alt="Review in cubic" src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a> <!-- End of auto-generated description by cubic. -->	2026-07-03 11:47:58 +02:00
martmull	25fe66565c	feat(applications): add type and options to application variables (#22157) ... ## Before <img width="1452" height="709" alt="image" src="https://github.com/user-attachments/assets/cd384ffa-cbe6-49d5-a807-ca8d580f55a9" /> <img width="1074" height="452" alt="image" src="https://github.com/user-attachments/assets/720d38db-3495-4032-8831-17d24ec6a7e7" /> ## After <img width="1421" height="865" alt="image" src="https://github.com/user-attachments/assets/2275c996-c895-4800-8324-2aa2ddfddd43" /> <img width="1348" height="870" alt="image" src="https://github.com/user-attachments/assets/3e1a891d-6db0-4cbd-870a-2a5bbde4929d" /> ## Summary Adds typed application variables with optional select **options**. This is the other half of #22059, split out from the custom-settings-tab removal. ## Changes - **Shared types**: `ApplicationVariable` / `ServerVariables` gain an optional `type` (a `FieldMetadataType` subset — `TEXT`, `BOOLEAN`, `NUMBER`, `DATE`, `SELECT`, `MULTI_SELECT`, `RAW_JSON`, `RICH_TEXT`, `ARRAY`, …) and select `options`. New `serializeApplicationVariableValue` / `deserializeApplicationVariableValue` helpers convert typed values to/from the encrypted string storage. - **Server**: `type`/`options` columns on `applicationVariable` and `applicationRegistrationVariable` (entities + DTOs), a fast `2-17` instance command, manifest processing via the serialization helpers, and a `QueryDeepPartialEntity` cast where the manifest JSON column is persisted. - **Frontend**: a polymorphic `SettingsApplicationVariableInput` that renders the native `Form*` field component for each type (boolean, number, date/date-time, select, multi-select, array, raw JSON, rich text, text); fragment/query updates to fetch `type`/`options`. - **SDK**: `defineApplication` validates that `SELECT`/`MULTI_SELECT` variables declare non-empty `options` at build time (since `options` is kept structurally optional for TypeORM/SDK compatibility). Variables default to `TEXT` when no type is given, so existing manifests are unaffected. ## Notes The generated GraphQL artifacts (`type`/`options` on the variable types) are regenerated by codegen; that change accompanies this PR. https://claude.ai/code/session_013Z7UB35V2mvUozh55QHG23 --- _Generated by [Claude Code](https://claude.ai/code/session_013Z7UB35V2mvUozh55QHG23)_ <!-- This is an auto-generated description by cubic. --> <a href="https://cubic.dev/pr/twentyhq/twenty/pull/22157?utm_source=github" target="_blank" rel="noopener noreferrer" data-no-image-dialog="true"><picture><source media="(prefers-color-scheme: dark)" srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source media="(prefers-color-scheme: light)" srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img alt="Review in cubic" src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a> <!-- End of auto-generated description by cubic. -->	2026-07-03 10:52:22 +02:00
Félix Malfait	9ef1af9799	fix(ai): make stream claims atomic via conditional UPDATEs with claim-or-queue send (#22481) ... ## Rationale `activeStreamId` is the mutex that guarantees one live stream per thread — but claiming it is a plain read-then-update. The resolver checks it, then `streamAgentChat` enqueues the job **before** writing the claim. Two racing sends both pass the check, both start jobs, and each job's `resetStreamState` wipes the other's Redis chunk list — tokens from two answers interleave into the visible message. The same window exists for retry vs. send, the queue drain vs. send, and `stopAgentChatStream`, which cleared the claim **unguarded** (`{ id, userWorkspaceId }`) and could wipe a newer stream's claim entirely. ## Why this is the root cause, not a symptom patch Ownership must live in the `activeStreamId` column regardless of any locking mechanism — the queue-behind gate, the thread DTO, and stop all read it. So the correct primitive is a single-row compare-and-set on that column: `UPDATE … WHERE "activeStreamId" IS NULL` checked via affected rows, claim **before** enqueue, release on enqueue failure. Every mutation of the claim is now guarded on the observed value. Alternatives evaluated and rejected: - **BullMQ jobId dedup by threadId**: the driver appends a `-${v4()}` suffix to custom ids and dedups via a non-atomic `getJobs(['waiting'])` scan that ignores active jobs — two racing sends still run concurrently, and it does nothing for stop/retry races. - **`SELECT FOR UPDATE` / Redis SETNX / advisory locks**: all add a second mechanism (transaction plumbing or a second source of truth) to protect a single-row write that Postgres can already do atomically. Path-specific claim predicates fall out naturally: send/drain claim with `pendingQuestionMessageId IS NULL`, retry claims with `lastStreamError IS NOT NULL` (and restores the error if its enqueue fails) — closing the double-retry race for free. ## User impact Double-send (impatient double-click, two tabs, retry racing a queued drain) can currently garble the assistant's answer with interleaved tokens from two model runs and strand one stream's claim. All of these become deterministic: exactly one winner streams; the loser queues politely. ## Test plan - [x] New claim spec: conditional claim before enqueue, race-loser queues, halted-backlog send queues at the back and kicks the drain front-first, enqueue-failure releases the claim - [x] Retry spec updated: rollback restores the prior `lastStreamError`; guarded shapes asserted - [ ] CI green https://claude.ai/code/session_01Lyi6zTema2FMVVh8MD6c38 --- _Generated by [Claude Code](https://claude.ai/code/session_01Lyi6zTema2FMVVh8MD6c38)_ <!-- This is an auto-generated description by cubic. --> <a href="https://cubic.dev/pr/twentyhq/twenty/pull/22481?utm_source=github" target="_blank" rel="noopener noreferrer" data-no-image-dialog="true"><picture><source media="(prefers-color-scheme: dark)" srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source media="(prefers-color-scheme: light)" srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img alt="Review in cubic" src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a> <!-- End of auto-generated description by cubic. -->	2026-07-03 10:40:03 +02:00
Félix Malfait	5057f14df9	test(ai): pin the queue-updated event that now follows stream-error on the failure path (#22500) ... ## Rationale main's `server-test` is red: 4 error-path tests in `stream-agent-chat.job.spec.ts` fail. #22494 made the stream failure path publish `queue-updated` **after** the terminal `stream-error` (so background tabs refetch the persisted partial transcript), but these specs still asserted `stream-error` is the *last* published event. Classic squash-merge semantic conflict — #22494's branch predated the spec assertions, both were green in isolation. ## Why this is the right fix (not patching a symptom) The job behavior is the intended one from #22494; the specs encode the old contract. Rather than loosening the assertions to "a stream-error was published somewhere", this pins the full intended terminal sequence — `stream-error` followed by `queue-updated` — so the convergence event itself is now regression-tested on all four failure paths (mid-stream provider error, setup rejection, persistence failure, missing workspace). ## Impact Unblocks `server-test` / `ci-server-status-check` for every open PR (including #22498, which is needed to fix the dev-cluster migration incident). Test-only change. https://claude.ai/code/session_01Lyi6zTema2FMVVh8MD6c38 --- _Generated by [Claude Code](https://claude.ai/code/session_01Lyi6zTema2FMVVh8MD6c38)_ <!-- This is an auto-generated description by cubic. --> <a href="https://cubic.dev/pr/twentyhq/twenty/pull/22500?utm_source=github" target="_blank" rel="noopener noreferrer" data-no-image-dialog="true"><picture><source media="(prefers-color-scheme: dark)" srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source media="(prefers-color-scheme: light)" srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img alt="Review in cubic" src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a> <!-- End of auto-generated description by cubic. -->	2026-07-02 23:32:49 +02:00
Félix Malfait	027a098099	fix(ai): fully roll back a question answer when the resume enqueue fails (#22492) ... ## Rationale `answerAgentChatQuestion` is a three-step transaction without the transaction: resolve the question (flip tool part to `answered`, clear `pendingQuestionMessageId`, claim the stream), then enqueue the resume job. If the **enqueue fails**, the catch restores only `activeStreamId`. What's left behind: tool part says `answered`, `pendingQuestionMessageId` is `null`, no job will ever run. The client's own error handler rolls its card back to *pending* — so the user sees an answerable question whose re-submission deterministically throws `QUESTION_NOT_PENDING`. The turn is stuck and state is divergent on three surfaces (DB part, DB thread, client). ## Why this is the root cause, not a symptom patch The failure path was rolling back one of three writes. This makes the rollback total and **exact**: `resolvePendingQuestion` now returns the part's precise previous `toolOutput` (no reconstruction guesswork — question tools can carry arbitrary output fields), and the failure path restores the part verbatim plus the thread's pending-question state, guarded on the observed streamId so a competing claim is never clobbered. After rollback, server and client agree again: the question is pending, answering retries cleanly. The audit's alternative — forward recovery (keep the answers, mark the turn interrupted, resume via Retry) — has nicer UX in isolation but contradicts the client's existing rollback-to-pending behavior; matching the established contract wins until the client changes. ## User impact A transient Redis/queue hiccup at answer time currently bricks the question turn permanently. With this, the user sees the question again and can just re-answer. ## Test plan - [ ] CI green - [ ] Manual: fail the enqueue (kill Redis briefly) at answer time → question card returns to pending, re-answer succeeds https://claude.ai/code/session_01Lyi6zTema2FMVVh8MD6c38 --- _Generated by [Claude Code](https://claude.ai/code/session_01Lyi6zTema2FMVVh8MD6c38)_ <!-- This is an auto-generated description by cubic. --> <a href="https://cubic.dev/pr/twentyhq/twenty/pull/22492?utm_source=github" target="_blank" rel="noopener noreferrer" data-no-image-dialog="true"><picture><source media="(prefers-color-scheme: dark)" srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source media="(prefers-color-scheme: light)" srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img alt="Review in cubic" src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a> <!-- End of auto-generated description by cubic. -->	2026-07-02 21:33:20 +02:00
Félix Malfait	087bee0036	fix(ai): notify all tabs when a pending question is answered (#22491) ... ## Rationale `resolvePendingQuestion` updates the question tool-part to `answered` and re-claims the thread — but publishes **nothing**. The answering tab converges via a local browser event; every other tab keeps rendering the question card as interactive until the resumed stream's first chunk happens to arrive. A second tab (or teammate view on shared context) can attempt to answer an already-answered question and hit a confusing `QUESTION_NOT_PENDING` error. ## Why this is the root cause, not a symptom patch Answering a question is a state transition every subscriber cares about — exactly like queue promotion, message persistence, and stream errors, all of which publish. This transition just never did. The fix publishes the existing refetch-trigger event (`queue-updated`, which every tab already handles by refetching messages + thread state) right after resolution — no new event type, no new client code path, consistent by construction with how every other transition converges tabs. A dedicated `question-answered` event carrying the answers would save one refetch round-trip; the audit's verdict was that's over-engineering for a rare interaction. Publishing *before* the resume-enqueue is deliberate: even if the enqueue fails, the question **is** answered server-side, and tabs should reflect server truth. ## User impact Second tabs stop offering an interactive question that will error when submitted; everyone sees the answered state within a refetch instead of whenever the stream resumes. ## Test plan - [ ] CI green - [ ] Manual: two tabs on one thread, answer the question in tab A → tab B's card flips to answered without interaction https://claude.ai/code/session_01Lyi6zTema2FMVVh8MD6c38 --- _Generated by [Claude Code](https://claude.ai/code/session_01Lyi6zTema2FMVVh8MD6c38)_ <!-- This is an auto-generated description by cubic. --> <a href="https://cubic.dev/pr/twentyhq/twenty/pull/22491?utm_source=github" target="_blank" rel="noopener noreferrer" data-no-image-dialog="true"><picture><source media="(prefers-color-scheme: dark)" srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source media="(prefers-color-scheme: light)" srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img alt="Review in cubic" src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a> <!-- End of auto-generated description by cubic. -->	2026-07-02 21:30:34 +02:00
Félix Malfait	d1a4e250cf	fix(ai): converge every tab's transcript when a stream fails with partial output (#22494) ... ## Rationale When a turn fails after emitting output, the partial assistant message **is persisted** — but only the success path publishes the event (`message-persisted`) that makes other tabs refetch. On failure, tabs that weren't watching the live stream keep a stale transcript until manual reload. (Error *visibility* itself already works — `stream-error` reaches every subscribed tab — the gap is purely the persisted-transcript sync. The original gap-analysis framing of this as an error-visibility problem was wrong; this is the corrected scope.) ## Why this is the root cause, not a symptom patch Turn settlement should converge subscribers regardless of *how* the turn settled — success and failure both persist state that tabs need. The failure path now publishes the same refetch-trigger event the rest of the lifecycle uses, right after the terminal `stream-error`. No new event type, no client changes: the existing guarded replay (`firstLiveSeq === null`) already ensures tabs with a live view keep their in-place error rendering while background tabs pick up the persisted partial message and error state. ## User impact Open the same thread in two tabs, have the turn die mid-answer in one: the other tab currently shows the conversation frozen pre-turn until reload. Now both converge to the persisted partial output plus the failed-turn state within one refetch. ## Test plan - [ ] CI green - [ ] Manual: two tabs, kill the provider mid-stream in tab A → tab B shows the partial message + error without reload https://claude.ai/code/session_01Lyi6zTema2FMVVh8MD6c38 --- _Generated by [Claude Code](https://claude.ai/code/session_01Lyi6zTema2FMVVh8MD6c38)_ <!-- This is an auto-generated description by cubic. --> <a href="https://cubic.dev/pr/twentyhq/twenty/pull/22494?utm_source=github" target="_blank" rel="noopener noreferrer" data-no-image-dialog="true"><picture><source media="(prefers-color-scheme: dark)" srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source media="(prefers-color-scheme: light)" srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img alt="Review in cubic" src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a> <!-- End of auto-generated description by cubic. -->	2026-07-02 21:29:56 +02:00
Félix Malfait	a505ed3245	feat(ai): typed CONTEXT_WINDOW_EXCEEDED error that hides the pointless Retry (#22488) ... ## Rationale When message pruning can't fit the conversation into the model's context window, `chat-execution.service.ts` throws a **raw `Error`**. `mapErrorToStreamError` classifies it as generic `STREAM_EXECUTION_FAILED`, so the client renders a standard failure with a **Retry button that deterministically fails again** — the conversation doesn't get shorter by retrying. Users loop on Retry against a permanently-failing thread. ## Why this is the root cause, not a symptom patch The failure is *terminal for the thread by construction*, and the error channel already distinguishes terminal-vs-retryable via typed `AiExceptionCode`s — this failure just never got one. Adding `CONTEXT_WINDOW_EXCEEDED` (typed exception → `UserInputError` mapping instead of a 500 → both error surfaces render the start-a-new-thread message without `onRetry`) puts it on the same rails as `API_KEY_NOT_CONFIGURED` and the other special-cased codes. Both frontend error surfaces route through `AiChatErrorRenderer`, so one case covers the in-message and under-list renderings. The deeper endgame (auto-summarize/compact older turns so threads never brick) is a multi-week feature — and this typed error remains necessary even then, as its terminal fallback. ## User impact Instead of an opaque error and a Retry that never works, users hitting the context limit get told exactly what happened and what to do (start a new thread), and monitoring stops counting a user-condition as a server error. ## Test plan - [ ] CI green - [ ] Manual: fill a thread past the model limit → typed message, no Retry on either error surface https://claude.ai/code/session_01Lyi6zTema2FMVVh8MD6c38 --- _Generated by [Claude Code](https://claude.ai/code/session_01Lyi6zTema2FMVVh8MD6c38)_ <!-- This is an auto-generated description by cubic. --> <a href="https://cubic.dev/pr/twentyhq/twenty/pull/22488?utm_source=github" target="_blank" rel="noopener noreferrer" data-no-image-dialog="true"><picture><source media="(prefers-color-scheme: dark)" srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source media="(prefers-color-scheme: light)" srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img alt="Review in cubic" src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a> <!-- End of auto-generated description by cubic. -->	2026-07-02 21:26:41 +02:00
Félix Malfait	56a6f9419b	chore(server): fix stream-agent-chat job spec formatting breaking main's lint (#22493) ... The stacked merges of #22479 and #22480 left `stream-agent-chat.job.spec.ts` on main failing the `oxfmt --check` gate (2 stray blank lines), which currently fails `server-lint-typecheck` on **every** open PR's merge ref. Two-line whitespace fix, no behavior change. Merging this first unblocks the rest of the queue. https://claude.ai/code/session_01Lyi6zTema2FMVVh8MD6c38 --- _Generated by [Claude Code](https://claude.ai/code/session_01Lyi6zTema2FMVVh8MD6c38)_ <!-- This is an auto-generated description by cubic. --> <a href="https://cubic.dev/pr/twentyhq/twenty/pull/22493?utm_source=github" target="_blank" rel="noopener noreferrer" data-no-image-dialog="true"><picture><source media="(prefers-color-scheme: dark)" srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source media="(prefers-color-scheme: light)" srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img alt="Review in cubic" src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a> <!-- End of auto-generated description by cubic. -->	2026-07-02 21:25:37 +02:00
Félix Malfait	3b76ec528f	fix(ai): route missing-workspace stream failures through the standard error path (#22480) ... ## Rationale When `StreamAgentChatJob` can't find the workspace, it publishes a transient `stream-error` event and **returns before the try/finally exists** (`stream-agent-chat.job.ts`). Consequences on main: - `activeStreamId` is never cleared → every subsequent send in that thread queues behind a dead claim, forever; - no `lastStreamError` is persisted → nothing renders after a reload, and Retry has nothing to retry; - nothing throws → **zero telemetry**. Sentry confirms: the "Workspace not found" issues that exist are all auth/Stripe paths — this path fails in complete silence. ## Why this is the root cause, not a symptom patch The job's catch/finally already implement the correct failure contract for *every other* error: persist a typed `lastStreamError`, publish the typed event, release the claim guarded on the observed streamId. The bug is that one code path bypasses that contract via an early return. The fix removes the bypass — the lookup moves inside the `try` and throws a typed `AiException(WORKSPACE_NOT_FOUND)` — rather than duplicating cleanup in the early-return branch (which would be the symptom patch, and would drift the next time the contract changes). The alternative "prevent the job from existing when the workspace is gone" isn't achievable: workspace deletion between enqueue and pickup is an inherent race, so the job must handle it regardless. ## User impact A workspace deleted/deactivated mid-flight currently bricks the thread silently (the user just sees sends vanish into a queue). With this, the failure is visible (typed error message), recoverable (standard failed-turn state), and observable (real exception in monitoring). ## Stack Based on #22479 (spec harness) — it extends the same spec file with the regression test. `WORKSPACE_NOT_FOUND` is a TypeScript enum member, not a GraphQL schema change: no client-sdk regeneration needed. ## Test plan - [x] Regression test: missing workspace → typed rejection, `lastStreamError` persisted, terminal event published, claim released - [ ] CI green https://claude.ai/code/session_01Lyi6zTema2FMVVh8MD6c38 --- _Generated by [Claude Code](https://claude.ai/code/session_01Lyi6zTema2FMVVh8MD6c38)_ <!-- This is an auto-generated description by cubic. --> <a href="https://cubic.dev/pr/twentyhq/twenty/pull/22480?utm_source=github" target="_blank" rel="noopener noreferrer" data-no-image-dialog="true"><picture><source media="(prefers-color-scheme: dark)" srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source media="(prefers-color-scheme: light)" srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img alt="Review in cubic" src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a> <!-- End of auto-generated description by cubic. -->	2026-07-02 21:11:02 +02:00
Félix Malfait	ca8ab32253	fix(ai): delete the Redis chunk list on credits-exhausted terminal events (#22477) ... ## Rationale The AI chat stream keeps every published chunk in a Redis list (`agent-chat-stream-chunks:<threadId>`, 1h TTL) so late subscribers can catch up. On `message-persisted` the list is deleted. On `credits-exhausted` — the *other* successful terminal event — it wasn't. Any reload/refetch within the TTL replayed the orphaned chunks, flipping the thread into a "streaming" state that no terminal event ever closes: an endless spinner until the user sends another message. **Production evidence (Sentry):** `Billing Credits Exhausted` fired for **290 users / 937 events in 90 days**, ongoing ([TWENTY-SERVER-G42](https://twenty-v7.sentry.io/issues/TWENTY-SERVER-G42)) — every one of those users who reloads the chat within an hour hits this. ## Why this is the root cause, not a symptom patch The chunk list's lifecycle contract is "cleared when the turn settles". `credits-exhausted` resolves the job successfully **without** persisting a `lastStreamError`, so unlike `stream-error` there is no persisted terminator for catchup to replay after the chunks — the replay is unconditionally un-closeable. Deleting on both settle events restores the contract exactly where it's already enforced for `message-persisted`. `stream-error` deliberately keeps the list: the persisted error acts as the replay terminator, letting a reloading client still see the failed turn's partial output. ## User impact Users who hit their billing cap mid-answer (~100/month) no longer come back to a permanently spinning thread after a reload — they see the settled conversation and the billing state. ## Test plan - [x] Unit spec: chunk accumulation with 1-based seq, deletion on both terminal events (`it.each`), retention on `stream-error` - [ ] CI green https://claude.ai/code/session_01Lyi6zTema2FMVVh8MD6c38 --- _Generated by [Claude Code](https://claude.ai/code/session_01Lyi6zTema2FMVVh8MD6c38)_ <!-- This is an auto-generated description by cubic. --> <a href="https://cubic.dev/pr/twentyhq/twenty/pull/22477?utm_source=github" target="_blank" rel="noopener noreferrer" data-no-image-dialog="true"><picture><source media="(prefers-color-scheme: dark)" srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source media="(prefers-color-scheme: light)" srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img alt="Review in cubic" src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a> <!-- End of auto-generated description by cubic. -->	2026-07-02 21:05:19 +02:00
Félix Malfait	6ac8ebcd18	test(ai): pin StreamAgentChatJob stream lifecycle with a reusable spec harness (#22479) ... ## Rationale `StreamAgentChatJob` is the most failure-sensitive path in the AI chat stack — it coordinates the model stream, Redis event publishing, message persistence, the thread's stream claim, and the queued-message flush — and it had **zero unit coverage**. Both historical hangs lived here: - a throw before the model stream merges bypassed `onFinish` entirely and hung the job until the **10-minute BullMQ lock** expired (thread stuck the whole time); - a throw inside `onFinish` (persistence failure) left trailing chunks published with **no terminal event** — the client spinner ran forever. Production still shows this class is live: `Query read timeout` thrown from inside `handleStreamFinish` ([TWENTY-SERVER-GV7](https://twenty-v7.sentry.io/issues/TWENTY-SERVER-GV7)). ## Why this shape, not something else Tests-only PR, zero production risk. The fake chat stream mirrors the one AI SDK contract the job's coordination depends on — verified against the installed `ai@6.0.97` dist: `toUIMessageStream` converts mid-stream errors into error parts and **always** fires `onFinish` when the stream ends (`handleUIMessageStreamFinish` invokes it from both `flush()` and `cancel()`). Pinning that contract in the fake means a future SDK upgrade that breaks it fails these tests instead of production. Six tests pin current behavior: chunk ordering with `message-persisted` last, opaque error-chunk suppression, mid-stream failure persisting `lastStreamError` + releasing the claim, the two hang regressions above, and cancel skipping the queued flush. Three sibling PRs extend this exact spec file (missing-workspace routing, halted queue, and — later — auto-retry), which is why the harness lands first. ## User impact None directly; it makes the two worst historical user-facing hangs (10-minute dead thread, infinite spinner) regression-proof before the stuck-state fix series touches this code. ## Test plan - [x] 6 unit tests, no production code changed - [ ] CI green https://claude.ai/code/session_01Lyi6zTema2FMVVh8MD6c38 --- _Generated by [Claude Code](https://claude.ai/code/session_01Lyi6zTema2FMVVh8MD6c38)_ <!-- This is an auto-generated description by cubic. --> <a href="https://cubic.dev/pr/twentyhq/twenty/pull/22479?utm_source=github" target="_blank" rel="noopener noreferrer" data-no-image-dialog="true"><picture><source media="(prefers-color-scheme: dark)" srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source media="(prefers-color-scheme: light)" srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img alt="Review in cubic" src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a> <!-- End of auto-generated description by cubic. -->	2026-07-02 21:01:32 +02:00
Félix Malfait	4aaf171d63	feat(ai): add ask_questions interactive clarifying-question tool (#22346) ... ## What & why Adds an `ask_questions` tool that lets the in-app **Ask AI** assistant **pause a turn to ask the user one or more multiple-choice questions** (per the [Figma design](https://www.figma.com/design/xt8O9mFeLl46C5InWwoMrN/Twenty?node-id=105959-117153)) and resume once answered — instead of guessing on ambiguous/consequential decisions. The tool is **harness-only**: an interactive question UI is meaningless without a user to answer it, so it must be absent from MCP and from head-less workflow agents. ## Design — true tool-result resume (not a synthetic user message) The user's answer is a **structured tool result bound to the `toolCallId`**, and the **same agent turn resumes** — exactly how Anthropic (`tool_result` by `tool_use_id`) and OpenAI (`function_call_output`) model human-in-the-loop. The naive form of this (leave the tool call in `input-available` to mean "pending") is **impossible** here: `finalizeDanglingToolParts` rewrites `input-available` → `output-error` ("Tool execution was interrupted") on both the persist path (`addMessage`) and the model-reload path (`chat-execution.service.ts`). That util is a load-bearing safety net, so weakening it is the wrong move. Instead: - `ask_questions` is an **inline, chat-only tool with an `execute` that returns a `status: 'pending'` result immediately**, so the tool part is always `output-available` and **immune to `finalizeDanglingToolParts`**. `stopWhen(hasToolCall('ask_questions'))` halts the turn right after the call (the model never sees the placeholder). - A nullable **`thread.pendingQuestionMessageId`** marker records that a turn is awaiting an answer. - The new **`answerAgentChatQuestion`** mutation atomically *claims* the question (clears the marker, marks the thread streaming), **writes the answer onto the same tool part** (`status: 'answered'`), and **re-enqueues the turn via the existing `existingTurnId` plumbing** (`isResume` bypasses the per-turn dedup guard). On resume `finalizeDanglingToolParts` leaves the `output-available` part untouched and `convertToModelMessages` emits `assistant(tool_use)` + `tool_result(answers)`, so the model continues. This achieves the platform-aligned semantics **without** weakening the finalize safety net or inventing a fragile new part state. ### Meets the two requirements - **Survives refresh, scoped per-thread** — the pending state is a normal persisted `output-available` part + the thread marker; the frontend card is derived per-thread from the loaded messages, so it re-appears on reload and only on its own thread. - **Takes priority over the queue** — a unified `isBlocked = activeStreamId || pendingQuestionMessageId` gate is applied in both `sendChatMessage` (new messages queue) and `flushNextQueuedMessage` (the drain). The queue cannot unpile until the question is answered and the resumed turn completes. ### Harness-only by construction `ask_questions` is added **only** to the chat's inline `activeTools` (like `learn_tools`/`execute_tool`/`load_skills`). It never enters the tool registry/catalog, so it is invisible to MCP and to workflow agents — no `MCP_EXCLUDED_TOOL_NAMES` entry needed. ## UX While a question is pending, the **composer is replaced by the question card** (matching the Figma): question title + pager (`1/2`), numbered option rows (`IconSquareNumber*`) with per-option info-icon descriptions and a "Recommended" badge, and the normal composer as the free-text fallback ("Type anything to do differently."). The transcript shows a compact "Asking questions…" status line that becomes an answered summary. ## Changes **twenty-shared** - `ai/types/AskQuestionsToolTypes.ts` — `AskQuestionItem/Option/Answer/Result`, `ASK_QUESTIONS_TOOL_NAME`. **twenty-server** - `ai-chat/tools/ask-questions.tool.ts` — inline tool factory (pending-result `execute`, zod schema, 1–4 questions × 2–4 options). - `chat-execution.service.ts` — add to `activeTools` + `preloadedToolNames`; `hasToolCall` in `stopWhen`. - `chat-system-prompts.const.ts` — when-to-use guidance. - `entities/agent-chat-thread.entity.ts` — `pendingQuestionMessageId` column. - `stream-agent-chat.job.ts` — set the marker on a question pause; bypass the dedup guard on resume; suppress the no-text warning for question pauses. - `agent-chat-streaming.service.ts` — gate `flushNextQueuedMessage`; `enqueueResumeStream`. - `agent-chat.resolver.ts` — gate `sendChatMessage`; `answerAgentChatQuestion` mutation. - `agent-chat.service.ts` — `resolvePendingQuestion` (atomic claim + write answer). - `dtos/agent-chat-question-answer.input.ts`, `ai.exception.ts` (`QUESTION_NOT_PENDING`), `utils/find-pending-question-part.util.ts`. **twenty-front** - `components/AiChatQuestionCard.tsx` — the interactive card (matches Figma tokens) + `__stories__/AiChatQuestionCard.stories.tsx`. - `components/AiChatEditorSection.tsx` — swap the composer for the card while pending. - `components/AiChatQuestionStatusRenderer.tsx` + branch in `AiChatAssistantMessageRenderer.tsx`. - `states/selectors/agentChatPendingQuestionComponentSelector.ts`, `types/AgentChatPendingQuestion.ts`. - `hooks/useSubmitQuestionAnswer.ts` + `utils/markQuestionAnswered.ts` (optimistic) + `graphql/mutations/answerAgentChatQuestion.ts`. A design doc lives at `packages/twenty-server/docs/ASK_USER_QUESTION_TOOL_PLAN.md`. ## Migration Adds a nullable `pendingQuestionMessageId` (uuid) column to `core.agentChatThread`. Needs a generated **fast instance command** (`database:migrate:generate --name addThreadPendingQuestion --type fast`) — see "Verification status". ## Tests - Server: `ask-questions.tool.spec.ts` (pending echo + schema bounds), `find-pending-question-part.util.spec.ts`. - Front: `markQuestionAnswered.test.ts`, plus the Storybook story. ## Verification status (please read) This branch was authored in an environment where the monorepo `yarn install` repeatedly failed on transient TLS resets from the package registry, so I could **not** locally run the mechanical gates. The logic was reviewed by hand and the `ai@6.0.97` exports used (`hasToolCall`, `stepCountIs`, `generateId`) were confirmed against the package's type defs. Still **TODO** (will rely on CI / a follow-up once deps install): - [ ] `nx run twenty-shared:generateBarrels` (the `ai/index.ts` export was added by hand; regen to reconcile) - [ ] `nx run twenty-front:graphql:generate` (new mutation + input type) - [ ] generate the fast instance command (migration) for the new column - [ ] `typecheck` + `lint:diff-with-main` (front + server) — expect minor import-ordering autofixes - [ ] run the unit tests **Screenshots:** reproducing the live flow needs an AI provider API key (to get the model to actually call `ask_questions`), which isn't available here. The card can be screenshotted from its **Storybook story** (`AiChatQuestionCard.stories.tsx`) with no API key — I'll add that image once deps install, or a reviewer can run `nx storybook twenty-front`. 🤖 Generated with [Claude Code](https://claude.com/claude-code) https://claude.ai/code/session_01AArS8H3y3Z1Qwm763xhPLB --- _Generated by [Claude Code](https://claude.ai/code/session_01AArS8H3y3Z1Qwm763xhPLB)_ <!-- This is an auto-generated description by cubic. --> <a href="https://cubic.dev/pr/twentyhq/twenty/pull/22346?utm_source=github" target="_blank" rel="noopener noreferrer" data-no-image-dialog="true"><picture><source media="(prefers-color-scheme: dark)" srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source media="(prefers-color-scheme: light)" srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img alt="Review in cubic" src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a> <!-- End of auto-generated description by cubic. -->	2026-07-02 15:32:18 +02:00
Félix Malfait	d709467902	feat(ai): surface AI chat stream failures through one typed error channel (#22434) ... ## Context Investigating a report where the AI chat showed only a `...` spinner while the network response clearly contained `No AI models are available`. Root cause: terminal stream failures reach the client on **two mismatched channels**. | Representation | Persisted (survives reload) | Rendered by client | |---|---|---| | AI-SDK `error` chunk (inside `stream-chunk`) | ✅ RPUSH'd to Redis | ❌ dropped by `readUIMessageStream` (no message part, no error state) | | typed `stream-error` event | ❌ never persisted | ✅ sets the error atom | Live, the `stream-error` event renders. But on reload, `chatStreamCatchupChunks` replays only the persisted **error chunk** — which the reducer discards — and the streaming indicator never clears. ## Change Collapse to a single typed error contract: - **Suppress the opaque `error` chunk** in the stream job; every failure is surfaced through the typed `stream-error` event. Errors are mapped via `mapErrorToStreamError` so an `AiException` keeps its `AiExceptionCode` (e.g. `API_KEY_NOT_CONFIGURED` → the existing "AI not configured" banner) instead of leaking a raw string. - **Persist the terminal error** next to the accumulated chunks and expose it as an explicit `error { code message }` field on `ChatStreamCatchupChunks`, so a client catching up after a reload recovers it — no dependency on the AI SDK's internal chunk shape. - **Reset per-thread stream state at job start**, so a failed turn's leftover chunks/error never replay on the next stream. - **Client replays the catchup error** as a terminal `stream-error` event, which clears the streaming indicator and renders the error (fixes the infinite spinner on a stream that ended in error). ## Notes - `ChatStreamError` is a new metadata GraphQL type; generated types (twenty-front metadata + client-sdk) were hand-updated to keep the tree consistent and will be reconciled by CI's `graphql:generate` check if anything differs. - Server unit test added for the error mapping. No schema/DB migration. ## Test plan - [ ] With no AI provider configured, send a chat message → error renders immediately (not a spinner). - [ ] Reload the thread → the error still renders (recovered from catchup), indicator not spinning. - [ ] Configure a provider and send again → normal streaming; no stale error from the previous failed turn. <!-- This is an auto-generated description by cubic. --> <a href="https://cubic.dev/pr/twentyhq/twenty/pull/22434?utm_source=github" target="_blank" rel="noopener noreferrer" data-no-image-dialog="true"><picture><source media="(prefers-color-scheme: dark)" srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source media="(prefers-color-scheme: light)" srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img alt="Review in cubic" src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a> <!-- End of auto-generated description by cubic. -->	2026-07-02 14:50:57 +02:00
Paul Rastoin	47689e676b	feat(server): improve traceability of flat-entity map mutation errors (#22396) ... ## Context cc @rashad Twenty applies metadata changes optimistically to in-memory *flat entity maps* before persisting them. The utils that mutate these maps throw `FlatEntityMapsException` on invariant violations, which surface in Sentry (e.g. during `InstallApplication`) as a **hardcoded, generic message with no identifying data**: ``` GraphQLError: addUniversalFlatEntityToUniversalFlatEntityMapsThroughMutationOrThrow: flat entity to add already exists ``` There was no way to know *which* entity collided — making triage impossible. ## What this does (two layers) **Layer 1 — leaf utils emit identifiers** - `FlatEntityMapsException` gains an optional structured `context` (`universalIdentifier` / `id` / `applicationId` / `metadataName` / `relatedMetadataName` / `operation`), read by the Sentry driver's existing `'context' in exception` → `setExtra` channel. - All **9 leaf throw sites** append their in-scope identifiers to the message **and** populate `context`. **Propagation — context survives the re-wraps** - On the install path the collision throws in the (unwrapped) `compute()` step, so the raw exception + context reaches app-sync intact. - For the run/build-phase paths, the migration runner and build-orchestrator re-wraps copy only `.message`; they now also **forward `context`** so structured data survives there too. **Layer 2 — human installation error** - `synchronizeFromManifest` catches flat-entity failures, resolves the offending `universalIdentifier` to a manifest **object/field label**, and rethrows `ApplicationException(APPLICATION_INSTALLATION_FAILED)` with a safe, human `userFriendlyMessage`. - The leaf `userFriendlyMessage` stays `STANDARD_ERROR_MESSAGE` — the detailed message never leaks to end users. - `APPLICATION_INSTALLATION_FAILED` surfaces with the dedicated `ErrorCode.APPLICATION_INSTALLATION_FAILED` GraphQL code (mirroring the workspace-migration runner formatter), not `INTERNAL_SERVER_ERROR`. ### Result — client-facing GraphQL error envelope ```json { "extensions": { "code": "APPLICATION_INSTALLATION_FAILED", "subCode": "APPLICATION_INSTALLATION_FAILED", "userFriendlyMessage": "We couldn't install \"Test Application\". Its Invoice could not be applied to your workspace." }, "message": "Installing application 'Test Application' failed [object: Invoice]: addUniversalFlatEntityToUniversalFlatEntityMapsThroughMutationOrThrow: flat entity to add already exists (universalIdentifier: ...)", "name": "GraphQLError" } ``` ## Where the identifier shows up (not just Sentry) The offending `universalIdentifier` reaches every consumer, not only Sentry: - **Sentry (server):** structured `context` extras + the enriched message (fingerprinted by `code`, so no issue fragmentation). - **GraphQL response `message`:** un-masked (no `useMaskedErrors`; the error-handler hook passes `BaseGraphQLError` through as-is), so it travels over the wire. - **App-author SDK/CLI terminal:** `twenty-sdk` captures `errors[0].message`; for this error `formatManifestValidationErrors` returns `null` (no `extensions.errors`/`summary`), so the orchestrator falls back to printing the full message, e.g.: ``` ✗ Sync failed with error: Installing application 'X' failed [object: Invoice]: … already exists (universalIdentifier: b1b2c3d4-…) ℹ Hint: a metadata conflict was detected. Preview the plan with `yarn twenty dev --once --dry-run`; … ``` The `already exists` / `universalidentifier` substrings also trigger `getSyncErrorRecoveryHint`, so the author gets an actionable next step. - **End-user (CRM UI):** only the safe rendered `userFriendlyMessage` (no UUIDs). ## Design note `userFriendlyMessage` behaviour of the leaf exceptions is intentionally unchanged (guardrail). Layer 2 resolves labels for **objects and fields** (the bulk of metadata); other manifest entity kinds fall back to an app-name-only human message to avoid brittle manifest-walking — easy to extend. A future first-class option would be structured `extensions` (like `METADATA_VALIDATION_FAILED`) + a dedicated SDK formatter; deferred since the message path already surfaces the detail in the terminal. ## Tests - **Unit:** existing through-mutation + runner-exception specs still pass (they assert on exception **code**, not message). Added a spec for the enrichment util. - **Response-format snapshot (verified, green):** `application-exception-filter.spec.ts` runs the exception filter and snapshots the exact client-facing GraphQL error envelope shown above. - **Integration:** `failing-sync-application-flat-entity-map-conflict.integration-spec.ts` syncs a manifest whose two objects share a `universalIdentifier` (collision during manifest map build, before validation) and snapshots the GraphQL error response via `expectOneNotInternalServerErrorSnapshot`. - ⚠️ The integration `.snap` was authored from the identical deterministic path (verified by the filter unit snapshot) because the integration suite couldn't be executed in the authoring sandbox. Please regenerate/confirm with `nx test:integration:with-db-reset` (or `-u`) in a seeded env. ## Status Draft — opening for review.	2026-07-02 12:39:48 +00:00
Félix Malfait	38fbff465f	chore(server): ship the 2.20 standardOverrides drop as a dormant command (#22448) ... Follow-up to #22417, per [this thread](https://github.com/twentyhq/twenty/pull/22417#discussion_r3512187719): migrate the `2-20/README.md` placeholder into a real command using the `TWENTY_NEXT_VERSIONS` mechanism. ### What - Add `DropMetadataStandardOverridesColumnFastInstanceCommand`, registered against `2.20.0`. It boots (`2.20.0` is in `TWENTY_ALL_VERSIONS`) but stays **dormant** — the upgrade sequence only runs `TWENTY_CROSS_UPGRADE_SUPPORTED_VERSIONS` (previous + current), so it never executes during the 2.19 deploy and activates automatically when `nx version:bump` promotes 2.20 to current. - Name constant + unit test (SQL parity, registration against `2.20.0`, name-constant parity). - Register it in `instance-commands.constant.ts`. - Update the `standardOverrides` `@deprecated` comments on object/field metadata to point at the shipped command. - Delete `2-20/README.md`. - Document the "ship a command for a future version" flow in `docs/UPGRADE_COMMANDS.md` and `.cursor/rules/server-migrations.mdc` (the mechanism was previously undocumented). ### Note / correction to the README's plan The old README implied both the command **and** `@WasRemovedInUpgrade` could be added at 2.20 time. Only the command can ship now: the decorator's validator runs against the active sequence, so referencing a still-dormant 2.20 step fails boot with `unknown-step-name`. So the entity keeps its `WasRemovedInUpgrade<T>` type wrapper for now; the decorator gets wired (one line, via the name constant) once 2.20 is current — same deferred-drop shape as `isUIReadOnly`. ### Verification Could not run `jest`/`typecheck`/`lint` in this environment: `yarn install` is blocked by egress policy on a git-based transitive dep (`github.com/electron/node-gyp.git`). Verified by review against the sibling 2-19 add-column and 2-12 drop commands. **Please let CI run before merge.** https://claude.ai/code/session_01KMArJvdEmsX3eAmJLbS1b6 --- _Generated by [Claude Code](https://claude.ai/code/session_01KMArJvdEmsX3eAmJLbS1b6)_ <!-- This is an auto-generated description by cubic. --> <a href="https://cubic.dev/pr/twentyhq/twenty/pull/22448?utm_source=github" target="_blank" rel="noopener noreferrer" data-no-image-dialog="true"><picture><source media="(prefers-color-scheme: dark)" srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source media="(prefers-color-scheme: light)" srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img alt="Review in cubic" src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a> <!-- End of auto-generated description by cubic. -->	2026-07-02 13:18:20 +02:00
Félix Malfait	5a4ebca226	refactor(server): unify the two metadata override mechanisms into one (#22417) ... ## Unify the two metadata override mechanisms into one Twenty had **two** override mechanisms: - **`standardOverrides`** — a bespoke JSONB column on `objectMetadata`/`fieldMetadata` with typed DTOs and a per-locale `translations` map, resolved by two i18n-aware resolvers. - **`OverridableEntity.overrides`** — a flat, registry-driven JSONB blob on view / view-field / view-field-group / command-menu-item / page-layout-tab / page-layout-widget, resolved by a plain spread. This PR collapses them into **one** concept: a single `overrides` blob, one registry-driven overridable set, one i18n-aware read path, and one write path (`computeMetadataOverridesBlob`, extracted in #22404). Object/field **stay on `SyncableEntity`** (not reparented to `OverridableEntity`) so their `isActive` default stays **FALSE** — this sidesteps the `isActive` default conflict entirely. ### GraphQL breaking change (accepted) The `standardOverrides` field is **removed** with no deprecation alias — `overrides` (a `JSON` scalar) is exposed instead on `Object` and `Field`. Product confirmed negligible external usage; the front-end has no hand-written consumer (only generated types), which are regenerated here. ### Commit structure (reviewable commit-by-commit) 1. **Unified resolver + parity harness** — `resolveEffectiveEntityProperty` is a strict superset of the three legacy resolvers; a corpus parity spec compares it against a *frozen reference* of the old logic across every locale, `isStandardApp` branch and override shape. 2. **Registry-driven** — object/field presentation props tagged `isOverridable` + `translatable`; the overridable/translatable sets are derived from the registry (a test asserts they equal the legacy hardcoded lists). 3. **Rename + swap + delete** — `standardOverrides` → `overrides` across entities, DTOs, flat/universal types, producers, the ~12 resolve/write/create/sync call sites, mocks and specs; the reconciler's two compare entries collapse to one; the three legacy resolvers, both DTOs and the hardcoded constants/types are deleted. 4. **Migration (zero-downtime, two-phase)** — split across two releases so a rolling deploy never drops a column a previous-release pod still `SELECT`s: - **2.19 fast** — add the `overrides` column (schema only). - **2.19 slow** — backfill `overrides` from `standardOverrides` in `runDataMigration` (kept out of the schema transaction so the bulk write doesn't hold the ACCESS EXCLUSIVE lock; skipped on fresh installs, which have no data to copy). - **2.20 fast** — drop the legacy `standardOverrides` column (gated by `TWENTY_NEXT_VERSIONS`, so it stays dormant until the instance reaches 2.20). 5. **Front/client-SDK regen** — regenerated metadata GraphQL types. 6. **Integration specs + i18n** — updated the standard object/field update integration specs + snapshots, and the reworded validator message catalog entry. ### Rolling-deploy safety `standardOverrides` is retained through 2.19 and only dropped in 2.20, mirroring the codebase's deferred-drop convention (`isUIReadOnly`/`isCustom`). During the 2.19 rollout both columns exist, so old and new pods coexist without "column does not exist" errors. The backfill lives in a slow `runDataMigration` (per the `no-data-mutation-in-fast-instance-command` rule) so it doesn't stall reads. ### `isActive` guard The migration never reads or writes `isActive`; the backfill asserts the active-row count is unchanged and aborts otherwise. Verified on a real DB: apply + revert preserves the blob **and** the nested `translations` map, with `isActive` counts identical before/after. ### Verification (local) - `nx typecheck twenty-server` + `nx typecheck twenty-front` — green - `nx lint:diff-with-main twenty-server` (oxlint `--type-aware` + oxfmt) — green - `nx test twenty-server` — green (unit + parity + registry + migration tests) - `nx run twenty-server:test:integration:with-db-reset` — green - `database:reset` applies the 2.19 phases and leaves **both** columns present (2.20 drop stays dormant); backfill + revert round-trip verified on a real DB - Metadata integration suites (standard object/field update, application sync) pass end-to-end against the two-column schema - Metadata GraphQL types regenerated against a booted server; zero `standardOverrides` references remain in application code (only the migration commands + the legacy schema baseline) --------- Co-authored-by: prastoin <paul@twenty.com>	2026-07-02 12:01:15 +02:00
Abdul Rahman	3bbc08d41f	refactor(schema): reorganize IndexField and related types (#22439) ... ## Summary Querying `indexMetadatas { indexFieldMetadatas { ... } }` on the `/metadata` GraphQL endpoint fails with a 500: > Nest could not find IndexFieldMetadataDTOAuthorizer element (this provider does not > exist in the current context) The `@CursorConnection('indexFieldMetadatas', ...)` decorator on `IndexMetadataDTO` makes nestjs-query auto-generate a relation resolver that injects an authorizer for `IndexFieldMetadataDTO`. That authorizer is never provided, because the DTO was never registered as a resolver in `IndexMetadataModule` — so the field has been broken since it was introduced in #7162. Since the working, DataLoader-backed `indexFieldMetadataList` field already exposes the same data (and is what the frontend uses), this PR removes the dead connection instead of wiring up the authorizer. ## Changes - Remove `@CursorConnection('indexFieldMetadatas', ...)` from `IndexMetadataDTO` - Regenerate frontend metadata GraphQL types (`twenty-front/src/generated-metadata`) - Regenerate client SDK metadata schema/types (`twenty-client-sdk/src/metadata/generated`) ## Notes - Not a breaking change in practice: the removed field always threw, so no consumer can have been relying on it. Callers now get a standard GraphQL validation error suggesting `indexFieldMetadataList` instead of an internal server error. - Verified locally: the failing query now returns `Cannot query field "indexFieldMetadatas" on type "Index". Did you mean "indexFieldMetadataList"?` and `indexFieldMetadataList` continues to work. Fixes [sonarly issue #54098](https://sonarly.com/issue/54098?type=bug) <!-- This is an auto-generated description by cubic. --> <a href="https://cubic.dev/pr/twentyhq/twenty/pull/22439?utm_source=github" target="_blank" rel="noopener noreferrer" data-no-image-dialog="true"><picture><source media="(prefers-color-scheme: dark)" srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source media="(prefers-color-scheme: light)" srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img alt="Review in cubic" src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a> <!-- End of auto-generated description by cubic. -->	2026-07-02 11:32:39 +02:00
Félix Malfait	2e7441380f	refactor(server): unify metadata override-blob computation (step 1 of override unification) (#22404) ... ## Context Twenty currently has **two override mechanisms** for metadata: - `standardOverrides` (a bespoke JSONB column on `objectMetadata` / `fieldMetadata`) — i18n-aware, typed DTO with a per-locale `translations` map, resolved via `resolve-object/field-metadata-standard-override.util.ts`. - `OverridableEntity.overrides` (base class: `view`, `view-field`, `view-field-group`, `command-menu-item`, `page-layout-tab`, `page-layout-widget`) — a flat, i18n-free `{...entity, ...overrides}` spread, registry-driven via `isOverridable`. "One concept, two code paths → drift & confusion; reconciliation has to special-case." This PR is **step 1** of collapsing them. ## What this PR does (small, behavior-preserving) The add / remove / null-collapse **override-blob write logic was triplicated** across: - `sanitizeOverridableEntityInput` (`overrides`) - `sanitizeRawUpdateObjectInput` (object `standardOverrides`) - `sanitizeRawUpdateFieldInput` (field `standardOverrides`) This extracts it into a single `computeMetadataOverridesBlob` helper that all three now call. This is genuine **cross-mechanism convergence of the write path** — the first concrete reduction of the "two code paths". - Behavior-preserving: same diff semantics. The object/field paths used strict `===` on their string standard-override props; `isEqual` subsumes that for strings, and the overridable path already used `isEqual`. - Type-casts are contained **inside** the one helper; the three call sites stay clean and type-preserving. - 4 files: 1 new util + 3 refactors. ## ⚠️ Draft — verification status I could **not** run `typecheck` / `lint` / tests locally: the sandbox this was authored in cannot complete `yarn install` (network aborts mid-install, no `node_modules`). The change is small and reasoned, but **please let CI validate it** — that's why this is a draft. If CI flags a type/lint nit in the contained casts, it's isolated to `compute-metadata-overrides-blob.util.ts`. Per request: no code comments were added; the design/tradeoff discussion lives here. ## The full unification plan (this PR is step 1) The remaining steps are deliberately **not** in this PR because they need a live DB (migration) and the front-end codegen pipeline to verify — neither is available in the authoring sandbox. Documented here for review before we proceed: | Step | Change | Why staged | |------|--------|-----------| | **(this PR)** | Unify the write-path blob logic | Safe, behavior-preserving, no DB/FE | | Read path | One i18n-aware `resolveEffectiveEntity` (superset of the flat spread + the two i18n resolvers) | The i18n resolvers are entangled with typed translation-key narrowing; merging cleanly needs the storage/i18n generalization below | | Registry | Make object/field presentation props registry-driven (`facet` + `translatable`), like the overridable set already is | Depends on the facet annotation | | Storage | Object/field extend `OverridableEntity`; `standardOverrides` → `overrides` (translations preserved); **one data migration** | Needs DB verification; changes schema | | GraphQL + FE | Remove the `standardOverrides` field, expose `overrides`; regen `twenty-front` / client-SDK types; update the Settings → Data-Model rename UI | Needs codegen; see tradeoff below | ## Key tradeoffs / decisions to confirm 1. **GraphQL break on `standardOverrides` — accepted.** Per product call, external usage is negligible, so the later step will **remove** the field outright (no deprecated alias). The one real consumer is the Settings → Data-Model rename-label UI, updated in the same step. This drops the most complex part of the original plan (a virtual-alias resolver + deprecation window). 2. **`isActive` default.** `OverridableEntity` defaults `isActive` to `true`; object/field default it to `false`. The storage step must **explicitly override the default** and assert in the migration that no existing row's `isActive` changes. 3. **Overrides stay anonymous single-slot blobs** (no per-app attribution / multi-contributor 3-way merge). That limitation is unchanged here and is only worth revisiting if a concrete use case needs owner-tagged layering (real schema work, sized separately). 4. **Parity harness is the safety net for the storage step.** Because the read-path/storage merge touches the hot object/field resolve path and i18n precedence, that PR should land a golden-corpus parity gate (all locales, `isStandardApp`, empty/partial/full overrides) proving the unified resolver reproduces today's output byte-for-byte, before any switch. ## Not included (per request) - No service tests added. - No code comments added (rationale/tradeoffs are here, in the PR). ## Test plan - CI: `typecheck` + `lint` + the existing `sanitize-overridable-entity-input.util.spec.ts` (which exercises the shared logic through `sanitizeOverridableEntityInput`). - The object/field write paths have no dedicated unit spec; they're covered by the metadata integration suites (`successful-update-one-standard-object/field-metadata`). https://claude.ai/code/session_01E1pGBDLC3gEBs1w45G2W5Z --- _Generated by [Claude Code](https://claude.ai/code/session_01E1pGBDLC3gEBs1w45G2W5Z)_ <!-- This is an auto-generated description by cubic. --> <a href="https://cubic.dev/pr/twentyhq/twenty/pull/22404?utm_source=github" target="_blank" rel="noopener noreferrer" data-no-image-dialog="true"><picture><source media="(prefers-color-scheme: dark)" srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source media="(prefers-color-scheme: light)" srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img alt="Review in cubic" src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a> <!-- End of auto-generated description by cubic. -->	2026-07-01 16:58:38 +02:00
Paul Rastoin	2b1417770e	SearchVector derivation via migration-scoped index (alt to #2622 __warmedUpCache) (#22389) ... ## What this is close https://github.com/twentyhq/core-team-issues/issues/2622 A **POC / discussion branch** implementing the runner-scoped alternative to the `__warmedUpCache` design in [core-team-issues#2622](https://github.com/twentyhq/core-team-issues/issues/2622). Not for merge as-is — meant to diff against that plan. ## Problem `deriveSearchVectorAsExpressionForTsVectorField` scans the entire `flatSearchFieldMetadataMaps` (`Object.values(...).filter(...)`) once per object created in a migration. On install that's `O(objectsCreated × totalSearchFields)` — the quadratic #2622 targets. Only **one** of the three call sites is actually hot: - `create-object` (runner) — global maps, called per created object → the quadratic - export DDL — maps already built **per object** (O(k)) - `update-field` rebuild — one field, gated on `rebuildSearchVector` ## Approach Instead of a private `__warmedUpCache` side-channel on `FlatEntityMaps<T>` + drain-on-hydration, this keeps the index in the **consumer**: 1. `derive` now takes `targetSearchFieldMetadatas` (already scoped to the tsVector field) instead of scanning the map itself. 2. The runner builds a `Map<tsVectorFieldMetadataId, searchFields[]>` **once per migration**, lazily, and threads it through the action context. Safe because `searchFieldMetadata` creates are ordered before `objectMetadata` creates (`computeOrderedMigrationActions`), so the map is complete on first use. → `O(totalSearchFields)`. 3. `getTargetSearchFieldMetadatasForTsVectorField` (O(total) filter) stays as the fallback for the one-off callers (export, field-update) and when the accessor isn't provided. ## Why this over `__warmedUpCache` - **No `FlatEntityMaps<T>` type widening**, no convention-only privacy, no id/universalIdentifier drain to keep in sync. - **No referential-integrity obligation.** The index only ever contains entities present in the map, so the "search field created-then-deleted before its object hydrates" case (deferred as an edge in #2622) can't put a stale id into an aggregator and crash `derive` via the `-orThrow` lookup. - **One `derive` path**, not "aggregator + direct-filter fallback for export". - Blast radius: ~220 lines, mostly a new util + test. ## Benchmark (micro, isolated function) Median of 7 trials, 10 search fields per object, running the real shipped utils — old = `getTargetSearchFieldMetadatasForTsVectorField` once per object (identical to the old inline scan), new = `buildSearchFieldMetadatasByTsVectorFieldId` once + N lookups (both assert they resolve the same fields): | objects | total search fields | old (scan/obj) | new (index once) | speedup | |--------:|--------------------:|---------------:|-----------------:|--------:| | 50 | 500 | 2.08 ms | 0.06 ms | 33× | | 100 | 1,000 | 9.26 ms | 0.12 ms | 77× | | 200 | 2,000 | 36.2 ms | 0.23 ms | 160× | | 400 | 4,000 | 151 ms | 0.40 ms | 379× | | 800 | 8,000 | 701 ms | 0.92 ms | 766× | Confirms the old path is quadratic (~4× per doubling of object count) and the new path is linear (sub-ms throughout). **Caveats — read these before trusting the speedup:** - This is the **isolated derivation function**, no DB / DDL / inserts. In a real `create-object` action the derive is a small fraction of per-action cost, so the end-to-end win is far smaller than the ratios above. - A default workspace has ~20–30 objects, where the **old** code already costs only ~1–2 ms total across the whole install. The quadratic only becomes material (>50 ms, the runner's slow-action threshold) around **200–400 objects**. - The measurement that should actually gate this — `[install-perf] create:objectMetadata` on a real install against a real DB with a few hundred objects — has **not** been run yet. The micro-benchmark bounds the upside and locates the knee of the curve; it does not prove end-to-end payoff. ## Not done on purpose - **No end-to-end benchmark yet** — step 0 should still be measuring `[install-perf] create:objectMetadata` on a real large install to confirm the quadratic is worth removing at all. - Relies on the ordering invariant (commented at the build site). The fully self-contained variant is to put the object's search fields on `FlatCreateObjectAction` (builder change) — deliberately left out to keep this runner-scoped. ## Checks `nx typecheck twenty-server`, `nx lint:diff-with-main twenty-server`, new util spec + existing `generate-workspace-schema-ddl` spec all green.	2026-07-01 14:29:19 +02:00
Weiko	fab0358df5	Handle field isNullable update (#22362) ... ## Context Setting isNullable on a field via the app SDK manifest was silently ignored when re-syncing an existing field. The first sync that creates a field honored isNullable correctly, but any later manifest change to isNullable had no effect, neither on the field metadata nor on the underlying Postgres column. Two compounding gaps caused this: The diff never detected the change. isNullable was configured with toCompare: false, so compareTwoFlatEntity excluded it from the diff and no update action was ever generated. There was no DDL to apply it. Even if detected, the update field action handler only altered name, options, defaultValue, and settings. The column manager had no way to alter a column's NOT NULL constraint. ## Fix - Set isNullable.toCompare: true so manifest changes are detected and persisted to the field metadata (via the existing executeForMetadata path). - Add WorkspaceSchemaColumnManagerService.alterColumnNullable(): emits SET NOT NULL / DROP NOT NULL, with an optional pre-serialized backfill (UPDATE … WHERE col IS NULL) applied only on the nullable → non-nullable transition. - Add handleFieldNullableUpdate() to the update field action handler, dispatched after the defaultValue block so the default is in place before NOT NULL is enforced. It is composite-aware (mirrors the per-sub-column parentIsNullable || !property.isRequired rule used at column creation) and skips relation/morph join columns and TS_VECTOR, which are always nullable by design. <!-- This is an auto-generated description by cubic. --> <a href="https://cubic.dev/pr/twentyhq/twenty/pull/22362?utm_source=github" target="_blank" rel="noopener noreferrer" data-no-image-dialog="true"><picture><source media="(prefers-color-scheme: dark)" srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source media="(prefers-color-scheme: light)" srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img alt="Review in cubic" src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a> <!-- End of auto-generated description by cubic. -->	2026-07-01 13:31:22 +02:00
github-actions[bot]	18c0d117a3	chore: sync AI model catalog from models.dev (#22387) ... Automated daily sync of `ai-providers.json` from [models.dev](https://models.dev). This PR updates pricing, context windows, and model availability based on the latest data. New models meeting inclusion criteria (tool calling, pricing data, context limits) are added automatically. Deprecated models are detected based on cost-efficiency within the same model family. **Please review before merging** — verify no critical models were incorrectly deprecated. <!-- This is an auto-generated description by cubic. --> <a href="https://cubic.dev/pr/twentyhq/twenty/pull/22387?utm_source=github" target="_blank" rel="noopener noreferrer" data-no-image-dialog="true"><picture><source media="(prefers-color-scheme: dark)" srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source media="(prefers-color-scheme: light)" srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img alt="Review in cubic" src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a> <!-- End of auto-generated description by cubic. --> Co-authored-by: FelixMalfait <6399865+FelixMalfait@users.noreply.github.com>	2026-07-01 09:23:37 +02:00
Etienne	c4a6446757	fix(navigation-menu-item): reject PAGE_LAYOUT items that don't reference a STANDALONE_PAGE layout (#22343) ... ## Issue A `PAGE_LAYOUT` navigation menu item can be created pointing at a page layout whose type is **not** `STANDALONE_PAGE` (e.g. a `DASHBOARD`). The sidebar always links such an item to `/page/<pageLayoutId>`, but that route only renders `STANDALONE_PAGE` layouts, anything else is redirected to 404. Result: a silently broken sidebar link (cc: https://discord.com/channels/1130383047699738754/1519045990047285288). ## Root cause - `/page/:pageLayoutId` is standalone-only by design (route guard in `usePageChangeEffectNavigateLocation`, and `StandalonePageLayoutPage` hardcodes `layoutType: STANDALONE_PAGE`). Dashboards/record pages are reached elsewhere (record show page). - A `PAGE_LAYOUT` nav item unconditionally computes `/page/<pageLayoutId>`. - No validation ensured the referenced layout is `STANDALONE_PAGE`: the migration/manifest validator only checked that `pageLayoutId` was present, the DB constraint only checked `NOT NULL`, and the runtime tool description even suggested pinning dashboards this way. So an app manifest pairing a `DASHBOARD` layout with a `PAGE_LAYOUT` nav item installed cleanly and produced a dead link. ## Fix (treat as invalid config — fail fast) - Cross-entity validation in `FlatNavigationMenuItemValidatorService` (both create and update): when `type === PAGE_LAYOUT`, resolve the referenced page layout from the optimistic page-layout maps and raise `INVALID_NAVIGATION_MENU_ITEM_INPUT` if its `type !== STANDALONE_PAGE`. Existence keeps being enforced by foreign-key resolution, so the type check only fires when the layout resolves. - Corrected the misleading `create_navigation_menu_item` tool description (no longer says "e.g. a dashboard"; states the target must be a `STANDALONE_PAGE`). - Added unit tests covering: `STANDALONE_PAGE` accepted; `DASHBOARD` rejected; `RECORD_PAGE` rejected; unresolved reference not flagged as a type error. ## Files changed - `flat-navigation-menu-item-validator.service.ts` — new `validatePageLayoutReference` + wired into create/update. - `create-navigation-menu-item.tool.ts` — tool description fix. - `__tests__/flat-navigation-menu-item-validator.service.spec.ts` — new tests (4 passing). ## Out of scope / follow-up - To open discussion, check https://github.com/twentyhq/twenty/pull/22255 <!-- This is an auto-generated description by cubic. --> <a href="https://cubic.dev/pr/twentyhq/twenty/pull/22343?utm_source=github" target="_blank" rel="noopener noreferrer" data-no-image-dialog="true"><picture><source media="(prefers-color-scheme: dark)" srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source media="(prefers-color-scheme: light)" srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img alt="Review in cubic" src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a> <!-- End of auto-generated description by cubic. -->	2026-06-30 11:41:26 +02:00
Paul Rastoin	b0d7516951	Deprecate asExpression from field metadata search_vector (#22287) ... ## Summary Fully deprecates the cached `asExpression` / `generatedType` settings on `TS_VECTOR` (searchVector) fields. Previously the generated-column expression was stored in `FieldMetadataSettings` and kept in sync via imperative recompute side-effects. It is now **derived at DDL time** from the `searchFieldMetadata` rows that describe which fields feed the search vector, making `searchFieldMetadata` the single source of truth and removing a whole class of cache-drift bugs. This is delivered across the milestones tracked in #2587 and coordinates with the frontend migration (#1428). ## Why - The searchVector expression lived in two places (stored `settings.asExpression` + the actual generated column), kept consistent by bespoke side-effects (`recompute-search-vector-on-field-rename`, label-identifier recompute, etc.). - The frontend reconstructed the searchable-fields list by **regex-parsing** the stored `asExpression`. - Both are brittle. Deriving the expression from `searchFieldMetadata` rows at build/run time removes the cache and the parsing. ## What changed ### Server - data model & derivation - Introduce the `tsVectorFieldMetadata` relation on `searchFieldMetadata` (`tsVectorFieldMetadataId` / universal identifier) linking each searchable-field row to its target `TS_VECTOR` field. - New runtime derivation `deriveSearchVectorAsExpressionForTsVectorField` (`flat-search-field-metadata/utils/...`) used by the create-object and update-field handlers to generate the column expression from `searchFieldMetadata` rows. - Remove `asExpression` / `generatedType` from stored settings: `FieldMetadataSettings.TS_VECTOR` is now `null`; the column builder (`generate-column-definitions.util.ts`) hardcodes `generatedType: 'STORED'` and requires the derived expression. - Delete the imperative recompute side-effects and the `compute-search-vector-universal-settings-from-object-manifest` path; drop the `settings` block from all 28 standard `compute-*-standard-flat-field-metadata` utils. ### Server - migration runner - New `rebuildSearchVector` marker on `update-field` actions: the orchestrator synthesizes targeted column rebuilds (`compute-search-vector-rebuild-target-universal-identifiers.util.ts` + the deprioritize aggregator) only when a searchFieldMetadata change or indexed-field rename actually requires it - instead of rebuilding on every settings touch. - Deferrable FKs + in-flight ID resolution so a `searchFieldMetadata` row and its `TS_VECTOR` field can be created in the same transaction (deterministic UUIDs). ### Frontend (contract change, #1428) - New `SearchFieldMetadataDTO` + dataloader exposing `searchFieldMetadataList` on object metadata. - `SettingsObjectSearchSection` now reads `objectMetadataItem.searchFieldMetadatas` instead of parsing `asExpression`; new `SearchFieldMetadataItem` type, fragment, and mapping updates. ### Upgrade commands (2.18) - `2-18-instance-command-fast-...-add-ts-vector-field-metadata-id-to-search-field-metadata` - `2-18-instance-command-fast-...-make-search-field-metadata-fks-deferrable` - `2-18-instance-command-slow-...-backfill-ts-vector-field-metadata-id-on-search-field-metadata` (These were relocated from 2.16 to 2.18 and re-timestamped into an ordered block - add column -> make FK deferrable -> backfill data - since 2.16/2.17 are released.) ### Tests - Updated search-vector side-effect integration specs to assert behavior (search works) rather than the now-removed `asExpression`; removed the obsolete expression-validation specs; refreshed the application-sync snapshot (`universalSettings: null`). ## Upgrade / compatibility notes - Existing workspaces keep their stored `settings` until a later cleanup; nothing reads it anymore. The new derivation drives all DDL going forward. - Schema changes are gated behind the 2.18 instance commands above. ## Known follow-up (separate PR) https://github.com/twentyhq/core-team-issues/issues/2620 - The column rebuild (`DROP`/`ADD` of the `searchVector` STORED column) cascade-drops its GIN index and does not recreate it - a pre-existing regression on `main` inherited here. A follow-up PR will fix the rebuild handler to recreate the GIN index and add a 2.18 workspace command to recompute every search vector + strip the deprecated settings. (Planned.) ## Test plan - [ ] `npx nx typecheck twenty-server` / `twenty-front` - [ ] `npx nx lint:diff-with-main twenty-server` / `twenty-front` - [ ] Server integration: create/update/delete field, rename indexed field, update object - search returns expected records - [ ] Run the 2.18 instance commands on a seeded DB; verify `tsVectorFieldMetadataId` backfilled and FKs deferrable - [ ] Frontend: object Search settings tab lists the correct searchable fields (no `asExpression` parsing) close https://github.com/twentyhq/core-team-issues/issues/2587 <!-- This is an auto-generated description by cubic. --> <a href="https://cubic.dev/pr/twentyhq/twenty/pull/22287?utm_source=github" target="_blank" rel="noopener noreferrer" data-no-image-dialog="true"><picture><source media="(prefers-color-scheme: dark)" srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source media="(prefers-color-scheme: light)" srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img alt="Review in cubic" src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a> <!-- End of auto-generated description by cubic. -->	2026-06-30 09:16:01 +00:00
Félix Malfait	b1a781fbd9	feat(server): resolve app translations across remaining metadata resolvers (#22237) ... ## Summary **PR 3/4** of the app-metadata-translations stack. Extends runtime translation resolution to the remaining Twenty-rendered metadata types so coverage is complete. - New shared `MetadataTranslationResolverService.getApplicationCatalog({ applicationId, workspaceId, locale })` — the single seam for fetching an app's per-locale catalog. - Wired into the **page-layout tab**, **page-layout widget**, **view-field-group**, **command-menu navigation item**, and **view name** resolvers, each extended with an optional `applicationCatalog` param (backward-compatible). Together with PR 1/4 (object + field), this covers all seven translatable metadata surfaces. ## Stack Stacks on #22236 (PR 2/4). Base branch: `claude/app-translation-2-sdk-manifest`. ## Verification note `yarn install` could not complete in the remote dev environment, so typecheck/lint/tests were not run locally — **CI is the source of truth**. https://claude.ai/code/session_01NiE7o3cd3zCLZarVkJa6UA --- _Generated by [Claude Code](https://claude.ai/code/session_01NiE7o3cd3zCLZarVkJa6UA)_ <!-- This is an auto-generated description by cubic. --> <a href="https://cubic.dev/pr/twentyhq/twenty/pull/22237?utm_source=github" target="_blank" rel="noopener noreferrer" data-no-image-dialog="true"><picture><source media="(prefers-color-scheme: dark)" srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source media="(prefers-color-scheme: light)" srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img alt="Review in cubic" src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a> <!-- End of auto-generated description by cubic. -->	2026-06-29 22:49:36 +02:00
Etienne	56deba351b	feat(ai): reliable bulk data import via code-interpreter (#22209) ... ## Summary Makes AI-assisted bulk data import (CSV/Excel/spreadsheets) reliable and token-efficient by letting an entire import run inside a single code-interpreter call, with a persistent sandbox session and server-side bulk helpers. Also includes supporting improvements to attachment handling, upsert reporting, and field-permission error messages. ## Changes ### Code interpreter - **Persistent per-session kernel** in `LocalDriver`: a long-lived Python process per `sessionId` keeps variables, imports, and files alive across calls (matching E2B behavior). Falls back to the existing ephemeral per-call path when no session is provided. CAN BE REMOVED, INTERESTING FOR DEV X - Idle watchdog that self-terminates the kernel, configurable via the new `CODE_INTERPRETER_IDLE_TIMEOUT_MS` config variable; the process also exits on parent shutdown (EOF on control fd). CAN BE REMOVED, INTERESTING FOR DEV X - New `bulk_upsert` and `lookup_by` helpers on the sandbox `twenty` object for idempotent batched writes (≤200/batch) and bounded relation-ID resolution. ### Records - `upsert_many_*` now reports a `created` / `updated` / `total` split in its result and log line (new `isFreshlyCreatedRecord` util). ### AI chat - `replaceUnsupportedFileParts`: user-attached files whose MIME type the model can't handle natively (and that aren't code-interpreter-supported) are downgraded to a descriptive text note instead of being sent as unsupported file parts. Modality→MIME mapping drives native support detection. - Finalize dangling tool parts before `convertToModelMessages` to avoid malformed model messages. - Extracted shared types/constants for code-interpreter file extraction. ### Permissions - Field permission-denied exceptions now include the field name and entity name for easier debugging. ### Skill docs - Added the bulk-import recipe ## To do in following PR - [ ] Skill command migration ## Test plan - [x] Unit tests for `getNativeMimeTypesForModalities` and `replaceUnsupportedFileParts` pass - [x] Run a bulk import (>50 rows) end-to-end through the code interpreter and verify a single sandbox call handles read → resolve relations → upsert → summary - [x] Verify session persistence: define a variable in one call, use it in the next within the same session - [x] Verify the kernel self-terminates after `CODE_INTERPRETER_IDLE_TIMEOUT_MS` - [x] Verify unsupported attachments are replaced with a text note for models lacking the modality - [x] Verify `upsert_many_*` returns correct created/updated counts - [x] Verify field-restricted role triggers a permission error naming the field and entity - [ ] Test with [hotel_business.xlsx](https://github.com/user-attachments/files/29376307/hotel_business.xlsx) and simple "import record" prompt <!-- This is an auto-generated description by cubic. --> <a href="https://cubic.dev/pr/twentyhq/twenty/pull/22209?utm_source=github" target="_blank" rel="noopener noreferrer" data-no-image-dialog="true"><picture><source media="(prefers-color-scheme: dark)" srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source media="(prefers-color-scheme: light)" srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img alt="Review in cubic" src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a> <!-- End of auto-generated description by cubic. -->	2026-06-29 09:54:27 +00:00
Félix Malfait	41c10b9ee7	feat(server): resolve app-owned metadata translations at runtime (#22235) ... ## Summary First of a **4-PR stack** that lets apps built with `twenty-sdk` translate their metadata, resolved at runtime. The standard Twenty app is modelled as "an app like any other" — `NULL applicationRegistrationId` ⟺ the standard app, no special-casing. This PR adds the server foundation and wires runtime resolution for **object** and **field** metadata: - New `applicationTranslation` core table + entity (nullable `applicationRegistrationId`, `locale`, `messages` jsonb), one row per (app, locale) to avoid multi-MB rows. - `ApplicationTranslationCacheService` (process-local, 30s TTL) + `ApplicationTranslationSyncService` (upsert + soft-delete from a manifest). - Shared `translateStandardLabel` util: application catalog → i18n bundle → source value. - Object/field resolvers + dataloaders prefetch and apply the per-app catalog. The new `applicationCatalog` param is **optional**, so standard behaviour is byte-unchanged. - Fast instance command to create the table. ## Stack **PR 1/4**, targets `main`. Followed by: (2) twenty-sdk extract/compile → `manifest.translations`, (3) resolution across the remaining metadata resolvers, (4) the per-locale standard-override editor. ## Tests Unit: `translateStandardLabel`, `resolveObjectMetadataStandardOverride` (including the application-catalog path). ## Verification note The remote dev environment for this branch could not complete `yarn install` (no package-registry egress), so typecheck/lint/tests were not run locally — **CI is the source of truth** for this stack. Changes follow existing patterns. https://claude.ai/code/session_01NiE7o3cd3zCLZarVkJa6UA --- _Generated by [Claude Code](https://claude.ai/code/session_01NiE7o3cd3zCLZarVkJa6UA)_ <!-- This is an auto-generated description by cubic. --> <a href="https://cubic.dev/pr/twentyhq/twenty/pull/22235?utm_source=github" target="_blank" rel="noopener noreferrer" data-no-image-dialog="true"><picture><source media="(prefers-color-scheme: dark)" srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source media="(prefers-color-scheme: light)" srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img alt="Review in cubic" src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a> <!-- End of auto-generated description by cubic. -->	2026-06-28 07:36:21 +02:00