de75be16e2d4e737075909e70f1bd84f260d22df
1148 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Paul Rastoin
|
de75be16e2 |
harden(server): backfill and enforce workspace.databaseSchema invariant with a check constraint (#22855)
## What & why `core.workspace.databaseSchema` is meant to be set for every workspace past the creation phase. It only started being written at creation time in 2.x (dual-write since 2026-03-28, direct write since 2026-04-10); older workspaces relied on the `1-21 backfill-datasource-to-workspace` instance command, which never effectively ran on some instances. On affected rows the column could be left `NULL`. A null value on a post-creation workspace is a real integrity problem — several paths trust the column: - **REST API**: `hydrateRestRequest` throws `No data sources found` for authenticated requests. - **GraphQL API**: `getOrComputeSchemaSDL` returns `null`, so `WorkspaceSchemaFactory` hands back an empty schema. - **GraphQL introspection** (direct execution) returns `null`. This PR makes the invariant impossible to silently violate, and repairs any instance still lagging. ### On the original "No data source, skipping" logs This investigation started from `BackfillActorSourceEnumValuesCommand` logging `No data source for workspace <id>, skipping` at high volume. **That symptom is not explained by this change, and this PR is not a fix for it.** Findings: - The workspace iterator only processes `ACTIVE` + `SUSPENDED` workspaces, and on the affected instance all of those already have `databaseSchema` set (only `PENDING_CREATION` rows are null, and those are never iterated). - `getGlobalWorkspaceDataSource()` never resolves to `undefined` (it returns a value or throws), so a defined-schema workspace should never hit the skip branch. - The upgrade-aware repository proxy was investigated as a possible cause (it can short-circuit `findOne` to `null` for entities marked unavailable during an upgrade) and **exonerated**: `WorkspaceEntity` and its `databaseSchema` column carry no `@WasIntroducedInUpgrade`/`@WasRemovedInUpgrade` decorators, so `resolveEntityShapeAtUpgradeCursor` always reports the entity available and the column visible at every cursor. In other words, current code should emit zero such skips for that instance's data, so the root cause of the observed logs remains undetermined and is tracked separately. See twentyhq/core-team-issues#2666. ## Changes - **Check constraint `workspace_requires_database_schema`** (the core of this PR): enforces `databaseSchema IS NOT NULL` for any workspace past creation (`activationStatus NOT IN ('PENDING_CREATION', 'ONGOING_CREATION')`). Declared on `WorkspaceEntity` and applied in the slow instance command's `up()`. Safe against the creation flow: `databaseSchema` is written in `WorkspaceManagerService.init` (right after schema creation) long before a workspace becomes `ACTIVE`. - **Defensive backfill** (`2-21` slow instance command): repopulates `databaseSchema` where it is `NULL`/empty, deriving the schema name deterministically from the workspace id (`getWorkspaceSchemaName`) and only setting it for workspaces whose schema actually exists in `information_schema.schemata` (so `PENDING_CREATION` rows without a provisioned schema are left untouched, and stay exempt via the constraint). No-op on instances already backfilled. - `runDataMigration` runs before `up()`, so the backfill repairs legacy rows before the constraint is enforced. Keeping both in the same slow command (rather than a standalone fast command) guarantees the constraint is never added ahead of the repair. - `checkSchemaExists` gets an explicit `: Promise<boolean>` return type. ## Notes - Backfill + constraint live in a **slow** instance command, so they only apply on upgrades run with `--include-slow`. - The constraint is added **`NOT VALID`**: the backfill repairs every workspace whose Postgres schema exists, but some legacy active/suspended workspaces (e.g. carried over from very old versions, as reproduced by the cross-version upgrade from v1.22) have a null `databaseSchema` with no schema to point at and are unrepairable. `NOT VALID` enforces the invariant on all future inserts/updates without failing the upgrade on that pre-existing corruption. - No production request path was changed — the iterator and `checkSchemaExists` keep trusting the (now backfilled + constrained) column. ## Test plan - [ ] Run `database:migrate:prod --include-slow` on an instance with null `databaseSchema` rows; verify rows whose schema exists get backfilled and `PENDING_CREATION` rows are left null. - [ ] Verify the `workspace_requires_database_schema` constraint exists on `core.workspace` and rejects nulling `databaseSchema` on an active workspace. - [ ] Verify a fresh workspace creation still succeeds (constraint does not fight the `PENDING_CREATION` → `ACTIVE` transition). |
||
|
Paul Rastoin
|
1b168ac1f7 |
fix(server): gate workspaceDiscoverability behind upgrade decorator (#22818)
## Context Needs to be patched on 2.20, will craft a 2.19 equivalent with fallback asap ( be it won't be merged unlike this one ) Fixes #22662. Follow-up to #22423, which introduced `workspaceDiscoverability`. A clean 2.18.x to 2.19 upgrade breaks on login with: ``` QueryFailedError: column workspaceDiscoverability does not exist ``` `workspaceDiscoverability` was added to `WorkspaceEntity` (in #22423) as a plain, always-selected, non-nullable column, so any workspace query (including the auth-path `findAvailableWorkspacesByEmail` lookup) fails as soon as the ORM selects it, before the `2.19` upgrade command that creates the column has run. Because the Docker entrypoint is fail-open, the API starts even if the upgrade is delayed, and users hit this on their first login. ## Changes - Add `@WasIntroducedInUpgrade` to `workspaceDiscoverability`, referencing the existing `2.19.0` fast instance command that creates the column. The upgrade-aware ORM then skips the column until the command has actually added it, keeping login working during the upgrade. - Keep the GraphQL `@Field` non-nullable and add a `workspaceDiscoverability` `@ResolveField` that falls back to `WorkspaceDiscoverability.PUBLIC` while the column is hidden, so the resolver never returns `null` for the non-nullable field during the upgrade window. This mirrors the existing pattern already applied to `FileEntity.status` and `FileEntity.applicationRegistrationId`, and the resolver-default pattern already used for `fastModel` / `smartModel` / `logo`. ## Cherry-pick This fix needs to be cherry-picked onto both the **2.19** and **2.20** release branches, since affected instances are upgrading into those versions. ## Test - `validate-upgrade-aware-entity-decorators` and `resolve-entity-shape-at-upgrade-cursor` unit tests pass (the referenced upgrade command name resolves correctly). - `upgrade-aware-repository.proxy` and `upgrade-aware-entity-metadata.adapter` specs pass. - `typecheck` and lint pass for `twenty-server` and `twenty-front`. - Regenerating the GraphQL schemas produces no diff (the field stays non-nullable). |
||
|
Paul Rastoin
|
652adc3c03 |
fix(server): backfill isSystemSideEffect on system fields provisioned before 2.15 (#22850)
## Context The `isSystemSideEffect` column was introduced in **2.15** via a fast instance command that added it with `DEFAULT false`. That stamped `false` onto every pre-existing `fieldMetadata` row — including the 8 engine-owned system fields (`id`, `createdAt`, `updatedAt`, `deletedAt`, `createdBy`, `updatedBy`, `position`, `searchVector`) of every object provisioned before 2.15, regardless of the creation path (API metadata **and** manifest sync). The per-workspace backfill that should have re-flagged those existing rows was explicitly deferred as out of scope in #21673 ("PR 2") and never shipped for `fieldMetadata`. Because `isSystemSideEffect` is configured with `toCompare: false`, no later sync ever repaired the stale value either. Since **2.20** the SDK no longer declares system fields in manifests. On an up-to-date instance, `twenty plan` against an unchanged app therefore diffs those stale-`false` system fields as **missing from the manifest**, and they fall through the `isSystemSideEffectFlatEntity` exclusion in `buildAllFlatEntityOperationRecordByMetadataNameFromFromTo`. Deletion inference then emits them as deletes, which the validator rejects: ``` Sync failed with 144 errors fieldMetadata: 144 errors 1..144. FIELD_MUTATION_NOT_ALLOWED: System fields cannot be deleted ``` (144 = 8 system fields × 18 custom objects, as reported on a production 2.20 instance.) ## What this PR does Adds a **2.21 workspace command** (`upgrade:2-21:backfill-system-field-is-system-side-effect`) that iterates active/suspended workspaces and flags the 8 system fields as `isSystemSideEffect: true`. - **Resolution by deterministic universal identifier**: for each object × reserved system field name it recomputes `getFieldUniversalIdentifier(applicationUID, objectUID, name)` and looks the row up in the flat maps. This is safe (and preferable to matching by `name`) because the 2.19 backfill already took over system field UIDs for every application, so an author-declared field reusing a reserved name keeps its own identifier and is never touched. An extra `isSystem` guard warn-and-skips any mismatch. - **All applications** are covered (installed apps, workspace custom app, twenty-standard): the stale flag is a function of *when* a row was provisioned, not *how*. Installed/custom apps are the acute `twenty plan` delete trap; twenty-standard has no trap today but flagging is a zero-diff no-op (`toCompare: false`) and a prerequisite for the end-state ownership invariant. - **`name` is intentionally excluded**: the 2.20 slow instance command deliberately flipped it to `false` (caller-provided default, not engine-owned); re-flagging it would undo that migration. - Supports `--dry-run`, updates only the collected rows, and invalidates the `flatFieldMetadataMaps` workspace cache after the write (a raw repository update does not invalidate it). ## Related - Resolves the pre-2.15 regression tail of twentyhq/core-team-issues#2635 - Follow-up to twentyhq/core-team-issues#2642 (system field side-effect engine migration) <!-- This is an auto-generated description by cubic. --> <a href="https://cubic.dev/pr/twentyhq/twenty/pull/22850?utm_source=github" target="_blank" rel="noopener noreferrer" data-no-image-dialog="true"><picture><source media="(prefers-color-scheme: dark)" srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source media="(prefers-color-scheme: light)" srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img alt="Review in cubic" src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a> <!-- End of auto-generated description by cubic. --> --------- Co-authored-by: Weiko <corentin@twenty.com> |
||
|
Thomas Trompette
|
c1b62334b7 |
fix(workflow): scope one-active-per-workflow index to workspace (#22795)
## Problem
The Phase 0 core index \`IDX_WORKFLOW_VERSION_ONE_ACTIVE_PER_WORKFLOW\`
is on \`(workflowId) WHERE status='ACTIVE'\`, with **no
\`workspaceId\`**. But \`core.workflowVersion\` is a shared multi-tenant
table, so this enforces "one active version per workflowId **globally
across all workspaces**" instead of per workspace.
The version backfill fails on staging with:
\`\`\`
duplicate key value violates unique constraint
"IDX_WORKFLOW_VERSION_ONE_ACTIVE_PER_WORKFLOW"
Detail: Key ("workflowId")=(8b213cac-...) already exists.
\`\`\`
across several different workspaces that share the same workflowId
(seeded/cloned data): workspace A's active version claims the
workflowId, and every other workspace's insert collides. Every other
index on this table includes \`workspaceId\`; this one dropped it when
copied from the per-tenant workspace entity.
## Fix
Index becomes \`(workspaceId, workflowId) WHERE status='ACTIVE'\` — one
active version per workflow **per workspace**, matching the table's
multi-tenant design and the intended invariant. New 2-20 fast instance
command drops and recreates the index (Phase 0's command is
merged/append-only).
## Test
Reset + reproduce the exact scenario against the fixed index:
- two workspaces with the same workflowId, both ACTIVE → **insert
succeeds** (previously collided)
- a second ACTIVE version for the same workflow within one workspace →
**still blocked** (invariant preserved)
Zero \`migrate:generate\` drift, typecheck + lint clean. After this
deploys, re-run \`upgrade:2-20:backfill-workflow-version-to-core\`.
<!-- This is an auto-generated description by cubic. -->
<a
href="https://cubic.dev/pr/twentyhq/twenty/pull/22795?utm_source=github"
target="_blank" rel="noopener noreferrer"
data-no-image-dialog="true"><picture><source
media="(prefers-color-scheme: dark)"
srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source
media="(prefers-color-scheme: light)"
srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img
alt="Review in cubic"
src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a>
<!-- End of auto-generated description by cubic. -->
|
||
|
martmull
|
23cae2040a |
Improve application asset management (#22564)
App manifests could point the logo and screenshots at either external
URLs or public folder paths, and that was handled inconsistently across
install, sync and the marketplace.
This makes assets always bundled files:
- Manifests now use `logo` and `galleryImages` (a `string[]` of public
folder paths) instead of `logoUrl` and `screenshots`. The old fields
still work but are deprecated. Gallery order comes from the array index.
Normalization (deprecated-field migration, and warning about + ignoring
external URLs) happens in `defineApplication`, so the warnings surface
at define time.
- Logo is stored as a File record (`logoFileId`).
- The registration gallery is configured via a `settings` jsonb column
on `applicationRegistration` (`{ galleryImages: string[] }`) — populated
from the manifest, read by the marketplace detail (falling back to the
legacy `screenshots` column, then the manifest). No dedicated gallery
table.
- The marketplace detail DTO and front now use `galleryImages`.
Verified against a local Postgres: the fast instance commands run with
no pending-migration diff, the schema is correct, and the server boots.
Typecheck, lint, codegen and the application unit tests pass.
Not included yet: rehosting assets into storage for npm catalog and
tarball registrations, versioned cache busting on the serving route, and
a backfill for existing installs.
<a
href="https://cubic.dev/pr/twentyhq/twenty/pull/22564?utm_source=github"
rel="nofollow noreferrer noopener" target="_blank">``<img alt="Review
in cubic"
src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg">``</a>
|
||
|
Thomas Trompette
|
b327ab09a7 |
feat(workflow): add universalIdentifier + applicationId to core workflowVersion (#22747)
## workflowVersion core: syncable columns Adds `universalIdentifier` + `applicationId` (nullable) to `core.workflowVersion`, plus the FK to `core.application` and the `(workspaceId, universalIdentifier)` unique index, via a 2.20 add-columns fast command gated with `@WasIntroducedInUpgrade`. Nullable for now: the already-merged Phase A backfill (#22663) inserts version rows without these columns, so `applicationId` can't be NOT NULL yet. Flipping to NOT NULL + `extends SyncableEntity` comes once they're populated (backfill + dual-write follow-ups). Schema captured and verified via `migrate:generate` (zero drift). Independent of the core-workflow PR, but both add 2.20 upgrade commands, so this one (ts `…480`) must merge **after** the core-workflow PR (ts `…479`), or it gets re-timestamped on rebase. <!-- This is an auto-generated description by cubic. --> <a href="https://cubic.dev/pr/twentyhq/twenty/pull/22747?utm_source=github" target="_blank" rel="noopener noreferrer" data-no-image-dialog="true"><picture><source media="(prefers-color-scheme: dark)" srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source media="(prefers-color-scheme: light)" srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img alt="Review in cubic" src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a> <!-- End of auto-generated description by cubic. --> |
||
|
Thomas Trompette
|
6210389221 |
feat(workflow): add core workflow entity (syncable) + create-table (#22746)
## Core `workflow` entity (syncable) Part of the app-workflows work. Adds a core `WorkflowEntity extends SyncableEntity` (`name`, `lastPublishedVersionId`, plus `universalIdentifier`/`applicationId`/workspace from the base class) and its 2.20 create-table fast command, gated with `@WasIntroducedInUpgrade`. Schema was captured and verified via `migrate:generate` (zero drift, FK/index hashes correct), then run against a live DB. Backfill (populate from workspace `workflow` records) and the dual-write listener land in follow-ups. Independent of the version-syncable-columns PR, but note: both add 2.20 upgrade commands, so this one (ts `…479`) must merge **before** the version PR (ts `…480`) to satisfy the append-only guard. <!-- This is an auto-generated description by cubic. --> <a href="https://cubic.dev/pr/twentyhq/twenty/pull/22746?utm_source=github" target="_blank" rel="noopener noreferrer" data-no-image-dialog="true"><picture><source media="(prefers-color-scheme: dark)" srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source media="(prefers-color-scheme: light)" srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img alt="Review in cubic" src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a> <!-- End of auto-generated description by cubic. --> |
||
|
Paul Rastoin
|
60fd322b49 |
Centralize system field side effects + search field metadata (#22594)
## Introduction Closes twentyhq/core-team-issues#2635 and twentyhq/core-team-issues#2642 and twentyhq/core-team-issues#2589 Object system fields (`searchVector` + its GIN index + `searchFieldMetadata`, the reserved system fields, default relations) were provisioned through several scattered, path-specific code paths. As a result the **app-manifest sync path** authored objects with an empty/`NULL` `searchVector` and **zero `searchFieldMetadata`**, so app-owned objects shipped a broken generated search column (see #22657). The generation logic also lived partly in imperative services rather than in the metadata side-effect engine, and relied on non-deterministic (`v4`) universal identifiers that `twenty apply` could not converge, destroying manually backfilled rows. This PR centralizes every object-creation system side effect into the **metadata side-effect engine**, extends the engine to keep search metadata consistent on field delete and object relabel, makes the standard app's search identifiers deterministic, and ships upgrade commands to reconcile existing workspaces. ## What changed ### Side effects moved into the metadata side-effect engine New dedicated, self-contained handlers — so every write path (API and app manifest) gets identical results, and side effects never trigger other side effects. **Object create / delete** (`handlers/object-metadata`) * **`objectSystemFieldsOnCreate`** — generates the 7 reserved system fields (`id`, `createdAt`, `updatedAt`, `deletedAt`, `createdBy`, `updatedBy`, `position`). * **`objectSearchVectorOnCreate`** — provisions the full-text search surface as one unit: the `searchVector` `TS_VECTOR` field, its backing GIN index, and the `searchFieldMetadata` row (for searchable objects whose label identifier is a searchable field) that keeps `searchVector` populated instead of `NULL`. * **`objectSystemSideEffectsOnDelete`** — tears the above down on object deletion. **Search-metadata consistency on relabel / field delete** (new — these are what close the manifest-path gaps) * **`objectSearchVectorOnUpdate`** (`handlers/object-metadata`) — when a searchable object is relabeled onto a new searchable field, provisions the `searchFieldMetadata` row that indexes it. Relabeling is **additive**: existing rows (e.g. the provisioned `name` row) are preserved, so the previous label identifier stays searchable. Mirrors the API update path so a manifest re-sync that changes the label identifier reaches search parity. No-ops for junction objects (`id` label identifier) and non-searchable field types. * **`fieldSearchFieldMetadataOnDelete`** (`handlers/field-metadata`) — when a field is deleted, cascade-deletes every `searchFieldMetadata` row that indexes it. `searchFieldMetadata` is excluded from manifest deletion inference, so this explicit cascade is what covers **both the API and manifest paths** (the object-scoped DB cascade only fires on object deletion). Uses the `searchFieldMetadataUniversalIdentifiers` aggregator on the flat field for an O(k) lookup instead of scanning all rows. The **default `name` field and default relations are now caller-provided default fields** (SDK autocomplete on the manifest path, input transpiler on the API path) rather than system side effects — removing duplicate name generation, the imperative `build-default-*-for-custom-object` utilities, and the ad-hoc system-field integrity validator. ### Deterministic identifiers for the standard app The twenty-standard search GIN index and `searchFieldMetadata` now derive deterministic universal identifiers (`getIndexUniversalIdentifier` / `getSearchFieldUniversalIdentifier`) instead of `v4`, so `twenty apply` converges instead of recreating. ### Upgrade commands (`2-20`) to reconcile existing workspaces **Instance commands** (run once per instance; ordered fast → slow → workspace): 1. **`AddIsSystemSideEffectToSearchFieldMetadata`** (fast) — adds the `isSystemSideEffect` column to `core.searchFieldMetadata`. Defaults to `true`, which also correctly backfills every existing row since `searchFieldMetadata` is always system-derived (never user-authored). 2. **`BackfillNameFieldIsSystemSideEffect`** (slow) — re-flags existing `name` fields from `isSystemSideEffect: true` → `false`, since the default `name` field is now a caller-provided default like any other user-owned field (it was provisioned as `true` in 2.15 → 2.19). This is a pure data backfill, so the bulk `UPDATE` lives in `runDataMigration()` rather than `up()` — keeping it out of the fast schema transaction avoids holding an `ACCESS EXCLUSIVE` lock that could stall reads during the deploy. Slow instance commands still run before every workspace command of the version, so the fresh value is in place before the search-reconcile workspace commands recompute the `fieldMetadata` flat-entity cache. Scoping by name alone is safe (no engine-owned field is named `name`); `down()` is best-effort (pre-2.15 `false` rows are indistinguishable from flipped ones). **Workspace commands** (idempotent, dry-run supported): 1. **`reconcile-search-vector-gin-index-universal-identifier`** — re-owns every searchVector GIN index UID to its deterministic value (all applications), then backfills the missing GIN index for installed-app objects. 2. **`reconcile-search-field-metadata`** — re-owns every `searchFieldMetadata` UID (all applications), then backfills the missing rows for installed-app searchable objects. 3. **`rebuild-installed-app-search-vectors`** — rebuilds the `searchVector` column of every installed-app `TS_VECTOR` field, once the index and rows exist. Design notes: * **Re-own is global** (twenty-standard, workspace-custom, installed) — a UID convergence keyed on each row's own application. * **Backfill is installed-app only** — standard/custom objects already have these rows via the manifest funnel. * Re-own runs **before** backfill and is transaction-guarded; a failure aborts that workspace to avoid a unique-identifier collision. ## Tests * Integration: app manifest sync now asserts system fields + searchable objects (searchVector, GIN index, searchFieldMetadata) are created; a new relabel suite drives three manifest syncs and asserts records stay searchable through the old + new label identifiers and lose searchability when a field is removed; removed the obsolete system-fields-integrity suite/snapshots. * Unit: per-handler side-effect specs (including the new `objectSearchVectorOnUpdate` and `fieldSearchFieldMetadataOnDelete` handlers), and per-util specs for the re-own / backfill operation builders and the GIN-index classifier. ## Upgrade / migration notes * Existing workspaces converge on the next upgrade run via the `2-20` instance + workspace commands (idempotent, dry-run supported). * Backfill and rebuild go through the workspace-migration runner (automatic cache invalidation); the re-own step invalidates only the affected flat-entity maps directly. * The cross-version upgrade CI now flushes the cache before running the upgrade, so the new version recomputes every flat-entity map from the database instead of reading blobs the old version serialized in an older shape. ## Follow-up * `object-metadata.service.ts` still carries a `TODO: remove once default view fields move to the metadata side effect engine` — default view fields are the next candidate to move into the engine. * A single manifest sync cannot yet both create a field and relabel the object onto it, because `objectMetadata.update` is ordered before `fieldMetadata.create` in the migration runner. Tracked in twentyhq/core-team-issues#2655; to be fixed in a follow-up. |
||
|
Thomas Trompette
|
ca90a9358f |
feat(workflow): backfill workspace workflowVersion into core (phase A) (#22663)
## workflowVersion -> core, Phase A Follows #21674 (Phase 0, merged). Base: `main`. Populates core `workflowVersion` and keeps it in sync with the workspace object, so a later phase can switch reads to core. Reads stay on the workspace object in this PR. ### 1. Backfill (upgrade command) `BackfillWorkflowVersionToCoreCommand`, a `@RegisteredWorkspaceCommand('2.20.0', ...)`. Per workspace, reads all workspace `workflowVersion` records and upserts them into core, preserving ids (idempotent), dry-run aware. ### 2. Dual-write (always on, not flag-gated) `WorkflowVersionCoreDualWriteListener` hooks `@OnDatabaseBatchEvent('workflowVersion', CREATED/UPDATED/DELETED)` (same mechanism as the existing workflow-version status listener) and mirrors every mutation into core. Sync failures are logged, never break the user's write; drift is repaired by re-running the backfill command. Dual-write is deliberately not behind a flag: reading from core (next phase) is only safe if core has been continuously in sync since the backfill. An always-on mirror makes "core is fresh" an invariant, so the read switch becomes a plain flag flip. The cost is one extra upsert on infrequent workflowVersion writes. `IS_WORKFLOW_VERSION_IN_CORE_ENABLED` is reserved for the read switch (Phase B): dispatch from the `workflowAutomatedTriggerMaps` cache, runner and builder reading trigger/steps from core. Until then it gates nothing. Both the backfill and the listener go through a single `WorkflowVersionCoreSyncService` (`upsertToCore`/`deleteFromCore`): the workspace-to-core mapping (`trigger` -> `triggers[]`, plus `steps`, `status`, `workflowId`) and `workflowAutomatedTriggerMaps` invalidation live in one place. ### Rollout plan (following phases) - **B, read switch (flag per workspace):** reads move to core; writes keep flowing workspace -> listener -> core. Rollback = flip the flag back, workspace never stopped being source of truth. - **C, contract (code change):** write paths write trigger/steps to core directly; workspace `workflowVersion` stays as a thin shell (nav/relations/search) but drops the trigger/steps columns; listener and flag removed. ### Not in this PR - Reconciliation tooling beyond re-running the backfill. - The read switch (Phase B). |
||
|
martmull
|
0ea0c23556 |
refactor(app-marketplace): rename featured to vetted (#22674)
## What Renames the application-registration "featured" flag to "vetted" across the backend, frontend, GraphQL schema/DTOs, and the marketplace UI. "Vetted" better describes what the flag actually does today: it marks an app as reviewed and approved by the Twenty team (a trust signal), rather than "featured" which reads as spotlighting/promotion. The admin toggle description was already "Mark this app as reviewed and approved". ## How - Renamed `isFeatured` -> `isVetted` on the `ApplicationRegistration` entity, DTOs (`MarketplaceApp`, `MarketplaceAppDetail`, `UpdateApplicationRegistrationPayload`), services, GraphQL fragments, and the settings/admin UI (labels: "Featured" -> "Vetted", "Featured only" -> "Vetted only", etc.). - Renamed the `MARKETPLACE_FEATURED_APPLICATIONS` constant/file to `MARKETPLACE_VETTED_APPLICATIONS`. - Regenerated GraphQL client artifacts (`generated-metadata`, `generated-admin`, `twenty-client-sdk`). ### Database The `isFeatured` column is renamed in place to `isVetted` via a single 2.20 fast instance command (`ALTER TABLE ... RENAME COLUMN`). No new column, no data-copy backfill. - Since all 2.19 commands (including the existing `isFeatured` backfill) complete before any 2.20 command runs, the rename carries over the values that backfill set. - The entity uses `@WasRenamedInUpgrade` so the upgrade-aware layer queries the old column name until the rename step runs during an upgrade. ## Testing - `nx typecheck` and `nx lint:diff-with-main` pass for twenty-server and twenty-front. - Ran `database:reset` on a fresh dev DB: the 2.19 `isFeatured` backfill runs first, then the 2.20 rename; the column ends up as `isVetted` (and `isFeatured` no longer exists), values preserved. - Booted the server: the `@WasRenamedInUpgrade` decorator validates against the upgrade sequence, and GraphQL introspection confirms all four types expose `isVetted` and none expose `isFeatured`. - Ran the three `graphql:generate` configs and the SDK metadata client generator so the committed generated files match the generator output (field ordering included). ## Notes - The `api-breaking-changes` check flags the removal of the `isFeatured` GraphQL field — that is expected and inherent to this rename. - Translation catalogs (`locales/`) are intentionally not touched here since they are managed via Crowdin; new English strings render via Lingui's default-message fallback until translated. |
||
|
neo773
|
674de0056b |
feat(messaging): message campaign delivery stats + views (#22661)
Re-land of #22452 (reverted in #22627). Rebuilt on fresh main with upgrade commands isolated to 2-20 only; no other version's commands touched. <!-- This is an auto-generated description by cubic. --> <a href="https://cubic.dev/pr/twentyhq/twenty/pull/22661?utm_source=github" target="_blank" rel="noopener noreferrer" data-no-image-dialog="true"><picture><source media="(prefers-color-scheme: dark)" srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source media="(prefers-color-scheme: light)" srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img alt="Review in cubic" src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a> <!-- End of auto-generated description by cubic. --> |
||
|
Thomas Trompette
|
48730df0d2 |
feat(workflow): scaffold core workflowVersion entity + trigger cache (phase 0) (#21674)
## What **Phase 0 (scaffold)** of migrating `workflowVersion` data to **core**. Gated by `IS_WORKFLOW_VERSION_IN_CORE_ENABLED` with **no behavior change** — nothing reads or writes the new core entity yet. ## Plan `workflowVersion` becomes a thin **workspace shell** over a core entity (the `dashboard`/`pageLayout` pattern), so navigation, the metadata relations, and the record UI keep working while the heavy data (`triggers`, `steps`) lives in core. Trigger dispatch will derive from active core versions via a per-workspace cache, letting us **eliminate** the denormalized `workflowAutomatedTrigger` object. `workflow` and `workflowRun` stay as workspace objects. Phases: **0 — scaffold (this PR)** → A — backfill + dual-write → B — switch reads to core → C — drop the workspace `trigger`/`steps` columns + the `workflowAutomatedTrigger` object. ## Included - **Core `WorkflowVersionEntity`** (`extends WorkspaceRelatedEntity`) — stores version data, with triggers as an **array** (`triggers: WorkflowTrigger[]`), a long-due shape change. Storage only: dispatch reads the primary trigger, so behavior stays single-trigger for now. - **Fast create-table instance command** for `core."workflowVersion"` (v2.19.0). - **`IS_WORKFLOW_VERSION_IN_CORE_ENABLED`** feature flag. - **Per-workspace automated-trigger cache provider** deriving CRON/DATABASE_EVENT dispatch from the active version's trigger — groundwork for removing `workflowAutomatedTrigger`. ## Notes - `WorkspaceRelatedEntity`, **not** `SyncableEntity`: this is user runtime data (like `connectedAccount`/`apiKey`/`file`), not application-manifest metadata. - No frontend behavior; the generated `FeatureFlagKey` enums are updated to include the new flag. |
||
|
Etienne
|
4b9f393167 |
fix: add workspace command to backfill missing AGENT source enum values (#22593)
**What** Adds a 2.19 workspace upgrade command that backfills missing FieldActorSource enum values (AGENT) into the Postgres enums backing every ACTOR composite field (createdBy, updatedBy) across all existing workspaces. Each ACTOR field stores its source sub-field as a Postgres enum scoped to its own table and schema (e.g. workspace_abc.company_createdBySource_enum). Workspaces created before these enum values were introduced are missing them, which causes runtime errors when those sources are used. **How** buildActorSourceEnumBackfillTargets collects all ACTOR fields from the workspace metadata cache and maps each enum sub-property to its (tableName, columnName, enumName, expectedValues) tuple. Before iterating over all targets, the command performs a single fast pg_catalog.pg_enum lookup on company.createdBySource as a representative sentinel. If AGENT is already present there, the workspace is skipped entirely (idempotency fast-path). For each remaining target, ALTER TYPE … ADD VALUE IF NOT EXISTS is issued per missing value via WorkspaceSchemaEnumManagerService.addEnumValue, making the command fully idempotent and safe to re-run. Fixes https://discord.com/channels/1130383047699738754/1522507190949118003 <!-- This is an auto-generated description by cubic. --> <a href="https://cubic.dev/pr/twentyhq/twenty/pull/22593?utm_source=github" target="_blank" rel="noopener noreferrer" data-no-image-dialog="true"><picture><source media="(prefers-color-scheme: dark)" srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source media="(prefers-color-scheme: light)" srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img alt="Review in cubic" src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a> <!-- End of auto-generated description by cubic. --> |
||
|
martmull
|
b733a79821 |
feat(server): support server-scoped files via nullable workspaceId on file table (#22587)
Part of the app settings architecture cleanup (twentyhq/core-team-issues#2456) — PR 1 of the server-level documents plan, reworked after the revert of #22560 (#22579). Same capability, different shape: **no new entity** — server-level documents live in the existing `file` table with a nullable `workspaceId`. ## Problem All file storage is workspace-scoped (`FileEntity.workspaceId NOT NULL`, `{workspaceId}/{app}/…` storage keys). Server-level data like application-registration manifests and tarballs for ownerless catalog registrations has no first-class home, forcing raw-driver bypasses (`DefaultAiCatalogService`, prototype #22556). ## Changes (core storage layer only — no HTTP serving, no GraphQL exposure) **`FileEntity` gains server scope** (mirrors `KeyValuePairEntity`, which already supports both instance-level and per-workspace rows): - `workspaceId` uuid becomes **nullable** — NULL means server-scoped; the entity no longer extends `WorkspaceRelatedEntity` and declares its columns directly - `applicationRegistrationId` nullable FK (`onDelete: CASCADE`) — registration-owned documents follow their registration - ownership checks: `workspaceId IS NOT NULL OR applicationRegistrationId IS NOT NULL` and `workspaceId IS NULL OR applicationRegistrationId IS NULL` — every row has exactly one owner - `IDX_FILE_APPLICATION_REGISTRATION_ID_PATH_UNIQUE` UNIQUE (`applicationRegistrationId`, `path`) — mirrors the workspace unique-constraint pattern; workspace rows are exempt via their NULL `applicationRegistrationId` **New `ServerFileStorageService`** (`file-storage/services/`, exported from the global `FileStorageModule`; `FileStorageService` moved alongside it): - storage keys `server/{fileFolder}/{applicationRegistrationId}/{resourcePath}` — the registration segment is injected by the service itself, so paths cannot collide across registrations; scope-validation util mirroring `validateStoragePathIsWithinWorkspaceOrThrow`; new `ServerFileFolder` enum in twenty-shared - `writeServerFile` (upsert on (`applicationRegistrationId`, `path`) + driver write; throws on failure), `readServerFile`/`readServerFileById` (missing row or bytes surfaces `FILE_NOT_FOUND`), `checkServerFileExists`, `deleteServerFile`/`deleteByServerFileId` (bytes best-effort, row authoritative), `deleteByApplicationRegistrationId` - rows are accessed through a plain repository pinned to `workspaceId: IsNull()` on every query; workspace-file code paths still go through `WorkspaceScopedRepository`, which never sees NULL rows **Null-safety ripples** (workspaceId is now `string | null`): - `WorkspaceScopedEntity` bound widened to `workspaceId: string | null` (the wrapper always filters with a concrete id) - `list-and-delete-orphaned-workspace-entities` now skips `workspaceId IS NULL` rows — previously `NOT EXISTS` would have flagged server rows as orphans and deleted them - `PendingFileCleanupService` sweeps only `workspaceId IS NOT NULL` rows; `application-package-fetcher` pins its tarball lookup to workspace rows (tarball migration to server scope is a follow-up PR) **Migration**: `allow-server-scoped-file` ships as a **2-20 fast instance command** (2.20.0 is current since #22639; re-slotted from 2-19 per review). Command runs are tracked by name, so instances that already executed the 2-20 `standardOverrides` drop command still pick this one up. Its realistic timestamp sorts before that drop command's fabricated `1825000000000`, which the `ci:allow-upgrade-command-timestamp-exception` label covers. ## Next PRs in the plan - PR 2: HTTP serving + token type for server files - PR 3: application-registration manifests stored as versioned server files (rework of draft #22556) - PR 4 (optional): registration tarballs migrate to server scope ## Verification - New spec `server-file-storage.service.spec.ts` (traversal table, upsert conflict semantics, row-before-bytes reads, best-effort byte deletion, registration cascade) + scope-validation util spec; affected suites all green - Typecheck (server + shared), `lint:diff-with-main`, full `oxfmt --check src/` on both packages clean - Fresh `database:reset` on the re-slotted branch: the 2-20 command executes, generator then reports **no schema drift**; both ownership checks and the composite unique verified live (dual-owner insert and duplicate registration+path both rejected) |
||
|
martmull
|
435073e9c5 |
Display featured applications in marketplace (#22635)
## After <img width="1060" height="589" alt="image" src="https://github.com/user-attachments/assets/74dfadcf-8698-4404-81c6-b309cc4cbf79" /> <img width="732" alt="image" src="https://github.com/user-attachments/assets/0e1a3644-04bc-4208-aa77-3842d9db9cc8" /> <img width="797" alt="image" src="https://github.com/user-attachments/assets/0456ecce-607a-4705-8a89-c77029bfb6ac" /> - Remove IS_MARKETPLACE_SETTING_TAB_VISIBLE feature flag - add vetted toggle in admin app tab - added people data labs, last contact and call recorder to default vetted applications <!-- This is an auto-generated description by cubic. --> <a href="https://cubic.dev/pr/twentyhq/twenty/pull/22635?utm_source=github" target="_blank" rel="noopener noreferrer" data-no-image-dialog="true"><picture><source media="(prefers-color-scheme: dark)" srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source media="(prefers-color-scheme: light)" srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img alt="Review in cubic" src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a> <!-- End of auto-generated description by cubic. --> --------- Co-authored-by: prastoin <paul@twenty.com> |
||
|
Paul Rastoin
|
bd8bf89653 |
Revert "feat(messaging): message campaign delivery stats + views" (#22452) (#22627)
Revert "feat(messaging): message campaign delivery stats + views
(#22452)"
This reverts commit
|
||
|
neo773
|
2e1117d442 |
feat(messaging): message campaign delivery stats + views (#22452)
<!-- This is an auto-generated description by cubic. --> <a href="https://cubic.dev/pr/twentyhq/twenty/pull/22452?utm_source=github" target="_blank" rel="noopener noreferrer" data-no-image-dialog="true"><picture><source media="(prefers-color-scheme: dark)" srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source media="(prefers-color-scheme: light)" srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img alt="Review in cubic" src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a> <!-- End of auto-generated description by cubic. --> |
||
|
Paul Rastoin
|
6c40c7b91a |
Deterministic system field universal identifier (#22565)
# Introduction Close twentyhq/core-team-issues#2641 Auto-provisioned field metadata used to get its `universalIdentifier` from three unrelated sources: random `v4()` on the server when creating custom objects, hardcoded values in `STANDARD_OBJECTS`, and an ad-hoc `v5` derivation in the SDK manifest build. This PR unifies all of them behind the shared `getFieldUniversalIdentifier` derivation: ``` universalIdentifier = f(applicationUniversalIdentifier, objectUniversalIdentifier, fieldName) ``` ## Ownership model The rollout is built on an explicit split of who owns a field's universal identifier: - **The 8 system fields** (`id`, `createdAt`, `updatedAt`, `deletedAt`, `createdBy`, `updatedBy`, `position`, `searchVector`) are **server-owned**. Their universal identifiers are always the deterministic derivation, on **every** application (standard, workspace-custom, installed). Clients cannot provide custom values: a temporary check in `validateObjectMetadataSystemFieldsIntegrity` rejects any non-derived system field identifier at migration build time. This check stands in until system fields are generated exclusively server side by the metadata side-effect engine and stripped from client inputs — at which point it becomes structurally impossible to send one. - **`name` is a default field, not a system field**: it is auto-provisioned when absent (server side for custom objects, SDK side for application objects) but authors can define their own. It is only derived where it is guaranteed to be auto-provisioned. In particular, standard objects keep their **historical hardcoded** `name` identifiers: the standard app authors its `name` fields like any installed app would, and moving those identifiers would break every installed application referencing them (e.g. views on `opportunity.name`). - **User-created and author-provided fields** keep random / explicit identifiers, untouched. ## Server - `validateObjectMetadataSystemFieldsIntegrity` now validates, on top of the existing type/`isSystem` checks, that each system field's `universalIdentifier` equals the deterministic derivation. Runs for every object creation going through the migration orchestrator: app sync, custom object creation, standard provisioning - `build-default-flat-field-metadatas-for-custom-object.util.ts` derives the system field identifiers (and the auto-provisioned `name`) with `getFieldUniversalIdentifier` instead of `v4()` - `build-default-relation-flat-field-metadatas-for-custom-object.util.ts` derives both the forward and the reverse default relation field identifiers deterministically - `generateMorphOrRelationFlatFieldMetadataPair` accepts optional `sourceFieldUniversalIdentifier` / `targetFieldUniversalIdentifier` so callers can inject deterministic values; user-created relations still default to `v4()` ## twenty-shared - `STANDARD_OBJECTS` system field identifiers (the 8) are now computed at module load via `buildStandardObjectSystemFields`; `name` and every other identifier keep their hardcoded values - New snapshot test pinning **every** universal identifier of `STANDARD_OBJECTS`: any identifier change now requires an explicit snapshot update and should ship with a coordinated backfill ## SDK (breaking, pre-GA) - `generateDefaultFieldUniversalIdentifier` delegates to `getFieldUniversalIdentifier` and now requires `applicationUniversalIdentifier` - Reverse default relation field identifiers are derived from the field's real coordinates (standard object UID + actual field name, e.g. `targetRocket` on `attachment`) instead of the legacy custom-object UID + synthetic `${fieldName}Inverse` hash input. Field *names* are unchanged - The manifest build threads the application universal identifier through default field injection (two-pass over object configs) - `twenty dev:add` now resolves the application universal identifier upfront and refuses to scaffold anything until `defineApplication` declares one — no more `fill-later` placeholder for the app UID in generated files ## Upgrade A 2.19 **workspace command** backfills existing `fieldMetadata.universalIdentifier` rows to the deterministic derivation. Coverage follows the ownership model: - **The 8 system fields**: taken over for **every application**, whatever value they currently hold. This is both safe and required now that sync rejects non-derived values — leaving a row unconverged would make its application unsyncable - **`name`**: workspace-custom app → always taken over (server-generated, no author to clobber); installed applications → only rows still carrying the legacy SDK derivation are recomputed, author-provided identifiers are never touched; standard app → never touched (hardcoded in `STANDARD_OBJECTS`) - **Default relation fields**: workspace-custom app → forward fields on custom objects and reverse fields on the standard relation objects; installed applications → legacy-derivation probe only All identifiers of a workspace are updated inside a single transaction, then the command flushes the field-metadata-related workspace caches and bumps the metadata version. Stored `applicationRegistration.manifest` snapshots are intentionally **not** rewritten: installs and upgrades always sync from the `manifest.json` inside the resolved package (npm/tarball), the stored column is only used for display/marketplace purposes. ## Breaking behavior for old packages (fail closed) Packages built with an older SDK carry legacy system field identifiers in their tarball `manifest.json`. Installing or upgrading such a package now fails with an explicit `INVALID_SYSTEM_FIELD` validation error ("universal identifier is not deterministic") instead of silently mismatching against the backfilled rows and triggering a destructive delete+create. The remediation is to rebuild the package with the new SDK; the backfill has already converged the installed rows, so the rebuilt manifest syncs cleanly. ## Test plan - [x] `twenty-sdk` unit tests (526 tests) and typecheck - [x] `twenty-shared` unit tests (1635 tests) including the `STANDARD_OBJECTS` snapshot; `name` identifiers verified byte-for-byte identical to `main` - [x] Lint and typecheck clean on all touched packages - [x] Integration: create a custom object and verify system + default relation field identifiers match the deterministic derivation (`create-one-object-metadata-deterministic-field-universal-identifiers`, 13 assertions passing) - [x] Integration: `failing-sync-application-object-system-fields` extended with a non-derived system field identifier case; all identifiers in the spec pinned deterministically so snapshots embedding expected/actual values are stable across runs (verified with a double run) - [x] Integration: all application sync suites pass with the derived system field identifiers now required by the `buildDefaultObjectManifest` test helper (9 suites, 20 tests) - [x] Full test-database reset: standard app provisioning and seeded workspaces pass the new validation - [x] SDK manifest build verified on the postcard example app: all auto-generated default field identifiers match the derivation - [ ] Run `upgrade:2-19:backfill-deterministic-field-universal-identifiers` (dry-run then real) on a seeded workspace and verify identifier convergence with a rebuilt app manifest |
||
|
martmull
|
2327ae7122 |
Revert "feat(server): add instance-level file storage layer" (#22579)
Reverts twentyhq/twenty#22560 <!-- This is an auto-generated description by cubic. --> <a href="https://cubic.dev/pr/twentyhq/twenty/pull/22579?utm_source=github" target="_blank" rel="noopener noreferrer" data-no-image-dialog="true"><picture><source media="(prefers-color-scheme: dark)" srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source media="(prefers-color-scheme: light)" srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img alt="Review in cubic" src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a> <!-- End of auto-generated description by cubic. --> |
||
|
martmull
|
0baf213fa4 |
feat(server): add instance-level file storage layer (#22560)
Part of the app settings architecture cleanup
(twentyhq/core-team-issues#2456) — PR 1 of the instance-level documents
plan. Today all file storage is workspace-scoped
(`FileEntity.workspaceId NOT NULL`, `{workspaceId}/{app}/…` storage
keys, workspace-anchored tokens); instance-level data like
application-registration manifests and tarballs for ownerless catalog
registrations has no first-class home, forcing raw-driver bypasses
(`DefaultAiCatalogService`, prototype #22556).
## Changes (core storage layer only — no HTTP serving, no GraphQL
exposure)
**New `instanceFile` table** (`InstanceFileEntity`) — deliberately
separate from the workspace-scoped `file` table so nothing about the
existing system changes:
- `id`, `path` (unique, `{fileFolder}/{relativePath}` mirroring
FileEntity's convention), `size`, `mimeType`, timestamps
- nullable `applicationRegistrationId` FK (`onDelete: CASCADE`) —
registration-owned documents follow their registration
- no `workspaceId`, no `applicationId`; plain repository (added to the
`prefer-workspace-scoped-repository` lint rule's global-table
exemptions, as the rule's own message directs)
**New `InstanceFileStorageService`** (exported from the global
`FileStorageModule`):
- storage keys under a literal `instance/{fileFolder}/…` prefix —
collision-free with workspace prefixes (UUIDs); scope-validation util
mirroring `validateStoragePathIsWithinWorkspaceOrThrow`
- `writeInstanceFile` (upsert row on `path` conflict + driver write;
throws on failure — no swallowing),
`readInstanceFile`/`readInstanceFileById` (missing file surfaces
`FILE_NOT_FOUND` like `FileStorageService.readFile`),
`checkInstanceFileExists`, `deleteInstanceFile`/`deleteByInstanceFileId`
(bytes best-effort, row authoritative),
`deleteByApplicationRegistrationId` (lifecycle hook for
registration-owned files)
- same driver path as `FileStorageService` (`FileStorageDriverFactory` →
`ValidatedStorageDriver`)
**Migration**: fast instance command `add-instance-file-table` (2.19,
generator-produced; post-command `database:migrate:generate` reports no
pending changes).
## Next PRs in the plan
- PR 2: HTTP serving + token type for instance files (new route + guard;
workspace file endpoints untouched)
- PR 3: application-registration manifests stored as versioned instance
files (supersedes draft #22556)
- PR 4 (optional): registration tarballs migrate to instance scope,
removing the cross-workspace `FileEntity` read in
`application-package-fetcher` and the `ownerWorkspaceId` requirement on
`uploadTarball`
## Verification
- New specs: scope-validation util (traversal cases) + service (upsert
conflict, missing-file error, best-effort byte deletion, registration
cascade) — 16/16; `npx jest "application"` still 31 suites / 160 green
- Typecheck, `lint:diff-with-main`, full `oxfmt --check src/` (6421
files) and full type-aware oxlint clean
- Fast command executed against the local DB — table, unique index, and
CASCADE FK verified via psql; generator then reports no schema drift
- Server boots with the new provider; `generate-metadata-client
--skip-nx-cache` zero diff (no GraphQL change)
---
_Generated by [Claude
Code](https://claude.ai/code/session_011sST4rPLU1Koi2oVGi84ei)_
<!-- This is an auto-generated description by cubic. -->
<a
href="https://cubic.dev/pr/twentyhq/twenty/pull/22560?utm_source=github"
target="_blank" rel="noopener noreferrer"
data-no-image-dialog="true"><picture><source
media="(prefers-color-scheme: dark)"
srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source
media="(prefers-color-scheme: light)"
srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img
alt="Review in cubic"
src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a>
<!-- End of auto-generated description by cubic. -->
|
||
|
martmull
|
e5e3fadbbb |
feat(files): reap stale pending direct-upload files via hourly cron (#22531)
## Context Follow-up to #22449 (direct-to-storage upload endpoints). That PR introduced the `PENDING` → `UPLOADED` file lifecycle: `createFileUpload` inserts a file record in `PENDING`, the client uploads the bytes directly to storage, then `completeFileUpload` flips it to `UPLOADED`. A client that initiates an upload but never confirms — a crash, a closed tab, an expired presigned URL — leaves a `PENDING` file record and a possibly-partial storage object behind forever. This PR reaps them. ## What this does Adds an hourly cron that hard-deletes `PENDING` files older than 24h together with their storage objects, in bounded batches. - **`PendingFileCleanupService`** — finds `PENDING` files with `createdAt` older than `PENDING_FILE_MAX_AGE_MS` (24h), capped at `PENDING_FILE_CLEANUP_BATCH_SIZE` (200) per run, and deletes each via `FileStorageService.deleteByFileId` (which tolerates a missing object). A failure on one file is logged and skipped so the rest of the batch still gets cleaned. - **`PendingFileCleanupCronJob`** — `@Processor(cronQueue)` job that runs the service and reports exceptions. - **`PendingFileCleanupCronCommand`** — registers the job on the hourly pattern (`0 * * * *`). - Wired into `FileUploadModule` (providers + export) and registered in `cron:register:all`. ### Why 24h The reaper threshold sits well past the presigned URL expiry, so a `PENDING` file only becomes reapable long after any legitimate in-flight upload could still complete — the cleanup can never race a real upload. A file that was never confirmed is referenced by nothing; the client recovery path is simply re-uploading under a fresh `fileId`, so we never promote to `UPLOADED`. ## Tests `pending-file-cleanup.service.spec.ts` covers: the query shape (status + age threshold + batch cap), deleting each stale file and returning the count, continuing past a per-file deletion failure, and the empty-batch no-op. ## Scope Server-only, non-breaking, no user-facing change. Part of the incremental direct-upload rollout being split into small PRs. https://claude.ai/code/session_015UH8KWmsB9zdYaog8MFG1d --- _Generated by [Claude Code](https://claude.ai/code/session_015UH8KWmsB9zdYaog8MFG1d)_ <!-- This is an auto-generated description by cubic. --> <a href="https://cubic.dev/pr/twentyhq/twenty/pull/22531?utm_source=github" target="_blank" rel="noopener noreferrer" data-no-image-dialog="true"><picture><source media="(prefers-color-scheme: dark)" srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source media="(prefers-color-scheme: light)" srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img alt="Review in cubic" src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a> <!-- End of auto-generated description by cubic. --> |
||
|
martmull
|
d6b6962604 |
feat(files): content-verify direct uploads and pin pending files to octet-stream (#22533)
## Context Follow-up to #22449 (direct-to-storage upload endpoints). In that flow `createFileUpload` inserts a `PENDING` file record before any bytes exist, and until now it guessed the mime type from the **filename extension** — an untrusted, client-controlled value. This PR makes a pending file opaque and only trusts a mime type that was verified against the actual stored bytes. ## What this does **1. A pending file is always `application/octet-stream`.** `createFileUpload` records the pending file — and signs the presigned PUT — as `application/octet-stream`. The extension is still kept on the stored object name so the content can be checked against it later. **2. Content verification at completion.** `completeFileUpload`, after the existing size check, reads a **bounded prefix** of the stored object (`readReadablePrefix`, capped at 64 KiB — a large object is never buffered in full) and runs the existing `extractFileInfoOrThrow` util to detect the real mime type from the content. It: - writes the detected type alongside `status = UPLOADED`, and - rejects a file whose bytes don't match its declared extension (the record stays `PENDING`, so it can never be served or attached, and is reaped by the pending-file cleanup cron). Serving already overrides `Content-Type` from the DB record, so storing the object as octet-stream is fine. **3. A database constraint as backstop.** `CHK_FILE_PENDING_MIME_OCTET_STREAM` — `"status" != 'PENDING' OR "mimeType" = 'application/octet-stream'` — added to `FileEntity` and applied by a fast instance command (`2-19`). It is added `NOT VALID` on purpose: an instance freshly upgraded past #22449 may still hold `PENDING` rows whose mime came from the old extension-guess path, and `NOT VALID` enforces the invariant on every new/updated row without failing on that legacy backlog (those rows get overwritten to octet-stream when completed — `status` flips to `UPLOADED`, so the check passes — or are reaped while pending). ## Tests - `read-readable-prefix.spec.ts` — prefix reader: short source, early stop on a large source (asserts it tears the stream down without draining it), error propagation, empty stream. - `file-upload.service.spec.ts` — create records octet-stream; complete sniffs and sets the detected type, overrides a spoofed extension with the real content type, and rejects content that can't be matched to the declared extension. - `direct-file-upload.integration-spec.ts` — end-to-end case rejecting a `.png` upload whose bytes are plain text. ## Verification `typecheck` green, `lint:diff-with-main` clean, unit suites pass (17 tests). No GraphQL schema change, so no codegen drift. ## Scope Server-only, part of the incremental direct-upload rollout being split into small PRs. Independent of the reaper-cron PR (#22531). https://claude.ai/code/session_015UH8KWmsB9zdYaog8MFG1d --- _Generated by [Claude Code](https://claude.ai/code/session_015UH8KWmsB9zdYaog8MFG1d)_ <!-- This is an auto-generated description by cubic. --> <a href="https://cubic.dev/pr/twentyhq/twenty/pull/22533?utm_source=github" target="_blank" rel="noopener noreferrer" data-no-image-dialog="true"><picture><source media="(prefers-color-scheme: dark)" srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source media="(prefers-color-scheme: light)" srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img alt="Review in cubic" src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a> <!-- End of auto-generated description by cubic. --> |
||
|
Paul Rastoin
|
43730d7748 |
Centralized side effects devxp basis (#22295)
# Introduction
This PR introduces a centralized, strictly-typed **metadata side-effect
engine** that unifies how system metadata side effects are derived and
applied across both metadata entry points — the **metadata GraphQL API**
and the **application sync / manifest** flow — and migrates the first
side effect end-to-end: **a unique scalar field owns its backing
single-field `UNIQUE` index** (full create / update / delete lifecycle).
## New conventions
- **Engine-owned companions**: metadata flagged `isSystemSideEffect:
true` is owned by the engine. Its deletion is never inferred from
absence in a manifest — it results from PG-level cascade or from a
delete side effect (a side effect always has a cause, its parent
metadata).
- **Reserved deterministic identifiers**: apps cannot declare metadata
reusing an engine-owned deterministic `universalIdentifier`. Doing so
fails validation with `RESERVED_SYSTEM_UNIVERSAL_IDENTIFIER` (until an
explicit override API exists).
- **Record-native operation matrix**: the operation matrix is keyed by
`universalIdentifier` (`AllFlatEntityOperationRecordByMetadataName`)
instead of arrays, making parent resolution and deduplication O(1).
Array-based API callers are transpiled to records at the
validate-build-and-run boundary.
- Twenty-sdk user-facing experience with system fields will only be
related to overrides.
# What this PR does
## 1. Side-effect engine (foundation)
- `MetadataSideEffectEngineService.expandWithSideEffects(...)` takes the
intention-carrying record matrix and returns it expanded with derived
side effects, or a structured failure.
- Handlers are registered via a typed **decorator + registry** pattern
(`MetadataSideEffectHandler({ operation, metadataName, name, description
})`), with runtime duplicate-name detection. Multiple handlers per
(operation, metadataName) are supported.
- Handler contract mirrors the validator pattern:
- receives the trigger flat entity, the live record matrix, and
**strictly-typed related flat entity maps**
(`MetadataFlatEntityAndRelatedFlatEntityMapsForSideEffect<P>`, derived
from declared companion metadata names — no loose
`Partial<AllFlatEntityMaps>` context)
- returns `MetadataSideEffectResult`: `success` (operations record) |
`noop` | `fail` (structured failure)
- **Non-recursion is structural**: triggers are read from the original
caller input, never from the expanded matrix, so a side effect can never
trigger another side effect.
- **Deduplication + collision detection**: side effects are deduped by
`universalIdentifier` per operation; a caller-declared entity colliding
with an engine-owned deterministic identifier is recorded as a
collision.
- **Unified failure channel**: handler failures and reserved-identifier
collisions are merged into the same `OrchestratorFailureReport` contract
as builder validation errors, and the run short-circuits (fail-closed,
nothing is applied).
## 2. First migrated side effect — unique field → backing unique index
Three handlers own the complete lifecycle of the deterministic
single-field `UNIQUE` index backing a unique scalar field:
- **create**: unique scalar field → generate the deterministic backing
index (`fieldUniqueBackingIndexOnCreate`)
- **update**: `isUnique` flips and renames of still-unique fields (the
index name — and therefore its deterministic identifier — derives from
the field name, so a rename drops the stale index and recreates the
deterministic one) (`fieldUniqueBackingIndexOnUpdate`)
- **delete**: cascade-delete the backing index
(`fieldUniqueBackingIndexOnDelete`)
Supporting rules:
- The primary key `id` field never spawns a backing index (uniqueness
comes from the PK constraint) — explicit `isPrimaryKeyFlatFieldMetadata`
guard.
- Parent object resolution is **optimistic-first**: an object created or
updated in the same batch wins over the workspace cache (so e.g.
renaming an object while flipping a field to unique builds the index
from the post-rename object), resolved in O(1) via the record matrix.
- A missing parent object is reported as a structured side-effect
failure, never silently skipped.
## 3. Path convergence — manifest and API share one flow
- The manifest sync now derives a from→to **record matrix** from the
cache and feeds `validateBuildAndRunWorkspaceMigrationFromRecord`, the
same flow the API uses — both paths converge on the engine.
- Manifest-side unique-index generation and API transpiler
system-unique-index handling were removed (declared/composite/relation
indexes stay untouched).
- New `WorkspaceMigrationFlatEntityMapsService` mutualizes
flat-entity-maps computation between the side-effect engine and the
builder: cache keys are derived from the caller metadata names (+
validation- and side-effect-related closures) instead of hardcoded
loads.
- App-scoping and pruning are folded into one shared primitive
(`getSubAllFlatEntityMapsByApplicationIdsOrThrow`): slicing dependency
maps to the involved applications always prunes dangling one-to-many
aggregators — callers can no longer forget it.
- **Behavior change**: an app extending another app's view with a view
field now syncs successfully (cross-app view-field extension), covered
by a dedicated integration test.
## 4. Backfill upgrade command (2.19)
`upgrade:2-19:backfill-system-unique-index-universal-identifier`
rewrites legacy system unique-index `universalIdentifier`s to their
deterministic value so the engine can own pre-existing indexes. The
backfill is **driven from `isUnique: true` fields** (mirroring the
engine ownership predicate — excludes PK / morph / relation fields) and
resolves each field's backing index in O(1).
# Bugs fixed along the way
- `database:reset` seeding failed with
`INDEX_FIELD_INVALID_DEFAULT_VALUE`: the engine derived a backing
`UNIQUE` index for the default `id` primary key. Fixed with the explicit
primary-key guard.
- `isUnique` updates on system-flagged standard fields (e.g.
auto-created `name`) did not trigger the backing-index side effect.
- Manifest sync crashed with "Could not find flat entity with universal
identifier ..." when app-scoped slices left dangling aggregator
references — fixed by centralizing pruning in the shared slice primitive
|
||
|
martmull
|
1a85b88d38 |
feat(files): direct-to-storage upload endpoints with pending file lifecycle (#22449)
<img width="1484" height="404" alt="image" src="https://github.com/user-attachments/assets/b2d363bf-d9e1-49fb-9811-8cc98041aa79" /> ## Context Uploading large files currently OOMs the server: every upload resolver buffers the whole file in memory (`streamToBuffer`) before writing it to storage. This PR is the first of a series introducing direct client-to-storage uploads. It adds the server-side endpoints and driver support only — it is non-breaking and nothing consumes the new flow yet. Follow-up PRs will migrate the frontend upload paths, add a stale-pending-file cleanup cron, and cap the legacy buffered resolvers. ## What it does **New upload flow (initiate → PUT → confirm):** - `createFileUpload(filename, size, fileFolder, fieldMetadataId?)` validates the request (folder allowlist: `FilesField`/`Workflow`, max size, extension-derived mime type), creates the file record in a new `PENDING` status, and returns an upload target: - **S3 with presign enabled** → a presigned PUT URL with `Content-Type`/`Content-Length` pinned in the signature, so the client uploads straight to the bucket; - **local storage, or S3 without presign** → a token-authenticated streaming endpoint on the server (`PUT /file-upload/:id?token=…`, new `FILE_UPLOAD` JWT type) that pipes the request body to the storage driver with constant memory usage and a declared-size cap. - `completeFileUpload(fileId)` verifies the bytes actually landed in storage (HEAD + size match against the declared size) and flips the record to `UPLOADED`. Idempotent. **Pending lifecycle safety:** - New `status` column on `core.file` (`PENDING`/`UPLOADED`, default `UPLOADED` so all existing rows and the legacy upload path are unaffected) + fast instance command. - Files are refused by the serving endpoints and by FILES-field sync while `PENDING`. **Driver support (both drivers):** - `getPresignedUploadUrl` (S3: presigned PUT; local: `null` → server-endpoint fallback) - `writeFileStream` (local: `fs` pipeline with the existing symlink/containment hardening, partial-file cleanup on error; S3: `@aws-sdk/lib-storage` `Upload` for bounded-memory streaming) - `getFileMetadata` (HEAD/stat for confirm-time verification) ## Tests - `file-upload.service.spec.ts`: initiate validation (folder allowlist, size), presigned vs fallback target, confirm verification (missing object, size mismatch, happy path, idempotency) - `local.driver.spec.ts`: `writeFileStream` (content, symlink rejection, partial-file cleanup on stream error), `getFileMetadata` - `s3.driver.spec.ts`: `getPresignedUploadUrl` (disabled → null, PUT command with signed content-type/content-length) - `direct-file-upload.integration-spec.ts`: full end-to-end flow against the local driver (initiate → PUT → complete → download), plus error paths (complete without upload, oversized PUT → 413, invalid token → 403, unsupported folder, size above max) ## Notes for reviewers - The upload-size ceiling for direct uploads is `settings.storage.maxDirectUploadFileSize` (1GB), separate from the 10MB `maxFileSize` used for pictures. - Since content can't be sniffed before it reaches storage, the mime type is derived from the file extension (with the existing `TWENTY_MIME_POLICY` override) and unknown extensions fall back to `application/octet-stream`; the serving path already forces `Content-Disposition: attachment` for anything not on the inline-safe allowlist. - Self-hosters using S3 presign will need a bucket CORS policy allowing `PUT` from the frontend origin (config variable description updated). https://claude.ai/code/session_015UH8KWmsB9zdYaog8MFG1d --- _Generated by [Claude Code](https://claude.ai/code/session_015UH8KWmsB9zdYaog8MFG1d)_ <!-- This is an auto-generated description by cubic. --> <a href="https://cubic.dev/pr/twentyhq/twenty/pull/22449?utm_source=github" target="_blank" rel="noopener noreferrer" data-no-image-dialog="true"><picture><source media="(prefers-color-scheme: dark)" srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source media="(prefers-color-scheme: light)" srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img alt="Review in cubic" src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a> <!-- End of auto-generated description by cubic. --> |
||
|
Paul Rastoin
|
0db4ddd46e |
fix(server): tolerate metadata-physical index drift in legacy index name normalization (#22472)
## Context The 2.18 `NormalizeLegacyIndexNames` workspace command (`1799200000000`, introduced in #22053) fails several production workspaces with `42P01 relation … does not exist`, rolling back the whole per-workspace upgrade transaction and marking the workspace `Failed`. ## Root cause The command assumes the physical index in the workspace schema is named exactly as recorded in `core."indexMetadata"."name"`. For workspaces where the physical index was already rebuilt/renamed under the v2 deterministic name (only targeted phone/relation rebuilds got new names after #14567) while metadata kept the legacy hash, the rename source no longer exists, so `ALTER INDEX … RENAME` aborts the entire workspace upgrade. (The duplicate-drop path uses `DROP INDEX IF EXISTS` and is unaffected.) ## Fix - **`WorkspaceSchemaIndexManagerService`**: new `doesIndexExist` and `getIndexDefinition` helpers querying `pg_indexes` for a `(schema, index)` pair. `renameIndexWithoutRebuild` keeps its strict semantics (no `IF EXISTS`) — drift tolerance lives in the command, which is the only caller that expects it. - **`NormalizeLegacyIndexNamesCommand`** — the rename operation now reconciles drift instead of blindly renaming: - Target name already exists physically, source gone → skip the rename, just point `indexMetadata.name` at it (the common "physical already v2, metadata still legacy" case). - Both source and target exist physically → compare their `pg_indexes.indexdef` ignoring the name: if identical, drop the legacy duplicate (it would otherwise be orphaned forever since metadata stops referencing it, adding permanent write/maintenance cost); if the definitions differ, keep it in place and log a warning. - Source exists, target free → rename as before, then update metadata. - Neither exists → log a warning and update metadata so a future rebuild recreates the index under the expected v2 name. In every branch the metadata name ends up on the recomputed v2 name, and no missing physical index can abort the workspace transaction anymore. ## Tests - Regression tests on the command spec for the four drift cases (target-already-renamed, both-missing, both-present-identical → drop, both-present-different → keep); existing rename/duplicate/dry-run/rollback tests updated to declare the physical indexes present. - New spec for `WorkspaceSchemaIndexManagerService` covering the rename SQL, the `pg_indexes` existence check, and the definition lookup. - New spec for `areIndexDefinitionsEquivalent` (name-only diff, uniqueness, columns, where clause, malformed input). `npx jest` on all three specs (20 passed), `lint:diff-with-main` and `typecheck` green. |
||
|
martmull
|
0992d0b748 |
feat(server): promote registration display fields to first-class columns (#22513)
Part of the application settings architecture work: https://github.com/twentyhq/core-team-issues/issues/2456 — follow-up to #22453, delivering the promised removal of the temporary manifest load. Display data (description, author, category, websiteUrl, aboutDescription, termsUrl, emailSupport, issueReportUrl, screenshots) only existed inside the `manifest` jsonb, forcing hot paths to load it. This PR: - Promotes those 9 fields to first-class columns on `applicationRegistration`, populated at every ingestion point (`updateFromManifest`, both `upsertFromCatalog` branches) — fast command creates the columns at deploy, slow command backfills them from the manifest. - `findManyListedCatalogCards()` (marketplace list) now selects only scalar columns — the manifest jsonb is no longer loaded there. - `findPublicByClientId()` (OAuth consent page) now selects `id, name, logo, websiteUrl, oAuthScopes` — no manifest. - The narrow select used by `findMany`/`findAll`/`findOneById`/`findOneByIdGlobal` includes the new columns. - GraphQL surface unchanged (no new fields); the marketplace detail endpoint still reads the manifest and is slimmed in the next PR. Verified: migration applied via the real runner (both commands recorded completed), backfill SQL exercised against live rows (full + minimal manifests), migration generator reports no pending schema changes, typecheck, lint, unit suites (application-registration + marketplace 15/15). 🤖 Generated with [Claude Code](https://claude.com/claude-code) https://claude.ai/code/session_011sST4rPLU1Koi2oVGi84ei --- _Generated by [Claude Code](https://claude.ai/code/session_011sST4rPLU1Koi2oVGi84ei)_ <!-- This is an auto-generated description by cubic. --> <a href="https://cubic.dev/pr/twentyhq/twenty/pull/22513?utm_source=github" target="_blank" rel="noopener noreferrer" data-no-image-dialog="true"><picture><source media="(prefers-color-scheme: dark)" srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source media="(prefers-color-scheme: light)" srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img alt="Review in cubic" src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a> <!-- End of auto-generated description by cubic. --> |
||
|
martmull
|
af1b89c788 |
feat(server): import application logo into file storage at install (#22437)
Installed apps stored the logo as the manifest's relative path but never imported the file, so the public-assets URL 404'd and logos went missing in the UI for npm/tarball sources. Import the logo (best-effort — a declared but unshipped logo is skipped, not fatal) and record it as a first-class logoFileId on the application so it can be served reliably. <!-- This is an auto-generated description by cubic. --> <a href="https://cubic.dev/pr/twentyhq/twenty/pull/22437?utm_source=github" target="_blank" rel="noopener noreferrer" data-no-image-dialog="true"><picture><source media="(prefers-color-scheme: dark)" srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source media="(prefers-color-scheme: light)" srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img alt="Review in cubic" src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a> <!-- End of auto-generated description by cubic. --> |
||
|
martmull
|
20f2e33702 |
feat(server): first-class logo on application registration + narrowed settings queries (#22453)
Part of the application settings architecture work: https://github.com/twentyhq/core-team-issues/issues/2456 Application-registration list queries loaded the entire `manifest` jsonb (potentially 100KB+/row) on every settings/marketplace list request because display data (logo, description, author, category) only exists inside it. This PR: - Adds a first-class nullable `logo` column on `applicationRegistration`, populated at every ingestion point (`updateFromManifest`, `upsertFromCatalog`) and backfilled from `manifest->application->>logoUrl` via a slow instance command (self-sufficient backfill since `runDataMigration` runs before `up`). - Backs the `logoUrl` GraphQL getter with the column (manifest fallback for un-backfilled rows) — **GraphQL surface unchanged**. - Narrows `findMany` / `findAll` / `findOneById` / `findOneByIdGlobal` to an explicit scalar select that excludes `manifest` and `oAuthClientSecretHash` (every caller audited — none needs them; OAuth verification paths are untouched). - Replaces `findManyListed()` with `findManyListedCatalogCards()`: a projection query that extracts the four display strings from the manifest in SQL (with explicit soft-delete filtering) instead of hydrating full entities, feeding `findManyMarketplaceApps`. Verified: typecheck, lint:diff-with-main, unit suites (application-registration 5/5, marketplace 10/10, instance-command 31/31), migration applied via the real runner, and the migration generator reports no pending schema changes. 🤖 Generated with [Claude Code](https://claude.com/claude-code) https://claude.ai/code/session_011sST4rPLU1Koi2oVGi84ei --- _Generated by [Claude Code](https://claude.ai/code/session_011sST4rPLU1Koi2oVGi84ei)_ <!-- This is an auto-generated description by cubic. --> <a href="https://cubic.dev/pr/twentyhq/twenty/pull/22453?utm_source=github" target="_blank" rel="noopener noreferrer" data-no-image-dialog="true"><picture><source media="(prefers-color-scheme: dark)" srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source media="(prefers-color-scheme: light)" srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img alt="Review in cubic" src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a> <!-- End of auto-generated description by cubic. --> |
||
|
martmull
|
25fe66565c |
feat(applications): add type and options to application variables (#22157)
## Before <img width="1452" height="709" alt="image" src="https://github.com/user-attachments/assets/cd384ffa-cbe6-49d5-a807-ca8d580f55a9" /> <img width="1074" height="452" alt="image" src="https://github.com/user-attachments/assets/720d38db-3495-4032-8831-17d24ec6a7e7" /> ## After <img width="1421" height="865" alt="image" src="https://github.com/user-attachments/assets/2275c996-c895-4800-8324-2aa2ddfddd43" /> <img width="1348" height="870" alt="image" src="https://github.com/user-attachments/assets/3e1a891d-6db0-4cbd-870a-2a5bbde4929d" /> ## Summary Adds typed application variables with optional select **options**. This is the other half of #22059, split out from the custom-settings-tab removal. ## Changes - **Shared types**: `ApplicationVariable` / `ServerVariables` gain an optional `type` (a `FieldMetadataType` subset — `TEXT`, `BOOLEAN`, `NUMBER`, `DATE`, `SELECT`, `MULTI_SELECT`, `RAW_JSON`, `RICH_TEXT`, `ARRAY`, …) and select `options`. New `serializeApplicationVariableValue` / `deserializeApplicationVariableValue` helpers convert typed values to/from the encrypted string storage. - **Server**: `type`/`options` columns on `applicationVariable` and `applicationRegistrationVariable` (entities + DTOs), a fast `2-17` instance command, manifest processing via the serialization helpers, and a `QueryDeepPartialEntity` cast where the manifest JSON column is persisted. - **Frontend**: a polymorphic `SettingsApplicationVariableInput` that renders the native `Form*` field component for each type (boolean, number, date/date-time, select, multi-select, array, raw JSON, rich text, text); fragment/query updates to fetch `type`/`options`. - **SDK**: `defineApplication` validates that `SELECT`/`MULTI_SELECT` variables declare non-empty `options` at build time (since `options` is kept structurally optional for TypeORM/SDK compatibility). Variables default to `TEXT` when no type is given, so existing manifests are unaffected. ## Notes The generated GraphQL artifacts (`type`/`options` on the variable types) are regenerated by codegen; that change accompanies this PR. https://claude.ai/code/session_013Z7UB35V2mvUozh55QHG23 --- _Generated by [Claude Code](https://claude.ai/code/session_013Z7UB35V2mvUozh55QHG23)_ <!-- This is an auto-generated description by cubic. --> <a href="https://cubic.dev/pr/twentyhq/twenty/pull/22157?utm_source=github" target="_blank" rel="noopener noreferrer" data-no-image-dialog="true"><picture><source media="(prefers-color-scheme: dark)" srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source media="(prefers-color-scheme: light)" srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img alt="Review in cubic" src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a> <!-- End of auto-generated description by cubic. --> |
||
|
Félix Malfait
|
024a9b4d94 |
fix(upgrade): re-slot all 2-19 upgrade commands to their real merge epochs and guard timestamps in CI (#22498)
# Fix broken 2-19 upgrade command sequence (dev incident: `lastStreamError` / `workspaceDiscoverability`) ## Incident The dev environment (tracking main) throws: - `Property "lastStreamError" was not found in "AgentChatThreadEntity"` - `Cannot return null for non-nullable field Workspace.workspaceDiscoverability` ## Root cause The 2-19 upgrade commands were committed with **fabricated future timestamps** (year-2027 epochs like `1820000000000`). The upgrade cursor (`upgrade-aware-entity-metadata.adapter.ts`) tracks a **single** most-recent applied step: it looks up the latest `core."upgradeMigration"` row's name in the sequence (sorted by timestamp within kind) and hides every `@WasIntroducedInUpgrade` column at or past that index. Two ways this breaks, and both were live on main: 1. **Cursor regression**: a command merged *later* with a *smaller* timestamp (e.g. `pendingQuestion` at `1811…` after `metadata-overrides` at `1820…` had run) sorts *before* already-applied steps. When migrate runs, the "latest" row now points earlier in the sequence, re-hiding columns that were already applied. 2. **Migrate not running at all** (Felix's hypothesis): if the deploy pipeline skipped `database:migrate:prod`, none of the 2-19 rows exist and every 2-19-gated column is hidden. Both hypotheses have the same fix path; the discriminating query is in the verification section below. ## Fix **Real timestamps** (per maintainer direction — no more fabricated epochs): | Command | Old (fabricated) | New (real merge epoch) | Introduced by | |---|---|---|---| | workspace: backfill-workspace-custom-application-registration | `1820000000000` | `1782853718000` | #22378 | | fast: add-metadata-overrides-column | `1820000100000` | `1782986475000` | #22417 | | slow: backfill-metadata-overrides | `1820000110000` | `1782986476000` (+1s to order after its fast pair) | #22417 | | fast: add-last-stream-error-to-agent-chat-thread | `1821000000000` | `1782996657000` | #22434 | | fast: add-pending-question-to-agent-chat-thread | `1811000000000` | `1782999138000` | #22346 | | fast: add-workspace-discoverability-to-workspace | `1820000001000` | `1783004140000` | #22423 | Each value is the committer epoch of the squash-merge commit that introduced the command on main (verified via `git log --diff-filter=A`). Sorted by real time, the fast sequence is strictly increasing, so the cursor can no longer regress. **Idempotency**: renaming a command changes its step name, so every one of these re-runs on any instance that already applied it under the old name (dev cluster, edge self-hosters — 2.19 is unreleased, so tagged releases are unaffected). All six are now safe to re-run: - `lastStreamError`, `pendingQuestion`, `metadata-overrides` fast: `ADD COLUMN IF NOT EXISTS` (already were) - `metadata-overrides` slow backfill: `WHERE … IS NULL` guard (already was) - workspace command: skips when `applicationRegistrationId` is already set (already did) - `workspaceDiscoverability`: **made idempotent in this PR** — `CREATE TYPE` wrapped in a `duplicate_object` handler, `ADD COLUMN IF NOT EXISTS` **CI guard** (replaces the append-only check added earlier on this branch): - Timestamps must be **real**: within `[now − 60 days, now + 2 days]`. This is the check that would have prevented the original sin — 2027 epochs can never pass. - Still **append-only** within the version directory, but computed from `git diff --name-status --find-renames` so renamed/copied files are checked too (cubic's P2), and files the PR deletes/renames away no longer count toward the existing max (otherwise a re-slotting PR like this one could never pass its own guard). - Covers `workspace-command-<ts>-` filenames, not just `instance-command-fast|slow-<ts>-`; skips `.spec.ts` files. - Failure message documents the escape path (re-slot the fabricated blocker to its real epoch + make it idempotent) and a bypass label `ci:allow-upgrade-command-timestamp-exception` for deliberate exceptions. ## Deploy sequencing (important) After this merges and deploys, `database:migrate:prod` **must run** before the API pods are relied on: the old step names no longer exist in the sequence, so until the renamed commands run once, the cursor resolves to 0 and *every* gated column is hidden. The commands are idempotent, so the re-run is harmless. Running API/worker pods only compute the cursor at boot — restart them after migrate. ## Verification / diagnosis on dev ```sql SELECT name, status, "createdAt" FROM core."upgradeMigration" WHERE "workspaceId" IS NULL ORDER BY "createdAt" DESC LIMIT 15; ``` - Latest rows named `…_182xxxxxxxxxx` (fabricated) and completed → migrate ran, cursor regressed (hypothesis 1). - No 2-19 rows at all → migrate never ran for 2-19 (hypothesis 2). - After the fix: latest row should be `2.19.0_AddWorkspaceDiscoverabilityToWorkspaceFastInstanceCommand_1783004140000`, status `completed`. https://claude.ai/code/session_01Lyi6zTema2FMVVh8MD6c38 |
||
|
Félix Malfait
|
a28887bba6 |
feat(server): workspace opt-out of root-domain directory listing (#22423)
## What Lets a workspace opt out of being surfaced in the multi-workspace root-domain (app.twenty.com) picker via **email-domain discovery**. Adds `isDirectoryListingEnabled` (default `true`) on the workspace. When `false`, the workspace is filtered out of the approved-access-domain branch of `findAvailableWorkspacesByEmail`, so a user whose email domain matches an approved access domain no longer sees the workspace in the sign-up picker. ## Scope of the opt-out (deliberately narrow) The filter is applied **only** to the approved-access-domain discovery source: - **Members** (`availableWorkspacesForSignIn`) — never filtered; they keep access. - **Explicit invitations** — never filtered; the intent is one-to-one. - **Approved-access-domain discovery** — the only "listing" source, gated by the flag. A hidden workspace stays fully reachable by members and invited users via the direct workspace subdomain; it just isn't advertised in the global picker. > Open question for review: do we also want a stronger mode that hides the workspace from the root-domain picker even for existing members (forcing them to use the subdomain directly)? That would additionally filter the member/invitation sources and is a larger behavior change — not included here. ## Changes **Backend** - `workspace.entity.ts` — new `isDirectoryListingEnabled` column (`@Field`, default `true`). - `user-workspace.service.ts` — filter the approved-access-domain branch on the flag. - `update-workspace-input.ts` — expose the field on `updateWorkspace`. - `workspace.service.ts` — `PermissionFlagType.SECURITY` (same as the other discovery/security toggles). - Fast instance command adding the column (default `true`, so no existing workspace is hidden). **Frontend** - Settings > Security: a **"List in workspace directory"** toggle (shown only in multi-workspace mode) that flips the flag via `updateWorkspace`, mirroring the existing `isInternalMessagesImportEnabled` toggle. - Threaded the field through the current-user fragment, `CurrentWorkspace` type, and mock data. - Regenerated the metadata + client-sdk GraphQL types (`generated-metadata`, `twenty-client-sdk/.../generated`) — generated against a server booted from this branch. ## Verification - `tsgo` typecheck: 0 errors. `oxlint`: 0/0. `oxfmt`: clean. - Codegen diff verified to contain **only** the new field (no unrelated drift). - Tests not run locally; CI covers unit/integration + the codegen/migration freshness checks. --------- Co-authored-by: claude[bot] <41898282+claude[bot]@users.noreply.github.com> |
||
|
Félix Malfait
|
4aaf171d63 |
feat(ai): add ask_questions interactive clarifying-question tool (#22346)
## What & why Adds an `ask_questions` tool that lets the in-app **Ask AI** assistant **pause a turn to ask the user one or more multiple-choice questions** (per the [Figma design](https://www.figma.com/design/xt8O9mFeLl46C5InWwoMrN/Twenty?node-id=105959-117153)) and resume once answered — instead of guessing on ambiguous/consequential decisions. The tool is **harness-only**: an interactive question UI is meaningless without a user to answer it, so it must be absent from MCP and from head-less workflow agents. ## Design — true tool-result resume (not a synthetic user message) The user's answer is a **structured tool result bound to the `toolCallId`**, and the **same agent turn resumes** — exactly how Anthropic (`tool_result` by `tool_use_id`) and OpenAI (`function_call_output`) model human-in-the-loop. The naive form of this (leave the tool call in `input-available` to mean "pending") is **impossible** here: `finalizeDanglingToolParts` rewrites `input-available` → `output-error` ("Tool execution was interrupted") on both the persist path (`addMessage`) and the model-reload path (`chat-execution.service.ts`). That util is a load-bearing safety net, so weakening it is the wrong move. Instead: - `ask_questions` is an **inline, chat-only tool with an `execute` that returns a `status: 'pending'` result immediately**, so the tool part is always `output-available` and **immune to `finalizeDanglingToolParts`**. `stopWhen(hasToolCall('ask_questions'))` halts the turn right after the call (the model never sees the placeholder). - A nullable **`thread.pendingQuestionMessageId`** marker records that a turn is awaiting an answer. - The new **`answerAgentChatQuestion`** mutation atomically *claims* the question (clears the marker, marks the thread streaming), **writes the answer onto the same tool part** (`status: 'answered'`), and **re-enqueues the turn via the existing `existingTurnId` plumbing** (`isResume` bypasses the per-turn dedup guard). On resume `finalizeDanglingToolParts` leaves the `output-available` part untouched and `convertToModelMessages` emits `assistant(tool_use)` + `tool_result(answers)`, so the model continues. This achieves the platform-aligned semantics **without** weakening the finalize safety net or inventing a fragile new part state. ### Meets the two requirements - **Survives refresh, scoped per-thread** — the pending state is a normal persisted `output-available` part + the thread marker; the frontend card is derived per-thread from the loaded messages, so it re-appears on reload and only on its own thread. - **Takes priority over the queue** — a unified `isBlocked = activeStreamId || pendingQuestionMessageId` gate is applied in both `sendChatMessage` (new messages queue) and `flushNextQueuedMessage` (the drain). The queue cannot unpile until the question is answered and the resumed turn completes. ### Harness-only by construction `ask_questions` is added **only** to the chat's inline `activeTools` (like `learn_tools`/`execute_tool`/`load_skills`). It never enters the tool registry/catalog, so it is invisible to MCP and to workflow agents — no `MCP_EXCLUDED_TOOL_NAMES` entry needed. ## UX While a question is pending, the **composer is replaced by the question card** (matching the Figma): question title + pager (`1/2`), numbered option rows (`IconSquareNumber*`) with per-option info-icon descriptions and a "Recommended" badge, and the normal composer as the free-text fallback ("Type anything to do differently."). The transcript shows a compact "Asking questions…" status line that becomes an answered summary. ## Changes **twenty-shared** - `ai/types/AskQuestionsToolTypes.ts` — `AskQuestionItem/Option/Answer/Result`, `ASK_QUESTIONS_TOOL_NAME`. **twenty-server** - `ai-chat/tools/ask-questions.tool.ts` — inline tool factory (pending-result `execute`, zod schema, 1–4 questions × 2–4 options). - `chat-execution.service.ts` — add to `activeTools` + `preloadedToolNames`; `hasToolCall` in `stopWhen`. - `chat-system-prompts.const.ts` — when-to-use guidance. - `entities/agent-chat-thread.entity.ts` — `pendingQuestionMessageId` column. - `stream-agent-chat.job.ts` — set the marker on a question pause; bypass the dedup guard on resume; suppress the no-text warning for question pauses. - `agent-chat-streaming.service.ts` — gate `flushNextQueuedMessage`; `enqueueResumeStream`. - `agent-chat.resolver.ts` — gate `sendChatMessage`; `answerAgentChatQuestion` mutation. - `agent-chat.service.ts` — `resolvePendingQuestion` (atomic claim + write answer). - `dtos/agent-chat-question-answer.input.ts`, `ai.exception.ts` (`QUESTION_NOT_PENDING`), `utils/find-pending-question-part.util.ts`. **twenty-front** - `components/AiChatQuestionCard.tsx` — the interactive card (matches Figma tokens) + `__stories__/AiChatQuestionCard.stories.tsx`. - `components/AiChatEditorSection.tsx` — swap the composer for the card while pending. - `components/AiChatQuestionStatusRenderer.tsx` + branch in `AiChatAssistantMessageRenderer.tsx`. - `states/selectors/agentChatPendingQuestionComponentSelector.ts`, `types/AgentChatPendingQuestion.ts`. - `hooks/useSubmitQuestionAnswer.ts` + `utils/markQuestionAnswered.ts` (optimistic) + `graphql/mutations/answerAgentChatQuestion.ts`. A design doc lives at `packages/twenty-server/docs/ASK_USER_QUESTION_TOOL_PLAN.md`. ## Migration Adds a nullable `pendingQuestionMessageId` (uuid) column to `core.agentChatThread`. Needs a generated **fast instance command** (`database:migrate:generate --name addThreadPendingQuestion --type fast`) — see "Verification status". ## Tests - Server: `ask-questions.tool.spec.ts` (pending echo + schema bounds), `find-pending-question-part.util.spec.ts`. - Front: `markQuestionAnswered.test.ts`, plus the Storybook story. ## Verification status (please read) This branch was authored in an environment where the monorepo `yarn install` repeatedly failed on transient TLS resets from the package registry, so I could **not** locally run the mechanical gates. The logic was reviewed by hand and the `ai@6.0.97` exports used (`hasToolCall`, `stepCountIs`, `generateId`) were confirmed against the package's type defs. Still **TODO** (will rely on CI / a follow-up once deps install): - [ ] `nx run twenty-shared:generateBarrels` (the `ai/index.ts` export was added by hand; regen to reconcile) - [ ] `nx run twenty-front:graphql:generate` (new mutation + input type) - [ ] generate the fast instance command (migration) for the new column - [ ] `typecheck` + `lint:diff-with-main` (front + server) — expect minor import-ordering autofixes - [ ] run the unit tests **Screenshots:** reproducing the live flow needs an AI provider API key (to get the model to actually call `ask_questions`), which isn't available here. The card can be screenshotted from its **Storybook story** (`AiChatQuestionCard.stories.tsx`) with no API key — I'll add that image once deps install, or a reviewer can run `nx storybook twenty-front`. 🤖 Generated with [Claude Code](https://claude.com/claude-code) https://claude.ai/code/session_01AArS8H3y3Z1Qwm763xhPLB --- _Generated by [Claude Code](https://claude.ai/code/session_01AArS8H3y3Z1Qwm763xhPLB)_ <!-- This is an auto-generated description by cubic. --> <a href="https://cubic.dev/pr/twentyhq/twenty/pull/22346?utm_source=github" target="_blank" rel="noopener noreferrer" data-no-image-dialog="true"><picture><source media="(prefers-color-scheme: dark)" srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source media="(prefers-color-scheme: light)" srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img alt="Review in cubic" src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a> <!-- End of auto-generated description by cubic. --> |
||
|
Félix Malfait
|
d709467902 |
feat(ai): surface AI chat stream failures through one typed error channel (#22434)
## Context Investigating a report where the AI chat showed only a `...` spinner while the network response clearly contained `No AI models are available`. Root cause: terminal stream failures reach the client on **two mismatched channels**. | Representation | Persisted (survives reload) | Rendered by client | |---|---|---| | AI-SDK `error` chunk (inside `stream-chunk`) | ✅ RPUSH'd to Redis | ❌ dropped by `readUIMessageStream` (no message part, no error state) | | typed `stream-error` event | ❌ never persisted | ✅ sets the error atom | Live, the `stream-error` event renders. But on reload, `chatStreamCatchupChunks` replays only the persisted **error chunk** — which the reducer discards — and the streaming indicator never clears. ## Change Collapse to a single typed error contract: - **Suppress the opaque `error` chunk** in the stream job; every failure is surfaced through the typed `stream-error` event. Errors are mapped via `mapErrorToStreamError` so an `AiException` keeps its `AiExceptionCode` (e.g. `API_KEY_NOT_CONFIGURED` → the existing "AI not configured" banner) instead of leaking a raw string. - **Persist the terminal error** next to the accumulated chunks and expose it as an explicit `error { code message }` field on `ChatStreamCatchupChunks`, so a client catching up after a reload recovers it — no dependency on the AI SDK's internal chunk shape. - **Reset per-thread stream state at job start**, so a failed turn's leftover chunks/error never replay on the next stream. - **Client replays the catchup error** as a terminal `stream-error` event, which clears the streaming indicator and renders the error (fixes the infinite spinner on a stream that ended in error). ## Notes - `ChatStreamError` is a new metadata GraphQL type; generated types (twenty-front metadata + client-sdk) were hand-updated to keep the tree consistent and will be reconciled by CI's `graphql:generate` check if anything differs. - Server unit test added for the error mapping. No schema/DB migration. ## Test plan - [ ] With no AI provider configured, send a chat message → error renders immediately (not a spinner). - [ ] Reload the thread → the error still renders (recovered from catchup), indicator not spinning. - [ ] Configure a provider and send again → normal streaming; no stale error from the previous failed turn. <!-- This is an auto-generated description by cubic. --> <a href="https://cubic.dev/pr/twentyhq/twenty/pull/22434?utm_source=github" target="_blank" rel="noopener noreferrer" data-no-image-dialog="true"><picture><source media="(prefers-color-scheme: dark)" srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source media="(prefers-color-scheme: light)" srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img alt="Review in cubic" src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a> <!-- End of auto-generated description by cubic. --> |
||
|
Félix Malfait
|
38fbff465f |
chore(server): ship the 2.20 standardOverrides drop as a dormant command (#22448)
Follow-up to #22417, per [this thread](https://github.com/twentyhq/twenty/pull/22417#discussion_r3512187719): migrate the `2-20/README.md` placeholder into a real command using the `TWENTY_NEXT_VERSIONS` mechanism. ### What - Add `DropMetadataStandardOverridesColumnFastInstanceCommand`, registered against `2.20.0`. It boots (`2.20.0` is in `TWENTY_ALL_VERSIONS`) but stays **dormant** — the upgrade sequence only runs `TWENTY_CROSS_UPGRADE_SUPPORTED_VERSIONS` (previous + current), so it never executes during the 2.19 deploy and activates automatically when `nx version:bump` promotes 2.20 to current. - Name constant + unit test (SQL parity, registration against `2.20.0`, name-constant parity). - Register it in `instance-commands.constant.ts`. - Update the `standardOverrides` `@deprecated` comments on object/field metadata to point at the shipped command. - Delete `2-20/README.md`. - Document the "ship a command for a future version" flow in `docs/UPGRADE_COMMANDS.md` and `.cursor/rules/server-migrations.mdc` (the mechanism was previously undocumented). ### Note / correction to the README's plan The old README implied both the command **and** `@WasRemovedInUpgrade` could be added at 2.20 time. Only the command can ship now: the decorator's validator runs against the active sequence, so referencing a still-dormant 2.20 step fails boot with `unknown-step-name`. So the entity keeps its `WasRemovedInUpgrade<T>` type wrapper for now; the decorator gets wired (one line, via the name constant) once 2.20 is current — same deferred-drop shape as `isUIReadOnly`. ### Verification Could not run `jest`/`typecheck`/`lint` in this environment: `yarn install` is blocked by egress policy on a git-based transitive dep (`github.com/electron/node-gyp.git`). Verified by review against the sibling 2-19 add-column and 2-12 drop commands. **Please let CI run before merge.** https://claude.ai/code/session_01KMArJvdEmsX3eAmJLbS1b6 --- _Generated by [Claude Code](https://claude.ai/code/session_01KMArJvdEmsX3eAmJLbS1b6)_ <!-- This is an auto-generated description by cubic. --> <a href="https://cubic.dev/pr/twentyhq/twenty/pull/22448?utm_source=github" target="_blank" rel="noopener noreferrer" data-no-image-dialog="true"><picture><source media="(prefers-color-scheme: dark)" srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source media="(prefers-color-scheme: light)" srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img alt="Review in cubic" src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a> <!-- End of auto-generated description by cubic. --> |
||
|
Félix Malfait
|
5a4ebca226 |
refactor(server): unify the two metadata override mechanisms into one (#22417)
## Unify the two metadata override mechanisms into one Twenty had **two** override mechanisms: - **`standardOverrides`** — a bespoke JSONB column on `objectMetadata`/`fieldMetadata` with typed DTOs and a per-locale `translations` map, resolved by two i18n-aware resolvers. - **`OverridableEntity.overrides`** — a flat, registry-driven JSONB blob on view / view-field / view-field-group / command-menu-item / page-layout-tab / page-layout-widget, resolved by a plain spread. This PR collapses them into **one** concept: a single `overrides` blob, one registry-driven overridable set, one i18n-aware read path, and one write path (`computeMetadataOverridesBlob`, extracted in #22404). Object/field **stay on `SyncableEntity`** (not reparented to `OverridableEntity`) so their `isActive` default stays **FALSE** — this sidesteps the `isActive` default conflict entirely. ### GraphQL breaking change (accepted) The `standardOverrides` field is **removed** with no deprecation alias — `overrides` (a `JSON` scalar) is exposed instead on `Object` and `Field`. Product confirmed negligible external usage; the front-end has no hand-written consumer (only generated types), which are regenerated here. ### Commit structure (reviewable commit-by-commit) 1. **Unified resolver + parity harness** — `resolveEffectiveEntityProperty` is a strict superset of the three legacy resolvers; a corpus parity spec compares it against a *frozen reference* of the old logic across every locale, `isStandardApp` branch and override shape. 2. **Registry-driven** — object/field presentation props tagged `isOverridable` + `translatable`; the overridable/translatable sets are derived from the registry (a test asserts they equal the legacy hardcoded lists). 3. **Rename + swap + delete** — `standardOverrides` → `overrides` across entities, DTOs, flat/universal types, producers, the ~12 resolve/write/create/sync call sites, mocks and specs; the reconciler's two compare entries collapse to one; the three legacy resolvers, both DTOs and the hardcoded constants/types are deleted. 4. **Migration (zero-downtime, two-phase)** — split across two releases so a rolling deploy never drops a column a previous-release pod still `SELECT`s: - **2.19 fast** — add the `overrides` column (schema only). - **2.19 slow** — backfill `overrides` from `standardOverrides` in `runDataMigration` (kept out of the schema transaction so the bulk write doesn't hold the ACCESS EXCLUSIVE lock; skipped on fresh installs, which have no data to copy). - **2.20 fast** — drop the legacy `standardOverrides` column (gated by `TWENTY_NEXT_VERSIONS`, so it stays dormant until the instance reaches 2.20). 5. **Front/client-SDK regen** — regenerated metadata GraphQL types. 6. **Integration specs + i18n** — updated the standard object/field update integration specs + snapshots, and the reworded validator message catalog entry. ### Rolling-deploy safety `standardOverrides` is retained through 2.19 and only dropped in 2.20, mirroring the codebase's deferred-drop convention (`isUIReadOnly`/`isCustom`). During the 2.19 rollout both columns exist, so old and new pods coexist without "column does not exist" errors. The backfill lives in a slow `runDataMigration` (per the `no-data-mutation-in-fast-instance-command` rule) so it doesn't stall reads. ### `isActive` guard The migration never reads or writes `isActive`; the backfill asserts the active-row count is unchanged and aborts otherwise. Verified on a real DB: apply + revert preserves the blob **and** the nested `translations` map, with `isActive` counts identical before/after. ### Verification (local) - `nx typecheck twenty-server` + `nx typecheck twenty-front` — green - `nx lint:diff-with-main twenty-server` (oxlint `--type-aware` + oxfmt) — green - `nx test twenty-server` — green (unit + parity + registry + migration tests) - `nx run twenty-server:test:integration:with-db-reset` — green - `database:reset` applies the 2.19 phases and leaves **both** columns present (2.20 drop stays dormant); backfill + revert round-trip verified on a real DB - Metadata integration suites (standard object/field update, application sync) pass end-to-end against the two-column schema - Metadata GraphQL types regenerated against a booted server; zero `standardOverrides` references remain in application code (only the migration commands + the legacy schema baseline) --------- Co-authored-by: prastoin <paul@twenty.com> |
||
|
Paul Rastoin
|
2b1417770e |
SearchVector derivation via migration-scoped index (alt to #2622 __warmedUpCache) (#22389)
## What this is close https://github.com/twentyhq/core-team-issues/issues/2622 A **POC / discussion branch** implementing the runner-scoped alternative to the `__warmedUpCache` design in [core-team-issues#2622](https://github.com/twentyhq/core-team-issues/issues/2622). Not for merge as-is — meant to diff against that plan. ## Problem `deriveSearchVectorAsExpressionForTsVectorField` scans the entire `flatSearchFieldMetadataMaps` (`Object.values(...).filter(...)`) once per object created in a migration. On install that's `O(objectsCreated × totalSearchFields)` — the quadratic #2622 targets. Only **one** of the three call sites is actually hot: - `create-object` (runner) — global maps, called per created object → the quadratic - export DDL — maps already built **per object** (O(k)) - `update-field` rebuild — one field, gated on `rebuildSearchVector` ## Approach Instead of a private `__warmedUpCache` side-channel on `FlatEntityMaps<T>` + drain-on-hydration, this keeps the index in the **consumer**: 1. `derive` now takes `targetSearchFieldMetadatas` (already scoped to the tsVector field) instead of scanning the map itself. 2. The runner builds a `Map<tsVectorFieldMetadataId, searchFields[]>` **once per migration**, lazily, and threads it through the action context. Safe because `searchFieldMetadata` creates are ordered before `objectMetadata` creates (`computeOrderedMigrationActions`), so the map is complete on first use. → `O(totalSearchFields)`. 3. `getTargetSearchFieldMetadatasForTsVectorField` (O(total) filter) stays as the fallback for the one-off callers (export, field-update) and when the accessor isn't provided. ## Why this over `__warmedUpCache` - **No `FlatEntityMaps<T>` type widening**, no convention-only privacy, no id/universalIdentifier drain to keep in sync. - **No referential-integrity obligation.** The index only ever contains entities present in the map, so the "search field created-then-deleted before its object hydrates" case (deferred as an edge in #2622) can't put a stale id into an aggregator and crash `derive` via the `-orThrow` lookup. - **One `derive` path**, not "aggregator + direct-filter fallback for export". - Blast radius: ~220 lines, mostly a new util + test. ## Benchmark (micro, isolated function) Median of 7 trials, 10 search fields per object, running the real shipped utils — old = `getTargetSearchFieldMetadatasForTsVectorField` once per object (identical to the old inline scan), new = `buildSearchFieldMetadatasByTsVectorFieldId` once + N lookups (both assert they resolve the same fields): | objects | total search fields | old (scan/obj) | new (index once) | speedup | |--------:|--------------------:|---------------:|-----------------:|--------:| | 50 | 500 | 2.08 ms | 0.06 ms | 33× | | 100 | 1,000 | 9.26 ms | 0.12 ms | 77× | | 200 | 2,000 | 36.2 ms | 0.23 ms | 160× | | 400 | 4,000 | 151 ms | 0.40 ms | 379× | | 800 | 8,000 | 701 ms | 0.92 ms | 766× | Confirms the old path is quadratic (~4× per doubling of object count) and the new path is linear (sub-ms throughout). **Caveats — read these before trusting the speedup:** - This is the **isolated derivation function**, no DB / DDL / inserts. In a real `create-object` action the derive is a small fraction of per-action cost, so the end-to-end win is far smaller than the ratios above. - A default workspace has ~20–30 objects, where the **old** code already costs only ~1–2 ms total across the whole install. The quadratic only becomes material (>50 ms, the runner's slow-action threshold) around **200–400 objects**. - The measurement that should actually gate this — `[install-perf] create:objectMetadata` on a real install against a real DB with a few hundred objects — has **not** been run yet. The micro-benchmark bounds the upside and locates the knee of the curve; it does not prove end-to-end payoff. ## Not done on purpose - **No end-to-end benchmark yet** — step 0 should still be measuring `[install-perf] create:objectMetadata` on a real large install to confirm the quadratic is worth removing at all. - Relies on the ordering invariant (commented at the build site). The fully self-contained variant is to put the object's search fields on `FlatCreateObjectAction` (builder change) — deliberately left out to keep this runner-scoped. ## Checks `nx typecheck twenty-server`, `nx lint:diff-with-main twenty-server`, new util spec + existing `generate-workspace-schema-ddl` spec all green. |
||
|
Félix Malfait
|
1df00698cf |
feat(server): make workspace Custom application carry an applicationRegistration so custom labels are translatable (#22378)
## Why Custom objects/fields belong to a per-workspace **Custom** application (`workspace.workspaceCustomApplicationId`). That application was created with `applicationRegistrationId = null`. Because the metadata label resolver loads a translation catalog from `core.applicationTranslation` **keyed by `applicationRegistrationId`** (`ApplicationTranslationCacheService.getCatalog` → `applicationTranslationCatalogLoader` → `resolveObjectMetadataStandardOverride` / `resolveFieldMetadataStandardOverride`), the Custom app had no catalog and custom labels always resolved to the raw source string. This is the foundational slice: it wires up the missing key so custom labels can be translated **exactly like any installed third-party app**. The read/resolve path already works once a catalog exists — confirmed end-to-end. `flatApplicationMaps` carries `applicationRegistrationId` straight from the entity column, so setting it + recomputing that cache is all that's needed. ## What changed - **`ApplicationService.createWorkspaceCustomApplication`** now creates a workspace-scoped `applicationRegistration` and links it to the Custom application. This covers both production creation sites (sign-in-up and the dev-seeder), which are the only callers. - **New idempotent workspace upgrade command** `upgrade:2-18:backfill-workspace-custom-application-registration` creates a registration for each existing workspace's Custom application that lacks one and links it. It delegates the registration lifecycle (create + link + `flatApplicationMaps` recompute) to `ApplicationService`, so the command only decides *which* workspaces need it. - New `WORKSPACE_CUSTOM_APPLICATION_NAME` constant; the registration creation lives in `ApplicationService.createWorkspaceCustomApplicationRegistration`. ## Design decisions - **Per-workspace registration (not a shared "custom" registration).** `applicationTranslation` is keyed *only* by `applicationRegistrationId` (cross-workspace). A shared registration would force every workspace's custom translations into one catalog keyed by `generateMessageId(sourceText)`, guaranteeing cross-workspace collisions and leakage (two workspaces both naming an object "Project" would clash). Each workspace's Custom app gets its own registration (`ownerWorkspaceId = workspaceId`, `universalIdentifier = the Custom app's per-workspace uuid`) and thus an isolated catalog — matching installed-app behaviour, where `application.universalIdentifier === registration.universalIdentifier`. - **Source-label keying kept** (`generateMessageId(sourceLabel)`). The resolve path and the third-party manifest pipeline both key catalogs this way. Re-keying by a stable `universalIdentifier` would require changing the shared resolver/dataloader for *all* apps and would break marketplace manifest translations — out of scope for this slice. Consequence: renaming a label orphans its catalog entry (it falls back to the source label until re-translated) — the same behaviour an installed app has when it changes a source string. Re-keying on rename can be handled later by the interactive write path. - **Workspace command (not instance command)** for the backfill: it is per-workspace data logic that must recompute the per-workspace `flatApplicationMaps` cache the resolver reads from. It is idempotent (skips Custom apps that already have a registration), supports `--dry-run`, and is forward-only by design. - **Interactive write path deferred** as an explicit follow-up. This slice proves the read/resolve path; an editor that writes custom translations into `applicationTranslation` (+ cache invalidation) is the natural next step. ## Tests - **Unit test** for the backfill command: creation + linking, idempotency, dry-run, and the skip paths. - **Integration test** (`custom-application-translation.integration-spec.ts`): on a freshly created workspace (so the registration's translation cache is guaranteed cold), it asserts the Custom application is created with a registration, seeds an `applicationTranslation` row, and verifies a custom object's label resolves from that catalog while a label with no catalog entry falls back to its source label. ## Notes for reviewers - No new entity columns or migrations beyond the workspace command — `ApplicationRegistrationEntity` already supports a workspace-scoped `workspaceId`. - The backfill follows the established upgrade-command pattern: it imports `ApplicationModule` and delegates to `ApplicationService` (consistent with the other version-command modules). https://claude.ai/code/session_018heTgu4ew4AJ99VVz4bjqd --- _Generated by [Claude Code](https://claude.ai/code/session_018heTgu4ew4AJ99VVz4bjqd)_ <!-- This is an auto-generated description by cubic. --> <a href="https://cubic.dev/pr/twentyhq/twenty/pull/22378?utm_source=github" target="_blank" rel="noopener noreferrer" data-no-image-dialog="true"><picture><source media="(prefers-color-scheme: dark)" srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source media="(prefers-color-scheme: light)" srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img alt="Review in cubic" src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a> <!-- End of auto-generated description by cubic. --> |
||
|
neo773
|
101b85db7b |
messaging remove dead workspace entities (#22366)
<!-- This is an auto-generated description by cubic. --> <a href="https://cubic.dev/pr/twentyhq/twenty/pull/22366?utm_source=github" target="_blank" rel="noopener noreferrer" data-no-image-dialog="true"><picture><source media="(prefers-color-scheme: dark)" srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source media="(prefers-color-scheme: light)" srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img alt="Review in cubic" src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a> <!-- End of auto-generated description by cubic. --> --------- Co-authored-by: prastoin <paul@twenty.com> |
||
|
Paul Rastoin
|
3d00dd4066 |
feat(server): add 2.18 recompute-search-vectors upgrade command (#22355)
## Summary closes https://github.com/twentyhq/core-team-issues/issues/2620 Adds the 2.18 `recompute-search-vectors` workspace upgrade command — Part 2 of https://github.com/twentyhq/core-team-issues/issues/2620 (Part 1, the GIN-index rebuild fix, merged in #22349). It uniformizes every workspace's `TS_VECTOR` (`searchVector`) columns onto the new derive-from-`searchFieldMetadata` model and drops the now-dead cached settings: - **Recomputes** every searchVector column (re-derives its generated expression from the `searchFieldMetadata` rows) and **recreates its GIN index** — relying on the Part 1 rebuild fix. - **Clears** the deprecated cached `TS_VECTOR` settings (`asExpression` / `generatedType`), which nothing reads anymore. ## How New command `RecomputeSearchVectorsCommand` (`@RegisteredWorkspaceCommand('2.18.0', 1799200001000)`), per active/suspended workspace: 1. Load `flatFieldMetadataMaps`, enumerate every `FieldMetadataType.TS_VECTOR` field. Skip (log) if none. 2. Support `--dry-run` (log the count, no writes). 3. Build one `update-field` action per TS_VECTOR field and run them in a **single migration** via `workspaceMigrationRunnerService.run(...)`: ```ts update: { universalSettings: null }, // clears the deprecated cached settings rebuildSearchVector: true, // re-derive column + recreate GIN index ``` `universalSettings: null` transpiles to `settings: null`, so one atomic action both clears settings and triggers the rebuild; `runner.run` invalidates the field-metadata cache afterward. ## Why these choices - **Single migration under the standard app** covers standard, custom, and installed-app search vectors at once — the runner operates per workspace-schema table regardless of a field's owning application, and the update-field handler doesn't use `flatApplication`. - **Ordering is safe**: the upgrade sequence runs fast-instance → slow-instance → workspace commands per version, so the 2.18 `tsVectorFieldMetadataId` backfill (which the expression derivation depends on) is guaranteed to have run first. - **Cost**: this is a deliberate full rebuild — it drops/re-adds every searchVector STORED column (table rewrite per searchable object) and recreates each GIN index, per workspace, under the workspace iterator. Intentional ("uniformize for everyone"), as noted in the issue. ## Test plan - [x] `npx nx typecheck twenty-server` - [x] `npx nx lint:diff-with-main twenty-server` Closes Part 2 of https://github.com/twentyhq/core-team-issues/issues/2620 <!-- This is an auto-generated description by cubic. --> <a href="https://cubic.dev/pr/twentyhq/twenty/pull/22355?utm_source=github" target="_blank" rel="noopener noreferrer" data-no-image-dialog="true"><picture><source media="(prefers-color-scheme: dark)" srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source media="(prefers-color-scheme: light)" srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img alt="Review in cubic" src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a> <!-- End of auto-generated description by cubic. --> |
||
|
Paul Rastoin
|
3e53a16b27 |
Clear orphan search field metadata backfill tsVectorFieldMetadataId (#22353)
Instead of invariant throw in instance slow, auto recover by deleting orphan search field metadata as in the end they would just end up as dead metadata <!-- This is an auto-generated description by cubic. --> <a href="https://cubic.dev/pr/twentyhq/twenty/pull/22353?utm_source=github" target="_blank" rel="noopener noreferrer" data-no-image-dialog="true"><picture><source media="(prefers-color-scheme: dark)" srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source media="(prefers-color-scheme: light)" srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img alt="Review in cubic" src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a> <!-- End of auto-generated description by cubic. --> |
||
|
neo773
|
9f3ebaaf22 |
feat(messaging): sync draft emails and edit them in the thread composer (#22178)
Stop excluding drafts from sync across all three providers (Gmail DRAFT label, Microsoft/IMAP Drafts folder) and add an isDraft boolean field on Message so drafts are queryable by the API and AI agents. Drafts render in the thread with a Draft tag; clicking one opens the existing reply composer pre-filled with the draft's recipients, subject and body, and Send reuses the existing send-email flow. <!-- This is an auto-generated description by cubic. --> <a href="https://cubic.dev/pr/twentyhq/twenty/pull/22178?utm_source=github" target="_blank" rel="noopener noreferrer" data-no-image-dialog="true"><picture><source media="(prefers-color-scheme: dark)" srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source media="(prefers-color-scheme: light)" srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img alt="Review in cubic" src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a> <!-- End of auto-generated description by cubic. --> --------- Co-authored-by: Félix Malfait <felix.malfait@gmail.com> |
||
|
Paul Rastoin
|
b0d7516951 |
Deprecate asExpression from field metadata search_vector (#22287)
## Summary Fully deprecates the cached `asExpression` / `generatedType` settings on `TS_VECTOR` (searchVector) fields. Previously the generated-column expression was stored in `FieldMetadataSettings` and kept in sync via imperative recompute side-effects. It is now **derived at DDL time** from the `searchFieldMetadata` rows that describe which fields feed the search vector, making `searchFieldMetadata` the single source of truth and removing a whole class of cache-drift bugs. This is delivered across the milestones tracked in #2587 and coordinates with the frontend migration (#1428). ## Why - The searchVector expression lived in two places (stored `settings.asExpression` + the actual generated column), kept consistent by bespoke side-effects (`recompute-search-vector-on-field-rename`, label-identifier recompute, etc.). - The frontend reconstructed the searchable-fields list by **regex-parsing** the stored `asExpression`. - Both are brittle. Deriving the expression from `searchFieldMetadata` rows at build/run time removes the cache and the parsing. ## What changed ### Server - data model & derivation - Introduce the `tsVectorFieldMetadata` relation on `searchFieldMetadata` (`tsVectorFieldMetadataId` / universal identifier) linking each searchable-field row to its target `TS_VECTOR` field. - New runtime derivation `deriveSearchVectorAsExpressionForTsVectorField` (`flat-search-field-metadata/utils/...`) used by the create-object and update-field handlers to generate the column expression from `searchFieldMetadata` rows. - Remove `asExpression` / `generatedType` from stored settings: `FieldMetadataSettings.TS_VECTOR` is now `null`; the column builder (`generate-column-definitions.util.ts`) hardcodes `generatedType: 'STORED'` and requires the derived expression. - Delete the imperative recompute side-effects and the `compute-search-vector-universal-settings-from-object-manifest` path; drop the `settings` block from all 28 standard `compute-*-standard-flat-field-metadata` utils. ### Server - migration runner - New `rebuildSearchVector` marker on `update-field` actions: the orchestrator synthesizes targeted column rebuilds (`compute-search-vector-rebuild-target-universal-identifiers.util.ts` + the deprioritize aggregator) only when a searchFieldMetadata change or indexed-field rename actually requires it - instead of rebuilding on every settings touch. - Deferrable FKs + in-flight ID resolution so a `searchFieldMetadata` row and its `TS_VECTOR` field can be created in the same transaction (deterministic UUIDs). ### Frontend (contract change, #1428) - New `SearchFieldMetadataDTO` + dataloader exposing `searchFieldMetadataList` on object metadata. - `SettingsObjectSearchSection` now reads `objectMetadataItem.searchFieldMetadatas` instead of parsing `asExpression`; new `SearchFieldMetadataItem` type, fragment, and mapping updates. ### Upgrade commands (2.18) - `2-18-instance-command-fast-...-add-ts-vector-field-metadata-id-to-search-field-metadata` - `2-18-instance-command-fast-...-make-search-field-metadata-fks-deferrable` - `2-18-instance-command-slow-...-backfill-ts-vector-field-metadata-id-on-search-field-metadata` (These were relocated from 2.16 to 2.18 and re-timestamped into an ordered block - add column -> make FK deferrable -> backfill data - since 2.16/2.17 are released.) ### Tests - Updated search-vector side-effect integration specs to assert behavior (search works) rather than the now-removed `asExpression`; removed the obsolete expression-validation specs; refreshed the application-sync snapshot (`universalSettings: null`). ## Upgrade / compatibility notes - Existing workspaces keep their stored `settings` until a later cleanup; nothing reads it anymore. The new derivation drives all DDL going forward. - Schema changes are gated behind the 2.18 instance commands above. ## Known follow-up (separate PR) https://github.com/twentyhq/core-team-issues/issues/2620 - The column rebuild (`DROP`/`ADD` of the `searchVector` STORED column) cascade-drops its GIN index and does not recreate it - a pre-existing regression on `main` inherited here. A follow-up PR will fix the rebuild handler to recreate the GIN index and add a 2.18 workspace command to recompute every search vector + strip the deprecated settings. (Planned.) ## Test plan - [ ] `npx nx typecheck twenty-server` / `twenty-front` - [ ] `npx nx lint:diff-with-main twenty-server` / `twenty-front` - [ ] Server integration: create/update/delete field, rename indexed field, update object - search returns expected records - [ ] Run the 2.18 instance commands on a seeded DB; verify `tsVectorFieldMetadataId` backfilled and FKs deferrable - [ ] Frontend: object Search settings tab lists the correct searchable fields (no `asExpression` parsing) close https://github.com/twentyhq/core-team-issues/issues/2587 <!-- This is an auto-generated description by cubic. --> <a href="https://cubic.dev/pr/twentyhq/twenty/pull/22287?utm_source=github" target="_blank" rel="noopener noreferrer" data-no-image-dialog="true"><picture><source media="(prefers-color-scheme: dark)" srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source media="(prefers-color-scheme: light)" srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img alt="Review in cubic" src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a> <!-- End of auto-generated description by cubic. --> |
||
|
Félix Malfait
|
107245e56d |
fix(server): add workspaceMember jobTitle field without view-field side effects (#22306)
## Problem The `2.17` `AddWorkspaceMemberJobTitleField` upgrade command failed for every workspace that has a **custom field on `workspaceMember`** (it aborted the per-workspace upgrade sequence). Root cause: The command used `FieldMetadataService.createManyFields`, which — besides creating the field metadata — also creates the new field's **view fields** across the object's existing views. Validating a view-field creation enumerates the target view's full `viewFieldUniversalIdentifiers`, which includes view fields owned by **another application** (a custom field a user added to `workspaceMember`). Those cross-application view fields are filtered out of the build's application-scoped dependency maps, so the build throws `Could not find flat entity with universal identifier ...`. ## Fix (command-scoped) Create the field metadata **only — no view fields** — sourced from the standard-application definition, mirroring `AddInactiveGenericStandardFieldsCommand` (which adds a standard field the same way and is unaffected by this bug). The standard view-field side effects are reconciled as code, so they should not be produced at runtime here. Concretely: source the standard `jobTitle` `FlatFieldMetadata` from `computeTwentyStandardApplicationAllFlatEntityMaps`, blank its `viewFieldIds` / `viewFieldUniversalIdentifiers`, and run a `fieldMetadata`-only `validateBuildAndRunWorkspaceMigration` instead of `createManyFields`. Also drops the now-unused `FieldMetadataModule` import from the 2-17 command module. ## Trade-off Existing workspaces get the `jobTitle` field + column but **no view field**, so it won't appear as a default column in `workspaceMember` views until the standard-application reconciliation adds view fields. `jobTitle` is `isSystem` / `isUIReadOnly`, so this is acceptable as the immediate unblock. ## Scope This is the small, immediate unblock for the failed `2.17` upgrade. The general framework fix (load cross-application children of involved parents into the build scope) is tracked separately in #22294. <!-- This is an auto-generated description by cubic. --> <a href="https://cubic.dev/pr/twentyhq/twenty/pull/22306?utm_source=github" target="_blank" rel="noopener noreferrer" data-no-image-dialog="true"><picture><source media="(prefers-color-scheme: dark)" srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source media="(prefers-color-scheme: light)" srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img alt="Review in cubic" src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a> <!-- End of auto-generated description by cubic. --> |
||
|
Marie
|
4eb28b73c7 |
fix(server): normalize legacy index names (command) (#22053)
## TL;DR Adds a workspace upgrade command that normalizes index names to the current v2 deterministic naming convention (IDX_ prefix). This will close https://github.com/twentyhq/twenty/issues/21383 : uniqueness constraint cannot be disabled (for users who set it before v2 determinist naming is enforced). ### Background The deterministic index name embeds the table name, columns, uniqueness and where clause. The naming convention changed on **2025-09-23** (#14567 - added the `IDX_`/`IDX_UNIQUE_` prefix and folded the table name into the hash). Index rows created before that kept their old name in `core."indexMetadata"`, and nothing rewrites it (only targeted phone/relation rebuilds got new names). Code paths that locate an index by recomputing its expected name then miss these legacy-named rows. The most visible symptom is #21383: toggling a field's `isUnique` from `true` → `false` recomputes the expected unique-index name, fails to find the legacy-named index, and silently no-ops — so uniqueness can't be disabled. ### What the command does Per workspace, for each index: - recomputes the expected name with the same generator the app uses (`generateFlatIndexMetadataWithNameOrThrow`); - if the stored name differs → **rename** it (metadata `UPDATE` + a new metadata-only `ALTER INDEX … RENAME`, which preserves the unique constraint with no rebuild/lock); - if a correctly-named twin already exists (the legacy + freshly-generated duplicate case) → **drop the redundant** one (physical + metadata, field rows cascade) and keep the canonical; - invalidates the metadata cache so the running app + `isUnique` derivation reflect the new names. Honors `--dry-run`, wraps writes in a transaction (rolls back on error), and skips (with a warning) any single index whose name can't be recomputed so it can't abort the whole workspace. ### Notable changes - New `renameIndex` on `WorkspaceSchemaIndexManagerService` (`ALTER INDEX IF EXISTS … RENAME`). - Planning logic extracted into a pure, unit-tested util (`planIndexNameNormalization`). |
||
|
Félix Malfait
|
41c10b9ee7 |
feat(server): resolve app-owned metadata translations at runtime (#22235)
## Summary First of a **4-PR stack** that lets apps built with `twenty-sdk` translate their metadata, resolved at runtime. The standard Twenty app is modelled as "an app like any other" — `NULL applicationRegistrationId` ⟺ the standard app, no special-casing. This PR adds the server foundation and wires runtime resolution for **object** and **field** metadata: - New `applicationTranslation` core table + entity (nullable `applicationRegistrationId`, `locale`, `messages` jsonb), one row per (app, locale) to avoid multi-MB rows. - `ApplicationTranslationCacheService` (process-local, 30s TTL) + `ApplicationTranslationSyncService` (upsert + soft-delete from a manifest). - Shared `translateStandardLabel` util: application catalog → i18n bundle → source value. - Object/field resolvers + dataloaders prefetch and apply the per-app catalog. The new `applicationCatalog` param is **optional**, so standard behaviour is byte-unchanged. - Fast instance command to create the table. ## Stack **PR 1/4**, targets `main`. Followed by: (2) twenty-sdk extract/compile → `manifest.translations`, (3) resolution across the remaining metadata resolvers, (4) the per-locale standard-override editor. ## Tests Unit: `translateStandardLabel`, `resolveObjectMetadataStandardOverride` (including the application-catalog path). ## Verification note The remote dev environment for this branch could not complete `yarn install` (no package-registry egress), so typecheck/lint/tests were not run locally — **CI is the source of truth** for this stack. Changes follow existing patterns. https://claude.ai/code/session_01NiE7o3cd3zCLZarVkJa6UA --- _Generated by [Claude Code](https://claude.ai/code/session_01NiE7o3cd3zCLZarVkJa6UA)_ <!-- This is an auto-generated description by cubic. --> <a href="https://cubic.dev/pr/twentyhq/twenty/pull/22235?utm_source=github" target="_blank" rel="noopener noreferrer" data-no-image-dialog="true"><picture><source media="(prefers-color-scheme: dark)" srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source media="(prefers-color-scheme: light)" srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img alt="Review in cubic" src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a> <!-- End of auto-generated description by cubic. --> |
||
|
Félix Malfait
|
538b180824 |
feat(dpa): self-serve Data Processing Agreement generator (#22243)
## What
A single, region-aware DPA that serves all customers, generated
automatically from the customer's deployment. Two layers:
1. **Click-through DPA** — recorded at signup (acceptance = execution),
resolving merge fields from the deployment region. Cloud only.
2. **In-app signed-PDF generator** — Settings → Legal → Generate DPA:
preview the agreement, enter legal entity + authorized signatory,
download a PDF pre-signed by Twenty, and store the executed copy against
the workspace with its template version + timestamp. Deep-linkable at
`/dpa` (login-gated) for `twenty.com/dpa`.
## How it resolves
A typed variable matrix (`dpa-region-config.constant.ts`) maps the
deployment region to the contracting Processor entity and terms:
- **EU (default)** → Twenty.com SAS, hosting EU/Frankfurt, governing law
France, SCC section dormant.
- **US (custom)** → Twenty, Inc., hosting US, SCC section active.
Region is a deployment-wide setting (`DPA_DEPLOYMENT_REGION`, default
EU) behind a `DpaRegionService` seam so it can later become
per-workspace without touching callers. The legal text is verbatim from
the template (generated into `dpa-template.constant.ts` directly from
the source `.docx`); only the 6 merge fields are filled and the SCC
sections (7.2–7.5) stay in the document for every region per the spec —
only field values branch. Sub-processors are deferred to
trust.twenty.com (not enumerated). Billing stays decoupled (Twenty, Inc.
remains merchant of record regardless of Processor).
## UI
Standard list + create-page pattern (mirrors API keys / webhooks): a
list of executed copies (with re-download) — or the agreement preview
when none exists — and a top-right blue **Generate DPA** CTA opening a
standard create page. The "Legal" item is intentionally **not** in the
settings menu; the page is reached via the `/dpa` deep link.
## Notable implementation details
- **PDF** is rendered server-side with `@react-pdf/renderer`. The
built-in standard-14 fonts only encode ASCII and crash on the template's
curly quotes / em–en dashes / accented Latin, so Liberation Sans (OFL)
is **subset to a Latin glyph set and embedded as base64 data: URLs** —
no font files to ship or resolve at runtime (works in dev, prod-Docker
and CI).
- New `core.dpaAgreement` table via a fast instance command (FK hash
reproduced to match TypeORM).
- Self-hosted deployments (billing disabled) skip click-through
recording and stamp a prominent "not a valid agreement" banner on the
preview and PDF.
## Tests
- Unit: resolver (per-region entity/law/SCC state, EU default, no
unresolved `{{ }}`, SCC sections present in both regions, self-hosted
notice) and HTML renderer.
- Integration (`test/integration/graphql/suites/dpa`): preview has no
unresolved fields; `generateSignedDpa` renders + persists + returns a
downloadable PDF (asserted with accented input to guard the font
regression); list re-download.
## ⚠ Needs legal input before go-live (marked `TODO_CONFIRM` in
`dpa-region-config.constant.ts`)
- Registered-office addresses for Twenty.com SAS and Twenty, Inc.
- US deployment governing law (the template only specifies France).
- DPO name and the Twenty pre-signed authorized signatory name/title.
## Out of scope (flagged per spec)
Intra-group legal agreement and any Stripe/billing-entity changes. A
future e-sign provider would plug in at `DpaService.generateSignedDpa` +
the signatory input.
> Draft until the integration test passes in CI and the legal
`TODO_CONFIRM` values are supplied.
https://claude.ai/code/session_01Ahjydxx6J1souz1s1NeA9a
---
_Generated by [Claude
Code](https://claude.ai/code/session_01Ahjydxx6J1souz1s1NeA9a)_
<!-- This is an auto-generated description by cubic. -->
<a
href="https://cubic.dev/pr/twentyhq/twenty/pull/22243?utm_source=github"
target="_blank" rel="noopener noreferrer"
data-no-image-dialog="true"><picture><source
media="(prefers-color-scheme: dark)"
srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source
media="(prefers-color-scheme: light)"
srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img
alt="Review in cubic"
src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a>
<!-- End of auto-generated description by cubic. -->
|
||
|
Raphaël Bosi
|
c891258f34 |
Add v2 onboarding create profile page (#22221)
<img width="3024" height="1498" alt="CleanShot 2026-06-26 at 15 30 23@2x" src="https://github.com/user-attachments/assets/8b4863a9-66ed-4da1-851b-473cedf71511" /> <img width="3022" height="1500" alt="CleanShot 2026-06-26 at 15 29 43@2x" src="https://github.com/user-attachments/assets/22fc0e94-f670-4638-975c-f06b2b2e25e8" /> Adds the v2 onboarding **Create profile** page, shown right after the import-contacts step (`PROFILE_CREATION`) for the onboarding-v2 cohort. It renders full-screen under `BlankLayout` via the shared `OnboardingV2Layout`, matching the Figma (340px column, inline round avatar uploader + First/Last row, Job Title, dark Continue). The v1 modal flow is untouched and still used for non-v2 users. Job Title is wired end-to-end: it adds a real `jobTitle` field to the `WorkspaceMember` standard object (shared metadata constant + flat field metadata + entity property) and a `2-17` workspace upgrade command to backfill the field on existing workspaces. Continue persists name + jobTitle through the existing `updateWorkspaceMemberSettings` mutation, whose allow-list picks up the new standard field automatically. Routing mirrors `SyncEmailsV2`: new `AppPath.CreateProfileV2`, lazy route, and an `isOnboardingV2`-gated branch in `usePageChangeEffectNavigateLocation` (+ tests and a Storybook story). Reviewer notes: - `jobTitle` is **write-only** for now (no read-back path: core DTO/transpiler/fragment unchanged), and the field is `isSystem`/non-UI-editable to match its siblings. Easy to surface later if wanted. - New `OnboardingProfilePictureUploader` is a compact round avatar uploader reusing the same upload mutation flow as `WorkspaceMemberPictureUploader`. <!-- This is an auto-generated description by cubic. --> <a href="https://cubic.dev/pr/twentyhq/twenty/pull/22221?utm_source=github" target="_blank" rel="noopener noreferrer" data-no-image-dialog="true"><picture><source media="(prefers-color-scheme: dark)" srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source media="(prefers-color-scheme: light)" srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img alt="Review in cubic" src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a> <!-- End of auto-generated description by cubic. --> |
||
|
neo773
|
9747e3a7a3 |
feat(messaging): link emails by Reply-To as a REPLY_TO participant (#22216)
Relay senders (e.g. a website form sending as a shared address with the real contact in Reply-To) never linked to the contact because matching only used From/To/Cc/Bcc. Record Reply-To addresses under a new REPLY_TO participant role across the Gmail, Microsoft and IMAP drivers, excluding any that just repeat the sender. Adds the REPLY_TO option to the messageParticipant role field and a 2.17 workspace command to backfill it for existing workspaces. QAed with real test run <!-- This is an auto-generated description by cubic. --> <a href="https://cubic.dev/pr/twentyhq/twenty/pull/22216?utm_source=github" target="_blank" rel="noopener noreferrer" data-no-image-dialog="true"><picture><source media="(prefers-color-scheme: dark)" srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source media="(prefers-color-scheme: light)" srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img alt="Review in cubic" src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a> <!-- End of auto-generated description by cubic. --> |
||
|
Félix Malfait
|
ea9e11581c |
feat(billing): replace Stripe trial emails with fair, well-timed reminders (#22186)
## Why We currently rely on Stripe's automated trial-ending email. It misfires: the global "remind 7 days before trial ends" setting lands the reminder on **signup day** for the 7‑day no‑card trial, and the "your card will be charged" copy makes no sense for a trial with no card. This replaces it with our own honest, well‑timed, Twenty‑branded emails. ## 🔒 Safety — these emails are OFF by default Because these reach real customers, the whole feature is gated behind a kill‑switch that **defaults to `false`**: - **`BILLING_REMINDER_EMAILS_ENABLED` (default `false`)** — checked **both** at cron registration **and** on every job run (defense in depth), so the emails can never be sent inadvertently (not on deploy, not in staging, not via a stray trigger). They only go out once an operator explicitly opts in. - Also gated on `IS_BILLING_ENABLED` (cloud‑only; self‑hosters unaffected). - In non‑prod the email driver is typically `logger`, so even if enabled there, nothing is actually sent. A unit test asserts that with the switch off, **zero** emails are produced. ## What it does A daily cron (`0 8 * * *`) sends three honest, Twenty‑branded emails: | Plan | Email | When | |---|---|---| | No‑card trial (7d) | "Add a card to keep your data" | **1 day before** trial ends | | Card‑on‑file trial (30d) | Upcoming‑charge heads‑up (cancel in one click) | **7 days before** first charge | | Yearly subscription | Renewal reminder (no surprise) | **7 days before** each renewal | - **Monthly renewals get no reminder** (avoids noise) — only the first charge and annual renewals do. - Branches no‑card vs with‑card on the customer's payment‑method flag (with a trial‑duration fallback), so someone who adds a card mid‑trial correctly gets the charge heads‑up instead of the add‑a‑card one. - **Idempotent** per `(workspace, boundary date)` via workspace‑level user vars — yearly reminders re‑fire each period, but the daily cron never double‑sends. - Offsets are configurable via new `BILLING_*_REMINDER_DAYS_BEFORE` variables. Also **warms up the tone** of the existing suspended / deleted workspace emails (less robotic, fair, loss‑aversion framing) — these already act as the "come back or lose your data" win‑back, so no extra win‑back email was added. ## Rollout 1. Merge. 2. Disable Stripe's automated trial/renewal customer emails in the Stripe dashboard. 3. Review copy/timing, then set `BILLING_REMINDER_EMAILS_ENABLED=true` to turn the cron on. ## Notes for reviewers - **i18n:** new English strings render via Lingui's msgid fallback; translation catalogs are intentionally **not** included to keep the diff focused (the repo extracts translations via its standard periodic `lingui extract` sync — `main` already carries catalog drift). Diff is 18 code files. - **Recipients:** reminders go to all workspace members, consistent with the existing suspension emails. Happy to scope the charge‑related ones to billing admins if preferred. - **Follow‑ups discussed:** in‑app trial banner, loss‑aversion with real record counts, and failed‑payment dunning are the higher‑leverage conversion levers beyond this. ## Test plan - [x] `typecheck` (twenty-server, twenty-emails) - [x] oxlint type‑aware + oxfmt - [x] Unit tests: no‑card path, with‑card path, idempotency, yearly renewal, billing‑disabled, **kill‑switch off → no send** (6/6 green) - [ ] Manual: set the flag on a staging instance with `logger` driver and confirm the right email is logged at each boundary https://claude.ai/code/session_0147ujzHv1X4vzimf4iGbnT4 --- _Generated by [Claude Code](https://claude.ai/code/session_0147ujzHv1X4vzimf4iGbnT4)_ <!-- This is an auto-generated description by cubic. --> <a href="https://cubic.dev/pr/twentyhq/twenty/pull/22186?utm_source=github" target="_blank" rel="noopener noreferrer" data-no-image-dialog="true"><picture><source media="(prefers-color-scheme: dark)" srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source media="(prefers-color-scheme: light)" srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img alt="Review in cubic" src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a> <!-- End of auto-generated description by cubic. --> |
||
|
nitin
|
f86bf637d1 |
[BREAKING CHANGE] remove call recording feature flag and backfill upgrade command for existing command menu items navigation command (#22176)
<!-- This is an auto-generated description by cubic. --> <a href="https://cubic.dev/pr/twentyhq/twenty/pull/22176?utm_source=github" target="_blank" rel="noopener noreferrer" data-no-image-dialog="true"><picture><source media="(prefers-color-scheme: dark)" srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source media="(prefers-color-scheme: light)" srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img alt="Review in cubic" src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a> <!-- End of auto-generated description by cubic. --> |
