5a4ebca2261899c8ef054c051a427092b9b9c4d4
224 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Félix Malfait
|
5a4ebca226 |
refactor(server): unify the two metadata override mechanisms into one (#22417)
## Unify the two metadata override mechanisms into one Twenty had **two** override mechanisms: - **`standardOverrides`** — a bespoke JSONB column on `objectMetadata`/`fieldMetadata` with typed DTOs and a per-locale `translations` map, resolved by two i18n-aware resolvers. - **`OverridableEntity.overrides`** — a flat, registry-driven JSONB blob on view / view-field / view-field-group / command-menu-item / page-layout-tab / page-layout-widget, resolved by a plain spread. This PR collapses them into **one** concept: a single `overrides` blob, one registry-driven overridable set, one i18n-aware read path, and one write path (`computeMetadataOverridesBlob`, extracted in #22404). Object/field **stay on `SyncableEntity`** (not reparented to `OverridableEntity`) so their `isActive` default stays **FALSE** — this sidesteps the `isActive` default conflict entirely. ### GraphQL breaking change (accepted) The `standardOverrides` field is **removed** with no deprecation alias — `overrides` (a `JSON` scalar) is exposed instead on `Object` and `Field`. Product confirmed negligible external usage; the front-end has no hand-written consumer (only generated types), which are regenerated here. ### Commit structure (reviewable commit-by-commit) 1. **Unified resolver + parity harness** — `resolveEffectiveEntityProperty` is a strict superset of the three legacy resolvers; a corpus parity spec compares it against a *frozen reference* of the old logic across every locale, `isStandardApp` branch and override shape. 2. **Registry-driven** — object/field presentation props tagged `isOverridable` + `translatable`; the overridable/translatable sets are derived from the registry (a test asserts they equal the legacy hardcoded lists). 3. **Rename + swap + delete** — `standardOverrides` → `overrides` across entities, DTOs, flat/universal types, producers, the ~12 resolve/write/create/sync call sites, mocks and specs; the reconciler's two compare entries collapse to one; the three legacy resolvers, both DTOs and the hardcoded constants/types are deleted. 4. **Migration (zero-downtime, two-phase)** — split across two releases so a rolling deploy never drops a column a previous-release pod still `SELECT`s: - **2.19 fast** — add the `overrides` column (schema only). - **2.19 slow** — backfill `overrides` from `standardOverrides` in `runDataMigration` (kept out of the schema transaction so the bulk write doesn't hold the ACCESS EXCLUSIVE lock; skipped on fresh installs, which have no data to copy). - **2.20 fast** — drop the legacy `standardOverrides` column (gated by `TWENTY_NEXT_VERSIONS`, so it stays dormant until the instance reaches 2.20). 5. **Front/client-SDK regen** — regenerated metadata GraphQL types. 6. **Integration specs + i18n** — updated the standard object/field update integration specs + snapshots, and the reworded validator message catalog entry. ### Rolling-deploy safety `standardOverrides` is retained through 2.19 and only dropped in 2.20, mirroring the codebase's deferred-drop convention (`isUIReadOnly`/`isCustom`). During the 2.19 rollout both columns exist, so old and new pods coexist without "column does not exist" errors. The backfill lives in a slow `runDataMigration` (per the `no-data-mutation-in-fast-instance-command` rule) so it doesn't stall reads. ### `isActive` guard The migration never reads or writes `isActive`; the backfill asserts the active-row count is unchanged and aborts otherwise. Verified on a real DB: apply + revert preserves the blob **and** the nested `translations` map, with `isActive` counts identical before/after. ### Verification (local) - `nx typecheck twenty-server` + `nx typecheck twenty-front` — green - `nx lint:diff-with-main twenty-server` (oxlint `--type-aware` + oxfmt) — green - `nx test twenty-server` — green (unit + parity + registry + migration tests) - `nx run twenty-server:test:integration:with-db-reset` — green - `database:reset` applies the 2.19 phases and leaves **both** columns present (2.20 drop stays dormant); backfill + revert round-trip verified on a real DB - Metadata integration suites (standard object/field update, application sync) pass end-to-end against the two-column schema - Metadata GraphQL types regenerated against a booted server; zero `standardOverrides` references remain in application code (only the migration commands + the legacy schema baseline) --------- Co-authored-by: prastoin <paul@twenty.com> |
||
|
Paul Rastoin
|
2b1417770e |
SearchVector derivation via migration-scoped index (alt to #2622 __warmedUpCache) (#22389)
## What this is close https://github.com/twentyhq/core-team-issues/issues/2622 A **POC / discussion branch** implementing the runner-scoped alternative to the `__warmedUpCache` design in [core-team-issues#2622](https://github.com/twentyhq/core-team-issues/issues/2622). Not for merge as-is — meant to diff against that plan. ## Problem `deriveSearchVectorAsExpressionForTsVectorField` scans the entire `flatSearchFieldMetadataMaps` (`Object.values(...).filter(...)`) once per object created in a migration. On install that's `O(objectsCreated × totalSearchFields)` — the quadratic #2622 targets. Only **one** of the three call sites is actually hot: - `create-object` (runner) — global maps, called per created object → the quadratic - export DDL — maps already built **per object** (O(k)) - `update-field` rebuild — one field, gated on `rebuildSearchVector` ## Approach Instead of a private `__warmedUpCache` side-channel on `FlatEntityMaps<T>` + drain-on-hydration, this keeps the index in the **consumer**: 1. `derive` now takes `targetSearchFieldMetadatas` (already scoped to the tsVector field) instead of scanning the map itself. 2. The runner builds a `Map<tsVectorFieldMetadataId, searchFields[]>` **once per migration**, lazily, and threads it through the action context. Safe because `searchFieldMetadata` creates are ordered before `objectMetadata` creates (`computeOrderedMigrationActions`), so the map is complete on first use. → `O(totalSearchFields)`. 3. `getTargetSearchFieldMetadatasForTsVectorField` (O(total) filter) stays as the fallback for the one-off callers (export, field-update) and when the accessor isn't provided. ## Why this over `__warmedUpCache` - **No `FlatEntityMaps<T>` type widening**, no convention-only privacy, no id/universalIdentifier drain to keep in sync. - **No referential-integrity obligation.** The index only ever contains entities present in the map, so the "search field created-then-deleted before its object hydrates" case (deferred as an edge in #2622) can't put a stale id into an aggregator and crash `derive` via the `-orThrow` lookup. - **One `derive` path**, not "aggregator + direct-filter fallback for export". - Blast radius: ~220 lines, mostly a new util + test. ## Benchmark (micro, isolated function) Median of 7 trials, 10 search fields per object, running the real shipped utils — old = `getTargetSearchFieldMetadatasForTsVectorField` once per object (identical to the old inline scan), new = `buildSearchFieldMetadatasByTsVectorFieldId` once + N lookups (both assert they resolve the same fields): | objects | total search fields | old (scan/obj) | new (index once) | speedup | |--------:|--------------------:|---------------:|-----------------:|--------:| | 50 | 500 | 2.08 ms | 0.06 ms | 33× | | 100 | 1,000 | 9.26 ms | 0.12 ms | 77× | | 200 | 2,000 | 36.2 ms | 0.23 ms | 160× | | 400 | 4,000 | 151 ms | 0.40 ms | 379× | | 800 | 8,000 | 701 ms | 0.92 ms | 766× | Confirms the old path is quadratic (~4× per doubling of object count) and the new path is linear (sub-ms throughout). **Caveats — read these before trusting the speedup:** - This is the **isolated derivation function**, no DB / DDL / inserts. In a real `create-object` action the derive is a small fraction of per-action cost, so the end-to-end win is far smaller than the ratios above. - A default workspace has ~20–30 objects, where the **old** code already costs only ~1–2 ms total across the whole install. The quadratic only becomes material (>50 ms, the runner's slow-action threshold) around **200–400 objects**. - The measurement that should actually gate this — `[install-perf] create:objectMetadata` on a real install against a real DB with a few hundred objects — has **not** been run yet. The micro-benchmark bounds the upside and locates the knee of the curve; it does not prove end-to-end payoff. ## Not done on purpose - **No end-to-end benchmark yet** — step 0 should still be measuring `[install-perf] create:objectMetadata` on a real large install to confirm the quadratic is worth removing at all. - Relies on the ordering invariant (commented at the build site). The fully self-contained variant is to put the object's search fields on `FlatCreateObjectAction` (builder change) — deliberately left out to keep this runner-scoped. ## Checks `nx typecheck twenty-server`, `nx lint:diff-with-main twenty-server`, new util spec + existing `generate-workspace-schema-ddl` spec all green. |
||
|
Weiko
|
fab0358df5 |
Handle field isNullable update (#22362)
## Context Setting isNullable on a field via the app SDK manifest was silently ignored when re-syncing an existing field. The first sync that creates a field honored isNullable correctly, but any later manifest change to isNullable had no effect, neither on the field metadata nor on the underlying Postgres column. Two compounding gaps caused this: The diff never detected the change. isNullable was configured with toCompare: false, so compareTwoFlatEntity excluded it from the diff and no update action was ever generated. There was no DDL to apply it. Even if detected, the update field action handler only altered name, options, defaultValue, and settings. The column manager had no way to alter a column's NOT NULL constraint. ## Fix - Set isNullable.toCompare: true so manifest changes are detected and persisted to the field metadata (via the existing executeForMetadata path). - Add WorkspaceSchemaColumnManagerService.alterColumnNullable(): emits SET NOT NULL / DROP NOT NULL, with an optional pre-serialized backfill (UPDATE … WHERE col IS NULL) applied only on the nullable → non-nullable transition. - Add handleFieldNullableUpdate() to the update field action handler, dispatched after the defaultValue block so the default is in place before NOT NULL is enforced. It is composite-aware (mirrors the per-sub-column parentIsNullable || !property.isRequired rule used at column creation) and skips relation/morph join columns and TS_VECTOR, which are always nullable by design. <!-- This is an auto-generated description by cubic. --> <a href="https://cubic.dev/pr/twentyhq/twenty/pull/22362?utm_source=github" target="_blank" rel="noopener noreferrer" data-no-image-dialog="true"><picture><source media="(prefers-color-scheme: dark)" srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source media="(prefers-color-scheme: light)" srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img alt="Review in cubic" src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a> <!-- End of auto-generated description by cubic. --> |
||
|
Paul Rastoin
|
fe644a0630 |
fix(server): recreate searchVector GIN index on rebuild (#22349)
## Summary Fixes a pre-existing regression where rebuilding a `TS_VECTOR` (`searchVector`) generated column drops its GIN index without recreating it, leaving search correct but **unindexed** (sequential scan). Changing a generated column's expression requires `DROP COLUMN` + `ADD COLUMN` (Postgres can't `ALTER` a generated expression). The `searchVector`'s GIN index is a separate index-metadata entity built on that column, so the `DROP COLUMN` cascade-drops the physical index — and the rebuild branch never re-issued `CREATE INDEX`. This existed on `main` (triggered by `asExpression`/`generatedType` settings changes) and was inherited by the `rebuildSearchVector` refactor in #22287. This is the first, self-contained part of https://github.com/twentyhq/core-team-issues/issues/2620. The 2.18 recompute/backfill workspace command is intentionally left for a follow-up PR. ## What changed ### Runner loads the maps a rebuild needs `workspace-migration-runner.service.ts` — `fieldMetadata` declares neither `searchFieldMetadata` nor `index` as a related metadata name, so a `fieldMetadata`-only rebuild action had neither `flatSearchFieldMetadataMaps` (needed by the expression derivation) nor `flatIndexMaps` (needed to recreate the index) in context. The runner now detects `update` actions carrying `rebuildSearchVector === true` and loads those two maps — **only** when a rebuild is present, so ordinary field operations are unaffected. ### Handler recreates the index after re-adding the column `update-field-action-handler.service.ts` — in the rebuild branch, after `addColumns`, recreate the field's single GIN index: ```ts const [searchVectorFlatIndexMetadata] = findFieldRelatedIndexes({ flatFieldMetadata: optimisticFlatFieldMetadata, flatObjectMetadata, flatIndexMaps, }); if (isDefined(searchVectorFlatIndexMetadata)) { await createIndexInWorkspaceSchema({ flatIndexMetadata: searchVectorFlatIndexMetadata, ... }); } ``` - **Narrow lookup, not a workspace-wide scan.** The flat field has no index back-reference (`fieldMetadata.indexFieldMetadatas` is `null` in `ALL_ONE_TO_MANY_METADATA_RELATIONS`). The *object* does aggregate its indexes (`indexMetadataIds`), so we reuse the existing `findFieldRelatedIndexes` helper — already used by `handle-index-changes-during-field-update.util.ts` and the morph-rename path — which resolves only this object's indexes and filters to the one on the field. - A `TS_VECTOR` field has exactly one index (the standard `searchVectorGinIndex`), so we retrieve that single index rather than iterating. `createIndex` emits `CREATE INDEX IF NOT EXISTS` (idempotent). This makes the rebuild self-contained (column + index move together) and fixes every rebuild path: rename, label-identifier change, and `searchFieldMetadata` changes. ### Regression test Extends `update-one-field-metadata-search-vector-side-effect.integration-spec.ts` to query `pg_indexes` before and after the rename and assert the GIN index on the `searchVector` column persists (not just that search still returns the record). Fails without the fix, passes with it. ## Test plan - [x] `npx nx lint:diff-with-main twenty-server` — 0 warnings, 0 errors - [x] `npx nx typecheck twenty-server` — clean for changed files - [ ] Integration: extended rename-rebuild spec (GIN index present post-rebuild) Part of https://github.com/twentyhq/core-team-issues/issues/2620 |
||
|
Etienne
|
c4a6446757 |
fix(navigation-menu-item): reject PAGE_LAYOUT items that don't reference a STANDALONE_PAGE layout (#22343)
## Issue A `PAGE_LAYOUT` navigation menu item can be created pointing at a page layout whose type is **not** `STANDALONE_PAGE` (e.g. a `DASHBOARD`). The sidebar always links such an item to `/page/<pageLayoutId>`, but that route only renders `STANDALONE_PAGE` layouts, anything else is redirected to 404. Result: a silently broken sidebar link (cc: https://discord.com/channels/1130383047699738754/1519045990047285288). ## Root cause - `/page/:pageLayoutId` is standalone-only by design (route guard in `usePageChangeEffectNavigateLocation`, and `StandalonePageLayoutPage` hardcodes `layoutType: STANDALONE_PAGE`). Dashboards/record pages are reached elsewhere (record show page). - A `PAGE_LAYOUT` nav item unconditionally computes `/page/<pageLayoutId>`. - No validation ensured the referenced layout is `STANDALONE_PAGE`: the migration/manifest validator only checked that `pageLayoutId` was present, the DB constraint only checked `NOT NULL`, and the runtime tool description even suggested pinning dashboards this way. So an app manifest pairing a `DASHBOARD` layout with a `PAGE_LAYOUT` nav item installed cleanly and produced a dead link. ## Fix (treat as invalid config — fail fast) - Cross-entity validation in `FlatNavigationMenuItemValidatorService` (both create and update): when `type === PAGE_LAYOUT`, resolve the referenced page layout from the optimistic page-layout maps and raise `INVALID_NAVIGATION_MENU_ITEM_INPUT` if its `type !== STANDALONE_PAGE`. Existence keeps being enforced by foreign-key resolution, so the type check only fires when the layout resolves. - Corrected the misleading `create_navigation_menu_item` tool description (no longer says "e.g. a dashboard"; states the target must be a `STANDALONE_PAGE`). - Added unit tests covering: `STANDALONE_PAGE` accepted; `DASHBOARD` rejected; `RECORD_PAGE` rejected; unresolved reference not flagged as a type error. ## Files changed - `flat-navigation-menu-item-validator.service.ts` — new `validatePageLayoutReference` + wired into create/update. - `create-navigation-menu-item.tool.ts` — tool description fix. - `__tests__/flat-navigation-menu-item-validator.service.spec.ts` — new tests (4 passing). ## Out of scope / follow-up - To open discussion, check https://github.com/twentyhq/twenty/pull/22255 <!-- This is an auto-generated description by cubic. --> <a href="https://cubic.dev/pr/twentyhq/twenty/pull/22343?utm_source=github" target="_blank" rel="noopener noreferrer" data-no-image-dialog="true"><picture><source media="(prefers-color-scheme: dark)" srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source media="(prefers-color-scheme: light)" srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img alt="Review in cubic" src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a> <!-- End of auto-generated description by cubic. --> |
||
|
Paul Rastoin
|
b0d7516951 |
Deprecate asExpression from field metadata search_vector (#22287)
## Summary Fully deprecates the cached `asExpression` / `generatedType` settings on `TS_VECTOR` (searchVector) fields. Previously the generated-column expression was stored in `FieldMetadataSettings` and kept in sync via imperative recompute side-effects. It is now **derived at DDL time** from the `searchFieldMetadata` rows that describe which fields feed the search vector, making `searchFieldMetadata` the single source of truth and removing a whole class of cache-drift bugs. This is delivered across the milestones tracked in #2587 and coordinates with the frontend migration (#1428). ## Why - The searchVector expression lived in two places (stored `settings.asExpression` + the actual generated column), kept consistent by bespoke side-effects (`recompute-search-vector-on-field-rename`, label-identifier recompute, etc.). - The frontend reconstructed the searchable-fields list by **regex-parsing** the stored `asExpression`. - Both are brittle. Deriving the expression from `searchFieldMetadata` rows at build/run time removes the cache and the parsing. ## What changed ### Server - data model & derivation - Introduce the `tsVectorFieldMetadata` relation on `searchFieldMetadata` (`tsVectorFieldMetadataId` / universal identifier) linking each searchable-field row to its target `TS_VECTOR` field. - New runtime derivation `deriveSearchVectorAsExpressionForTsVectorField` (`flat-search-field-metadata/utils/...`) used by the create-object and update-field handlers to generate the column expression from `searchFieldMetadata` rows. - Remove `asExpression` / `generatedType` from stored settings: `FieldMetadataSettings.TS_VECTOR` is now `null`; the column builder (`generate-column-definitions.util.ts`) hardcodes `generatedType: 'STORED'` and requires the derived expression. - Delete the imperative recompute side-effects and the `compute-search-vector-universal-settings-from-object-manifest` path; drop the `settings` block from all 28 standard `compute-*-standard-flat-field-metadata` utils. ### Server - migration runner - New `rebuildSearchVector` marker on `update-field` actions: the orchestrator synthesizes targeted column rebuilds (`compute-search-vector-rebuild-target-universal-identifiers.util.ts` + the deprioritize aggregator) only when a searchFieldMetadata change or indexed-field rename actually requires it - instead of rebuilding on every settings touch. - Deferrable FKs + in-flight ID resolution so a `searchFieldMetadata` row and its `TS_VECTOR` field can be created in the same transaction (deterministic UUIDs). ### Frontend (contract change, #1428) - New `SearchFieldMetadataDTO` + dataloader exposing `searchFieldMetadataList` on object metadata. - `SettingsObjectSearchSection` now reads `objectMetadataItem.searchFieldMetadatas` instead of parsing `asExpression`; new `SearchFieldMetadataItem` type, fragment, and mapping updates. ### Upgrade commands (2.18) - `2-18-instance-command-fast-...-add-ts-vector-field-metadata-id-to-search-field-metadata` - `2-18-instance-command-fast-...-make-search-field-metadata-fks-deferrable` - `2-18-instance-command-slow-...-backfill-ts-vector-field-metadata-id-on-search-field-metadata` (These were relocated from 2.16 to 2.18 and re-timestamped into an ordered block - add column -> make FK deferrable -> backfill data - since 2.16/2.17 are released.) ### Tests - Updated search-vector side-effect integration specs to assert behavior (search works) rather than the now-removed `asExpression`; removed the obsolete expression-validation specs; refreshed the application-sync snapshot (`universalSettings: null`). ## Upgrade / compatibility notes - Existing workspaces keep their stored `settings` until a later cleanup; nothing reads it anymore. The new derivation drives all DDL going forward. - Schema changes are gated behind the 2.18 instance commands above. ## Known follow-up (separate PR) https://github.com/twentyhq/core-team-issues/issues/2620 - The column rebuild (`DROP`/`ADD` of the `searchVector` STORED column) cascade-drops its GIN index and does not recreate it - a pre-existing regression on `main` inherited here. A follow-up PR will fix the rebuild handler to recreate the GIN index and add a 2.18 workspace command to recompute every search vector + strip the deprecated settings. (Planned.) ## Test plan - [ ] `npx nx typecheck twenty-server` / `twenty-front` - [ ] `npx nx lint:diff-with-main twenty-server` / `twenty-front` - [ ] Server integration: create/update/delete field, rename indexed field, update object - search returns expected records - [ ] Run the 2.18 instance commands on a seeded DB; verify `tsVectorFieldMetadataId` backfilled and FKs deferrable - [ ] Frontend: object Search settings tab lists the correct searchable fields (no `asExpression` parsing) close https://github.com/twentyhq/core-team-issues/issues/2587 <!-- This is an auto-generated description by cubic. --> <a href="https://cubic.dev/pr/twentyhq/twenty/pull/22287?utm_source=github" target="_blank" rel="noopener noreferrer" data-no-image-dialog="true"><picture><source media="(prefers-color-scheme: dark)" srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source media="(prefers-color-scheme: light)" srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img alt="Review in cubic" src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a> <!-- End of auto-generated description by cubic. --> |
||
|
martmull
|
b49225df4b |
Reorder validation execution to match migration action order (#22200)
## Summary Reorders the validation execution sequence in the workspace entity migration builder to match the actual execution order of migration actions (delete → create → update). This ensures that optimistic entity maps accurately simulate the post-migration state during validation. ## Key Changes - **Moved creation validation before update validation** in `WorkspaceEntityMigrationBuilderService`: Creation validation now executes immediately after deletion validation, allowing updates to reference entities created in the same migration without validators needing to peek into to-be-created maps. - **Removed `remainingFlatEntityMapsToValidate` parameter from update validation**: Since creation validation now completes before update validation begins, the optimistic maps already contain all created entities. Updates can safely reference newly created entities through the optimistic maps without needing access to remaining-to-create maps. - **Simplified `FlatNavigationMenuItemValidatorService`**: Removed the logic that combined remaining-to-create maps with optimistic maps, now relying solely on the optimistic maps which contain all previously validated creations. - **Updated type definition**: Modified `FlatEntityUpdateValidationArgs` type to exclude `remainingFlatEntityMapsToValidate` since it's no longer needed. ## Implementation Details This change enables a more intuitive validation flow where: 1. Deletions are validated first 2. Creations are validated next (in topological order for self-referential FKs) 3. Updates are validated last (can safely reference newly created entities) The optimistic maps are progressively built during creation validation, so by the time update validation runs, they faithfully represent the post-migration state, eliminating the need for validators to access separate remaining-to-create maps. <!-- This is an auto-generated description by cubic. --> <a href="https://cubic.dev/pr/twentyhq/twenty/pull/22200?utm_source=github" target="_blank" rel="noopener noreferrer" data-no-image-dialog="true"><picture><source media="(prefers-color-scheme: dark)" srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source media="(prefers-color-scheme: light)" srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img alt="Review in cubic" src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a> <!-- End of auto-generated description by cubic. --> |
||
|
Abdul Rahman
|
db5338cf32 |
Feat: group records by many to one relation (#22123)
## Group records by relation (Kanban + Table)
Adds grouping by `MANY_TO_ONE` relation fields on both board and table
views, reusing the existing `ViewGroup` storage (`fieldValue = related
record id`).
- **New group** record picker to create relation-backed groups (board
column + table row)
- Relation-aware group headers (name/avatar), filtering, and drag-drop
(writes the FK join column) — all using one canonical `${name}Id` column
- Sort menu hides alphabetical options when grouping by a relation (no
comparable title)
- A group whose backing record no longer exists renders a "Deleted" chip
instead of a blank header
- **Backend:** allow `MANY_TO_ONE` relations as the Kanban
`mainGroupByField` in the flat-view validator
https://github.com/user-attachments/assets/267077a6-2667-4506-b178-eee420a16f20
<!-- This is an auto-generated description by cubic. -->
<a
href="https://cubic.dev/pr/twentyhq/twenty/pull/22123?utm_source=github"
target="_blank" rel="noopener noreferrer"
data-no-image-dialog="true"><picture><source
media="(prefers-color-scheme: dark)"
srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source
media="(prefers-color-scheme: light)"
srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img
alt="Review in cubic"
src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a>
<!-- End of auto-generated description by cubic. -->
|
||
|
Weiko
|
108cc5b6a5 |
API key creation triggers unnecessary ORMEntityMetadatas cache recomputation (#22168)
## Context
Creating an API key only changes role/apiKey-related data, but the
workspace migration runner was also invalidating and recomputing the
metadata caches (`ORMEntityMetadatas` and `graphQLResolverNameMap`) on
it.
The root cause is a single `if` block in
`getLegacyCacheInvalidationPromises` that gated **all** caches with an
`||` condition combining the metadata and role/permission conditions:
```ts
if (
shouldIncrementMetadataGraphqlSchemaVersion ||
shouldInvalidateRoleMapCache ||
shouldInvalidateRolesPermissionsCache
) {
// recomputes role caches AND ORMEntityMetadatas + graphQLResolverNameMap
}
```
So any role-only change (such as API key creation, which sets
`shouldInvalidateRoleMapCache`) also recomputed `ORMEntityMetadatas` —
an expensive recomputation that does not depend on role data.
## Fix
Split the combined block into two independent blocks, each gated by its
own condition:
- `shouldIncrementMetadataGraphqlSchemaVersion` → invalidate/recompute
only the metadata-derived caches: `ORMEntityMetadatas` and
`graphQLResolverNameMap`
- `shouldInvalidateRoleMapCache ||
shouldInvalidateRolesPermissionsCache` → invalidate/recompute only the
role/permissions caches
`graphQLResolverNameMap` is built from `flatObjectMetadataMaps` (see
`WorkspaceResolverNameMapCacheService`), so it is grouped with
`ORMEntityMetadatas` in the metadata block rather than the role block.
## Result
- Role-only changes (e.g. API key creation) no longer trigger
unnecessary `ORMEntityMetadatas` / `graphQLResolverNameMap`
recomputation.
- Metadata-only changes no longer recompute the role/permissions caches.
|
||
|
martmull
|
29e0327063 |
fix(server): allow moving menu items into a folder created in the same sync (#22130)
## Context Fixes [core-team-issues#2593](https://github.com/twentyhq/core-team-issues/issues/2593). When reorganizing navigation menu items by moving existing items into a **newly created folder** within a single deploy, the sync failed with `Parent navigation menu item not found`, forcing a two-step deploy (create the folder first, then move the items into it). ## Root cause Migration entities are validated in the fixed order **delete → update → create** (`workspace-entity-migration-builder.service.ts`). When items are moved into a new folder in one sync, the items are *updated* (adding `folderUniversalIdentifier`) while the folder is *created* — but the update phase runs before the create phase, so the folder isn't yet in the optimistic maps. The **creation** validator already handles "parent doesn't exist yet" by also checking `remainingFlatEntityMapsToValidate`. The **update** validator couldn't: `FlatEntityUpdateValidationArgs` explicitly omitted that field, so it only looked at the optimistic maps and threw. ## Changes - `universal-flat-entity-update-validation-args.type.ts` — stop omitting `remainingFlatEntityMapsToValidate` from the update args. - `workspace-entity-migration-builder.service.ts` — pass `createdFlatEntityMaps` (entities being created in the same migration) into update validation. - `flat-navigation-menu-item-validator.service.ts` — resolve the parent folder against both the optimistic maps and the to-be-created entities, mirroring the creation validator. - Integration test — sync an item, then in a second sync create a folder and move the item into it, asserting it succeeds in a single deploy. The change is generic and type-safe: all other update validators receive the new field and simply ignore it. `createdFlatEntityMaps` is `MetadataUniversalFlatEntityMaps<T>`, matching the field's type. ## Test plan - [x] Added integration test `should move existing menu items into a folder created in the same sync` - [ ] CI green https://claude.ai/code/session_017pmBkho9Fh6Vjv8WA4m9YE --- _Generated by [Claude Code](https://claude.ai/code/session_017pmBkho9Fh6Vjv8WA4m9YE)_ <!-- This is an auto-generated description by cubic. --> <a href="https://cubic.dev/pr/twentyhq/twenty/pull/22130?utm_source=github" target="_blank" rel="noopener noreferrer" data-no-image-dialog="true"><picture><source media="(prefers-color-scheme: dark)" srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source media="(prefers-color-scheme: light)" srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img alt="Review in cubic" src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a> <!-- End of auto-generated description by cubic. --> |
||
|
Abhinav A P
|
cf91b87892 |
fix(server): skip defaultValue null check for relation/morph fields on update (#21875)
## Description
Updating any metadata property (e.g. `description`, `label`) of an
existing **non-nullable RELATION** field fails with:
```
INVALID_FIELD_INPUT: Default value cannot be null for non-nullable fields
```
A relation field has no literal `defaultValue` (it's always `null`), so
the update-path validator rejects every required relation. **Creating**
the same field is fine — only **updates** fail.
This also blocks any incremental app re-sync (`yarn twenty dev --once`)
whose diff touches a required relation field.
## Fix
Added a guard in
`FlatFieldMetadataValidatorService.validateFlatFieldMetadataUpdate()`
using the already-imported `isMorphOrRelationUniversalFlatFieldMetadata`
utility to skip the `defaultValue === null` check for relation/morph
field types:
```diff
if (
+ !isMorphOrRelationUniversalFlatFieldMetadata(
+ flatFieldMetadataToValidate,
+ ) &&
flatFieldMetadataToValidate.isNullable === false &&
flatFieldMetadataToValidate.defaultValue === null
) {
```
### Why this works:
- Relation fields represent foreign key relationships, not columns with
literal defaults
- The same guard is already used at line 144 in the same method for
relation-specific validation
- The create path (`validateFlatFieldMetadataCreation`) never had this
check, which is why creation always worked
- No new imports needed — `isMorphOrRelationUniversalFlatFieldMetadata`
is already imported on line 14
## Verification
- `npx nx build twenty-server` ✅ compiles successfully
Fixes #21751
<!-- This is an auto-generated description by cubic. -->
<a
href="https://cubic.dev/pr/twentyhq/twenty/pull/21875?utm_source=github"
target="_blank" rel="noopener noreferrer"
data-no-image-dialog="true"><picture><source
media="(prefers-color-scheme: dark)"
srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source
media="(prefers-color-scheme: light)"
srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img
alt="Review in cubic"
src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a>
<!-- End of auto-generated description by cubic. -->
---------
Co-authored-by: Charles Bochet <charles@twenty.com>
Co-authored-by: prastoin <paul@twenty.com>
|
||
|
Paul Rastoin
|
d2387430a1 |
Factorize from entity to flat entity utils (#21972)
## What Factorizes the two responsibilities that were copy‑pasted across every `from-<entity>-entity-to-flat-<entity>` util into two reusable tools. ### `fromEntityToScalarEntity` Projects a TypeORM entity into its scalar flat shape using an **allow‑list** driven by `ALL_ENTITY_PROPERTIES_CONFIGURATION_BY_METADATA_NAME` (plus the base columns `id`/`workspaceId`/`applicationId`/`universalIdentifier`). Only registered scalar columns are forwarded, `Date`s are serialized to ISO strings, and absent values are normalized to `null`. Replaces the previous deny‑list (`removePropertiesFromRecord`) approach, so unregistered/deprecated columns can no longer silently leak into the flat entity. ### `resolveManyToOneRelationIdsToUniversalIdentifiers` Resolves an entity's many‑to‑one foreign keys to their universal identifiers, driven by `ALL_MANY_TO_ONE_METADATA_RELATIONS`. Handles the always‑present `application`, nullable relations, and throws a `FlatEntityMapsException` when a referenced id is missing from its identifier map. Mirrors `resolveUniversalRelationIdentifiersToIds` in the opposite direction. Each `from-<entity>` util now reduces to: scalar spread + relation spread (+ explicit one‑to‑many id/universalIdentifier arrays where applicable). ### Note The allow‑list drops `isUIReadOnly` (a `WasRemovedInUpgrade` column not in the config) from `fieldMetadata`, which is the only integration‑snapshot change. <!-- This is an auto-generated description by cubic. --> <a href="https://cubic.dev/pr/twentyhq/twenty/pull/21972?utm_source=github" target="_blank" rel="noopener noreferrer" data-no-image-dialog="true"><picture><source media="(prefers-color-scheme: dark)" srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source media="(prefers-color-scheme: light)" srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img alt="Review in cubic" src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a> <!-- End of auto-generated description by cubic. --> |
||
|
Paul Rastoin
|
e9d5d71cd3 |
Wire up search field metadata (#21964)
## Part 1 - Exact scope of the current PR (#21964) close https://github.com/twentyhq/core-team-issues/issues/2586 This PR introduces `searchFieldMetadata` as a first-class flat metadata entity and migrates the existing search surface onto it, with **no change to which records are searchable** (ISO with `main`). In scope (what the PR does): - New flat entity `searchFieldMetadata` (universalIdentifier, applicationId, **`position`**, maps, conversions), registered in the central flat-entity constants and the migration build orchestrator. - `searchVector.asExpression` is **derived server-side** from `searchFieldMetadata` rows (validated by `isSafeTsVectorExpression`); never trusted from client input. - **Derivation order is deterministic, driven by each row's `position`** ([compute-search-vector-as-expression-from-search-field-metadatas.util.ts](packages/twenty-server/src/engine/metadata-modules/flat-search-field-metadata/utils/compute-search-vector-as-expression-from-search-field-metadatas.util.ts)), replacing the previous non-deterministic `(createdAt, id)` sort. That sort collapsed to random UUIDs for standard fields (same `createdAt`), so any rename/relabel rewrote the `STORED` generated column to a logically-identical-but-textually-different expression and produced a permanent per-workspace diff vs the standard definition. Ordering now equals provisioning order; ties break on `universalIdentifier`. - Provisioning at object creation mirrors the existing surface exactly **and seeds `position`**: - custom objects -> the `name` field only, at `position: 0` ([build-default-search-field-metadatas-for-custom-object.util.ts](packages/twenty-server/src/engine/metadata-modules/object-metadata/utils/build-default-search-field-metadatas-for-custom-object.util.ts)) - standard objects -> their curated `SEARCH_FIELDS_FOR_*` sets, `position` = the curated index - Backfill (instance + workspace commands in `2-16`) provisions rows for existing workspaces with the same surface **and the same positions** (standard from the curated standard maps, custom `name` = `0`), scoped to the workspace's own custom application ([build-search-field-metadata-backfill-operations.util.ts](packages/twenty-server/src/database/commands/upgrade-version-command/2-16/utils/build-search-field-metadata-backfill-operations.util.ts)). The `position` column is added in the same `2-16` fast instance command as `universalIdentifier`/`applicationId`. - Field rename of an already-indexed field recomputes `asExpression` (positions preserved, so order is stable) ([recompute-search-vector-on-field-rename.util.ts](packages/twenty-server/src/engine/metadata-modules/flat-field-metadata/utils/recompute-search-vector-on-field-rename.util.ts)). - Field delete drops the matching row(s) and recomputes; remaining rows keep their relative order (no renumber) ([from-delete-field-input-to-flat-field-metadatas-to-delete.util.ts](packages/twenty-server/src/engine/metadata-modules/flat-field-metadata/utils/from-delete-field-input-to-flat-field-metadatas-to-delete.util.ts)). - Object relabel is **additive** and ISO/regression-fix only: it indexes the new label identifier **appended last (`position = max(existing) + 1`)** without dropping `name` ([recompute-search-vector-on-label-identifier-update.util.ts](packages/twenty-server/src/engine/metadata-modules/flat-object-metadata/utils/recompute-search-vector-on-label-identifier-update.util.ts)). This is a deliberate, temporary bridge. Explicitly OUT of scope (deferred): - No API to edit `searchFieldMetadata` (no user-facing search-field configuration, including `position` — it is internal and only written by provisioning/backfill/recompute). - No auto-indexing of arbitrary searchable fields. Creating a custom TEXT/EMAILS/etc. field does NOT add it to search (the `computeSearchFieldMetadataCreationForFields` behavior was removed in `e6820ad`). - No field-type-transition handling (field type is immutable - not in `FLAT_FIELD_METADATA_EDITABLE_PROPERTIES`, so that path was dead code). - No `position` validation (uniqueness/range) and no multi-vector / per-field `weight` config — deferred to the configurable-search follow-up (#1428). Net: `searchFieldMetadata` becomes the source of truth for the *same* surface as `main`. The only intentional divergences from `main` are "relabel preserves `name`" (additive) and the deterministic `position`-ordered `asExpression` (a correctness/perf fix that is byte-identical to provisioning order, so it does not change the searchable surface). --------- Co-authored-by: Félix Malfait <felix@twenty.com> |
||
|
Paul Rastoin
|
a5aac3c21e |
Logic function handler name hardened validation (#21956)
# Introduction Introduce centralized handlerName validation for the logic function handlerName inside the flat logic function validator Even if not safe by definition, avoid string interpolation inside the local driver executor when retrieving the handler name from the parent module <!-- This is an auto-generated description by cubic. --> <a href="https://cubic.dev/pr/twentyhq/twenty/pull/21956?utm_source=github" target="_blank" rel="noopener noreferrer" data-no-image-dialog="true"><picture><source media="(prefers-color-scheme: dark)" srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source media="(prefers-color-scheme: light)" srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img alt="Review in cubic" src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a> <!-- End of auto-generated description by cubic. --> |
||
|
Paul Rastoin
|
0b8368cd6c |
Refactor search vector field (#21947)
# Introduction Refactoring the search vector field validation <!-- This is an auto-generated description by cubic. --> <a href="https://cubic.dev/pr/twentyhq/twenty/pull/21947?utm_source=github" target="_blank" rel="noopener noreferrer" data-no-image-dialog="true"><picture><source media="(prefers-color-scheme: dark)" srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source media="(prefers-color-scheme: light)" srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img alt="Review in cubic" src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a> <!-- End of auto-generated description by cubic. --> |
||
|
Félix Malfait
|
a682c8fa62 |
feat(sdk): declare row-level permission predicates in the role manifest (#21919)
## Why Apps can declare object and field permissions on a role via `defineRole`, but **not row-level security**. The RLS engine and the metadata-sync machinery already support predicates fully — they're first-class universal flat entities, the `FlatRole` already carries `rowLevelPermissionPredicateUniversalIdentifiers`, and the workspace-migration layer has builders/validators/handlers for them. The only gap was the **manifest layer**: `RoleManifest` had no field for predicates, so the sync converter always left them empty. As a result, the only way to ship RLS with an app was a post-install script that pushed predicates through the `upsertRowLevelPermissionPredicates` mutation. That mutation assigns predicates to the workspace's **generic custom application**, not the app that owns the role — so a single role's definition ends up split across two applications and drifts on every upgrade (you have to remember to re-run the script). The Partner app does exactly this today via `configure-partner-rls.ts`. ## What Adds `rowLevelPermissionPredicates` and `rowLevelPermissionPredicateGroups` to `RoleManifest` / `RoleConfig`, mirroring how `objectPermissions` / `fieldPermissions` already flow end-to-end: - **twenty-shared** — predicate + predicate-group manifest types on `RoleManifest` (referencing objects/fields by `universalIdentifier`, operand/logical-operator from the existing GraphQL enums). - **twenty-sdk** — `defineRole` accepts and validates them; the build derives deterministic predicate `universalIdentifier`s (groups keep an explicit one so predicates can reference them). - **twenty-server** — two converters turn manifest predicates/groups into universal flat entities during application-manifest sync, so they are created/updated/deleted together with the role and **owned by the app that ships it**. ### Bug fix found along the way The migration build order ran the `rowLevelPermissionPredicate(Group)` builders **before** the `role` builder, so a predicate declared alongside a brand-new role failed validation with `ROLE_NOT_FOUND`. They now run **after** the role builder, exactly like object/field permissions. ## Partner app (second commit) Converts `partner.role.ts` to declare its five predicates inline and **deletes `configure-partner-rls.ts`** + the `rls:configure` scripts — the workaround this PR is meant to retire. The predicates are byte-for-byte the same semantics as the script produced. > Live-deployment note: the existing script-created predicates are owned by the *custom* application, so the Partner app sync won't touch them. Clear them once (e.g. an empty upsert on the Partner role) around deploy to avoid duplicates. Kept as a **separate commit** so it can be split out if reviewers prefer. ## Testing - **Integration (full app):** new `successful-manifest-sync-row-level-permission-predicate.integration-spec.ts` — installs an app whose role declares a predicate and asserts the predicate row is created (and **owned by the app**, not the custom app), updated in place on re-sync, removed when dropped from the manifest, and removed on uninstall. Ran locally against a seeded test DB ✅. - Re-ran the existing cross-app permission + view-field manifest suites to confirm the build-order change doesn't regress object/field-permission sync (13/13 ✅). - **Unit (utils only):** `defineRole` validation and `fromRoleConfigToRoleManifest` deterministic-id derivation. - Docs: new "Row-level security" section in `apps/config/roles.mdx`. ## Scope notes / possible follow-ups - Surfacing RLS in the app-install permission summary UI was intentionally left out (predicates *restrict* rather than grant, and typically live on a non-default role) — easy follow-up if wanted. - The `upsertRowLevelPermissionPredicates` mutation still homes out-of-band predicates on the custom app for app-owned roles; making that consistent (or rejecting it, like field permissions already do) is a sensible follow-up. https://claude.ai/code/session_01MipAis9z9okd4oCm9HCKEf --- _Generated by [Claude Code](https://claude.ai/code/session_01MipAis9z9okd4oCm9HCKEf)_ <!-- This is an auto-generated description by cubic. --> <a href="https://cubic.dev/pr/twentyhq/twenty/pull/21919?utm_source=github" target="_blank" rel="noopener noreferrer" data-no-image-dialog="true"><picture><source media="(prefers-color-scheme: dark)" srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source media="(prefers-color-scheme: light)" srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img alt="Review in cubic" src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a> <!-- End of auto-generated description by cubic. --> |
||
|
Paul Rastoin
|
ee5b3a65f4 |
Workspace migration post transaction commit side effect (#21845)
# Introduction Avoid side effect in transaction to reduce lock duration As discussed we're going to introduce in migration runner side effect later through jobs and metadata boolean state tracker in db <!-- This is an auto-generated description by cubic. --> <a href="https://cubic.dev/pr/twentyhq/twenty/pull/21845?utm_source=github" target="_blank" rel="noopener noreferrer" data-no-image-dialog="true"><picture><source media="(prefers-color-scheme: dark)" srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source media="(prefers-color-scheme: light)" srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img alt="Review in cubic" src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a> <!-- End of auto-generated description by cubic. --> |
||
|
Félix Malfait
|
23f5ba9ebf |
feat: add resizable kanban column width (#21828)
## What & why Lets users resize the columns of a Kanban (record board) view. Requested by a user; the design avoids the "ragged board" problem by making the width a **single shared value**. ## Behaviour - A drag handle appears on the right edge of every column header. - Because all columns read **one** width value, dragging any handle resizes **every** column together — they can never end up mismatched. - Width is clamped between **150px** and **400px** (default **200px**). - The width is **persisted per view** and restored on reload. ## Approach **Backend** — a new nullable `View.kanbanColumnWidth` field, threaded through the existing view-level setting pattern (the same one `kanbanAggregateOperation` / `shouldHideEmptyGroups` use), so it gets create/update/manifest/override support for free: - entity column + `ViewOverrides` + `@WasIntroducedInUpgrade` - `CreateViewInput` / `UpdateViewInput` (`Int`, `@Min(150)`/`@Max(400)`) + `ViewDTO` - flat-view editable properties, entity-properties config, compare-type, standard-view + manifest converters - a fast instance command adding the `core.view` column **Frontend** — the value hydrates into a view-scoped atom and drives a single CSS variable set on the board container, which both column headers and bodies read. Live dragging only writes that CSS variable (no per-move React re-render); the final width is committed to the atom and persisted via `updateView` on pointer-up. ## Nullability / defaults `kanbanColumnWidth` is nullable — `null` means "never resized" and the UI falls back to the 200px default, so existing rows need no backfill. ## Validation - `nx typecheck twenty-server` ✅ and `nx typecheck twenty-front` ✅ - `nx lint:diff-with-main twenty-server` ✅; frontend lint fixes applied (split constants to one-per-file, removed `useRef`-for-state in favour of `useState`). - Draft pending a final green CI run (the dev container reclaimed `node_modules` mid-session; re-running locally). ## Test plan - [ ] Drag a kanban column edge → all columns resize together, clamped 150–400px - [ ] Reload → width persists for that view; other views unaffected - [ ] A view that was never resized still renders at 200px https://claude.ai/code/session_016Qe6oDBkhVbrq2QkXBJ5nE --- _Generated by [Claude Code](https://claude.ai/code/session_016Qe6oDBkhVbrq2QkXBJ5nE)_ <!-- This is an auto-generated description by cubic. --> <a href="https://cubic.dev/pr/twentyhq/twenty/pull/21828?utm_source=github" target="_blank" rel="noopener noreferrer" data-no-image-dialog="true"><picture><source media="(prefers-color-scheme: dark)" srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source media="(prefers-color-scheme: light)" srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img alt="Review in cubic" src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a> <!-- End of auto-generated description by cubic. --> --------- Co-authored-by: Claude <noreply@anthropic.com> |
||
|
Paul Rastoin
|
98c35ea804 |
App uninstall lambda, layers cleanup (#21749)
# Introduction On a logic function deletion also remove the driver entry On a app uninstall also remove the sdk layer ( keeps the dep one as it can be shared across several lambda ) | Resource | Scope | Before this PR | After | |---|---|---|---| | DB metadata (functions, objects, fields…) | per-app | deleted | deleted | | Source folder (`FileFolder.Source`) | per-function | deleted | deleted | | Built handler file (`FileFolder.BuiltLogicFunction`) | per-function | deleted | deleted | | **Lambda function** | per-function | **leaked** | **deleted** (driver `delete`) | | **SDK layer** `sdk-<wsId>-<appId>` (all versions) | per-app | **leaked** | **deleted** (driver `deleteApplicationResources` → `deleteSdkLayer`) | | Deps layer `deps-<checksum>` | shared across apps/workspaces | not deleted | **intentionally not deleted** (content-addressed, GC'd) | ## What I don't like about all that Right now there's some non reversible side effect inside the workspace migration transaction - If the transaction fails we're facing data loss - It also slows down everything I'm about to create a new PR that allow population post transaction commit side effect / cleanup to be run later <!-- This is an auto-generated description by cubic. --> <a href="https://cubic.dev/pr/twentyhq/twenty/pull/21749?utm_source=github" target="_blank" rel="noopener noreferrer" data-no-image-dialog="true"><picture><source media="(prefers-color-scheme: dark)" srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source media="(prefers-color-scheme: light)" srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img alt="Review in cubic" src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a> <!-- End of auto-generated description by cubic. --> |
||
|
Charles Bochet
|
0064ff6741 |
fix(ai): validate AI agent output field names against schema-key constraint (#21834)
## Problem
On a self-hosted instance, an AI Agent workflow action fails at run time
with an opaque model error:
```
The model returned the following errors: tools.0.custom.input_schema.properties:
Property keys should match pattern '^[a-zA-Z0-9_.-]{1,64}$'
```
This is Anthropic's validation on tool `input_schema` **property keys**.
An AI Agent's structured **Output** fields are turned into a JSON schema
and passed to the model as a tool; each output **variable name** becomes
a property key. Anthropic rejects any key that does not match
`^[a-zA-Z0-9_.-]{1,64}$` — most commonly a name containing a **space**
(e.g. `meetings brief`), but also names over 64 characters or with other
symbols.
Until now nothing validated this: `fieldsToSchema` writes
`properties[field.name]` verbatim, so a bad name only failed once the
workflow executed, with an error that gives the user no idea what to
fix. It doesn't reproduce on every instance — it depends purely on how
the workflow's output variables happen to be named.
## Fix
Introduce a single shared check,
`isValidAgentResponseSchemaPropertyKey`, and enforce it in two places:
- **Backend** — `validateAgentResponseFormat` now rejects invalid output
field names at agent **save time** with a clear `userFriendlyMessage`,
instead of letting the broken schema reach the model. This also gates
agents created via the API and re-saves of existing bad data.
- **Frontend** — the output schema builder shows an inline error on the
Variable Name field as soon as an invalid name is entered.
## Tests
- Unit test for the shared validity check (valid + invalid cases:
spaces, leading space, empty, > 64 chars, symbols, unicode).
- Unit test for `validateAgentResponseFormat` covering text/json
formats, valid names, a space in a name, an over-length name, and
reporting multiple invalid names at once.
## Notes for the reporter
The immediate unblock for an affected workflow is to rename the output
variable to remove the space (e.g. `meetings brief` → `meetings_brief`)
and retry the run. With this change the bad name is caught up front with
an explanation rather than failing mid-run.
<!-- This is an auto-generated description by cubic. -->
<a
href="https://cubic.dev/pr/twentyhq/twenty/pull/21834?utm_source=github"
target="_blank" rel="noopener noreferrer"
data-no-image-dialog="true"><picture><source
media="(prefers-color-scheme: dark)"
srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source
media="(prefers-color-scheme: light)"
srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img
alt="Review in cubic"
src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a>
<!-- End of auto-generated description by cubic. -->
|
||
|
Weiko
|
a1f79c4f40 |
fix(server): enforce lowercase universalIdentifier in sync (#21754)
## Context App sync fails when a manifest defines an entity with an uppercase UUID `universalIdentifier`. Postgres `uuid` columns normalize to lowercase on write, but the sync diff matches `universalIdentifier` strings case-sensitively. So an uppercase-defined entity never matches its lowercased DB row and is seen as delete + create on every sync, which trips downstream guards like "Parent navigation menu item not found". ## Change Reject non-lowercase `universalIdentifier`s at validation time in `WorkspaceEntityMigrationBuilderService.validateUniversalIdentifier` (right after the existing UUID-v4 check). This lives in the abstract base builder, so it covers every syncable entity type. App authors now get a clear "must be lowercase" error on the first sync instead of confusing downstream failures. Validation is sufficient here, no normalization needed — because the DB side is always lowercase, so rejecting uppercase input guarantees both sides of the diff match. <!-- This is an auto-generated description by cubic. --> <a href="https://cubic.dev/pr/twentyhq/twenty/pull/21754?utm_source=github" target="_blank" rel="noopener noreferrer" data-no-image-dialog="true"><picture><source media="(prefers-color-scheme: dark)" srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source media="(prefers-color-scheme: light)" srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img alt="Review in cubic" src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a> <!-- End of auto-generated description by cubic. --> |
||
|
Weiko
|
607d9ee6e5 |
fix(server): allow app-defined permission flags to be referenced by a role in the same sync (#21742)
## Context When an application defined custom permission flags and a role referencing them in the same sync, installation failed, first at validation (Permission flag not found) and then at execution (Migration action 'create' for 'rolePermissionFlag' failed). Root cause: both the migration builder order and the runner execution order processed rolePermissionFlag before permissionFlag, so the role's flag assignments were validated/inserted before the flags they reference existed. ## Changes - Builder order: run the permissionFlag builder before rolePermissionFlag so newly created flags are visible in the optimistic maps when assignments are validated. - Execution order: order the permission-flag actions so definitions are created before assignments, and assignments deleted before definitions, keeping the FK satisfied in both directions. - In-use check: move the "flag still assigned to a role" guard out of the per-entity deletion validator (order-dependent, false-positived when a flag and its assignments were deleted together) into a new order-independent validatePermissionFlagNotInUseCrossEntity (aligned with existing validateObjectMetadataCrossEntity, validateViewFieldLabelIdentifierCrossEntity, ...), run after all builders against the migration's final state. This fixes both the create path (define flag + reference it in one sync) and the teardown path (delete flag + its assignments in one sync). <!-- This is an auto-generated description by cubic. --> <a href="https://cubic.dev/pr/twentyhq/twenty/pull/21742?utm_source=github" target="_blank" rel="noopener noreferrer" data-no-image-dialog="true"><picture><source media="(prefers-color-scheme: dark)" srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source media="(prefers-color-scheme: light)" srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img alt="Review in cubic" src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a> <!-- End of auto-generated description by cubic. --> |
||
|
Weiko
|
3ee93b5ec9 |
feat(server): add isSystemSideEffect & merge createOneObject/createOneField side-effect migrations (#21673)
## Context When an object is created via the metadata API, `createOneObject` creates its side-effect entities (INDEX view + viewFields, indexes, navigation menu item, "go to" command menu item, record-page fields view, page layout/tabs/widgets) across **three separate `validateBuildAndRunWorkspaceMigration` calls**, purely because the protection behavior (mutations → overrides, delete → deactivate, reset → reactivate) was keyed on *"owned by the standard app"*, forcing the side effects into batches with different application owners. This misrepresents ownership and breaks atomicity. This PR separates two orthogonal concepts: - **Ownership** (`applicationId`), the true owner: the caller's application (the workspace custom app today, 3rd-party apps later). - **Protection** (`isSystemSideEffect`), the row was generated by the system, so user mutations route to overrides, deletion becomes deactivation, and reset restores defaults. Once side effects are re-owned to the caller, the old `applicationId === standardApp` check can no longer tell an original side-effect row from a user-added one so a dedicated `isSystemSideEffect` flag carries the protection instead. This is **PR 1 of 2** (forward-only). It makes newly created objects and fields correct; existing workspaces are handled by a follow-up backfill (see *Out of scope*). ## What this PR does - **`isSystemSideEffect` column** on the 8 affected entities (`view`, `viewField`, `indexMetadata`, `commandMenuItem`, `pageLayout`, `pageLayoutTab`, `pageLayoutWidget`, `fieldMetadata`), with `@WasIntroducedInUpgrade` + an entry in the flat-entity property configuration (`toCompare: true`, read-only). - **Single atomic migration in `createOneObject`**: the three `validateBuildAndRunWorkspaceMigration` calls are merged into one, owned by the caller (`resolvedOwnerFlatApplication`) and the record-page view/fields, page layout, and navigation command item are re-owned to the caller and flagged `isSystemSideEffect: true`. `buildNavigationFlatCommandMenuItem` is parameterized with `applicationUniversalIdentifier` (no longer hardcoded to the standard app). - **Field-creation side effects** (`createManyFields`/`createOneField` already run as a single caller-owned migration, so no re-ownership/merge was needed): the auto-created viewField is flagged `isSystemSideEffect: true`, and a new field now also propagates to the object's **INDEX/table view** (added there as a **hidden** column, `isVisible: false`) in addition to the record-page FIELDS widget. The INDEX view is targeted directly by `key = INDEX` (it is not a page-layout widget), de-duplicated per `(viewId, fieldMetadataUniversalIdentifier)` to respect the per-view unique index. The unique-field index is likewise flagged the inverse relation field stays unflagged (`isSystem: false`). - **Protection predicate** extended: `isCallerOverridingEntity` and the removal/reset split strategies now treat `isSystemSideEffect` rows as protected even when caller-owned (route to overrides / deactivate / reset) and the page-layout-reset guards allow resetting flagged entities. - **Standard compute maps** set the flag consistently so a re-sync produces no diff (standard-object side effects stay `false`; per-object nav command items and custom-object base fields are `true`). - **Read-only GraphQL exposure** of `isSystemSideEffect` on the view / view-field / page-layout / tab / widget / command-menu-item DTOs (not exposed on create/update inputs). => Todo: needs to take this new flag into account. This is fine for now because isSystem remains on object/field. - **Fast instance command** (`2-14`) adding the 8 columns (`NOT NULL DEFAULT false`). ## Scope decisions - **`pageLayout` is not an `OverridableEntity`**, its own row has nothing user-overridable (all customization lives on tabs/widgets). It's dual-purpose (`RECORD_PAGE` side-effect vs. user `DASHBOARD`), so it gets `isSystemSideEffect` for protection only, no `overrides` jsonb. - **`navigationMenuItem` is out of scope.**: Those are side effects only for the metadata API and not marked as "system" (they can be deleted/updated etc...) - **`viewFieldGroup` is not a side effect**, it's only created via the explicit view-field-group API, never by object/field creation, so it gets no flag. ## Out of scope (follow-ups) **PR 2** — slow per-workspace backfill (re-own + flag existing side effects, recreate missing ones) and deterministic v5 identifiers for base fields / pageLayout / tab. <!-- This is an auto-generated description by cubic. --> <a href="https://cubic.dev/pr/twentyhq/twenty/pull/21673?utm_source=github" target="_blank" rel="noopener noreferrer" data-no-image-dialog="true"><picture><source media="(prefers-color-scheme: dark)" srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source media="(prefers-color-scheme: light)" srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img alt="Review in cubic" src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a> <!-- End of auto-generated description by cubic. --> |
||
|
Charles Bochet
|
97871131a1 |
fix(server): mitigate integration-test OOM flakiness (#21588)
## Problem `server-integration-test` shards have been failing intermittently across unrelated PRs with a distinctive signature: the shard exits code 1 with **no jest assertion failure, no `Test Suites:` summary, and no V8 `JavaScript heap out of memory` error** — the process just dies mid-run. Failures hit random shards and clear on re-run (e.g. an unrelated branch failed shard 6 once, then passed 3× on identical code), while the `merge_group` gate stays green. ### Root cause Each shard runs a **single in-band jest process** that boots one shared NestJS app (`globalSetup` → `app.listen`) and holds it for the entire shard, driving heavy metadata migrations + cache rebuilds in that one process. `NODE_OPTIONS=--max-old-space-size=12288` let V8 grow to 12 GB — *above* the `ubuntu-latest` runner's available RAM (16 GB, shared with Postgres/Redis/ClickHouse). V8 therefore deferred aggressive GC and grew past physical memory, so the **OS OOM-killer killed the process before V8 hit its own ceiling** — which is why there's no heap error and no jest summary, just a silent exit. ## Changes (CI/test-only — prod runtime untouched) - **Lower the integration jest heap cap `12288` → `6144`** so V8 self-limits below physical headroom instead of being OS-killed. Counterintuitively safer: a real leak now surfaces as a *visible* heap error naming the test, rather than a silent death. (`database:reset` keeps 12288 — it runs alone, before jest.) - **Add `--logHeapUsage`** to the integration jest runs to expose the per-file heap trend for confirming/pinpointing the growth. - **Split integration tests across 16 shards (was 10)** to lower the peak working set per shard. - **Make perf logging a first-class `LoggerService` tool** (per @prastoin's review): add `LoggerService.perf()` and unify the existing `time()`/`timeEnd()` helpers into `perfTime()`/`perfTimeEnd()` (now routed through the driver), all gated by a new `PERF_LOG_ENABLED` config var. It **defaults on** so real environments keep emitting the install-perf logs, and `.env.test` sets it `false` to mute the per-action flood in integration tests. The `application-manifest` and `validate-build` services were moved from the built-in `Logger` to `LoggerService` to use it. ## Notes - `--max-old-space-size` lives only in the `test:integration` nx target; it is **not** the prod server heap setting, so prod is unaffected. - This is mitigation. If `--logHeapUsage` shows monotonic growth across files, there's a real accumulation in the long-lived app (retained flat-maps / metadata cache) worth a follow-up heap-snapshot fix. <!-- This is an auto-generated description by cubic. --> <a href="https://cubic.dev/pr/twentyhq/twenty/pull/21588?utm_source=github" target="_blank" rel="noopener noreferrer" data-no-image-dialog="true"><picture><source media="(prefers-color-scheme: dark)" srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source media="(prefers-color-scheme: light)" srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img alt="Review in cubic" src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a> <!-- End of auto-generated description by cubic. --> |
||
|
Félix Malfait
|
2f6a267b68 |
chore(server): remove stray comment in flat-entity-maps spec (#21599)
Follow-up to #21585: removes an explanatory comment in the test that slipped through before merge. <!-- This is an auto-generated description by cubic. --> <a href="https://cubic.dev/pr/twentyhq/twenty/pull/21599?utm_source=github" target="_blank" rel="noopener noreferrer" data-no-image-dialog="true"><picture><source media="(prefers-color-scheme: dark)" srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source media="(prefers-color-scheme: light)" srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img alt="Review in cubic" src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a> <!-- End of auto-generated description by cubic. --> |
||
|
Félix Malfait
|
02d6e2d76f |
perf(server): avoid O(n²) when building flat entity maps (#21585)
## Problem
After 2.13, server CPU stepped up and stayed up. Sentry profiling of
`POST /metadata` pins it on
`addFlatEntityToFlatEntityMapsThroughMutationOrThrow` — ~26% self-time
plus a long tail, turning metadata-write requests into multi-second
(~18s observed) operations.
The hot stack is:
```
WorkspaceMigrationValidateBuildAndRunService.computeAllRelatedFlatEntityMaps
└ getSubFlatEntityMapsByApplicationIdsOrThrow
└ addFlatEntityToFlatEntityMapsThroughMutationOrThrow
```
Every metadata migration rebuilds the twenty-standard application's flat
sub-maps — thousands of entities, across every involved metadata type —
through this util.
## Root cause
`addFlatEntityToFlatEntityMapsThroughMutationOrThrow` maintains
`universalIdentifiersByApplicationId` and deduped each insert with
`Array.includes`:
```ts
if (!existingUniversalIdentifiers.includes(flatEntity.universalIdentifier)) {
existingUniversalIdentifiers.push(flatEntity.universalIdentifier);
}
```
That scan is O(n) per insert, so building a map for an application with
`n` entities is **O(n²)**. The twenty-standard application groups
thousands of standard entities under one `applicationId`, so its sub-map
rebuild dominates.
The dedup is also redundant: the function throws `ENTITY_ALREADY_EXISTS`
at the top if the `universalIdentifier` is already in
`byUniversalIdentifier`, and every id pushed to the per-application list
is also written there. So reaching the push guarantees the id is new —
the `.includes()` is always `false`.
## Fix
Drop the scan and push directly → map building is **O(n)**. Behavior is
unchanged (the early throw already enforces uniqueness).
## Test
Adds a unit spec covering indexing, the no-`applicationId` case, the
duplicate throw, and a 20k-entity build that completes instantly (guards
against re-introducing the quadratic).
## Follow-up (separate PR)
This is the bleed-stopper. The deeper issue is that
`computeAllInvolvedApplicationIds` pulls the **entire** twenty-standard
application into the dependency set of every migration and rebuilds
those sub-maps per request instead of caching them. Worth scoping the
dependency set to referenced entities (or caching the standard-app
sub-maps), which I'll raise separately.
<!-- This is an auto-generated description by cubic. -->
<a
href="https://cubic.dev/pr/twentyhq/twenty/pull/21585?utm_source=github"
target="_blank" rel="noopener noreferrer"
data-no-image-dialog="true"><picture><source
media="(prefers-color-scheme: dark)"
srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source
media="(prefers-color-scheme: light)"
srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img
alt="Review in cubic"
src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a>
<!-- End of auto-generated description by cubic. -->
|
||
|
Weiko
|
f63f053444 |
CommandMenuItem overridable entity (#21486)
## Context Second PR of the overridable-entities track (after #21436 for views): command menu items become overridable so that edits on non-owned items are stored as overrides instead of mutating the row, and deletion/deactivation becomes reversible. ## What this does - `CommandMenuItemEntity` now extends `OverridableEntity<CommandMenuItemOverrides>` (adds `isActive` + `overrides`). All editable properties are overridable for now (to discuss). - **Update**: mutations on a command item not owned by the caller (standard items) are written into `overrides`; reads merge them in the DTO. The command palette edit mode (pin, reorder, shortLabel) now preserves standard values, "Reset label to default" gains true post-save semantics. - **Delete**: protected items are deactivated (`isActive = false`) instead of deleted; custom items still hard-delete. - **Object deactivate/enable toggle**: now flips `isActive` on the command item (merged into the main migration call) instead of delete/recreate; a create-if-missing fallback covers legacy deactivated objects. - **Front**: inactive command items are filtered out of the palette selector. <!-- This is an auto-generated description by cubic. --> <a href="https://cubic.dev/pr/twentyhq/twenty/pull/21486?utm_source=github" target="_blank" rel="noopener noreferrer" data-no-image-dialog="true"><picture><source media="(prefers-color-scheme: dark)" srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source media="(prefers-color-scheme: light)" srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img alt="Review in cubic" src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a> <!-- End of auto-generated description by cubic. --> --------- Co-authored-by: Charles Bochet <charles@twenty.com> |
||
|
Félix Malfait
|
1efa3567ef |
Rename isUIReadOnly to isUIEditable, add isUICreatable, expose both to app developers (#21504)
<!-- CURSOR_AGENT_PR_BODY_BEGIN -->
# UI capability flags: `isUIEditable` + `isUICreatable`
## Per-verb capability model
This PR replaces the negative `isUIReadOnly` metadata flag with
positive, per-verb capability flags (à la Salesforce
`createable`/`updateable`):
- **`isUIEditable: boolean`, default `true`** — rename of `isUIReadOnly`
with inverted polarity, on **both** `objectMetadata` and
`fieldMetadata`. It is one concept ("can the user edit this through the
generic UI?") at two altitudes, so it carries one name at both levels.
- **`isUICreatable: boolean`, default `true`** — new, **object-level
only** (fields have no create verb). When `false`, no generic UI
affordance to create a record of this object appears anywhere (table "+"
buttons, board column add, calendar add, relation-section "Add new",
record picker "Add new", command-menu create action and its keyboard
shortcut).
Both flags are **UI-affordance flags only**: the server does not block
create/edit mutations based on them, so the system, API, and workflows
continue to mutate these records freely. They are orthogonal statements
about the object's nature with no implication rule in the data model.
Because today's inline creation UX creates a blank record the user must
then edit, the frontend create predicate currently requires both
`isUICreatable` and effective editability.
There is no CREATE permission in `ObjectPermissions`; the frontend keeps
gating creation on `canUpdateObjectRecords` as a proxy, ANDed with the
new flags.
## Unified create predicate
All generic creation entry points now flow through one predicate,
`canCreateRecordsForObjectMetadataItem` (`isUICreatable` && not
`isSystem` && not effectively read-only, where effective read-only
covers `isUIEditable`, `isRemote`, and the `canUpdateObjectRecords`
proxy via `isObjectMetadataReadOnly`). This deletes the previously
hardcoded suppression lists:
- `isRecordTableCreateDisabled.ts` and its hardcoded
`WorkflowRun`/`WorkflowVersion` list — deleted; those objects (plus
`workspaceMember`) now declare `isUICreatable: false` in the standard
application instead.
- The hardcoded `workspaceMember` guard inside
`useAddNewRecordAndOpenSidePanel.ts` — deleted.
- The `CREATE_NEW_RECORD` command menu item's availability expression
now checks `objectMetadataItem.isUICreatable`, `isUIEditable`,
`isSystem`, and `isRemote`; a workspace upgrade command re-syncs the
expression in existing workspaces.
Component-local conditions (soft-delete filter active, layout
customization mode) stay in their components.
## GraphQL compatibility and removal plan
The schema delta versus main is **purely additive plus deprecations —
zero breaking changes**:
- `isUIReadOnly` remains on both the ObjectMetadata and FieldMetadata
GraphQL output types for **one release** as a deprecated field computed
as `!isUIEditable` (`deprecationReason: 'Use isUIEditable'`). The Twenty
frontend no longer queries it.
- `isUIReadOnly` also remains on the **input side** for one release
(`CreateFieldInput`, `UpdateFieldInput`, `FieldFilter`, `ObjectFilter`),
keeping the schema shape identical to main for those members. On create
it acts as a legacy alias mapped to `!isUIReadOnly` (`isUIEditable` wins
when both are provided); on update it is ignored, exactly as on main (it
was never an editable property). Filtering on the deprecated member
keeps working until the column is dropped at upgrade time; after that it
is a deprecated no-op surface kept only for schema compatibility.
**Removal plan for next release: drop `isUIReadOnly` from the output
DTOs (and resolvers' `@ResolveField`s), from the input/filter types,
from the create-input mapping, and the `@WasRemovedInUpgrade`-retained
entity columns and decorators.**
## ⚠️ Webhook / database-event payload shape change
The `database-event-payload` type in `twenty-shared` got a clean rename
(no alias): metadata snapshots in webhook and database-event payloads
now carry `isUIEditable` (and `isUICreatable` at object level) **instead
of** `isUIReadOnly`, with inverted polarity. Consumers of these payloads
that read `isUIReadOnly` must switch to `isUIEditable`.
## New manifest properties (app-developer DX)
Application developers can now set these flags in their app manifests
(purely additive — existing manifests and older `twenty-sdk` versions
are unaffected, defaults apply when omitted):
- `objects[].isUICreatable?: boolean` (default `true`)
- `objects[].isUIEditable?: boolean` (default `true`)
- `fields[].isUIEditable?: boolean` (default `true`)
The manifest converters previously hardcoded `isUIReadOnly: false`; they
now read the manifest values with `?? true` defaults. The types are
re-exported through `twenty-sdk` from `twenty-shared`.
## Migration & backfill
- One fast instance command: adds `isUIEditable` (NOT NULL default
`true`) on `core."objectMetadata"` and `core."fieldMetadata"`, backfills
`isUIEditable = false` exactly where `isUIReadOnly = true`, drops
`isUIReadOnly`, and adds `isUICreatable` (default `true`) on
`objectMetadata`. The `down` is the exact inverse. Uses `ADD/DROP COLUMN
IF (NOT) EXISTS`, matching the 2-12 drop-`isCustom` precedent. Verified
up and down in separate transactions against a dev database with exact
backfill counts.
- **Cross-version upgrade safety (multi-version self-hosted jumps):**
the upgrade sequence interleaves per version (instance → workspace
commands), so pre-2.13 workspace commands run **before** the 2.13 rename
when an old instance jumps several versions. Following the `isCustom`
precedent: `isUIEditable`/`isUICreatable` are marked
`@WasIntroducedInUpgrade` and `isUIReadOnly` stays on both entities as
`@WasRemovedInUpgrade`, so the upgrade-aware entity metadata adapter
hides the not-yet-existing columns (and keeps the legacy column live) at
pre-2.13 cursors. **No committed upgrade command outside the 2-13
directory is modified**: the old 1-21/2-8/2-9 commands keep their
original `isUIReadOnly: true` inputs, which still compile (entity
property retained, deprecated create-input alias mapped) and still
produce the correct legacy column writes pre-rename.
- A 2-13 workspace command (`sync-standard-ui-capability-flags`)
re-syncs `isUICreatable` **and** `isUIEditable` on standard objects and
`isUIEditable` on standard fields from the standard-application
definitions. This backfills `isUICreatable: false` on
`workflowRun`/`workflowVersion`/`workspaceMember` and heals fields
created mid-cross-upgrade by pre-2.13 commands (whose hidden
`isUIEditable` value cannot reach the insert). Both 2-13 sync commands
pass `isSystemBuild: true` — the flat metadata validator otherwise
rejects direct updates to system objects (verified against a
deliberately drifted dev database; the run is idempotent).
- A second 2-13 workspace command re-syncs the create-record command
availability expression.
## Testing
- Unit tests for `canCreateRecordsForObjectMetadataItem`
(flag/permission/system combinations) and for the manifest converters
(flags set / omitted → defaults).
- Full `upgrade --dry-run` boots the sequence (107 steps) and validates
the upgrade-aware decorator references; both 2-13 sync commands verified
end to end against real drift and re-run idempotently.
- Schema verified by live introspection after the input-alias restore:
all four input/filter members match main, output deprecations intact;
frontend metadata types and `twenty-client-sdk` schema regenerated from
the running server.
- Read-only-related and touched jest suites pass on both packages;
typecheck and lint pass on `twenty-server` and `twenty-front`.
<!-- CURSOR_AGENT_PR_BODY_END -->
<div><a
href="https://cursor.com/agents/bc-0f3e04cb-b04a-40be-8330-5609c4538e8a"><picture><source
media="(prefers-color-scheme: dark)"
srcset="https://cursor.com/assets/images/open-in-web-dark.png"><source
media="(prefers-color-scheme: light)"
srcset="https://cursor.com/assets/images/open-in-web-light.png"><img
alt="Open in Web" width="114" height="28"
src="https://cursor.com/assets/images/open-in-web-dark.png"></picture></a> <a
href="https://cursor.com/background-agent?bcId=bc-0f3e04cb-b04a-40be-8330-5609c4538e8a"><picture><source
media="(prefers-color-scheme: dark)"
srcset="https://cursor.com/assets/images/open-in-cursor-dark.png"><source
media="(prefers-color-scheme: light)"
srcset="https://cursor.com/assets/images/open-in-cursor-light.png"><img
alt="Open in Cursor" width="131" height="28"
src="https://cursor.com/assets/images/open-in-cursor-dark.png"></picture></a> </div>
<!-- This is an auto-generated description by cubic. -->
<a
href="https://cubic.dev/pr/twentyhq/twenty/pull/21504?utm_source=github"
target="_blank" rel="noopener noreferrer"
data-no-image-dialog="true"><picture><source
media="(prefers-color-scheme: dark)"
srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source
media="(prefers-color-scheme: light)"
srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img
alt="Review in cubic"
src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a>
<!-- End of auto-generated description by cubic. -->
---------
Co-authored-by: Cursor Agent <cursoragent@cursor.com>
|
||
|
Weiko
|
cfb9772179 |
feat(server): convert view to overridable entity (#21436)
## Context Every entity created as a side effect of object creation must support the overridable pattern (`isActive` + `overrides` + override routing) before we can re-own side effects to their true application. Starting with View. viewField, viewFieldGroup, pageLayoutTab and pageLayoutWidget already extend `OverridableEntity`. This PR brings `view` to the same pattern. ## What this does - `ViewEntity` now extends `OverridableEntity<ViewOverrides>` (adds `isActive` boolean + `overrides` jsonb). All editable view properties are overridable; the 3 fieldMetadata foreign keys are converted to/from universal identifiers like viewField's `viewFieldGroupId`. - **Update**: mutations on a view not owned by the caller (e.g. standard views like "All Companies") are written into `overrides` instead of mutating the row. Reads merge overrides in the DTO. - **Delete/destroy**: views not owned by the caller are deactivated (`isActive = false`) instead of deleted. ~~- **INDEX invariant**: `key = INDEX` views can only be created via object-creation side effect. The API now rejects creating, deleting or destroying INDEX views (object-deletion cascade is unaffected). This was not really needed for this migration but was flagged during implementation.~~ - **Front**: views with `isActive = false` are filtered out of the views selector. - Fast instance command adds the two columns (`2-12-instance-command-fast-...-view-overridable-entity.ts`). ## Notes - Custom (caller-owned) views behave exactly as before: direct updates, soft delete. - View-group side effects (kanban groups) are computed on the override-merged view so overridden `mainGroupByFieldMetadataId` works. |
||
|
Weiko
|
9c66975520 |
isCustom deprecation for Objects and Fields (#21228)
## Context
`isCustom` was a legacy denormalized boolean on `ObjectMetadataEntity`
and `FieldMetadataEntity`.
Now that every metadata row carries `applicationId` (via
`SyncableEntity`), "is this custom" is fully derivable, and the stored
boolean was a redundant second source of truth that could drift.
The real meaning of `isCustom` is **"the owning application is not the
twenty-standard application"** — i.e. `!belongsToTwentyStandardApp`.
Note this is *not* "belongs to the workspace custom app" as I initially
thought: third-party-application
objects/fields are custom too.
The standard application has a globally stable `universalIdentifier`, so
the value derives with no per-workspace lookup.
## Changed
## `isCustom` checks — before → after
`isCustom` is no longer a stored column. The table below lists every
site that branched on it and how it resolves now. The unifying rule:
`isCustom ≡
!isTwentyStandardApplicationUniversalIdentifier(applicationUniversalIdentifier)`.
### Server — behavioural checks
| Location | Purpose | Before | Now |
|---|---|---|---|
| `utils/compute-object-target-table.util.ts` | Physical table name `_`
prefix | `computeTableName(nameSingular, objectMetadata.isCustom)` |
derives from `applicationUniversalIdentifier` (single source for all
table-name callers) |
| `twenty-orm/factories/entity-schema.factory.ts` +
`…/entity-schema-metadata.type.ts` | ORM table name (hot path) |
`object.isCustom` | `object.applicationId !== standardApplicationId`
(computed in `buildEntitySchemaMetadataMaps`) |
|
`twenty-orm/repository/workspace-{delete,soft-delete,update}-query-builder.ts`
| Table name for mutations | `computeTableName(nameSingular,
objectMetadata.isCustom)` | `computeObjectTargetTable(objectMetadata)` |
| `index-metadata/utils/generate-deterministic-index-name-v2.ts` | Index
name hash (must stay bit-identical) | `flatObjectMetadata.isCustom` |
derives from `applicationUniversalIdentifier` |
| `object-metadata/object-record-count.service.ts` | Table name for
record count | `computeTableName(nameSingular, isCustom)` |
`computeObjectTargetTable(flatObjectMetadata)` |
|
`workspace-manager/dev-seeder/data/services/dev-seeder-data.service.ts`
| Match seed config by table name | `computeTableName(item.nameSingular,
item.isCustom)` | `computeObjectTargetTable(item)` |
| `commands/workspace-export/workspace-export.service.ts` +
`…/utils/generate-workspace-schema-ddl.util.ts` | Export table name (raw
entity) | `objectMetadata.isCustom` |
`!isTwentyStandard…(objectMetadata.application?.universalIdentifier)` |
|
`flat-field-metadata/services/flat-field-metadata-type-validator.service.ts`
| Block users creating reserved field types |
`args.flatEntityToValidate.isCustom` |
`!args.flatEntityToValidate.isSystem` |
| `api/common/.../common-create-many-query-runner.service.ts` | Don't
let client overwrite system `createdBy` |
`createdByFieldMetadata.isCustom === false` |
`createdByFieldMetadata.isSystem === true` |
|
`field-metadata/utils/resolve-field-metadata-standard-override.util.ts`
| Skip i18n/overrides for custom fields | `if (fieldMetadata.isCustom)
return raw` | **removed** — falls through on
`isDefined(standardOverrides)` |
|
`object-metadata/utils/resolve-object-metadata-standard-override.util.ts`
| Skip i18n/overrides for custom objects | `if (objectMetadata.isCustom)
return raw` | **removed** — same fall-through |
|
`command-menu-item/utils/build-navigation-interpolation-context.util.ts`
| Override context for nav labels | passed `isCustom` into resolver |
dropped (resolver no longer needs it) |
| `api/common/.../data-arg-processor.service.ts` | `isCustom` for
record-position table name | `flatObjectMetadata.isCustom` | derives
from `applicationUniversalIdentifier` |
| `metadata-modules/minimal-metadata/minimal-metadata.service.ts` |
Minimal DTO + override context | `flatObjectMetadata.isCustom` | derives
from `applicationUniversalIdentifier` |
|
`commands/upgrade-version-command/1-23/…backfill-record-page-layouts.command.ts`
| Filter to custom objects | `objectMetadata.isCustom` |
`!isTwentyStandard…(applicationUniversalIdentifier)` |
### Server — DTO / API population
| Location | Before | Now |
|---|---|---|
|
`flat-object-metadata/utils/from-flat-object-metadata-to-object-metadata-dto.util.ts`
| passthrough `isCustom` | derives from `applicationUniversalIdentifier`
|
|
`flat-field-metadata/utils/from-flat-field-metadata-to-field-metadata-dto.util.ts`
| passthrough `isCustom` | derives from `applicationUniversalIdentifier`
|
|
`object-metadata/utils/from-object-metadata-entity-to-object-metadata-dto.util.ts`
(REST) | `entity.isCustom` | `entity.applicationId !==
standardApplicationId` |
|
`field-metadata/utils/from-field-metadata-entity-to-field-metadata-dto.util.ts`
(REST) | `entity.isCustom` | `entity.applicationId !==
standardApplicationId` |
| `dataloaders/dataloader.service.ts` | passed
`flatFieldMetadata.isCustom` into override resolver | dropped (resolver
no longer needs it) |
> REST controllers (`object-metadata.controller.ts`,
`field-metadata.controller.ts`) resolve `standardApplicationId` once per
request from the cached `flatApplicationMaps`.
### Frontend
| Location | Purpose | Before | Now |
|---|---|---|---|
| `settings/.../SettingsObjectFieldDisabledActionDropdown.tsx` | Whether
an inactive field is deletable | `isDeletable = isCustomField` |
`isDeletable = isCustomField && !isSystemField` |
### Unchanged (out of scope)
`isCustom` on `IndexMetadata` / `View` / `Skill` / `Agent` and their
guards still read the persisted column.
Breaking change is on the isCustom filter on field and object APIs, this
is never used in the FE and unlikely used by external consumers
|
||
|
Charles Bochet
|
fcaf2b4d9b |
chore(twenty-server): temporary instrumentation for app-install 504 (#21365)
## Why App installs on cloud intermittently fail with a 504, surfacing in Sentry as `Migration action 'update' for 'logicFunction' failed` + `Failed to rollback transaction: Query runner already released`. This is **temporary instrumentation** to pin down where the time goes — to be reverted once the bottleneck is fixed. Everything is greppable via `[install-perf]` and marked `// TODO(install-perf)`. ## What the local repro already told us I instrumented the manifest-sync/migration path and ran a local harness (new skipped spec) installing **1 / 8 / 30 logic functions**, for both create and the checksum-bump **update** (the incident path): | stage (N=30, update) | ms | |---|---| | flat-maps recompute | ~1 | | build migration | ~11 | | transaction (all actions + commit) | ~79 | | post-commit cache invalidate | ~6 | | **full sync** | **~135** | Nothing approached 1s, let alone 10s; no slow queries logged. So the migration/cache code is **not** the algorithmic cause. Given the in-transaction `UPDATE ... WHERE id=?` is intrinsically fast, a >10s in prod almost certainly means it was **blocked on a lock**, and the 10s node-pg `query_timeout` (`core.datasource.ts`) then killed the connection → the observed errors + 504. Local can't reproduce prod lock contention / table sizes, hence this instrumentation. ## What this adds (all `TODO`-marked) - **hrtime per-stage timing** — flat-maps recompute, build vs run, per-action (`>50ms`), transaction summary, post-commit cache invalidation. Uses `process.hrtime` because the integration harness enables fake timers (so `Date.now()` is useless there). - **`maxQueryExecutionTime`** slow-query logging on the core datasource (logs the offending SQL). - **Scoped `SET LOCAL lock_timeout = '8s'`** on the migration transaction (below the 10s `query_timeout`) → a blocked action fails fast with a clear *"canceling statement due to lock timeout"* instead of the opaque connection kill. - **Best-effort `pg_stat_activity` snapshot on failure** (on a fresh pooled connection) to identify the blocking session, plus a **guarded rollback** so a released connection stops masking the real error. - **Skipped local perf harness** (`logic-function-install-performance.integration-spec.ts`) — run manually with `nx test:integration:with-db-reset -- --testPathPattern "logic-function-install-performance"`. ## How we'll use it Deploy, reproduce the failing install, and read the `[install-perf]` logs: the per-action timing names the action, the `lock_timeout` message + `pg_stat_activity` snapshot name the **blocking** query/PID. Then revert this PR and fix the actual contention. Typecheck (`nx typecheck twenty-server`) is clean. |
||
|
martmull
|
77d1e8ced6 |
feat(app-dev): sync error hints, flatEntity labels, dev-mode summary UI, and docs (#21252)
Split out of #21240 — all remaining app-dev improvements. Stacked on #21251 (review/merge that first). - Actionable recovery hints on failed syncs; unified diff renderer; `--dry-run` guard. - Return `flatEntity` on update/delete sync actions and unify the diff label. - Summarize the dev-mode entity list unless `--verbose`. - Docs: syncing & recovery guide + dry-run + open-an-issue prompt. - Live execution mode for synced logic functions; clearer manifest warnings. <img width="1018" height="700" alt="image" src="https://github.com/user-attachments/assets/5e9ce19e-0f1d-4f99-8524-4e118bde932b" /> |
||
|
martmull
|
6c65d26ced |
feat(app-dev): add dry-run preview to dev sync (#21251)
Split out of #21240. Stacked on #21250 (review/merge that first). `yarn twenty dev --once --dry-run` computes the migration plan and prints the diff **without applying anything** (no migration, no app-record update, no SDK generation). Also renders the diff on a normal `dev --once` sync. <img width="646" height="179" alt="image" src="https://github.com/user-attachments/assets/59f3ddcd-2a5b-4b8a-b21a-c659abe16af0" /> |
||
|
neo773
|
bfb83e93b2 |
fix(metadata): resolve junction targets order-independently during mgration (#21193)
A junction relation points at a target field on the join object that another action may create later (two junctions into the same join reference each other). The builder validator now also looks up the target in the to be created set, and the runner mints every field id up front so the target resolves regardless of action order, the same way relation pairs are already handled. --------- Co-authored-by: Félix Malfait <felix@twenty.com> |
||
|
martmull
|
f20d04eb6e |
feat(app-dev): surface metadata diff in dev sync and name failing migration actions (#21249)
Split out of #21240. - Render the applied metadata changes (created/updated/deleted + identifiers) in the dev sync output instead of a bare `✓ Synced`. - Include the failing entity's `universalIdentifier` in `WorkspaceMigrationRunnerException` messages so conflicts are diagnosable. <img width="637" height="114" alt="image" src="https://github.com/user-attachments/assets/61422a16-370c-4e9b-a2f6-c29ce17f3b1b" /> <img width="497" height="104" alt="image" src="https://github.com/user-attachments/assets/d493c398-da29-49c9-ac5e-aa0f26cd7389" /> <img width="593" height="127" alt="image" src="https://github.com/user-attachments/assets/15e26edc-c0e4-4427-bd34-909040e970c9" /> --------- Co-authored-by: Charles Bochet <charles@twenty.com> |
||
|
nitin
|
e485b679ea |
[Call Recording] Add standard object (#21158)
Adds **Call Recording** as a first-class standard object (Twenty's
flat-metadata
standard-object system), with a hidden junction to calendar events and a
backfill
command for existing workspaces. Everything is gated behind the
`IS_CALL_RECORDING_ENABLED` feature flag.
### What's included
- **`CallRecording`**: audio/video files, transcript, status, recording
policy,
timing, external bot/recording ids. Label identifier is
`meetingOccurrenceKey`.
- **`CallRecordingCalendarEventAssociation`**: hidden junction linking a
recording
to a calendar event (dedupes one bot to many subscribers of the same
meeting).
- Full metadata graph via the flat-metadata builders: fields, indexes,
views,
view fields/groups, record page layout, and navigation items.
- **Metadata-only reverse relation** on `CalendarEvent`: present in
standard
metadata, omitted from the TS entity class to avoid expanding recursive
nested-insert types.
- **Upgrade command (2.9.0)** backfilling active/suspended workspaces:
- Creates the full graph; idempotent (skips when it already exists).
- Moves a colliding custom `callRecording` object aside to
`callRecordingOld`
(numeric suffix if that name is also taken).
- Navigation items (commands) are flag-gated by `universalIdentifier`,
so a custom object
reusing the name is never gated.
### QA
Run locally against existing workspaces (with and without a name
collision) and a
freshly created workspace:
- [x] Backfill, collision: custom `callRecording` renamed to
`callRecordingOld`;
standard graph created.
- [x] Backfill, no collision: standard graph created; unrelated custom
object untouched.
- [x] Idempotent: re-run is a no-op, with no duplicate metadata and
counts unchanged.
- [x] New workspace via `init()` produces an identical graph to the
backfill
(`universalIdentifier` set-diff = 0).
- [x] Label identifier (`meetingOccurrenceKey`) holds position 0 in
non-widget views.
- [x] Nav items gated behind the feature flag; collision-renamed
object's nav
expression re-pointed to its new name.
- [x] Unit tests cover collision name resolution and nav-gating logic.
|
||
|
Paul Rastoin
|
3d49642d12 |
[AUDIT] Run knip over twenty-server (#21159)
# Introduction Run [knip](https://knip.dev/) over twenty-server Used config: ```json { "$schema": "https://unpkg.com/knip@5/schema.json", "workspaces": { "packages/twenty-server": { "entry": [ "src/main.ts", "src/command/command.ts", "src/queue-worker/queue-worker.ts", "src/database/scripts/setup-db.ts", "src/database/scripts/truncate-db.ts", "src/database/clickHouse/migrations/run-migrations.ts", "src/database/clickHouse/seeds/run-seeds.ts", "src/instrument.ts", "lingui.config.ts", "test/integration/graphql/codegen/index.ts", "test/integration/utils/setup-test.ts", "test/integration/utils/teardown-test.ts", "scripts/**/*.ts", "**/*.spec.ts", "**/*.integration-spec.ts" ], "project": ["src/**/*.ts", "test/**/*.ts", "scripts/**/*.ts"], "ignore": [ "src/database/typeorm/**/migrations/**", "src/database/typeorm/**/*.entity.ts", "**/*.workspace-entity.ts", "**/logic-function-resource/constants/seed-project/**" ], "ignoreDependencies": ["@types/psl", "@types/aws-lambda"], "ignoreBinaries": ["nest", "lingui", "typeorm"] } } } ``` |
||
|
Charles Bochet
|
cb744b2eeb |
refactor(twenty-server): collapse workspace migration build orchestrator (#21144)
## Why `WorkspaceMigrationBuildOrchestratorService.buildWorkspaceMigration` was 1117 lines of 30 near-identical copy-pasted blocks (one per metadata entity) plus a 165-line ordered `actions: [...]` literal. Adding a new entity meant copying ~30 lines and hoping the boilerplate stayed in sync; the actual per-entity execution order was buried in the noise. ## Gain - Orchestrator: **1117 → 388 lines** - Net diff: **-558 lines** - Adding a new entity is now a single line in the registry - Execution order is visible at a glance |
||
|
Charles Bochet
|
58907b733c |
feat(logic-function): add LIVE / PREBUILT execution modes (#20873)
## Summary
### Why
1. Sending the code to the lambda (~1Mb usually) is heavy on network and
results to a constant traffic of ~30Mb/s on AWS which results into TB of
network data every month
2. eval(1MB of code) is not that fast, it's heavy on memory and CPU on
lambda side
### High level
Adds two execution modes for logic functions, gated behind the new
`IS_LOGIC_FUNCTION_PREBUILT_MODE_ENABLED` workspace feature flag (off
everywhere by default):
- **LIVE** (current behavior, preserved bit-for-bit): the compiled
bundle is read from object storage and shipped in every Lambda invoke
payload. Used for fast iteration in the workflow editor / Settings test
runs.
- **PREBUILT** (new): the bundle is installed onto the per-function
Lambda alongside the unified executor, and invocations carry only `{
params, env, handlerName }` — saving JSON payload egress and warm-start
`import()` cost on every call.
### Key design choices
- **Unified Lambda handler** (`constants/executor/index.mjs`) dispatches
at runtime: `event.code` present ? LIVE (write to `/tmp`, dynamic
import) : `import('./prebuilt-logic-function.mjs')`. Both code paths
always coexist on the deployment package, so the same Lambda can serve
either mode without redeploying.
- **Install runs inside the `validateBuildAndRun` migration pipeline**,
not at execute time. `Create/UpdateLogicFunctionActionHandlerService`
calls `driver.installPrebuiltBundle` when `executionMode` flips
LIVE?PREBUILT or `checksum` changes while PREBUILT, gated on
`isBuildUpToDate=true` and a fresh checksum.
- **Strict execute, no reconciliation**:
`LogicFunctionExecutorService.execute` resolves `effectiveExecutionMode`
(caller override > feature flag > entity column). For PREBUILT it asks
the driver `getInstalledBundleChecksum` (Lambda `twenty:bundle-checksum`
tag for AWS, sidecar file locally) and throws
`LOGIC_FUNCTION_PREBUILT_BUNDLE_NOT_INSTALLED` on mismatch.
- **Feature flag gates every side effect**: with the flag off the
executor forces LIVE, the action-handler install hooks bail before AWS,
and workflow activation does not flip the mode. Rollback is just turning
the flag off.
### Lifecycle
- New workflow CODE step ? `LIVE`, no install.
- Workflow activated ? build + activation flips `executionMode=PREBUILT`
? action-handler installs the bundle + sets the Lambda tag.
- Draft from active version ? duplicated logic function reset to `LIVE`.
- App install ? manifest converter sets `PREBUILT`, create-action
handler installs.
- Test runs (`executeOneFromSource`, workflow editor) pass
`executionMode=LIVE` explicitly.
### Observability
`[lambda-timing]` log lines now include `effectiveExecutionMode` and
`payloadBytes`; the action handler logs `install_duration_ms` for each
install.
## Test plan
- [x] `npx nx typecheck twenty-server` ? passes
- [x] `npx oxlint --type-aware` on all changed files ? 0 warnings, 0
errors
- [x] `npx nx test twenty-server` ? 588 suites / 5009 tests pass (no
regressions vs main)
- [x] New unit suite `flat-logic-function-validator.service.spec.ts` ?
9/9
- [x] Existing
`workflow-version-step-operations.workspace-service.spec.ts` ? 8/8
(verified the new token-based DI avoids a circular-import regression)
- [x] Snapshot for
`ALL_UNIVERSAL_FLAT_ENTITY_PROPERTIES_TO_COMPARE_AND_STRINGIFY` updated
to include `executionMode`
- [x] Integration suite `logic-function-execution.integration-spec.ts`
extended to assert `executionMode=LIVE` on newly-created functions and
continues to exercise the LIVE happy path
- [ ] Manual staging rollout: flip
`IS_LOGIC_FUNCTION_PREBUILT_MODE_ENABLED` per workspace, observe
`[lambda-timing]` `payloadBytes` drop + `install_duration_ms`, then ramp
in prod.
|
||
|
Paul Rastoin
|
989b45db15 |
Strictly type encryption rotation key site maps constants through entity type derivation (#21085)
# Introduction Followup https://github.com/twentyhq/twenty/pull/21001 Now that the typeorm entities provide grains over their `encryptedString` value, we can strictly type the sitemaps of the encrypted string to rotate in case of encryption key rotation and also the integration tests tests cases |
||
|
nitin
|
13f09d8946 |
[Dashboards] Remove gauge chart types and code (#20410)
Follow-up cleanup to #20172. |
||
|
Paul Rastoin
|
ebfaca5b3d |
EncryptedString PlaintextString branded string types (#21001)
## Summary closes https://github.com/twentyhq/core-team-issues/issues/2464 Introduces compile-time branded types to distinguish encrypted ciphertext from plaintext strings, preventing mix-ups like the one fixed in #20819 — but at the type level rather in addition to the one existing at runtime. ### Branded string primitives - Created `EncryptedString` and `PlaintextString` as hard nominal brands using `z.string().brand(...)`, making them non-assignable to each other or to raw `string` - Created `isEncryptedString` type predicate to narrow `string` to `EncryptedString` based on the `enc:v2:` envelope prefix - Retyped `SecretEncryptionService`: `encryptVersioned` accepts `PlaintextString`, `decryptVersioned` returns `PlaintextString` ### Entity typing - Typed encrypted columns across entities: `SigningKeyEntity.privateKey`, `TwoFactorAuthenticationMethodEntity.secret`, `ApplicationRegistrationVariableEntity.encryptedValue`, `ApplicationVariableEntity.value` - Parameterized JSONB types for connected account connection parameters (`ImapSmtpCaldavParams<Pwd>`) with reusable aliases `EncryptedImapSmtpCaldavParams` / `DecryptedImapSmtpCaldavParams` - Typed DTOs (`CreateApplicationRegistrationVariableInput`, `UpdateApplicationRegistrationVariablePayload`, `UpdateApplicationVariableEntityInput`) with `PlaintextString` ### ApplicationVariable always-encrypt uniformization - Retyped `ApplicationVariableEntity.value` to `EncryptedString | ''` — all values are now encrypted regardless of `isSecret` - Updated `ApplicationVariableEntityService` to always encrypt on write and always decrypt on read - Simplified `UpdateApplicationVariableActionHandlerService` by removing conditional encrypt/decrypt-on-isSecret-toggle logic - Added slow instance command (`2.9.0`) to backfill-encrypt existing `isSecret=false` plaintext rows and tighten the `CHECK` constraint ### ConfigStorageService refactor - Split `convertAndSecureValue` (which used `any`) into two well-typed methods: `convertAndDecrypt` and `convertAndEncrypt` - Introduced `isSensitiveStringValue` type predicate to narrow values before encryption/decryption ### What's next - Typeorm entity derivation to strictly type sitemap configuration as code + handler logic for encryption rotation - https://github.com/twentyhq/core-team-issues/issues/2465 |
||
|
nitin
|
863f3f29a2 |
Fix page layout widget tab moves (#20915)
Fixes widget moves between page layout tabs by making pageLayoutTabId part of the flat-entity diff, so the save mutation no longer silently drops the new tab assignment. https://discord.com/channels/1130383047699738754/1508737039128985680 |
||
|
Weiko
|
2f358a1775 |
Add a Table display mode to relation field widgets (#20929)
## Context Adds a new Table layout to the FIELD widget for to-many relation fields. On a record page, a relation can now be displayed as a full record table (the same component used for record indexes and dashboard table widgets) scoped to the records related to the current record. https://github.com/user-attachments/assets/320b24dc-f019-4d0e-bc71-3e64d032d75a https://github.com/user-attachments/assets/2f6d4f8e-de26-4fc1-ae12-c9b9c19654dc https://github.com/user-attachments/assets/3fb6d512-f83c-4818-823e-46ad2644fbc2 |
||
|
Weiko
|
90f711361c |
Add definePermissionFlag for app-defined permission flags (#20887)
## Context
Adds the SDK plumbing for apps to declare custom permission flags and
the server-side manifest pipeline to persist them.
```typescript
import { definePermissionFlag } from 'twenty-sdk/define';
export const MANAGE_INVOICES_PERMISSION_FLAG_UNIVERSAL_IDENTIFIER = '…';
export default definePermissionFlag({
universalIdentifier: MANAGE_INVOICES_PERMISSION_FLAG_UNIVERSAL_IDENTIFIER,
key: 'MANAGE_INVOICES',
label: 'Manage Invoices',
description: 'Create, edit, and delete invoices',
icon: 'IconReceipt',
});
```
```typescript
import { defineApplicationRole, SystemPermissionFlag } from 'twenty-sdk/define';
import { MANAGE_INVOICES_PERMISSION_FLAG_UNIVERSAL_IDENTIFIER } from './permission-flags/manage-invoices';
export default defineApplicationRole({
universalIdentifier: DEFAULT_ROLE_UNIVERSAL_IDENTIFIER,
label: `${APP_DISPLAY_NAME} default function role`,
// ...
permissionFlagUniversalIdentifiers: [
SystemPermissionFlag.UPLOAD_FILE,
MANAGE_INVOICES_PERMISSION_FLAG_UNIVERSAL_IDENTIFIER,
],
});
```
The flag can then be referenced by UUID in a role's
permissionFlagUniversalIdentifiers. On sync, the catalog row lands in
core.permissionFlag and the link in core.rolePermissionFlag.
## Not in this PR
- Runtime permission checks.
PermissionsService.getUserWorkspacePermissions still builds its result
from Object.values(PermissionFlagType), so custom flags are stored but
not yet enforced, code asking "does this role have MANAGE_INVOICES?"
won't get a meaningful answer. Widening PermissionsService and
UserWorkspacePermissions.permissionFlags to support arbitrary flag keys
is the next PR.
- PermissionFlag from apps can only define "tool" permissions and not
"settings" as a permissionType, this parameter is not mutable. This is
because "settings" are for settings page (until we might decide to
separate both type of permissions into 2 different entities) and apps
can't declare settings page or interact with them so this parameter
would be unnecessary.
|
||
|
Félix Malfait
|
d602f35cbd |
feat(data-model): custom-indexes management UI and mutations (#20846)
## Summary
Brings indexes management into the per-object Settings tab as a section
under Search (no feature flag, advanced mode only). Admins can create /
delete non-unique indexes with the UI; apps can declare indexes in code
with `defineIndex`. Composite-typed fields are now indexable by picking
a specific sub-column (e.g. `Address > City`).
A few related polish items also land here (invite-user dropdown lands on
the Invite tab; standard warning callout above the new-index form).
## What ships
### UI — custom indexes on per-object Settings
- New section directly under Search, wrapped in
`AdvancedSettingsWrapper`.
- Filter dropdown on the search bar toggles system-index visibility
(shown by default since advanced mode).
- **+ Add Index** button (disabled with tooltip once the per-object cap
is reached) navigates to a dedicated `SettingsObjectNewIndex` page
(matches the field-creation pattern, not a modal):
- Field picker mirrors the webhook event-form layout (rows of dropdowns,
implicit trailing empty row).
- Composite fields surface their sub-properties (`Address > City`,
`Currency > Amount`, …).
- BTREE / GIN type selector.
- Standard warning Callout: "Use indexes sparingly — each one speeds
reads but slows writes."
- Trash icon on `isCustom: true` rows → confirmation modal →
`deleteOneIndex`.
### Server — `createOneIndex` / `deleteOneIndex` mutations
- Gated by `SettingsPermissionGuard(DATA_MODEL)`.
- `IndexMetadataService` wraps the existing migration runner via
`WorkspaceMigrationValidateBuildAndRunService` so the metadata row and
the SQL index land atomically.
- Validation: rejects empty fields, duplicate `(fieldMetadataId,
subFieldName)` pairs, fields not on the object, requires `subFieldName`
for composite parents, forbids `subFieldName` on scalar/relation,
enforces `MAX_CUSTOM_INDEXES_PER_OBJECT = 10`.
- Delete refuses on `isCustom: false` rows so system indexes can't be
removed via this API.
- Dedicated GraphQL exception handler maps each typed error to the right
transport error class.
### Composite sub-field indexing
- Adds `subFieldName: string | null` column to
`IndexFieldMetadataEntity` (fast instance command).
- The flat-entity flow (`UniversalFlatIndexFieldMetadata`,
`FlatIndexFieldMetadata`, `from-universal-flat-index-to-flat-index`,
runner column resolution) all carry `subFieldName` through.
- For composite parents, the runner uses
`computeCompositeColumnName({...}, property)` for the picked sub-column;
for non-composite parents, behavior is unchanged.
- The `'::'` separator encodes `(fieldMetadataId, subFieldName)` for
dedup on the wire; the frontend uses the same separator inside the
Select component's string value.
### Apps can declare indexes in code (`defineIndex`)
- New `IndexManifest` + `IndexFieldManifest` types in
`twenty-shared/application` wired into the `Manifest` type.
- `defineIndex` SDK helper + `IndexConfig`. CLI manifest builder +
extractor recognize `defineIndex` / `ManifestEntityKey.Indexes`.
- Server: `from-index-manifest-to-universal-flat-index` converter
resolves field IDs, validates composite/scalar `subFieldName` rules, and
delegates to `generateFlatIndexMetadataWithNameOrThrow` for the
deterministic name.
- Orchestrator wires the loop after the field-resolution pass;
per-object cap enforced inline against the manifest.
- Cascade on uninstall is automatic — when an app disappears its indexes
drop with it (universal-flat-entity diff handles it).
- Rich-app fixture ships a real `defineIndex` on `PostCard.status`,
exercising the full manifest → install path in CI.
### Closed for now (open later if needed)
- Apps cannot declare `isUnique` indexes — unique constraints stay with
the field-creation flow.
- Apps cannot use a partial-`indexWhereClause` — the UI surface keeps
the framework's hardcoded allowlist.
- UI cannot create unique or partial indexes either; same reasons.
### Cleanups along the way
- Reused the existing `getCompositeSubFieldLabel` +
`COMPOSITE_FIELD_SUB_FIELD_LABELS` (deleted the duplicates I'd created
early in the PR).
- Moved `MAX_CUSTOM_INDEXES_PER_OBJECT` to `twenty-shared/constants`
(single source for FE + BE).
- Replaced inline `isDefined(x) && x !== ''` with `isNonEmptyString`
(from `@sniptt/guards`).
- Hoisted the per-object fields Map + inlined the cap counter into the
indexes orchestrator loop (drops the install scan from O(indexes ×
totalFields) to O(totalFields + indexes)).
- Per design-feedback: page-based create flow (not a modal), filter
dropdown on the SearchInput (not a separate toggle), webhook-style
picker, field icons.
### Unrelated polish that lands here
- "Invite user" link in the multi-workspace dropdown now lands on the
Invite tab directly (`#invite`) instead of the first tab of the members
page.
## Test plan
- [ ] `npx nx typecheck twenty-server / twenty-front / twenty-sdk /
twenty-shared` — passes
- [ ] `npx nx lint:diff-with-main twenty-server / twenty-front` — clean
- [ ] `npx jest index-metadata.service.spec` — green
- [ ] `npx jest from-index-manifest-to-universal-flat-index` — green
(new converter spec, 8 cases)
- [ ] `npx vitest run
src/sdk/define/indexes/__tests__/define-index.spec.ts` (twenty-sdk) —
green (6 cases)
- [ ] `npx vitest run --config vitest.integration.config.ts -t
"rich-app"` — green (rich-app app-dev integration exercises the new
manifest path with the PostCard.status index)
- [ ] Advanced mode → Settings → any object → Settings tab → Indexes
section is visible under Search
- [ ] Create a single-field BTREE index, confirm SQL index exists
(verify via `pg_indexes`)
- [ ] Create a composite-field index (`Address > City`) and confirm the
column is `addressAddressCity`
- [ ] Create an index spanning two columns; column order matches the
picker order
- [ ] Attempt to create an 11th custom index → button is disabled with
tooltip
- [ ] Delete a custom index → confirmation modal → row disappears, PG
index dropped
- [ ] System indexes have no trash icon and are hidden by default
|
||
|
Paul Rastoin
|
b8b115f4e3 |
FileStorageService Dedicated file and folder code flow + integrity check (#20831)
# Introduction
Next handling mimetype integrity check and checksum integrity check for
s3 storage type
Always expecting a trailing end slash when deleting a folder etc
## Application
Uninstalling an application now deletes all its related files
## File storage service
Making a distincton between folder path and file path
## Validation Pipeline
Every file operation in `FileStorageService.buildOnStoragePath` runs
through `validateResourcePath`, which chains three validators in order:
**1. `validateSafeRelativePath`** -- rejects path traversal attacks
| Input | Result | Error |
|---|---|---|
| `../../../etc/passwd` | Rejected | `Resource path must not contain
path traversal (..)` |
| `/etc/passwd` | Rejected | `Resource path must be relative, not
absolute` |
| `file\0.txt` | Rejected | `Resource path contains null bytes` |
| `..\\..\\etc\\passwd` | Rejected | `Resource path must not contain
backslashes` |
| _(empty)_ | Rejected | `Resource path must not be empty` |
**2. `validateFilenameIntegrity`** -- enforces safe characters, length
limits, extension required
| Input | Result | Error |
|---|---|---|
| `my folder/file.mjs` | Rejected | `A path segment contains invalid
characters...` |
| `Makefile` | Rejected | `Filename must have an extension` |
| `aaa...(256 chars).mjs` | Rejected | `A path segment exceeds the
maximum length of 255 characters` |
| `a/b/.../file.mjs` (1025+ chars) | Rejected | `Resource path exceeds
maximum length of 1024 characters` |
| `src/handlers/index.mjs` | Accepted | -- |
| `my-app/my_file.tsx` | Accepted | -- |
| `v1.0/module.config.mjs` | Accepted | -- |
Allowed characters per segment: `a-z`, `A-Z`, `0-9`, `.`, `-`, `_`
**3. `validateResourceExtension`** -- checks extension against the
`FileFolder` allowlist
| Input | FileFolder | Result | Error |
|---|---|---|---|
| `handler.js` | `BuiltLogicFunction` | Rejected | `Invalid file
extension. Allowed extensions: .mjs` |
| `card.tsx` | `BuiltFrontComponent` | Rejected | `Invalid file
extension. Allowed extensions: .mjs` |
| `script.js` | `PublicAsset` | Rejected | `Invalid file extension.
Allowed extensions: .png, .jpg, ...` |
| `index.mjs` | `BuiltLogicFunction` | Accepted | -- |
| `app.tsx` | `Source` | Accepted | -- |
| `photo.png` | `CorePicture` | Accepted | -- (unconfigured folder,
passes through) |
## Consumers
- **`FileStorageService`** -- calls `validateResourcePath`, throws
`FileStorageException` on failure (last-resort defense)
- **Resolver (`uploadApplicationFile`)** -- calls
`validateResourcePath`, throws `ApplicationException` on failure
(user-facing)
- **Flat validators** -- call `validateResourcePath`, push the error to
`validationResult.errors` (non-throwing, collects all errors)
All error messages are translated via Lingui `t` and returned in a
discriminated union `{ isValid: true } | { isValid: false, error: string
}`, letting each consumer decide how to handle failures.
|
||
|
Paul Rastoin
|
76e144e85a |
Deprecate messageChannel messageFolder calendarChannel standard objects (#20836)
# Introduction Removing old standard objects `messageChannel` and `messageFolder` and `calendarChannel` --------- Co-authored-by: Cursor <cursoragent@cursor.com> |
||
|
Félix Malfait
|
084fa8eaba |
fix(server): auto-index target<X>Id join columns on polymorphic standard objects (#20820)
Closes #20726 ## The bug `timelineActivity` (and the three other polymorphic standard objects — `attachment`, `noteTarget`, `taskTarget`) store relations as N nullable `target<X>Id` columns, one per related object. Each one is a join key queried as `WHERE target<X>Id IN (...) AND deletedAt IS NULL`. For **built-in** related objects (Person, Company, Opportunity, …), each `target<X>Id` column gets a BTREE index, declared statically in `compute-{timelineActivity,attachment,noteTarget,taskTarget}-standard-flat-index-metadata.util.ts`. For **custom** related objects, the same `target<CustomObject>Id` column was added — **without an index**. On a `timelineActivity` table at issue-reporter scale (~21.9M rows, 7.1 GB), this turned record loads into 20–40s sequential scans and produced `QueryFailedError: Query read timeout` for end users. ## Diagnosis The morph/relation field generator (`generateMorphOrRelationFlatFieldMetadataPair`) already creates a BTREE index for the field that owns the join column and returns it alongside the field metadata pair. The two user-driven entry points (`fromRelationCreateFieldInput…`, `fromMorphRelationCreateFieldInput…`) correctly destructure and propagate that index. But the **custom-object creation path** — `buildDefaultRelationFlatFieldMetadatasForCustomObject`, called when a user creates a new custom object — destructured only `{ flatFieldMetadatas }` and threw away `indexMetadatas`. So every `target<CustomObject>Id` column added to the four polymorphic standard objects has been shipping unindexed since custom morph relations went in. ## The fix Three commits. ### 1. `fix(server): index target<CustomObject>Id columns on standard polymorphic objects` 13 lines across 2 files. - `build-default-relation-flat-field-metadatas-for-custom-object.util.ts` — also destructure `indexMetadatas` from the pair generator and accumulate them into the returned record (new field `standardTargetFlatIndexMetadatas`). - `from-create-object-input-to-flat-object-metadata-and-flat-field-metadatas-to-create.util.ts` — append the accumulated indexes to `flatIndexMetadataToCreate`. The migration pipeline at `object-metadata.service.ts:559–562` already passes `flatIndexMetadataToCreate` to the migration runner, so no further wiring is needed. From now on, creating a custom object also creates the four BTREE indexes — one per polymorphic standard object's new `target<CustomObject>Id` column — atomically with the rest of the migration. ### 2. `feat(server): backfill workspace command for relation join column indexes` For existing workspaces whose custom objects were created before the forward-fix. `upgrade:2-8:backfill-relation-join-column-indexes` is a `@RegisteredWorkspaceCommand('2.8.0', 1798100000000)` matching the pattern from `2-7-workspace-command-…-drop-connected-account-standard-object.command.ts`. Per workspace: 1. Load `flatObjectMetadataMaps`, `flatFieldMetadataMaps`, `flatIndexMaps` from the workspace cache. 2. Resolve the four polymorphic standard object IDs by `nameSingular` against `DEFAULT_RELATIONS_OBJECTS_STANDARD_IDS`. 3. Collect every field ID that's already covered by any existing index. 4. Filter `flatFieldMetadataMaps` to MORPH_RELATION fields on those four objects whose `settings.relationType === MANY_TO_ONE` (i.e. owns a join column) and whose ID isn't in the indexed set. 5. Generate a BTREE `UniversalFlatIndexMetadata` for each via `generateIndexForFlatFieldMetadata` (same helper the forward-fix uses). 6. Create the indexes in the workspace schema with **CONCURRENTLY** (see commit 3). 7. Submit the metadata through `WorkspaceMigrationValidateBuildAndRunService` so it lands in `indexMetadata` and the cache — same pipeline as a normal metadata change. The pipeline's own `CREATE INDEX IF NOT EXISTS` no-ops because the index already exists. Properties: - **Idempotent.** Re-running is a no-op once indexes exist. - **Scoped.** Only the four polymorphic standard objects, only their MANY_TO_ONE morph relation fields, only those with no covering index. - **Same code path as the forward-fix.** The backfill produces exactly the indexes the forward-fix would have created at custom-object creation time. - **`--dry-run` supported** via the base `ActiveOrSuspendedWorkspaceCommandRunner`. ### 3. `feat(server): create index CONCURRENTLY in relation join column backfill` Adds an opt-in `concurrently` flag to `WorkspaceSchemaIndexManagerService.createIndex` (threaded through `createIndexInWorkspaceSchema`). When `true`, emits `CREATE INDEX CONCURRENTLY IF NOT EXISTS …`. Defaults to `false` — every existing caller keeps the current transactional `CREATE INDEX` behavior. The backfill command opts in. It creates a QueryRunner **without** `startTransaction()`, issues the CONCURRENTLY indexes one-by-one (each waits for the previous to finish), then submits the metadata through the normal migration pipeline whose own `CREATE INDEX IF NOT EXISTS` is now a no-op. Why not flip the default for the helper: - `CREATE INDEX CONCURRENTLY` cannot run inside a transaction — Postgres errors out. The migration pipeline calls `createIndex` from inside a transactional schema migration. - CONCURRENTLY doesn't roll back with the transaction. If the surrounding migration fails, the index remains and you end up with metadata/schema drift. - Failed CONCURRENTLY builds leave an INVALID index behind that needs manual `DROP`. - UNIQUE indexes have different failure semantics under CONCURRENTLY (deferred, not immediate). So CONCURRENTLY is opt-in, used only where it's the right tool (post-hoc backfills on populated tables). ## Decisions / tradeoffs - **Single-column BTREE vs partial `WHERE deletedAt IS NULL` vs composite.** Twenty's queries always include `deletedAt IS NULL`. A partial index would be slightly better than a plain BTREE (smaller, no wasted seeks on soft-deleted rows). This PR ships single-column to match the existing built-in target index pattern, which already covers >95% of the available speedup (the 20s→4ms drop the reporter saw comes from having any index — composite/partial is a second-order effect). Switching all relation indexes to partial is a separate, broader change. - **CONCURRENTLY operator caveat.** If a CONCURRENTLY build is interrupted (kill, connection drop, OOM), Postgres leaves the index as INVALID. We deliberately don't probe `pg_index` for invalid leftovers on every create — catalog-table queries can be slow at multi-tenant scale and the failure mode is rare. Recovery is manual: `DROP INDEX <name>` and re-run the backfill. - **Forward-fix is not gated** behind a feature flag. The change is metadata-pipeline-internal; before, custom-object creation silently produced a degraded state. After, it produces the correct state. No new public API, no behavioural change for end users besides the indexes existing. ## Risk - Forward-fix: changes only the metadata produced during custom-object creation. New objects get four extra `FlatIndexMetadata` rows and four extra `CREATE INDEX` statements during their creation migration. Tables are empty at that point so the index builds in microseconds. - Helper change: API-compatible, default behavior unchanged. The new `concurrently` parameter is optional. - Backfill: read-only state probe → CONCURRENTLY index creation (no write blocking) → metadata insert via the normal migration pipeline. Idempotent. Reverting is `DROP INDEX`. ## Test plan - [ ] Verify forward-fix: create a custom object, confirm four new BTREE indexes appear on `timelineActivity`, `attachment`, `noteTarget`, `taskTarget` for the new `target<CustomObject>Id` columns, and that `flatIndexMaps` has matching entries. - [ ] Verify backfill on a workspace that had custom objects created before the fix: run `--dry-run` first, confirm the expected indexes are listed; then run for real, confirm the indexes appear in pg (and as `indisvalid = true` in `pg_index`) and in `flatIndexMaps`. Re-run; confirm no-op. - [ ] Verify backfill on a clean workspace: should log "no missing indexes" and exit. - [ ] Verify CONCURRENTLY behavior under load: run backfill against a workspace with active writes on `timelineActivity`; confirm inserts/updates keep working during index build (no `ShareLock` waits in `pg_stat_activity`). - [ ] On the affected reporter-scale workspace, confirm `EXPLAIN ANALYZE` switches from sequential scan to index scan and timeline activity timeouts go away. |
||
|
nitin
|
068d365731 |
feat(sdk): error on incompatible view filter operand at sync time (#20763)
view filters with mismatched operand + field type now error at sync -- was silently failing before |
