## Unify the two metadata override mechanisms into one
Twenty had **two** override mechanisms:
- **`standardOverrides`** — a bespoke JSONB column on
`objectMetadata`/`fieldMetadata` with typed DTOs and a per-locale
`translations` map, resolved by two i18n-aware resolvers.
- **`OverridableEntity.overrides`** — a flat, registry-driven JSONB blob
on view / view-field / view-field-group / command-menu-item /
page-layout-tab / page-layout-widget, resolved by a plain spread.
This PR collapses them into **one** concept: a single `overrides` blob,
one registry-driven overridable set, one i18n-aware read path, and one
write path (`computeMetadataOverridesBlob`, extracted in #22404).
Object/field **stay on `SyncableEntity`** (not reparented to
`OverridableEntity`) so their `isActive` default stays **FALSE** — this
sidesteps the `isActive` default conflict entirely.
### GraphQL breaking change (accepted)
The `standardOverrides` field is **removed** with no deprecation alias —
`overrides` (a `JSON` scalar) is exposed instead on `Object` and
`Field`. Product confirmed negligible external usage; the front-end has
no hand-written consumer (only generated types), which are regenerated
here.
### Commit structure (reviewable commit-by-commit)
1. **Unified resolver + parity harness** —
`resolveEffectiveEntityProperty` is a strict superset of the three
legacy resolvers; a corpus parity spec compares it against a *frozen
reference* of the old logic across every locale, `isStandardApp` branch
and override shape.
2. **Registry-driven** — object/field presentation props tagged
`isOverridable` + `translatable`; the overridable/translatable sets are
derived from the registry (a test asserts they equal the legacy
hardcoded lists).
3. **Rename + swap + delete** — `standardOverrides` → `overrides` across
entities, DTOs, flat/universal types, producers, the ~12
resolve/write/create/sync call sites, mocks and specs; the reconciler's
two compare entries collapse to one; the three legacy resolvers, both
DTOs and the hardcoded constants/types are deleted.
4. **Migration (zero-downtime, two-phase)** — split across two releases
so a rolling deploy never drops a column a previous-release pod still
`SELECT`s:
- **2.19 fast** — add the `overrides` column (schema only).
- **2.19 slow** — backfill `overrides` from `standardOverrides` in
`runDataMigration` (kept out of the schema transaction so the bulk write
doesn't hold the ACCESS EXCLUSIVE lock; skipped on fresh installs, which
have no data to copy).
- **2.20 fast** — drop the legacy `standardOverrides` column (gated by
`TWENTY_NEXT_VERSIONS`, so it stays dormant until the instance reaches
2.20).
5. **Front/client-SDK regen** — regenerated metadata GraphQL types.
6. **Integration specs + i18n** — updated the standard object/field
update integration specs + snapshots, and the reworded validator message
catalog entry.
### Rolling-deploy safety
`standardOverrides` is retained through 2.19 and only dropped in 2.20,
mirroring the codebase's deferred-drop convention
(`isUIReadOnly`/`isCustom`). During the 2.19 rollout both columns exist,
so old and new pods coexist without "column does not exist" errors. The
backfill lives in a slow `runDataMigration` (per the
`no-data-mutation-in-fast-instance-command` rule) so it doesn't stall
reads.
### `isActive` guard
The migration never reads or writes `isActive`; the backfill asserts the
active-row count is unchanged and aborts otherwise. Verified on a real
DB: apply + revert preserves the blob **and** the nested `translations`
map, with `isActive` counts identical before/after.
### Verification (local)
- `nx typecheck twenty-server` + `nx typecheck twenty-front` — green
- `nx lint:diff-with-main twenty-server` (oxlint `--type-aware` + oxfmt)
— green
- `nx test twenty-server` — green (unit + parity + registry + migration
tests)
- `nx run twenty-server:test:integration:with-db-reset` — green
- `database:reset` applies the 2.19 phases and leaves **both** columns
present (2.20 drop stays dormant); backfill + revert round-trip verified
on a real DB
- Metadata integration suites (standard object/field update, application
sync) pass end-to-end against the two-column schema
- Metadata GraphQL types regenerated against a booted server; zero
`standardOverrides` references remain in application code (only the
migration commands + the legacy schema baseline)
---------
Co-authored-by: prastoin <paul@twenty.com>
## Context
When an object is created via the metadata API, `createOneObject`
creates its side-effect entities (INDEX view + viewFields, indexes,
navigation menu item, "go to" command menu item, record-page fields
view, page layout/tabs/widgets) across **three separate
`validateBuildAndRunWorkspaceMigration` calls**, purely because the
protection behavior (mutations → overrides, delete → deactivate, reset →
reactivate) was keyed on *"owned by the standard app"*, forcing the side
effects into batches with different application owners. This
misrepresents ownership and breaks atomicity.
This PR separates two orthogonal concepts:
- **Ownership** (`applicationId`), the true owner: the caller's
application (the workspace custom app today, 3rd-party apps later).
- **Protection** (`isSystemSideEffect`), the row was generated by the
system, so user mutations route to overrides, deletion becomes
deactivation, and reset restores defaults.
Once side effects are re-owned to the caller, the old `applicationId ===
standardApp` check can no longer tell an original side-effect row from a
user-added one so a dedicated `isSystemSideEffect` flag carries the
protection instead.
This is **PR 1 of 2** (forward-only). It makes newly created objects and
fields correct; existing workspaces are handled by a follow-up backfill
(see *Out of scope*).
## What this PR does
- **`isSystemSideEffect` column** on the 8 affected entities (`view`,
`viewField`, `indexMetadata`, `commandMenuItem`, `pageLayout`,
`pageLayoutTab`, `pageLayoutWidget`, `fieldMetadata`), with
`@WasIntroducedInUpgrade` + an entry in the flat-entity property
configuration (`toCompare: true`, read-only).
- **Single atomic migration in `createOneObject`**: the three
`validateBuildAndRunWorkspaceMigration` calls are merged into one, owned
by the caller (`resolvedOwnerFlatApplication`) and the record-page
view/fields, page layout, and navigation command item are re-owned to
the caller and flagged `isSystemSideEffect: true`.
`buildNavigationFlatCommandMenuItem` is parameterized with
`applicationUniversalIdentifier` (no longer hardcoded to the standard
app).
- **Field-creation side effects** (`createManyFields`/`createOneField`
already run as a single caller-owned migration, so no re-ownership/merge
was needed): the auto-created viewField is flagged `isSystemSideEffect:
true`, and a new field now also propagates to the object's **INDEX/table
view** (added there as a **hidden** column, `isVisible: false`) in
addition to
the record-page FIELDS widget. The INDEX view is targeted directly by
`key = INDEX` (it is not a page-layout widget), de-duplicated per
`(viewId, fieldMetadataUniversalIdentifier)` to respect the per-view
unique index.
The unique-field index is likewise flagged the inverse relation field
stays unflagged (`isSystem: false`).
- **Protection predicate** extended: `isCallerOverridingEntity` and the
removal/reset split strategies now treat `isSystemSideEffect` rows as
protected even when caller-owned (route to
overrides / deactivate / reset) and the page-layout-reset guards allow
resetting flagged entities.
- **Standard compute maps** set the flag consistently so a re-sync
produces no diff (standard-object side effects stay `false`; per-object
nav command items and custom-object base fields are `true`).
- **Read-only GraphQL exposure** of `isSystemSideEffect` on the view /
view-field / page-layout / tab / widget / command-menu-item DTOs (not
exposed on create/update inputs). => Todo: needs to take this new flag
into account. This is fine for now because isSystem remains on
object/field.
- **Fast instance command** (`2-14`) adding the 8 columns (`NOT NULL
DEFAULT false`).
## Scope decisions
- **`pageLayout` is not an `OverridableEntity`**, its own row has
nothing user-overridable (all customization lives on tabs/widgets). It's
dual-purpose (`RECORD_PAGE` side-effect vs. user `DASHBOARD`), so it
gets `isSystemSideEffect` for protection only, no `overrides` jsonb.
- **`navigationMenuItem` is out of scope.**: Those are side effects only
for the metadata API and not marked as "system" (they can be
deleted/updated etc...)
- **`viewFieldGroup` is not a side effect**, it's only created via the
explicit view-field-group API, never by object/field creation, so it
gets no flag.
## Out of scope (follow-ups)
**PR 2** — slow per-workspace backfill (re-own + flag existing side
effects, recreate missing ones) and deterministic v5 identifiers for
base fields / pageLayout / tab.
<!-- This is an auto-generated description by cubic. -->
<a
href="https://cubic.dev/pr/twentyhq/twenty/pull/21673?utm_source=github"
target="_blank" rel="noopener noreferrer"
data-no-image-dialog="true"><picture><source
media="(prefers-color-scheme: dark)"
srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source
media="(prefers-color-scheme: light)"
srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img
alt="Review in cubic"
src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a>
<!-- End of auto-generated description by cubic. -->
## Summary
`PermissionsException` thrown by `SettingsPermissionGuard` (and other
permission code paths) was bubbling up through every typed REST
exception filter and landing in the global `UnhandledExceptionFilter`,
which falls back to **500** for anything that isn't an `HttpException`.
So a forbidden user (e.g. an API key whose role doesn't have
`DATA_MODEL`) calling `GET /rest/metadata/objects` got:
```
HTTP/1.1 500 Internal Server Error "Entity performing the request does not have permission"
```
GraphQL already had the right plumbing via
`permissionGraphqlApiExceptionHandler` (`ForbiddenError` → 403,
`UserInputError` → 400, `NotFoundError` → 404). This PR mirrors it on
the REST side.
## What
- New util `permissionRestApiExceptionCodeToHttpStatus` mapping every
`PermissionsExceptionCode` → HTTP status, with `assertUnreachable` to
force explicit handling of future codes.
- New filter `PermissionsRestApiExceptionFilter`
(`@Catch(PermissionsException)`) that delegates to
`HttpExceptionHandlerService.handleError(...)` with the resolved status.
- Wired `PermissionsRestApiExceptionFilter` (placed first, so the typed
filter wins over any sibling catch-all) into `@UseFilters(...)` of every
REST controller that uses `SettingsPermissionGuard` or whose service can
throw `PermissionsException`:
- `object-metadata`, `field-metadata`, `webhook`, `api-key`
- `view`, `view-sort`, `view-group`, `view-filter`, `view-filter-group`,
`view-field`
- `page-layout`, `page-layout-widget`, `page-layout-tab`
- `front-component`, `ai-generate-text`
- Unit tests covering 403 / 400 / 404 / 500 mappings.
## Mapping
| Code | Status |
|------|--------|
| `PERMISSION_DENIED`, `NO_AUTHENTICATION_CONTEXT`,
`ROLE_LABEL_ALREADY_EXISTS`, `CANNOT_UNASSIGN_LAST_ADMIN`,
`CANNOT_UPDATE_SELF_ROLE`, `CANNOT_DELETE_LAST_ADMIN_USER`,
`ROLE_NOT_EDITABLE`, `CANNOT_ADD_OBJECT_PERMISSION_ON_SYSTEM_OBJECT`,
`CANNOT_ADD_FIELD_PERMISSION_ON_SYSTEM_OBJECT` | **403** |
| `INVALID_ARG`, `INVALID_SETTING`,
`CANNOT_GIVE_WRITING_PERMISSION_ON_NON_READABLE_OBJECT`,
`CANNOT_GIVE_WRITING_PERMISSION_WITHOUT_READING_PERMISSION`,
`ONLY_FIELD_RESTRICTION_ALLOWED`,
`FIELD_RESTRICTION_ONLY_ALLOWED_ON_READABLE_OBJECT`,
`FIELD_RESTRICTION_ON_UPDATE_ONLY_ALLOWED_ON_UPDATABLE_OBJECT`,
`EMPTY_FIELD_PERMISSION_NOT_ALLOWED`,
`ROLE_MUST_HAVE_AT_LEAST_ONE_TARGET`, `ROLE_CANNOT_BE_ASSIGNED_TO_USERS`
| **400** |
| `ROLE_NOT_FOUND`, `OBJECT_METADATA_NOT_FOUND`,
`FIELD_METADATA_NOT_FOUND`, `FIELD_PERMISSION_NOT_FOUND`,
`PERMISSION_NOT_FOUND` | **404** |
| All remaining "internal" codes (rethrown as-is in GraphQL) | **500** |
## Before
<img width="507" height="216" alt="Screenshot 2026-05-19 at 19 26 07"
src="https://github.com/user-attachments/assets/21d633aa-7ee8-4923-94e4-7ad57258a29e"
/>
## After
<img width="610" height="385" alt="Screenshot 2026-05-19 at 19 26 01"
src="https://github.com/user-attachments/assets/0103b7ee-7df7-4aef-999a-73c22901afd2"
/>
## Context
Calling `POST /rest/views` (and other metadata mutations) currently
returns a generic `500` for user-input failures:
Ex:
1. **Invalid `objectMetadataId`** —
`resolveEntityRelationUniversalIdentifiers` throws
`FlatEntityMapsException(RELATION_UNIVERSAL_IDENTIFIER_NOT_FOUND)`.
Should be `404`.
2. **Missing required field** (e.g. `icon`) — Postgres raises a `NOT
NULL` violation, wrapped as
`WorkspaceMigrationRunnerException(EXECUTION_FAILED)` carrying a
`QueryFailedError`. Should be `400`.
Neither was caught by `ViewRestApiExceptionFilter`, so both fell through
to `UnhandledExceptionFilter` and were emitted as `500`s without
reaching Sentry.
Same gap existed on most metadata GraphQL resolvers — only
`page-layout*` and `role` resolvers covered
`WorkspaceMigrationRunnerException` via
`WorkspaceMigrationGraphqlApiExceptionInterceptor`.
## Changes
### New filters
REST (`HttpExceptionHandlerService` + Sentry-aware):
- `FlatEntityMapsRestApiExceptionFilter` — maps
`RELATION_UNIVERSAL_IDENTIFIER_NOT_FOUND` / `ENTITY_NOT_FOUND` → `404`,
`ENTITY_ALREADY_EXISTS` → `409`, others → `500`.
- `WorkspaceMigrationRunnerRestApiExceptionFilter` — for
`EXECUTION_FAILED`, unwraps the underlying `metadata` /
`workspaceSchema` / `actionTranspilation` error; if it's a
`QueryFailedError` it gets remapped to `400` via
`HttpExceptionHandlerService`. `APPLICATION_NOT_FOUND` → `404`,
`DDL_LOCKED` → `503`, otherwise `500`.
GraphQL (graphql-errors + existing formatter):
- `FlatEntityMapsGraphqlApiExceptionFilter` — kept as the GraphQL-shaped
counterpart (`NotFoundError` / `InternalServerError`).
- `WorkspaceMigrationRunnerGraphqlApiExceptionFilter` — reuses
`workspaceMigrationRunnerExceptionFormatter` for parity with the
existing interceptor.
### Wiring
Filters are now declared **per controller / resolver** via `@UseFilters`
(no global `APP_FILTER` registration) so they participate in the normal
NestJS filter chain instead of being preempted by
`UnhandledExceptionFilter`.
REST:
- `view.controller.ts` — adds `FlatEntityMapsRestApiExceptionFilter` and
`WorkspaceMigrationRunnerRestApiExceptionFilter`.
GraphQL (14 resolvers, all that mutate flat entities):
- `FlatEntityMapsGraphqlApiExceptionFilter` added to: `view`,
`view-field`, `view-field-group`, `view-sort`, `view-group`,
`view-filter`, `view-filter-group`, `page-layout`, `page-layout-tab`,
`page-layout-widget`, `role`, `object-metadata`, `field-metadata`,
`index-metadata`.
- `WorkspaceMigrationRunnerGraphqlApiExceptionFilter` added to the same
list **except** the four already covered by
`WorkspaceMigrationGraphqlApiExceptionInterceptor` (`page-layout`,
`page-layout-tab`, `page-layout-widget`, `role`) — to avoid
double-handling.
## Why per-resolver / per-controller instead of global
Earlier attempt to register the filters globally via `APP_FILTER`
regressed: NestJS reverses the global filter list and
`selectExceptionFilterMetadata` is first-match-wins, so
`UnhandledExceptionFilter` (registered last via `app.useGlobalFilters`
in `main.ts`) ended up first in the iteration order and preempted every
domain-specific filter. The per-resolver / per-controller approach is
explicit and predictable.
## Before
<img width="953" height="450" alt="Screenshot 2026-05-12 at 15 31 40"
src="https://github.com/user-attachments/assets/3c3bc6a8-f6bc-4032-97d0-7243540cfb90"
/>
## After
<img width="1050" height="598" alt="Screenshot 2026-05-12 at 15 31 17"
src="https://github.com/user-attachments/assets/c66c9ce5-d1ea-4f1d-b2fe-07979e2261f7"
/>
<img width="1068" height="503" alt="Screenshot 2026-05-12 at 15 31 09"
src="https://github.com/user-attachments/assets/ddd9eed8-812b-47d6-96cb-b019b807991b"
/>
## Context
Phase 1 of removing the legacy `gridPosition` field from
`PageLayoutWidget` in favor of the new `position` discriminated union
(`grid` / `vertical-list` / `canvas`). This PR is purely additive —
`gridPosition` is still required and read everywhere; we just guarantee
that every widget now also has a non-null `position` so a follow-up PR
can drop `gridPosition` cleanly.
## Changes
- **Slow instance command**
`BackfillPageLayoutWidgetPositionSlowInstanceCommand` (2.1.0):
for every `core.pageLayoutWidget` row where `position IS NULL`, copies
`gridPosition`
into `position` with `layoutMode: 'GRID'`. Historically only grid
widgets used
`gridPosition`, so a single SQL update covers every existing row.
- **`PageLayoutDuplicationService`**: when duplicating a widget, also
forwards
`originalWidget.position` (was previously only forwarding
`gridPosition`).
- **Frontend default layouts** (10 `Default*PageLayout.ts` files): added
a `position`
sibling to every widget, matching the parent tab's `layoutMode` —
`VERTICAL_LIST` widgets
get `{ layoutMode, index }`, `CANVAS` widgets get `{ layoutMode }`
<img width="338" height="226" alt="Screenshot 2026-04-24 at 16 23 17"
src="https://github.com/user-attachments/assets/c3319318-f1b8-4271-96b4-196b209a1f5e"
/>
## Context
Tab duplication was broken after the view creation logic was deferred
and moved to the FE.
- Duplicating a tab or a FIELDS widget now produces a fully independent
copy: new view, new view field groups, new view fields — all with fresh
IDs — while preserving any unsaved edits
from the source widget.
- Removed the backend auto-seed of default view fields / view field
groups in ViewService.createOne for FIELDS_WIDGET views. The frontend
always sends the complete layout via
upsertFieldsWidget, so the auto-seed was both redundant and the source
of potential bugs.
- Extracted a shared useDuplicateFieldsWidgetForPageLayout hook used by
both tab and widget duplication paths, plus a small
useCloneViewInMetadataStore helper that clones the FlatView
in the metadata store and returns the copied flat view fields/groups for
the caller.
Add support for standalone pages: a new `PageLayout` type
(`STANDALONE_PAGE`) that can be rendered independently at
`/page/:pageLayoutId`, not tied to any record or object context.
- New `STANDALONE_PAGE` page layout type
- New `PAGE_LAYOUT` navigation menu item type: adds a `pageLayoutId`
foreign key to `NavigationMenuItemEntity`, allowing sidebar items to
link directly to standalone pages
- New `GLOBAL_OBJECT_CONTEXT` command menu availability type: separates
object-context-dependent commands (Create Record, Import, Export, See
Deleted, Create View, Hide Deleted) from truly global ones, so
standalone pages only show relevant commands
- Frontend routing & rendering: adds a `/page/:pageLayoutId` route with
its own page component, header, and command menu
- Widget rendering refactor
- Instance commands: two fast 1.22 migrations: `pageLayoutId` column +
`STANDALONE_PAGE` enum, and `GLOBAL_OBJECT_CONTEXT` availability type
enum
- Workspace command: backfills existing command menu items from `GLOBAL`
to `GLOBAL_OBJECT_CONTEXT` where appropriate
- Dev seeds: adds a sample "Star History" standalone page with an iframe
widget for local development
- Add resetPageLayoutToDefault GraphQL mutation that resets an entire
page layout (all tabs, widgets, view field groups, and view fields) to
their default state in a single operation
- Add a "Reset to default" button in the settings Layout tab
(/settings/objects/:object#layout) with a confirmation modal
<img width="673" height="426" alt="Screenshot 2026-04-14 at 13 13 53"
src="https://github.com/user-attachments/assets/002d33c9-9ea1-49f2-bef6-179ce034c126"
/>
## Context
Moving isActive filtering to the frontend for page layout tabs and
widgets, hiding inactive entities from the UI while keeping them in
state for future reactivation
Next we will implement deactivated standard tab re-activation during tab
creation (cc @Devessier)
<img width="234" height="303" alt="📋 Menu (Slots)"
src="https://github.com/user-attachments/assets/17a25ac6-55e2-4778-b7f0-e7554ed69704"
/>
## Context
- Add "Reset to default" for page layout tabs and widgets — backend
mutation resets overrides, reactivates deactivated children, and deletes
custom children
- Fix override write bug where mutating an overridable property (e.g.
widget title) on a standard-app entity incorrectly overwrote the base
column instead of writing to the overrides JSONB —
PageLayoutUpdateService now uses resolveFlatEntityOverridableProperties
for accurate diffing and routes properties through
sanitizeOverridableEntityInput
- Deprecate isOverridden - We need more time to think about this
feature. Currently this adds too much complexity for a very small
benefit
https://github.com/user-attachments/assets/a84546c8-1e15-4d9e-a489-0825cf8b8ed2
- Replace soft-deletion (deletedAt) with isActive boolean for
overridable entities (tabs, widgets, viewFieldGroups, viewFields).
Standard entities are deactivated (isActive: false) when removed from
update payloads, while custom entities are hard-deleted.
- When a viewFieldGroup is deactivated/deleted, its viewFields are
reassigned to the next section by position (or null if none remain).
- Add isActive: true filters to viewFieldGroup and viewField API queries
so deactivated entities are excluded from responses.
Next:
- fields-widget-upsert.service.ts should be refactored a bit
- Add restore logic
- Fix duplication of tabs in dashboards that didn't open the duplicated
tab in the side panel: replaced the logic that used Math.round with an
algorithm that switches the positions of the elements. We will keep
relying on integers, but it will work well in all cases.
- Make dnd work with record page layouts, where there is a pinned tab
- Make tab movements work with pinned tab, too
- Allow user to set a tab as the pinned tab
- Make backend changes to be able to save tabs with `layoutMode`
https://github.com/user-attachments/assets/ce1130fa-71df-49ba-ba2f-6f971e15dd49
---------
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
## Refactor page layout widget types into shared package and expose from
SDK
### Why
Widget configuration types were defined only on the server, forcing SDK
consumer apps to import from deep internal `twenty-shared/dist` paths —
fragile and breaks on structural changes. Server DTOs also had no
compile-time guarantee they matched the canonical types.
### What changed
- **`twenty-shared`**: Migrated `ChartFilter`, `GridPosition`,
`RatioAggregateConfig` and all 20 widget configuration variants into
`twenty-shared/types`. `PageLayoutWidgetConfiguration` (base, with
`SerializedRelation`) and `PageLayoutWidgetUniversalConfiguration`
(derived via `FormatRecordSerializedRelationProperties`) are now the
single source of truth.
- **`twenty-sdk`**: Re-exported `AggregateOperations`,
`ObjectRecordGroupByDateGranularity`, `PageLayoutTabLayoutMode`, and
`PageLayoutWidgetUniversalConfiguration` so consumer apps import from
`twenty-sdk` directly.
- **`twenty-server`**: All widget DTOs now `implements` their shared
type for compile-time enforcement. Added helpers to convert nested
`fieldMetadataId` ↔ `fieldMetadataUniversalIdentifier` inside chart
filters. Removed redundant local type re-exports.
## Context
Introducing "NewFieldDefaultConfiguration" to FIELDS widget
configurations
```typescript
{
isVisible: boolean;
viewFieldGroupId: string | null;
}
```
This configuration will define where a new field should be added (which
section) and its default visibility inside FIELDS widget views.
The new field position should always be at the end (meaning the last
position for the view fields OR the last position of a viewFieldGroup)
See "New fields" on this screenshot
<img width="401" height="724" alt="Layout V1"
src="https://github.com/user-attachments/assets/4969bcaa-f244-4504-8947-778a02c24c47"
/>
# Introduction
In this PR we start returning a workspace migration post sync so it can
committed and provided within the tarball
## Universal aggregators utils
Created two utils
### deleteUniversalFlatEntityForeignKeyAggregators
Used when building a universal create action, a newly created actions
should not contain any aggregated foreign key so they won't be codegen
in the workspace migration but also they are overriden at uninversal to
flat transpilation anw
### resetUniversalFlatEntityForeignKeyAggregators
Used before validating a new flat entity creation, some validator will
consume the fk aggregator in order to validate integrity, but of
optimstically provided it can result to errors. To avoid caller
responsability we override them here
## create-field-action refactor
Refactored the universal and flat field create action to be following
the base actions in order to ease typing
Also it was tailored to handle unlimited amount of flat field metadata
in the same actions whereas in the reality we were always only sending
at max 2 ( for relation fields )
Note: relation field has to be provided at the same as if not optimistic
would fail to retrieve circular universal identifiers
## ObjectManifest
Now always expect a `labelIdentifierFieldMetadataUniversalIdentifier`
## Integration test
Created an integration test that creates an app, sync a first manifest
and a second implying update workspace migration action generation
## Fix resolver schema leaking between `/metadata` and `/graphql`
endpoints
### Summary
- Patch `@nestjs/graphql` to support a `resolverSchemaScope` option that
filters resolvers at both schema generation and runtime, preventing
cross-endpoint leaking
- Introduce `@CoreResolver()` and `@MetadataResolver()` decorators to
explicitly scope each resolver to its endpoint
- Move most resolvers (auth, billing, workspace, user, etc.) to the
metadata schema where the frontend expects them; only workflow and
timeline calendar/messaging resolvers remain on `/graphql`
- Fix frontend `SSEQuerySubscribeEffect` to use the default (metadata)
Apollo client instead of the core client
### Problem
NestJS GraphQL's module-based resolver discovery traverses transitive
imports, causing resolvers from `/metadata` modules to leak into the
`/graphql` schema and vice versa. This made the schemas unpredictable
and tightly coupled to module import order.
### Approach
- Added `resolverSchemaScope` to `GqlModuleOptions` via a patch on
`@nestjs/graphql`, filtering in both `filterResolvers()` (runtime
binding) and `getAllCtors()` (schema generation)
- Each resolver is explicitly decorated with `@CoreResolver()` or
`@MetadataResolver()`
- Organized decorator, constant, and type files under `graphql-config/`
following project conventions
Core GQL Schema: (see: no more fields!)
<img width="827" height="894" alt="image"
src="https://github.com/user-attachments/assets/668f3f0f-485e-43f0-92be-4345aeccacb6"
/>
Metadata GQL Schema (see no more getTimelineCalendarEventsFromCompany)
<img width="827" height="894" alt="image"
src="https://github.com/user-attachments/assets/443913db-e5fe-4161-b0e7-4a971cc80a71"
/>
# What this PR does
Overall naming `universal` versus `flat` is not always the most updated
and so on
Will make a big cleaning tour after I've finished the whole migration
Migrating all `view` and ( filter fields etc ) `field` `object` `index`
to the universal pattern on all `services`, `builder` and `runner`
levels
## Universal and flat optimistic tooling
`addUniversalFlatEntityToUniversalFlatEntityAndRelatedEntityMapsThroughMutationOrThrow`
and its delete counterpart maintain the consistency of
`UniversalFlatEntityMaps` when an entity is created or removed. Beyond
inserting/removing the entity from its own maps, they walk through
`ALL_UNIVERSAL_METADATA_RELATIONS` to update the **aggregator arrays**
on related parent entities — the add appends the new entity's
`universalIdentifier` to the parent's aggregator (e.g. a new viewField's
identifier gets appended to its parent view's
`viewFieldUniversalIdentifiers`), and the delete filters it out. This
keeps the maps in sync so that diff computations and relation lookups
remain accurate throughout the migration building process.
## ALL_UNIVERSAL_METADATA_RELATIONS
`ALL_UNIVERSAL_METADATA_RELATIONS` is the universal counterpart of
`ALL_METADATA_RELATIONS`. It maps each metadata entity to its
many-to-one and one-to-many relations using universal foreign keys
(`*UniversalIdentifier`) instead of database IDs (`*Id`). This allows
migration actions to reference related entities in a workspace-agnostic
way. Relations that are workspace-specific (e.g. `workspace`,
`dataSource`, `userWorkspace`) are set to `null` and skipped during
resolution.
## `workspaceMigrationCreateIdEnrichment`
Reserved to API metadata ( will be able to validate at app installation
lvl )
- Workspace migration `create` actions now carry an optional `id` (and
`fieldIdByUniversalIdentifier` for object/field actions) so that
caller-provided IDs flow through the entire build-validate-run pipeline.
- New `enrichCreateWorkspaceMigrationActionsWithIds` utility resolves
`universalIdentifier → id` mappings after the builder runs and injects
them into the migration actions before the runner persists entities.
- Runner action handlers use the provided IDs instead of generating new
UUIDs, enabling deterministic entity creation for synchronization
workflows.
## `resolveUniversalUpdateRelationIdentifiersToIds`
`resolveUniversalUpdateRelationIdentifiersToIds` converts universal
identifiers (workspace-agnostic, stable keys) in a migration update
payload into concrete database UUIDs, so the update can be applied to a
specific workspace.
It iterates over the many-to-one relations defined in
`ALL_UNIVERSAL_METADATA_RELATIONS` for the given entity type, replaces
each `*UniversalIdentifier` property with its corresponding `*Id` by
looking up the target entity in `allFlatEntityMaps`, and throws if a
non-null identifier can't be resolved.
Used by all `update` action handlers in
`transpileUniversalActionToFlatAction`, avoiding duplicated resolution
logic across handlers.
## What this PR does not
- Migrating twenty-standard declaration to universal
- Migrating all the inputs transpilers to universal
- Migrating all metadata to be fully universal ( we still need to
de-scope the type of all of them and refactor their validator very close
)
---------
Co-authored-by: Charles Bochet <charles@twenty.com>
# Introduction
Requiring the spreaded `__universal` record that aggregates all the
universal identifier ( relations fk and aggregators ) of an entity to
its root
It's blockin for https://github.com/twentyhq/twenty/pull/17687 to be
finalized because if we don't we would have to migrated all related
entities at once in order for them to always have the universal
properties
## `resolveEntityRelationUniversalIdentifiers`
Introduced `resolveEntityRelationUniversalIdentifiers` a centralized
utility that resolves foreign key IDs to universal identifiers using
ALL_METADATA_RELATIONS metadata. It provides strict typing for both
input (foreign keys) and output (universal identifiers), with
nullability dynamically inferred from entity relation types.
Strictly and dynamically typed for both output and input
To do so added a new type and const/runtime grain to
ALL_METADATA_RELATIONS `isNullable`to many-to-one entries, derived from
the entity relation property types. And fixed incorrectly typed typeorm
entities
### Usage
```ts
const {
availabilityObjectMetadataUniversalIdentifier,
frontComponentUniversalIdentifier,
} = resolveEntityRelationUniversalIdentifiers({
metadataName: 'commandMenuItem',
foreignKeyValues: {
availabilityObjectMetadataId:
createCommandMenuItemInput.availabilityObjectMetadataId,
frontComponentId: createCommandMenuItemInput.frontComponentId,
},
flatEntityMaps: { flatObjectMetadataMaps, flatFrontComponentMaps },
});
```
# Introduction
In preparation of the workspace agnostic builder, we're migrating
`FlatEntityMaps` to be universal identifier oriented and based
As in the builder context there're won't be any ids at all
Please also note that the FlatEntity is a UniversalFlatEntity superset
From
```ts
import { type SyncableFlatEntity } from 'src/engine/metadata-modules/flat-entity/types/flat-entity-from.type';
export type FlatEntityMaps<T extends SyncableFlatEntity> = {
byId: Partial<Record<string, T>>;
idByUniversalIdentifier: Partial<Record<string, string>>;
universalIdentifiersByApplicationId: Partial<Record<string, string[]>>;
};
```
To
```ts
export type FlatEntityMaps<
T extends SyncableFlatEntity | UniversalSyncableFlatEntity,
> = {
byUniversalIdentifier: Partial<Record<string, T>>;
universalIdentifierById: Partial<Record<string, string>>;
universalIdentifiersByApplicationId: Partial<Record<string, string[]>>; // this might make more sense to be migrated to universalIdentifiersByApplicationUniversalIdentifier but it's the main topic of this PR
};
```
## Low level maps tools
Had to refactor find | create | delete | replace | find-many | get-sub
tools ( through mutations and or throw equivalent )
# Introduction
Important note: This PR officially deprecates the `standardId`, about to
drop col and entity property after this has been merged
Important note2: Haven't updated the optimistic tool to also update the
universal identifier aggregators only the ids one, they should not be
consumed in the runner context -> need to improve typing or either the
optimistic tooling
In this PR we're introducing all the devxp allowing future metadata
incremental universal migration -> this has an impact on all existing
metadata actions handler ( explaining its size )
This PR also introduce workspace agnostic create update actions runner
for both field and object metadata in order to battle test the described
above devxp
Noting that these two metadata are the most complex to handle
Notes:
- A workspace migration is now highly bind to a
`applicationUniversalIdentifier`. Though we don't strictly validate
application scope for the moment
## Next
Migrate both object and field builder to universal comparison
## Universal Actions vs Flat Actions Architecture
### Concept
The migration system uses a two-phase action model:
1. **Universal Actions** - Actions defined using `universalIdentifier`
(stable, portable identifiers like `standardId` + `applicationId`)
2. **Flat Actions** - Actions defined using database `entityId` (UUIDs
specific to a workspace)
### Why This Separation?
- **Universal actions are portable**: They can be serialized, stored,
and replayed across different workspaces
- **Flat actions are executable**: They contain the actual database IDs
needed to perform operations
- **Decoupling**: The builder produces universal actions; the runner
transpiles them to flat actions at execution time
### Transpiler Pattern
Each action handler must implement
`transpileUniversalActionToFlatAction()`:
```typescript
@Injectable()
export class CreateFieldActionHandlerService extends WorkspaceMigrationRunnerActionHandler(
'create',
'fieldMetadata',
) {
override async transpileUniversalActionToFlatAction(
context: WorkspaceMigrationActionRunnerArgs<UniversalCreateFieldAction>,
): Promise<FlatCreateFieldAction> {
// Resolve universal identifiers to database IDs
const flatObjectMetadata = findFlatEntityByUniversalIdentifierOrThrow({
flatEntityMaps: allFlatEntityMaps.flatObjectMetadataMaps,
universalIdentifier: action.objectMetadataUniversalIdentifier,
});
return {
type: action.type,
metadataName: action.metadataName,
objectMetadataId: flatObjectMetadata.id, // Resolved ID
flatFieldMetadatas: /* ... transpiled entities ... */,
};
}
}
```
### Action Handler Base Class
`BaseWorkspaceMigrationRunnerActionHandlerService<TActionType,
TMetadataName>` provides:
- **`transpileUniversalActionToFlatAction()`** - Abstract method each
handler must implement
- **`transpileUniversalDeleteActionToFlatDeleteAction()`** - Shared
helper for delete actions
## FlatEntityMaps custom properties
Introduced a `TWithCustomMapsProperties` generic parameter to control
whether custom indexing structures are included:
- **`false` (default)**: Returns `FlatEntityMaps<MetadataFlatEntity<T>>`
- used in builder/runner contexts
- **`true`**: Returns the full maps type with custom properties (e.g.,
`byUserWorkspaceIdAndFolderId`) - used in cache contexts
## Create Field Actions Refactor
Refactored create-field actions to support relation field pairs
bundling.
**Problem:** Relation fields (e.g., `Attachment.targetTask` ↔
`Task.attachments`) couldn't resolve each other's IDs during
transpilation because they were in separate actions with independent
`fieldIdByUniversalIdentifier` maps.
**Solution:**
- Removed `objectMetadataUniversalIdentifier` from
`UniversalCreateFieldAction` and `objectMetadataId` from
`FlatCreateFieldAction` - each field now carries its own
- Runner groups fields by object internally and processes each table
separately
- Split aggregator into two focused utilities:
- `aggregateNonRelationFieldsIntoObjectActions` - merges non-relation
fields into object actions
- `aggregateRelationFieldPairs` - bundles relation pairs with shared
`fieldIdByUniversalIdentifier`
https://github.com/user-attachments/assets/c4e63edb-6e98-44bc-841f-ee110ae712d4
How it works
- `manifest.json` files are now committed when apps are published to our
repo
- to display available apps, from the server, we read into our github
repo, using a pod-scoped cache
- feature flagged
- app installation will be behind permission gate MARKETPLACE_APPS
Limitations and what is yet to develop
- content and settings tabs
- installed apps tab
- app installation
- test and potentially fix reading from .manifest.json once [Reshape
manifest
structure](https://github.com/twentyhq/core-team-issues/issues/2183) is
done. additional work is expected on assets notably. (couldnt properly
do it here as manifest.json will only be committed after this pr)
- we only read in community/ folder for now - we may want tochange that
- the cache is rather artisanal for now and scoped by pod - we may want
to change that
## Context
Introduces a middleware that automatically sets the workspace auth
context in AsyncLocalStorage for HTTP requests, making it available
throughout the request lifecycle without explicit parameter passing.
The motivation behind this change is to reduce boilerplate and simplify
the developer experience when working with workspace data in HTTP
request handlers.
The Problem (Before)
Every HTTP request handler that needed to access workspace data had to:
- Extract auth-related info from decorators (@AuthWorkspace(),
@AuthUserWorkspaceId(), etc.) in controller/resolver and pass down to
services
- Build or pass the authContext explicitly (sometimes with type
assertion which was flaky)
Then call executeInWorkspaceContext(authContext, async () => { ... })
## Changes
- Add WorkspaceAuthContextMiddleware that extracts auth context from the
request and stores it in AsyncLocalStorage
- Register middleware for GraphQL, metadata, and REST routes (runs after
hydration middlewares)
- Simplify executeInWorkspaceContext signature: fn is now the first
parameter, authContext is optional second
- If authContext is not provided, it's automatically retrieved from the
storage (set by middleware)
- Update all callers (~120 files) to use the new parameter order
- Fixes a bug in search where system auth context was used, bypassing
RLS feature.
# Introduction
In this PR we catch all the runner errors coming from a single workspace
migration action execution.
**1. `WorkspaceMigrationActionExecutionException`** (low-level,
action-specific)
- Thrown from action handlers, utils, and helper functions
- Contains specific error codes: `FIELD_METADATA_NOT_FOUND`,
`OBJECT_METADATA_NOT_FOUND`, `ENUM_OPERATION_FAILED`, `NOT_SUPPORTED`,
etc.
- Simple structure: `message`, `code`, `userFriendlyMessage`
- No action context - just describes what went wrong
**2. `WorkspaceMigrationRunnerException`** (high-level, runner-scoped)
- Only two codes: `INTERNAL_SERVER_ERROR` and `EXECUTION_FAILED`
- `EXECUTION_FAILED` **requires** `action` + `errors` (contains the
action context)
- `INTERNAL_SERVER_ERROR` **requires** `message` (no action context)
## Refactor
- Removed the `relatedFlatEntityMapsKeys` from the `WorkspaceMigration`
type as they're directly inferred from passed actions
- Swallowing actions rollbacks errors in order to iterate over all of
them
## Testing
Created a very straigthforward install application from workspace
migration endpoint in order to start testing the introduced
`WorkspaceMigrationActionExecutionException`
Introduced a feature flag that stop the access to the endpoint if not
enabled
Whole taken direction are totally subjective and highly prone to
mutations ( endpoint location, naming and input schema see
`ts-expect-error` comment )
cc @martmull
## Response error
```ts
{
"eventId": "evt_a1b2c3d4-5678-90ab-cdef-1234567890ab",
"extensions": {
"action": {
"metadataName": "fieldMetadata",
"type": "delete",
"universalIdentifier": "20202020-6110-4547-9fd0-2525257a2c3f"
},
"code": "APPLICATION_INSTALLATION_FAILED",
"errors": {
"metadata": {
"code": "ENTITY_NOT_FOUND",
"message": "Could not find flat entity with universal identifier 20202020-6110-4547-9fd0-2525257a2c3f"
},
"workspaceSchema": {
"code": "ENTITY_NOT_FOUND",
"message": "Could not find flat entity in maps"
}
},
"exceptionEventId": "exc_f9e8d7c6-5432-10ba-fedc-ba0987654321",
"userFriendlyMessage": "Migration execution failed."
},
"message": "Migration action 'delete' for 'fieldMetadata' failed",
"name": "GraphQLError"
}
```
This PR introduces a few changes:
- Add three actions: see deleted dashboards, destroy dashboard and
restore dashboard
- Remove the soft delete and restore on all the page layout entities
- Cascade the destruction of a dashboard to a page layout
Video QA:
https://github.com/user-attachments/assets/ab993b11-dd9c-4e88-880c-92691a521cc2
# Introduction
Followup of
https://github.com/twentyhq/twenty/pull/17001#pullrequestreview-3638508738
close https://github.com/twentyhq/core-team-issues/issues/1910
We've completely decom the `sync-metadata` in production. We're now then
removing its implementation in favor of the v2.
## TODO:
- [x] Remove sync-metadata implem and commands
- [x] Remove workspace decorators
- [x] Type each deprecated field to deprecated on their workspaceEntity
- [x] Remove the `workspace-sync-metadata` folder entirely
- [x] remove workspace migration
- [x] workspace migration removal migration
- [x] remove the `v2` references from workspace manager file names
- [x] remove the `v2` references from workspace manager modules
- [ ] Double check impact on translation file path updates
## Note
- Removed the gate logic
- Remains some service v2 naming, serverless needs to be migrated on v2
fully
- Removed workspaceMigration service app health consumption, making it
always returning up ( no more down ) cc @FelixMalfait ( quite obsolete
health check now, will require complete refactor once we introduce inter
app dependency etc )
This PR migrates the workflow CRUD services to use the Common API
(CommonQueryRunners) instead of directly accessing TwentyORM.
## Changes
- Created CommonApiContextBuilderService to build context for Common API
- Migrated CreateRecordService to use CommonCreateOneQueryRunnerService
- Migrated UpdateRecordService to use CommonUpdateOneQueryRunnerService
- Migrated DeleteRecordService to use CommonDeleteOneQueryRunnerService
- Migrated FindRecordsService to use CommonFindManyQueryRunnerService
- Migrated UpsertRecordService to use Common API with upsert flag
- Removed unused get-selected-columns-from-restricted-fields.util.ts
- Updated module dependencies
## Benefits
- Consistent permission checking via Common API
- Query hooks (before/after execution)
- Automatic input transformation
- Same behavior as REST/GraphQL APIs
- Reduced code duplication
# Introduction
Added a `WorkspaceRelated` and `AllNonWorkspaceRelatedEntity` to
simplify the `FlatEntityFrom` that now do not expect a string literal to
omit and itself builds the related many to one entities foreign key
aggregators
We now have the type grain over relation to syncable or just workspace
related entities
Added a migrations that sets the fk on missing entities
## Next
In upcoming PR we will be able to introduce such below type
```ts
import { type CastRecordTypeOrmDatePropertiesToString } from 'src/engine/metadata-modules/flat-entity/types/cast-record-typeorm-date-properties-to-string.type';
import { type ExtractEntityManyToOneEntityRelationProperties } from 'src/engine/metadata-modules/flat-entity/types/extract-entity-many-to-one-entity-relation-properties.type';
import { type ExtractEntityOneToManyEntityRelationProperties } from 'src/engine/metadata-modules/flat-entity/types/extract-entity-one-to-many-entity-relation-properties.type';
import { type ExtractEntityRelatedEntityProperties } from 'src/engine/metadata-modules/flat-entity/types/extract-entity-related-entity-properties.type';
import { type RemoveSuffix } from 'src/engine/workspace-manager/workspace-migration-v2/workspace-migration-builder-v2/types/remove-suffix.type';
import { type SyncableEntity } from 'src/engine/workspace-manager/workspace-sync/types/syncable-entity.interface';
export type UniversalFlatEntityFrom<TEntity extends SyncableEntity> = Omit<
TEntity,
| `${ExtractEntityManyToOneEntityRelationProperties<TEntity> & string}Id`
| ExtractEntityRelatedEntityProperties<TEntity>
| 'application'
| 'workspaceId'
| 'applicationId'
| keyof CastRecordTypeOrmDatePropertiesToString<TEntity>
> &
CastRecordTypeOrmDatePropertiesToString<TEntity> & {
[P in ExtractEntityManyToOneEntityRelationProperties<TEntity> &
string as `${RemoveSuffix<P, 's'>}UniversalIdentifier`]: string;
} & {
[P in ExtractEntityOneToManyEntityRelationProperties<
TEntity,
SyncableEntity
> &
string as `${RemoveSuffix<P, 's'>}UniversalIdentifiers`]: string[];
};
```
Two challenges with error messages
- always provide a useful/meaningful error message for the end user
instead of the generic one. eg: show "Wrong password" and not "An error
occured"
- avoid technical details unless error regards a technical feature. eg:
show "An error occured" and not "Invalid post-hook payload."; but do
show "Invalid issuer URL." as it occurs while configuring SSO
What this PR does
- Make userFriendlyMessage mandatory for widely used
GraphqlQueryRunnerException and CommonQueryRunnerException, so that
developers are forced to ask themselves what the error message should
be, and as it contains very wide error codes (eg: "Bad request") which
should not be mapped to just one default message
- Keep userFriendlyMessage optional for service-specific exceptions (eg:
workflowStepExecutorException), but convert the error code to
userFriendlyMessage mapper to a switch case function with a typecheck
ensuring that all codes are mapped to a message. These default messages
are still overridable where they are thrown.
# Introduction
In this pull-request we're refactoring the page layout widget
configuration entity to be containing its discriminated key simplifying
underlying code and maintainability
- Made the configuration and title non nullable
- Introduced a generic predicate for the `widgetConfigurationType`
- Upgraded command to remove `graphType` and insert new
`configurationType` to existing entries
- Migrated frontend to new type system
---------
Co-authored-by: bosiraphael <raphael.bosi@gmail.com>
Co-authored-by: Raphaël Bosi <71827178+bosiraphael@users.noreply.github.com>
# Introduction
Creating dedicated folders and module for both `page-layout-tab` and
`page-layout-widget`
The addition diff with deletion is due to the module being added
Félix Malfait
