Commit Graph

11 Commits

Author SHA1 Message Date
Félix Malfait 5a4ebca226 refactor(server): unify the two metadata override mechanisms into one (#22417)
## Unify the two metadata override mechanisms into one

Twenty had **two** override mechanisms:

- **`standardOverrides`** — a bespoke JSONB column on
`objectMetadata`/`fieldMetadata` with typed DTOs and a per-locale
`translations` map, resolved by two i18n-aware resolvers.
- **`OverridableEntity.overrides`** — a flat, registry-driven JSONB blob
on view / view-field / view-field-group / command-menu-item /
page-layout-tab / page-layout-widget, resolved by a plain spread.

This PR collapses them into **one** concept: a single `overrides` blob,
one registry-driven overridable set, one i18n-aware read path, and one
write path (`computeMetadataOverridesBlob`, extracted in #22404).

Object/field **stay on `SyncableEntity`** (not reparented to
`OverridableEntity`) so their `isActive` default stays **FALSE** — this
sidesteps the `isActive` default conflict entirely.

### GraphQL breaking change (accepted)

The `standardOverrides` field is **removed** with no deprecation alias —
`overrides` (a `JSON` scalar) is exposed instead on `Object` and
`Field`. Product confirmed negligible external usage; the front-end has
no hand-written consumer (only generated types), which are regenerated
here.

### Commit structure (reviewable commit-by-commit)

1. **Unified resolver + parity harness** —
`resolveEffectiveEntityProperty` is a strict superset of the three
legacy resolvers; a corpus parity spec compares it against a *frozen
reference* of the old logic across every locale, `isStandardApp` branch
and override shape.
2. **Registry-driven** — object/field presentation props tagged
`isOverridable` + `translatable`; the overridable/translatable sets are
derived from the registry (a test asserts they equal the legacy
hardcoded lists).
3. **Rename + swap + delete** — `standardOverrides` → `overrides` across
entities, DTOs, flat/universal types, producers, the ~12
resolve/write/create/sync call sites, mocks and specs; the reconciler's
two compare entries collapse to one; the three legacy resolvers, both
DTOs and the hardcoded constants/types are deleted.
4. **Migration (zero-downtime, two-phase)** — split across two releases
so a rolling deploy never drops a column a previous-release pod still
`SELECT`s:
   - **2.19 fast** — add the `overrides` column (schema only).
- **2.19 slow** — backfill `overrides` from `standardOverrides` in
`runDataMigration` (kept out of the schema transaction so the bulk write
doesn't hold the ACCESS EXCLUSIVE lock; skipped on fresh installs, which
have no data to copy).
- **2.20 fast** — drop the legacy `standardOverrides` column (gated by
`TWENTY_NEXT_VERSIONS`, so it stays dormant until the instance reaches
2.20).
5. **Front/client-SDK regen** — regenerated metadata GraphQL types.
6. **Integration specs + i18n** — updated the standard object/field
update integration specs + snapshots, and the reworded validator message
catalog entry.

### Rolling-deploy safety

`standardOverrides` is retained through 2.19 and only dropped in 2.20,
mirroring the codebase's deferred-drop convention
(`isUIReadOnly`/`isCustom`). During the 2.19 rollout both columns exist,
so old and new pods coexist without "column does not exist" errors. The
backfill lives in a slow `runDataMigration` (per the
`no-data-mutation-in-fast-instance-command` rule) so it doesn't stall
reads.

### `isActive` guard

The migration never reads or writes `isActive`; the backfill asserts the
active-row count is unchanged and aborts otherwise. Verified on a real
DB: apply + revert preserves the blob **and** the nested `translations`
map, with `isActive` counts identical before/after.

### Verification (local)

- `nx typecheck twenty-server` + `nx typecheck twenty-front` — green
- `nx lint:diff-with-main twenty-server` (oxlint `--type-aware` + oxfmt)
— green
- `nx test twenty-server` — green (unit + parity + registry + migration
tests)
- `nx run twenty-server:test:integration:with-db-reset` — green
- `database:reset` applies the 2.19 phases and leaves **both** columns
present (2.20 drop stays dormant); backfill + revert round-trip verified
on a real DB
- Metadata integration suites (standard object/field update, application
sync) pass end-to-end against the two-column schema
- Metadata GraphQL types regenerated against a booted server; zero
`standardOverrides` references remain in application code (only the
migration commands + the legacy schema baseline)

---------

Co-authored-by: prastoin <paul@twenty.com>
2026-07-02 12:01:15 +02:00
Weiko 9f30915f6f fix(metadata): remove deprecated isCustom from Objects and Fields (#21799)
## Context

Follow-up to #21228, which deprecated `isCustom` on object/field
metadata but kept it exposed because the frontend still relied on it.
This removes it from the GraphQL API and the frontend entirely.

## Implementation

### Server
- Remove `isCustom` `@Field` from the `Object`, `Field`, and
`MinimalObjectMetadata` GraphQL types
- Remove the `isCustom` `@ResolveField` resolvers and the
`isCustomLoader` dataloader (+ payload/interface)
- Remove `isCustom` as an internal `@HideField()` on the Object/Field
DTOs used by the i18n standard-override gate > Use an explicit
isStandard instead (which is the correct gating)

### Frontend
- Add `getIsMetadataItemCustom` helper + `useGetIsMetadataItemCustom`
hook: an item is custom when `applicationId ===
currentWorkspace.workspaceCustomApplication.id`
- Migrate all consumers off `objectMetadataItem.isCustom` /
`fieldMetadataItem.isCustom`; `isRecordFieldReadOnly` now takes a
precomputed `isFieldCustom`
- Drop `isCustom` from the metadata fragment/mutations/minimal query, FE
types, zod schemas, and mock generators; regenerate GraphQL types

## Notes
- Breaking change on the (already-deprecated) `Object.isCustom` /
`Field.isCustom` GraphQL fields and the `isCustom` filter
- FE semantic is "belongs to the workspace custom app" (third-party-app
objects/fields are treated as non-custom)
- `isCustom` on IndexMetadata / View / Skill / Agent is a separate
column and is untouched
- Breaking changes on REST metadata API
2026-06-19 01:19:49 +02:00
Félix Malfait 69a7e614ff fix: restore isCustom gate in metadata label resolvers (#21432)
## Context

#21228 removed the stored `isCustom` column and, with it, the `isCustom`
early-return in `resolveObjectMetadataStandardOverride` /
`resolveFieldMetadataStandardOverride`, on the assumption that falling
through the `standardOverrides` checks was equivalent.

It isn't: custom object/field labels now reach the Lingui lookup. A
custom label that collides with a standard catalog string (e.g. a custom
field labeled "Status") gets translated for non-English locales against
the user's intent, and every other custom label pays a hash + catalog
miss — and, in production, an "Uncompiled message detected" warning
(#21415) — on each metadata resolution.

## Fix

Restore the gate. `isCustom` is no longer stored, so call sites that
build the resolver input from flat entities (dataloader,
minimal-metadata, view controller, command-menu-item navigation context)
derive it via `belongsToTwentyStandardApp`; GraphQL resolvers keep
passing DTOs, which already carry the derived value.

## Testing

- Unit tests for both resolvers, including a new regression test: a
custom label matching a standard catalog entry is returned verbatim,
Lingui never called.

---------

Co-authored-by: Weiko <corentin@twenty.com>
2026-06-11 16:10:53 +00:00
Weiko 9c66975520 isCustom deprecation for Objects and Fields (#21228)
## Context

`isCustom` was a legacy denormalized boolean on `ObjectMetadataEntity`
and `FieldMetadataEntity`.
Now that every metadata row carries `applicationId` (via
`SyncableEntity`), "is this custom" is fully derivable, and the stored
boolean was a redundant second source of truth that could drift.

The real meaning of `isCustom` is **"the owning application is not the
twenty-standard application"** — i.e. `!belongsToTwentyStandardApp`.
Note this is *not* "belongs to the workspace custom app" as I initially
thought: third-party-application
objects/fields are custom too. 
The standard application has a globally stable `universalIdentifier`, so
the value derives with no per-workspace lookup.

## Changed
## `isCustom` checks — before → after

`isCustom` is no longer a stored column. The table below lists every
site that branched on it and how it resolves now. The unifying rule:
`isCustom ≡
!isTwentyStandardApplicationUniversalIdentifier(applicationUniversalIdentifier)`.

### Server — behavioural checks

| Location | Purpose | Before | Now |
|---|---|---|---|
| `utils/compute-object-target-table.util.ts` | Physical table name `_`
prefix | `computeTableName(nameSingular, objectMetadata.isCustom)` |
derives from `applicationUniversalIdentifier` (single source for all
table-name callers) |
| `twenty-orm/factories/entity-schema.factory.ts` +
`…/entity-schema-metadata.type.ts` | ORM table name (hot path) |
`object.isCustom` | `object.applicationId !== standardApplicationId`
(computed in `buildEntitySchemaMetadataMaps`) |
|
`twenty-orm/repository/workspace-{delete,soft-delete,update}-query-builder.ts`
| Table name for mutations | `computeTableName(nameSingular,
objectMetadata.isCustom)` | `computeObjectTargetTable(objectMetadata)` |
| `index-metadata/utils/generate-deterministic-index-name-v2.ts` | Index
name hash (must stay bit-identical) | `flatObjectMetadata.isCustom` |
derives from `applicationUniversalIdentifier` |
| `object-metadata/object-record-count.service.ts` | Table name for
record count | `computeTableName(nameSingular, isCustom)` |
`computeObjectTargetTable(flatObjectMetadata)` |
|
`workspace-manager/dev-seeder/data/services/dev-seeder-data.service.ts`
| Match seed config by table name | `computeTableName(item.nameSingular,
item.isCustom)` | `computeObjectTargetTable(item)` |
| `commands/workspace-export/workspace-export.service.ts` +
`…/utils/generate-workspace-schema-ddl.util.ts` | Export table name (raw
entity) | `objectMetadata.isCustom` |
`!isTwentyStandard…(objectMetadata.application?.universalIdentifier)` |
|
`flat-field-metadata/services/flat-field-metadata-type-validator.service.ts`
| Block users creating reserved field types |
`args.flatEntityToValidate.isCustom` |
`!args.flatEntityToValidate.isSystem` |
| `api/common/.../common-create-many-query-runner.service.ts` | Don't
let client overwrite system `createdBy` |
`createdByFieldMetadata.isCustom === false` |
`createdByFieldMetadata.isSystem === true` |
|
`field-metadata/utils/resolve-field-metadata-standard-override.util.ts`
| Skip i18n/overrides for custom fields | `if (fieldMetadata.isCustom)
return raw` | **removed** — falls through on
`isDefined(standardOverrides)` |
|
`object-metadata/utils/resolve-object-metadata-standard-override.util.ts`
| Skip i18n/overrides for custom objects | `if (objectMetadata.isCustom)
return raw` | **removed** — same fall-through |
|
`command-menu-item/utils/build-navigation-interpolation-context.util.ts`
| Override context for nav labels | passed `isCustom` into resolver |
dropped (resolver no longer needs it) |
| `api/common/.../data-arg-processor.service.ts` | `isCustom` for
record-position table name | `flatObjectMetadata.isCustom` | derives
from `applicationUniversalIdentifier` |
| `metadata-modules/minimal-metadata/minimal-metadata.service.ts` |
Minimal DTO + override context | `flatObjectMetadata.isCustom` | derives
from `applicationUniversalIdentifier` |
|
`commands/upgrade-version-command/1-23/…backfill-record-page-layouts.command.ts`
| Filter to custom objects | `objectMetadata.isCustom` |
`!isTwentyStandard…(applicationUniversalIdentifier)` |

### Server — DTO / API population

| Location | Before | Now |
|---|---|---|
|
`flat-object-metadata/utils/from-flat-object-metadata-to-object-metadata-dto.util.ts`
| passthrough `isCustom` | derives from `applicationUniversalIdentifier`
|
|
`flat-field-metadata/utils/from-flat-field-metadata-to-field-metadata-dto.util.ts`
| passthrough `isCustom` | derives from `applicationUniversalIdentifier`
|
|
`object-metadata/utils/from-object-metadata-entity-to-object-metadata-dto.util.ts`
(REST) | `entity.isCustom` | `entity.applicationId !==
standardApplicationId` |
|
`field-metadata/utils/from-field-metadata-entity-to-field-metadata-dto.util.ts`
(REST) | `entity.isCustom` | `entity.applicationId !==
standardApplicationId` |
| `dataloaders/dataloader.service.ts` | passed
`flatFieldMetadata.isCustom` into override resolver | dropped (resolver
no longer needs it) |

> REST controllers (`object-metadata.controller.ts`,
`field-metadata.controller.ts`) resolve `standardApplicationId` once per
request from the cached `flatApplicationMaps`.

### Frontend

| Location | Purpose | Before | Now |
|---|---|---|---|
| `settings/.../SettingsObjectFieldDisabledActionDropdown.tsx` | Whether
an inactive field is deletable | `isDeletable = isCustomField` |
`isDeletable = isCustomField && !isSystemField` |

### Unchanged (out of scope)

`isCustom` on `IndexMetadata` / `View` / `Skill` / `Agent` and their
guards still read the persisted column.

Breaking change is on the isCustom filter on field and object APIs, this
is never used in the FE and unlikely used by external consumers
2026-06-09 13:57:19 +00:00
james LI 25b0e0d091 fix: correct typo occurence -> occurrence in metadata-event-emitter.ts (#21036)
## Summary

Fixes a spelling typo in
`packages/twenty-server/src/engine/subscriptions/metadata-event/metadata-event-emitter.ts`:

- Variable name `occurence` → `occurrence` (4 references on lines 101,
103, 114, 115)

## Changes

-
`packages/twenty-server/src/engine/subscriptions/metadata-event/metadata-event-emitter.ts`
— rename misspelled variable

Co-authored-by: james <li@jamesdeMacBook-Pro.local>
2026-05-29 05:48:57 +00:00
Thomas des Francs 01b4754e62 New name not appearing when renaming "Stages" in data-model settings (#20246)
## Summary
- Resolve standard field `label`, `description`, and `icon` overrides
through dedicated GraphQL field resolvers.
- Fall back to the source locale safely when the request locale is
missing, and allow direct overrides to apply for non-source locales when
translations are absent.
- Enrich metadata subscription payloads for both `before` and `after`,
reusing the same override application path for field and object
metadata.
- Update and extend tests to cover the revised override behavior.

## Testing
- Updated unit coverage for standard override resolution, including the
non-source-locale fallback path.
- Not run (not requested).

---------

Co-authored-by: Charles Bochet <charles@twenty.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
2026-05-04 20:16:00 +00:00
Weiko 47bdcb11d8 Move is active to fe (#19649)
## Context
Moving isActive filtering to the frontend for page layout tabs and
widgets, hiding inactive entities from the UI while keeping them in
state for future reactivation

Next we will implement deactivated standard tab re-activation during tab
creation (cc @Devessier)
<img width="234" height="303" alt="📋 Menu (Slots)"
src="https://github.com/user-attachments/assets/17a25ac6-55e2-4778-b7f0-e7554ed69704"
/>
2026-04-14 10:18:09 +00:00
Raphaël Bosi 19f11b1216 [COMMAND MENU ITEMS] Add dynamic label and icon to command menu navigation items (#19452) 2026-04-09 12:00:49 +00:00
Weiko 8aa208fc93 Fix overridable entities logic for SSE (#19433)
## Context
SSE metadata events for overridable entities (viewField, viewFieldGroup,
pageLayoutWidget, pageLayoutTab) were sending raw flat entity data
without resolving overrides into base properties or filtering isActive:
false entities. This caused the frontend to display stale/incorrect
values (e.g., a hidden viewField still appearing visible).

## Implementation
Add a sanitization step in
MetadataEventPublisher.enrichMetadataEventBatch that resolves overrides
and converts isActive transitions into the appropriate event types
(deactivated -> delete, reactivated -> create), matching what GraphQL
resolvers already do
2026-04-08 11:53:55 +00:00
Félix Malfait fd7387928c feat: queue messages + replace AI SDK with GraphQL SSE subscription (#19203)
## Summary

- **Queue messages while streaming**: Messages sent during active AI
streaming are queued server-side and auto-flushed when the current
stream completes. Frontend renders queued messages optimistically in a
dedicated queue UI.
- **Drop `@ai-sdk/react` + `resumable-stream`**: Replace the dual HTTP
SSE + AI SDK client architecture with a single GraphQL SSE subscription
per thread. All events (token chunks, message persistence, queue
updates, errors) flow through Redis PubSub → GraphQL subscription.
- **Server-driven architecture**: The server decides whether to queue or
stream (via `POST /:threadId/message`). The frontend mirrors this
decision for optimistic rendering but defers to the server response.
- **Reuse AI SDK accumulation logic**: `readUIMessageStream` from the
`ai` package handles chunk-to-message accumulation on the frontend,
avoiding a custom 780-line accumulator.

## Key files

**Backend:**
- `agent-chat-event-publisher.service.ts` — publishes events to Redis
PubSub
- `agent-chat-subscription.resolver.ts` — GraphQL subscription resolver
- `stream-agent-chat.job.ts` — publishes chunks via PubSub instead of
resumable-stream
- `agent-chat.controller.ts` — unified `POST /:threadId/message`
endpoint

**Frontend:**
- `useAgentChatSubscription.ts` — subscribes to `onAgentChatEvent`,
bridges to `readUIMessageStream`
- `useAgentChat.ts` — send/stop/optimistic rendering (no more AI SDK)
- `AgentChatStreamSubscriptionEffect.tsx` — replaces
`AgentChatAiSdkStreamEffect.tsx`

## Test plan

- [ ] Send message on new thread → optimistic render, streaming response
appears
- [ ] Send message while streaming → queued instantly (no flash in main
thread)
- [ ] Queued message auto-flushes after current stream completes
- [ ] Remove queued message via queue UI
- [ ] Stop streaming mid-response
- [ ] Leave chat idle for several minutes → streaming still works after
(SSE client recycling)
- [ ] Token refresh during session → requests succeed (authenticated
fetch)
- [ ] Switch threads while streaming → clean subscription handoff


Made with [Cursor](https://cursor.com)

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-02 10:10:13 +02:00
Thomas Trompette 5526d2e5d0 Separate metadata and object record publisher (#18740)
- Separate both publishers
- Renaming
- Removal of the old SSE endpoint

---------

Co-authored-by: Charles Bochet <charles@twenty.com>
2026-03-19 13:24:38 +00:00