Centralize system field side effects + search field metadata (#22594)

## Introduction

Closes twentyhq/core-team-issues#2635 and twentyhq/core-team-issues#2642
and twentyhq/core-team-issues#2589

Object system fields (`searchVector` + its GIN index +
`searchFieldMetadata`, the reserved system fields, default relations)
were provisioned through several scattered, path-specific code paths. As
a result the **app-manifest sync path** authored objects with an
empty/`NULL` `searchVector` and **zero `searchFieldMetadata`**, so
app-owned objects shipped a broken generated search column (see #22657).
The generation logic also lived partly in imperative services rather
than in the metadata side-effect engine, and relied on non-deterministic
(`v4`) universal identifiers that `twenty apply` could not converge,
destroying manually backfilled rows.

This PR centralizes every object-creation system side effect into the
**metadata side-effect engine**, extends the engine to keep search
metadata consistent on field delete and object relabel, makes the
standard app's search identifiers deterministic, and ships upgrade
commands to reconcile existing workspaces.

## What changed

### Side effects moved into the metadata side-effect engine

New dedicated, self-contained handlers — so every write path (API and
app manifest) gets identical results, and side effects never trigger
other side effects.

**Object create / delete** (`handlers/object-metadata`)

* **`objectSystemFieldsOnCreate`** — generates the 7 reserved system
fields (`id`, `createdAt`, `updatedAt`, `deletedAt`, `createdBy`,
`updatedBy`, `position`).
* **`objectSearchVectorOnCreate`** — provisions the full-text search
surface as one unit: the `searchVector` `TS_VECTOR` field, its backing
GIN index, and the `searchFieldMetadata` row (for searchable objects
whose label identifier is a searchable field) that keeps `searchVector`
populated instead of `NULL`.
* **`objectSystemSideEffectsOnDelete`** — tears the above down on object
deletion.

**Search-metadata consistency on relabel / field delete** (new — these
are what close the manifest-path gaps)

* **`objectSearchVectorOnUpdate`** (`handlers/object-metadata`) — when a
searchable object is relabeled onto a new searchable field, provisions
the `searchFieldMetadata` row that indexes it. Relabeling is
**additive**: existing rows (e.g. the provisioned `name` row) are
preserved, so the previous label identifier stays searchable. Mirrors
the API update path so a manifest re-sync that changes the label
identifier reaches search parity. No-ops for junction objects (`id`
label identifier) and non-searchable field types.
* **`fieldSearchFieldMetadataOnDelete`** (`handlers/field-metadata`) —
when a field is deleted, cascade-deletes every `searchFieldMetadata` row
that indexes it. `searchFieldMetadata` is excluded from manifest
deletion inference, so this explicit cascade is what covers **both the
API and manifest paths** (the object-scoped DB cascade only fires on
object deletion). Uses the `searchFieldMetadataUniversalIdentifiers`
aggregator on the flat field for an O(k) lookup instead of scanning all
rows.

The **default `name` field and default relations are now caller-provided
default fields** (SDK autocomplete on the manifest path, input
transpiler on the API path) rather than system side effects — removing
duplicate name generation, the imperative
`build-default-*-for-custom-object` utilities, and the ad-hoc
system-field integrity validator.

### Deterministic identifiers for the standard app

The twenty-standard search GIN index and `searchFieldMetadata` now
derive deterministic universal identifiers
(`getIndexUniversalIdentifier` / `getSearchFieldUniversalIdentifier`)
instead of `v4`, so `twenty apply` converges instead of recreating.

### Upgrade commands (`2-20`) to reconcile existing workspaces

**Instance commands** (run once per instance; ordered fast → slow →
workspace):

1. **`AddIsSystemSideEffectToSearchFieldMetadata`** (fast) — adds the
`isSystemSideEffect` column to `core.searchFieldMetadata`. Defaults to
`true`, which also correctly backfills every existing row since
`searchFieldMetadata` is always system-derived (never user-authored).
2. **`BackfillNameFieldIsSystemSideEffect`** (slow) — re-flags existing
`name` fields from `isSystemSideEffect: true` → `false`, since the
default `name` field is now a caller-provided default like any other
user-owned field (it was provisioned as `true` in 2.15 → 2.19). This is
a pure data backfill, so the bulk `UPDATE` lives in `runDataMigration()`
rather than `up()` — keeping it out of the fast schema transaction
avoids holding an `ACCESS EXCLUSIVE` lock that could stall reads during
the deploy. Slow instance commands still run before every workspace
command of the version, so the fresh value is in place before the
search-reconcile workspace commands recompute the `fieldMetadata`
flat-entity cache. Scoping by name alone is safe (no engine-owned field
is named `name`); `down()` is best-effort (pre-2.15 `false` rows are
indistinguishable from flipped ones).

**Workspace commands** (idempotent, dry-run supported):

1. **`reconcile-search-vector-gin-index-universal-identifier`** —
re-owns every searchVector GIN index UID to its deterministic value (all
applications), then backfills the missing GIN index for installed-app
objects.
2. **`reconcile-search-field-metadata`** — re-owns every
`searchFieldMetadata` UID (all applications), then backfills the missing
rows for installed-app searchable objects.
3. **`rebuild-installed-app-search-vectors`** — rebuilds the
`searchVector` column of every installed-app `TS_VECTOR` field, once the
index and rows exist.

Design notes:

* **Re-own is global** (twenty-standard, workspace-custom, installed) —
a UID convergence keyed on each row's own application.
* **Backfill is installed-app only** — standard/custom objects already
have these rows via the manifest funnel.
* Re-own runs **before** backfill and is transaction-guarded; a failure
aborts that workspace to avoid a unique-identifier collision.

## Tests

* Integration: app manifest sync now asserts system fields + searchable
objects (searchVector, GIN index, searchFieldMetadata) are created; a
new relabel suite drives three manifest syncs and asserts records stay
searchable through the old + new label identifiers and lose
searchability when a field is removed; removed the obsolete
system-fields-integrity suite/snapshots.
* Unit: per-handler side-effect specs (including the new
`objectSearchVectorOnUpdate` and `fieldSearchFieldMetadataOnDelete`
handlers), and per-util specs for the re-own / backfill operation
builders and the GIN-index classifier.

## Upgrade / migration notes

* Existing workspaces converge on the next upgrade run via the `2-20`
instance + workspace commands (idempotent, dry-run supported).
* Backfill and rebuild go through the workspace-migration runner
(automatic cache invalidation); the re-own step invalidates only the
affected flat-entity maps directly.
* The cross-version upgrade CI now flushes the cache before running the
upgrade, so the new version recomputes every flat-entity map from the
database instead of reading blobs the old version serialized in an older
shape.

## Follow-up

* `object-metadata.service.ts` still carries a `TODO: remove once
default view fields move to the metadata side effect engine` — default
view fields are the next candidate to move into the engine.
* A single manifest sync cannot yet both create a field and relabel the
object onto it, because `objectMetadata.update` is ordered before
`fieldMetadata.create` in the migration runner. Tracked in
twentyhq/core-team-issues#2655; to be fixed in a follow-up.

packages/twenty-server/src/engine/core-modules/application/application-manifest/converters/from-field-manifest-to-universal-flat-field-metadata.util.ts

@@ -9,8 +9,8 @@ import {
   ApplicationExceptionCode,
 } from 'src/engine/core-modules/application/application.exception';
 import { type CompositeFieldMetadataType } from 'src/engine/metadata-modules/field-metadata/types/composite-field-metadata-type.type';
-import { isCompositeFieldMetadataType } from 'src/engine/metadata-modules/field-metadata/utils/is-composite-field-metadata-type.util';
 import { generateDefaultValue } from 'src/engine/metadata-modules/field-metadata/utils/generate-default-value';
+import { isCompositeFieldMetadataType } from 'src/engine/metadata-modules/field-metadata/utils/is-composite-field-metadata-type.util';
 import { nullifyEmptyCompositeDefaultValue } from 'src/engine/metadata-modules/flat-field-metadata/utils/nullify-empty-composite-default-value.util';
 import { PARTIAL_SYSTEM_FLAT_FIELD_METADATAS } from 'src/engine/metadata-modules/object-metadata/constants/partial-system-flat-field-metadatas.constant';
 import { isMorphOrRelationFieldMetadataType } from 'src/engine/utils/is-morph-or-relation-field-metadata-type.util';

packages/twenty-server/src/engine/core-modules/application/application-manifest/utils/build-all-flat-entity-operation-record-by-metadata-name-from-from-to.util.ts

@@ -11,7 +11,7 @@ import {
 } from 'src/engine/metadata-modules/flat-entity/types/all-flat-entity-operation-record-by-metadata-name.type';
 import { type MetadataUniversalFlatEntity } from 'src/engine/metadata-modules/flat-entity/types/metadata-universal-flat-entity.type';
 import { getMetadataFlatEntityMapsKey } from 'src/engine/metadata-modules/flat-entity/utils/get-metadata-flat-entity-maps-key.util';
-import { isSystemUniqueFlatIndexMetadata } from 'src/engine/metadata-modules/flat-index-metadata/utils/is-system-unique-flat-index-metadata.util';
+import { isSystemSideEffectFlatEntity } from 'src/engine/metadata-modules/flat-entity/utils/is-system-side-effect-flat-entity.util';
 import { type MetadataUniversalFlatEntityMaps } from 'src/engine/workspace-manager/workspace-migration/universal-flat-entity/types/metadata-universal-flat-entity-maps.type';
 import { compareTwoFlatEntity } from 'src/engine/workspace-manager/workspace-migration/universal-flat-entity/utils/compare-two-universal-flat-entity.util';
 import { shouldInferDeletionFromMissingEntities } from 'src/engine/workspace-manager/workspace-migration/utils/should-infer-deletion-from-missing-entities.util';
@@ -60,18 +60,12 @@ const buildFlatEntityOperationRecordForMetadata = <T extends AllMetadataName>({
               toByUniversalIdentifier[fromFlatEntity.universalIdentifier],
             ),
         )
-        .filter((fromFlatEntity) => {
-          if (metadataName !== ALL_METADATA_NAME.index) {
-            return true;
-          }
-
-          return !isSystemUniqueFlatIndexMetadata(
-            fromFlatEntity as unknown as {
-              isSystemSideEffect: boolean;
-              isUnique: boolean;
-            },
-          );
-        })
+        .filter(
+          (fromFlatEntity) =>
+            !isSystemSideEffectFlatEntity(
+              fromFlatEntity as unknown as MetadataUniversalFlatEntity<AllMetadataName>,
+            ),
+        )
     : [];
 
   const flatEntityToUpdate = Object.values(fromByUniversalIdentifier)