Centralize system field side effects + search field metadata (#22594)

## Introduction

Closes twentyhq/core-team-issues#2635 and twentyhq/core-team-issues#2642
and twentyhq/core-team-issues#2589

Object system fields (`searchVector` + its GIN index +
`searchFieldMetadata`, the reserved system fields, default relations)
were provisioned through several scattered, path-specific code paths. As
a result the **app-manifest sync path** authored objects with an
empty/`NULL` `searchVector` and **zero `searchFieldMetadata`**, so
app-owned objects shipped a broken generated search column (see #22657).
The generation logic also lived partly in imperative services rather
than in the metadata side-effect engine, and relied on non-deterministic
(`v4`) universal identifiers that `twenty apply` could not converge,
destroying manually backfilled rows.

This PR centralizes every object-creation system side effect into the
**metadata side-effect engine**, extends the engine to keep search
metadata consistent on field delete and object relabel, makes the
standard app's search identifiers deterministic, and ships upgrade
commands to reconcile existing workspaces.

## What changed

### Side effects moved into the metadata side-effect engine

New dedicated, self-contained handlers — so every write path (API and
app manifest) gets identical results, and side effects never trigger
other side effects.

**Object create / delete** (`handlers/object-metadata`)

* **`objectSystemFieldsOnCreate`** — generates the 7 reserved system
fields (`id`, `createdAt`, `updatedAt`, `deletedAt`, `createdBy`,
`updatedBy`, `position`).
* **`objectSearchVectorOnCreate`** — provisions the full-text search
surface as one unit: the `searchVector` `TS_VECTOR` field, its backing
GIN index, and the `searchFieldMetadata` row (for searchable objects
whose label identifier is a searchable field) that keeps `searchVector`
populated instead of `NULL`.
* **`objectSystemSideEffectsOnDelete`** — tears the above down on object
deletion.

**Search-metadata consistency on relabel / field delete** (new — these
are what close the manifest-path gaps)

* **`objectSearchVectorOnUpdate`** (`handlers/object-metadata`) — when a
searchable object is relabeled onto a new searchable field, provisions
the `searchFieldMetadata` row that indexes it. Relabeling is
**additive**: existing rows (e.g. the provisioned `name` row) are
preserved, so the previous label identifier stays searchable. Mirrors
the API update path so a manifest re-sync that changes the label
identifier reaches search parity. No-ops for junction objects (`id`
label identifier) and non-searchable field types.
* **`fieldSearchFieldMetadataOnDelete`** (`handlers/field-metadata`) —
when a field is deleted, cascade-deletes every `searchFieldMetadata` row
that indexes it. `searchFieldMetadata` is excluded from manifest
deletion inference, so this explicit cascade is what covers **both the
API and manifest paths** (the object-scoped DB cascade only fires on
object deletion). Uses the `searchFieldMetadataUniversalIdentifiers`
aggregator on the flat field for an O(k) lookup instead of scanning all
rows.

The **default `name` field and default relations are now caller-provided
default fields** (SDK autocomplete on the manifest path, input
transpiler on the API path) rather than system side effects — removing
duplicate name generation, the imperative
`build-default-*-for-custom-object` utilities, and the ad-hoc
system-field integrity validator.

### Deterministic identifiers for the standard app

The twenty-standard search GIN index and `searchFieldMetadata` now
derive deterministic universal identifiers
(`getIndexUniversalIdentifier` / `getSearchFieldUniversalIdentifier`)
instead of `v4`, so `twenty apply` converges instead of recreating.

### Upgrade commands (`2-20`) to reconcile existing workspaces

**Instance commands** (run once per instance; ordered fast → slow →
workspace):

1. **`AddIsSystemSideEffectToSearchFieldMetadata`** (fast) — adds the
`isSystemSideEffect` column to `core.searchFieldMetadata`. Defaults to
`true`, which also correctly backfills every existing row since
`searchFieldMetadata` is always system-derived (never user-authored).
2. **`BackfillNameFieldIsSystemSideEffect`** (slow) — re-flags existing
`name` fields from `isSystemSideEffect: true` → `false`, since the
default `name` field is now a caller-provided default like any other
user-owned field (it was provisioned as `true` in 2.15 → 2.19). This is
a pure data backfill, so the bulk `UPDATE` lives in `runDataMigration()`
rather than `up()` — keeping it out of the fast schema transaction
avoids holding an `ACCESS EXCLUSIVE` lock that could stall reads during
the deploy. Slow instance commands still run before every workspace
command of the version, so the fresh value is in place before the
search-reconcile workspace commands recompute the `fieldMetadata`
flat-entity cache. Scoping by name alone is safe (no engine-owned field
is named `name`); `down()` is best-effort (pre-2.15 `false` rows are
indistinguishable from flipped ones).

**Workspace commands** (idempotent, dry-run supported):

1. **`reconcile-search-vector-gin-index-universal-identifier`** —
re-owns every searchVector GIN index UID to its deterministic value (all
applications), then backfills the missing GIN index for installed-app
objects.
2. **`reconcile-search-field-metadata`** — re-owns every
`searchFieldMetadata` UID (all applications), then backfills the missing
rows for installed-app searchable objects.
3. **`rebuild-installed-app-search-vectors`** — rebuilds the
`searchVector` column of every installed-app `TS_VECTOR` field, once the
index and rows exist.

Design notes:

* **Re-own is global** (twenty-standard, workspace-custom, installed) —
a UID convergence keyed on each row's own application.
* **Backfill is installed-app only** — standard/custom objects already
have these rows via the manifest funnel.
* Re-own runs **before** backfill and is transaction-guarded; a failure
aborts that workspace to avoid a unique-identifier collision.

## Tests

* Integration: app manifest sync now asserts system fields + searchable
objects (searchVector, GIN index, searchFieldMetadata) are created; a
new relabel suite drives three manifest syncs and asserts records stay
searchable through the old + new label identifiers and lose
searchability when a field is removed; removed the obsolete
system-fields-integrity suite/snapshots.
* Unit: per-handler side-effect specs (including the new
`objectSearchVectorOnUpdate` and `fieldSearchFieldMetadataOnDelete`
handlers), and per-util specs for the re-own / backfill operation
builders and the GIN-index classifier.

## Upgrade / migration notes

* Existing workspaces converge on the next upgrade run via the `2-20`
instance + workspace commands (idempotent, dry-run supported).
* Backfill and rebuild go through the workspace-migration runner
(automatic cache invalidation); the re-own step invalidates only the
affected flat-entity maps directly.
* The cross-version upgrade CI now flushes the cache before running the
upgrade, so the new version recomputes every flat-entity map from the
database instead of reading blobs the old version serialized in an older
shape.

## Follow-up

* `object-metadata.service.ts` still carries a `TODO: remove once
default view fields move to the metadata side effect engine` — default
view fields are the next candidate to move into the engine.
* A single manifest sync cannot yet both create a field and relabel the
object onto it, because `objectMetadata.update` is ordered before
`fieldMetadata.create` in the migration runner. Tracked in
twentyhq/core-team-issues#2655; to be fixed in a follow-up.

packages/twenty-server/src/engine/metadata-modules/object-metadata/object-metadata.exception.ts

@@ -15,8 +15,6 @@ export enum ObjectMetadataExceptionCode {
   INVALID_ORM_OUTPUT = 'INVALID_ORM_OUTPUT',
   INTERNAL_SERVER_ERROR = 'INTERNAL_SERVER_ERROR',
   NAME_CONFLICT = 'NAME_CONFLICT',
-  MISSING_SYSTEM_FIELD = 'MISSING_SYSTEM_FIELD',
-  INVALID_SYSTEM_FIELD = 'INVALID_SYSTEM_FIELD',
 }
 
 const getObjectMetadataExceptionUserFriendlyMessage = (
@@ -41,10 +39,6 @@ const getObjectMetadataExceptionUserFriendlyMessage = (
       return STANDARD_ERROR_MESSAGE;
     case ObjectMetadataExceptionCode.NAME_CONFLICT:
       return msg`A name conflict occurred.`;
-    case ObjectMetadataExceptionCode.MISSING_SYSTEM_FIELD:
-      return msg`A system field is missing.`;
-    case ObjectMetadataExceptionCode.INVALID_SYSTEM_FIELD:
-      return msg`A system field has invalid properties.`;
     default:
       assertUnreachable(code);
   }

packages/twenty-server/src/engine/metadata-modules/object-metadata/object-metadata.service.ts

@@ -42,6 +42,7 @@ import {
   ObjectMetadataException,
   ObjectMetadataExceptionCode,
 } from 'src/engine/metadata-modules/object-metadata/object-metadata.exception';
+import { buildReservedSystemFlatFieldMetadatasForCustomObject } from 'src/engine/metadata-modules/object-metadata/utils/build-reserved-system-flat-field-metadatas-for-custom-object.util';
 import { computeFlatDefaultRecordPageLayoutToCreate } from 'src/engine/metadata-modules/object-metadata/utils/compute-flat-default-record-page-layout-to-create.util';
 import { computeFlatRecordPageFieldsViewToCreate } from 'src/engine/metadata-modules/object-metadata/utils/compute-flat-record-page-fields-view-to-create.util';
 import { computeFlatViewFieldsToCreate } from 'src/engine/metadata-modules/object-metadata/utils/compute-flat-view-fields-to-create.util';
@@ -483,22 +484,43 @@ export class ObjectMetadataService extends TypeOrmQueryService<ObjectMetadataEnt
       ownerFlatApplication ?? workspaceCustomFlatApplication;
 
     const { flatObjectMetadataMaps: existingFlatObjectMetadataMaps } =
-      await this.workspaceCacheService.getOrRecompute(workspaceId, [
-        'flatObjectMetadataMaps',
-      ]);
+      await this.flatEntityMapsCacheService.getOrRecomputeManyOrAllFlatEntityMaps(
+        {
+          workspaceId,
+          flatMapsKeys: ['flatObjectMetadataMaps'],
+        },
+      );
 
     const {
       flatObjectMetadataToCreate,
-      flatIndexMetadataToCreate,
-      flatSearchFieldMetadataToCreate,
       flatFieldMetadataToCreateOnObject,
       relationTargetFlatFieldMetadataToCreate,
+      flatIndexMetadataToCreate,
     } = fromCreateObjectInputToFlatObjectMetadataAndFlatFieldMetadatasToCreate({
       createObjectInput,
       flatApplication: resolvedOwnerFlatApplication,
       flatObjectMetadataMaps: existingFlatObjectMetadataMaps,
     });
 
+    // Default view fields reference the caller-provided fields (reused as-is from
+    // the transpiler output) plus the engine-owned reserved system fields, whose
+    // deterministic identifiers we re-derive here (searchVector is never shown).
+    // TODO: remove once default view fields move to the metadata side effect engine.
+    const defaultFlatFieldMetadatasForViewFields: UniversalFlatFieldMetadata[] =
+      [
+        ...flatFieldMetadataToCreateOnObject,
+        ...Object.values(
+          buildReservedSystemFlatFieldMetadatasForCustomObject({
+            flatObjectMetadata: {
+              applicationUniversalIdentifier:
+                resolvedOwnerFlatApplication.universalIdentifier,
+              universalIdentifier:
+                flatObjectMetadataToCreate.universalIdentifier,
+            },
+          }),
+        ),
+      ];
+
     const flatDefaultViewToCreate = this.computeFlatViewToCreate({
       objectMetadata: flatObjectMetadataToCreate,
       flatApplication: resolvedOwnerFlatApplication,
@@ -506,7 +528,7 @@ export class ObjectMetadataService extends TypeOrmQueryService<ObjectMetadataEnt
 
     const flatDefaultViewFieldsToCreate = computeFlatViewFieldsToCreate({
       flatApplication: resolvedOwnerFlatApplication,
-      objectFlatFieldMetadatas: flatFieldMetadataToCreateOnObject,
+      objectFlatFieldMetadatas: defaultFlatFieldMetadatasForViewFields,
       labelIdentifierFieldMetadataUniversalIdentifier:
         flatObjectMetadataToCreate.labelIdentifierFieldMetadataUniversalIdentifier,
       viewUniversalIdentifier: flatDefaultViewToCreate.universalIdentifier,
@@ -549,7 +571,7 @@ export class ObjectMetadataService extends TypeOrmQueryService<ObjectMetadataEnt
     const flatRecordPageFieldsViewFieldsToCreate =
       computeFlatViewFieldsToCreate({
         flatApplication: resolvedOwnerFlatApplication,
-        objectFlatFieldMetadatas: flatFieldMetadataToCreateOnObject,
+        objectFlatFieldMetadatas: defaultFlatFieldMetadatasForViewFields,
         labelIdentifierFieldMetadataUniversalIdentifier:
           flatObjectMetadataToCreate.labelIdentifierFieldMetadataUniversalIdentifier,
         viewUniversalIdentifier:
@@ -574,6 +596,19 @@ export class ObjectMetadataService extends TypeOrmQueryService<ObjectMetadataEnt
               flatEntityToDelete: [],
               flatEntityToUpdate: [],
             },
+            fieldMetadata: {
+              flatEntityToCreate: [
+                ...flatFieldMetadataToCreateOnObject,
+                ...relationTargetFlatFieldMetadataToCreate,
+              ],
+              flatEntityToDelete: [],
+              flatEntityToUpdate: [],
+            },
+            index: {
+              flatEntityToCreate: flatIndexMetadataToCreate,
+              flatEntityToDelete: [],
+              flatEntityToUpdate: [],
+            },
             view: {
               flatEntityToCreate: [
                 flatDefaultViewToCreate,
@@ -590,19 +625,6 @@ export class ObjectMetadataService extends TypeOrmQueryService<ObjectMetadataEnt
               flatEntityToDelete: [],
               flatEntityToUpdate: [],
             },
-            fieldMetadata: {
-              flatEntityToCreate: [
-                ...flatFieldMetadataToCreateOnObject,
-                ...relationTargetFlatFieldMetadataToCreate,
-              ],
-              flatEntityToDelete: [],
-              flatEntityToUpdate: [],
-            },
-            index: {
-              flatEntityToCreate: flatIndexMetadataToCreate,
-              flatEntityToDelete: [],
-              flatEntityToUpdate: [],
-            },
             commandMenuItem: {
               flatEntityToCreate: [flatCommandMenuItemToCreate],
               flatEntityToDelete: [],
@@ -626,11 +648,6 @@ export class ObjectMetadataService extends TypeOrmQueryService<ObjectMetadataEnt
               flatEntityToDelete: [],
               flatEntityToUpdate: [],
             },
-            searchFieldMetadata: {
-              flatEntityToCreate: flatSearchFieldMetadataToCreate,
-              flatEntityToDelete: [],
-              flatEntityToUpdate: [],
-            },
             ...(isDefined(flatNavigationMenuItemToCreate)
               ? {
                   navigationMenuItem: {

packages/twenty-server/src/engine/metadata-modules/object-metadata/utils/build-default-flat-field-metadatas-for-custom-object.util.ts

@@ -1,186 +0,0 @@
-import { getFieldUniversalIdentifier } from 'twenty-shared/application';
-import { FieldMetadataType } from 'twenty-shared/types';
-
-import { PARTIAL_SYSTEM_FLAT_FIELD_METADATAS } from 'src/engine/metadata-modules/object-metadata/constants/partial-system-flat-field-metadatas.constant';
-import { type UniversalFlatFieldMetadata } from 'src/engine/workspace-manager/workspace-migration/universal-flat-entity/types/universal-flat-field-metadata.type';
-import { type UniversalFlatObjectMetadata } from 'src/engine/workspace-manager/workspace-migration/universal-flat-entity/types/universal-flat-object-metadata.type';
-
-type BuildDefaultFlatFieldMetadataForCustomObjectArgs = {
-  flatObjectMetadata: Pick<
-    UniversalFlatObjectMetadata,
-    'universalIdentifier' | 'applicationUniversalIdentifier'
-  >;
-  skipNameField?: boolean;
-};
-
-export type DefaultFlatFieldForCustomObjectMaps = ReturnType<
-  typeof buildDefaultFlatFieldMetadatasForCustomObject
->;
-
-const buildObjectSystemFlatFieldMetadatas = ({
-  applicationUniversalIdentifier,
-  objectMetadataUniversalIdentifier,
-  now,
-  searchVectorUniversalSettings,
-}: {
-  applicationUniversalIdentifier: string;
-  objectMetadataUniversalIdentifier: string;
-  now: string;
-  searchVectorUniversalSettings: UniversalFlatFieldMetadata<FieldMetadataType.TS_VECTOR>['universalSettings'];
-}) => {
-  const {
-    createdAt,
-    createdBy,
-    deletedAt,
-    id,
-    position,
-    searchVector,
-    updatedAt,
-    updatedBy,
-  } = PARTIAL_SYSTEM_FLAT_FIELD_METADATAS;
-
-  const computeFieldUniversalIdentifier = (name: string) =>
-    getFieldUniversalIdentifier({
-      applicationUniversalIdentifier,
-      objectUniversalIdentifier: objectMetadataUniversalIdentifier,
-      name,
-    });
-
-  return {
-    id: {
-      ...id,
-      universalIdentifier: computeFieldUniversalIdentifier(id.name),
-      applicationUniversalIdentifier,
-      objectMetadataUniversalIdentifier,
-      createdAt: now,
-      updatedAt: now,
-    },
-    createdAt: {
-      ...createdAt,
-      universalIdentifier: computeFieldUniversalIdentifier(createdAt.name),
-      applicationUniversalIdentifier,
-      objectMetadataUniversalIdentifier,
-      createdAt: now,
-      updatedAt: now,
-    },
-    createdBy: {
-      ...createdBy,
-      universalIdentifier: computeFieldUniversalIdentifier(createdBy.name),
-      applicationUniversalIdentifier,
-      objectMetadataUniversalIdentifier,
-      createdAt: now,
-      updatedAt: now,
-    },
-    deletedAt: {
-      ...deletedAt,
-      universalIdentifier: computeFieldUniversalIdentifier(deletedAt.name),
-      applicationUniversalIdentifier,
-      objectMetadataUniversalIdentifier,
-      createdAt: now,
-      updatedAt: now,
-    },
-    position: {
-      ...position,
-      universalIdentifier: computeFieldUniversalIdentifier(position.name),
-      applicationUniversalIdentifier,
-      objectMetadataUniversalIdentifier,
-      createdAt: now,
-      updatedAt: now,
-    },
-    searchVector: {
-      ...searchVector,
-      universalIdentifier: computeFieldUniversalIdentifier(searchVector.name),
-      applicationUniversalIdentifier,
-      objectMetadataUniversalIdentifier,
-      createdAt: now,
-      updatedAt: now,
-      universalSettings: searchVectorUniversalSettings,
-    },
-    updatedAt: {
-      ...updatedAt,
-      universalIdentifier: computeFieldUniversalIdentifier(updatedAt.name),
-      applicationUniversalIdentifier,
-      objectMetadataUniversalIdentifier,
-      createdAt: now,
-      updatedAt: now,
-    },
-    updatedBy: {
-      ...updatedBy,
-      universalIdentifier: computeFieldUniversalIdentifier(updatedBy.name),
-      applicationUniversalIdentifier,
-      objectMetadataUniversalIdentifier,
-      createdAt: now,
-      updatedAt: now,
-    },
-  } as const satisfies Record<string, UniversalFlatFieldMetadata>;
-};
-
-// This could be replaced totally by an import schema + its transpilation when it's ready
-export const buildDefaultFlatFieldMetadatasForCustomObject = ({
-  flatObjectMetadata: {
-    applicationUniversalIdentifier,
-    universalIdentifier: objectMetadataUniversalIdentifier,
-  },
-  skipNameField = false,
-}: BuildDefaultFlatFieldMetadataForCustomObjectArgs) => {
-  const now = new Date().toISOString();
-
-  const nameField: UniversalFlatFieldMetadata<FieldMetadataType.TEXT> | null =
-    skipNameField
-      ? null
-      : {
-          type: FieldMetadataType.TEXT,
-          isLabelSyncedWithName: false,
-          isUnique: false,
-          universalIdentifier: getFieldUniversalIdentifier({
-            applicationUniversalIdentifier,
-            objectUniversalIdentifier: objectMetadataUniversalIdentifier,
-            name: 'name',
-          }),
-          name: 'name',
-          label: 'Name',
-          icon: 'IconAbc',
-          description: 'Name',
-          isNullable: true,
-          isActive: true,
-          isSystem: false,
-          isSystemSideEffect: true,
-          isUIEditable: true,
-          defaultValue: null,
-          createdAt: now,
-          updatedAt: now,
-          options: null,
-          overrides: null,
-          morphId: null,
-          applicationUniversalIdentifier,
-          objectMetadataUniversalIdentifier,
-          relationTargetObjectMetadataUniversalIdentifier: null,
-          relationTargetFieldMetadataUniversalIdentifier: null,
-          viewFilterUniversalIdentifiers: [],
-          viewFieldUniversalIdentifiers: [],
-          kanbanAggregateOperationViewUniversalIdentifiers: [],
-          calendarViewUniversalIdentifiers: [],
-          mainGroupByFieldMetadataViewUniversalIdentifiers: [],
-          fieldPermissionUniversalIdentifiers: [],
-          universalSettings: null,
-          viewSortUniversalIdentifiers: [],
-          searchFieldMetadataUniversalIdentifiers: [],
-        };
-
-  const searchVectorUniversalSettings: UniversalFlatFieldMetadata<FieldMetadataType.TS_VECTOR>['universalSettings'] =
-    null;
-
-  return {
-    fields: {
-      ...(nameField && { nameField }),
-      ...buildObjectSystemFlatFieldMetadatas({
-        applicationUniversalIdentifier,
-        objectMetadataUniversalIdentifier,
-        now,
-        searchVectorUniversalSettings,
-      }),
-    },
-  } as const satisfies {
-    fields: Record<string, UniversalFlatFieldMetadata>;
-  };
-};

packages/twenty-server/src/engine/metadata-modules/object-metadata/utils/build-default-index-for-custom-object.util.ts

@@ -1,58 +0,0 @@
-import { v4 } from 'uuid';
-
-import { IndexType } from 'src/engine/metadata-modules/index-metadata/types/indexType.types';
-import { generateFlatIndexMetadataWithNameOrThrow } from 'src/engine/metadata-modules/index-metadata/utils/generate-flat-index.util';
-import { type DefaultFlatFieldForCustomObjectMaps } from 'src/engine/metadata-modules/object-metadata/utils/build-default-flat-field-metadatas-for-custom-object.util';
-import { type UniversalFlatFieldMetadata } from 'src/engine/workspace-manager/workspace-migration/universal-flat-entity/types/universal-flat-field-metadata.type';
-import { type UniversalFlatIndexMetadata } from 'src/engine/workspace-manager/workspace-migration/universal-flat-entity/types/universal-flat-index-metadata.type';
-import { type UniversalFlatObjectMetadata } from 'src/engine/workspace-manager/workspace-migration/universal-flat-entity/types/universal-flat-object-metadata.type';
-
-export const buildDefaultIndexesForCustomObject = ({
-  flatObjectMetadata,
-  defaultFlatFieldForCustomObjectMaps,
-  objectFlatFieldMetadatas,
-}: {
-  flatObjectMetadata: UniversalFlatObjectMetadata & { id: string };
-  objectFlatFieldMetadatas: UniversalFlatFieldMetadata[];
-  defaultFlatFieldForCustomObjectMaps: DefaultFlatFieldForCustomObjectMaps;
-}) => {
-  const tsFlatVectorIndexUniversalIdentifier = v4();
-  const createdAt = new Date();
-  const tsVectorFlatIndex = generateFlatIndexMetadataWithNameOrThrow({
-    objectFlatFieldMetadatas,
-    flatIndex: {
-      createdAt: createdAt.toISOString(),
-      universalFlatIndexFieldMetadatas: [
-        {
-          createdAt: createdAt.toISOString(),
-          fieldMetadataUniversalIdentifier:
-            defaultFlatFieldForCustomObjectMaps.fields.searchVector
-              .universalIdentifier,
-          indexMetadataUniversalIdentifier:
-            tsFlatVectorIndexUniversalIdentifier,
-          order: 0,
-          subFieldName: null,
-          updatedAt: createdAt.toISOString(),
-        },
-      ],
-
-      indexType: IndexType.GIN,
-      indexWhereClause: null,
-      isCustom: false,
-      isUnique: false,
-      isSystemSideEffect: true,
-      objectMetadataUniversalIdentifier: flatObjectMetadata.universalIdentifier,
-      universalIdentifier: tsFlatVectorIndexUniversalIdentifier,
-      updatedAt: createdAt.toISOString(),
-      applicationUniversalIdentifier:
-        flatObjectMetadata.applicationUniversalIdentifier,
-    },
-    flatObjectMetadata,
-  });
-
-  return {
-    indexes: {
-      tsVectorFlatIndex,
-    },
-  } as const satisfies { indexes: Record<string, UniversalFlatIndexMetadata> };
-};

packages/twenty-server/src/engine/metadata-modules/object-metadata/utils/build-default-search-field-metadatas-for-custom-object.util.ts

@@ -1,38 +0,0 @@
-import { isDefined } from 'twenty-shared/utils';
-
-import { buildFlatSearchFieldMetadataForField } from 'src/engine/metadata-modules/flat-search-field-metadata/utils/build-flat-search-field-metadata-for-field.util';
-import { type DefaultFlatFieldForCustomObjectMaps } from 'src/engine/metadata-modules/object-metadata/utils/build-default-flat-field-metadatas-for-custom-object.util';
-import { type UniversalFlatObjectMetadata } from 'src/engine/workspace-manager/workspace-migration/universal-flat-entity/types/universal-flat-object-metadata.type';
-import { type UniversalFlatSearchFieldMetadata } from 'src/engine/workspace-manager/workspace-migration/universal-flat-entity/types/universal-flat-search-field-metadata.type';
-
-// Mirrors the custom-object searchVector, which indexes the name field only. Junction
-// objects (skipNameField) have no name field and therefore get no row.
-export const buildDefaultSearchFieldMetadatasForCustomObject = ({
-  flatObjectMetadata,
-  defaultFlatFieldForCustomObjectMaps,
-}: {
-  flatObjectMetadata: UniversalFlatObjectMetadata & { id: string };
-  defaultFlatFieldForCustomObjectMaps: DefaultFlatFieldForCustomObjectMaps;
-}): {
-  searchFieldMetadatas: UniversalFlatSearchFieldMetadata[];
-} => {
-  const nameField = defaultFlatFieldForCustomObjectMaps.fields.nameField;
-
-  if (!isDefined(nameField)) {
-    return {
-      searchFieldMetadatas: [],
-    };
-  }
-
-  const nameSearchFieldMetadata = buildFlatSearchFieldMetadataForField({
-    flatObjectMetadata,
-    flatFieldMetadata: nameField,
-    tsVectorFlatFieldMetadata:
-      defaultFlatFieldForCustomObjectMaps.fields.searchVector,
-    position: 0,
-  });
-
-  return {
-    searchFieldMetadatas: [nameSearchFieldMetadata],
-  };
-};

packages/twenty-server/src/engine/metadata-modules/object-metadata/utils/build-name-flat-field-metadata-for-custom-object.util.ts

@@ -0,0 +1,60 @@
+import { getFieldUniversalIdentifier } from 'twenty-shared/application';
+import { FieldMetadataType } from 'twenty-shared/types';
+
+import { type UniversalFlatFieldMetadata } from 'src/engine/workspace-manager/workspace-migration/universal-flat-entity/types/universal-flat-field-metadata.type';
+import { type UniversalFlatObjectMetadata } from 'src/engine/workspace-manager/workspace-migration/universal-flat-entity/types/universal-flat-object-metadata.type';
+
+type BuildNameFlatFieldMetadataForCustomObjectArgs = {
+  flatObjectMetadata: Pick<
+    UniversalFlatObjectMetadata,
+    'universalIdentifier' | 'applicationUniversalIdentifier'
+  >;
+};
+
+export const buildNameFlatFieldMetadataForCustomObject = ({
+  flatObjectMetadata: {
+    applicationUniversalIdentifier,
+    universalIdentifier: objectMetadataUniversalIdentifier,
+  },
+}: BuildNameFlatFieldMetadataForCustomObjectArgs): UniversalFlatFieldMetadata<FieldMetadataType.TEXT> => {
+  const now = new Date().toISOString();
+
+  return {
+    type: FieldMetadataType.TEXT,
+    isLabelSyncedWithName: false,
+    isUnique: false,
+    universalIdentifier: getFieldUniversalIdentifier({
+      applicationUniversalIdentifier,
+      objectUniversalIdentifier: objectMetadataUniversalIdentifier,
+      name: 'name',
+    }),
+    name: 'name',
+    label: 'Name',
+    icon: 'IconAbc',
+    description: 'Name',
+    isNullable: true,
+    isActive: true,
+    isSystem: false,
+    isSystemSideEffect: false,
+    isUIEditable: true,
+    defaultValue: null,
+    createdAt: now,
+    updatedAt: now,
+    options: null,
+    overrides: null,
+    morphId: null,
+    applicationUniversalIdentifier,
+    objectMetadataUniversalIdentifier,
+    relationTargetObjectMetadataUniversalIdentifier: null,
+    relationTargetFieldMetadataUniversalIdentifier: null,
+    viewFilterUniversalIdentifiers: [],
+    viewFieldUniversalIdentifiers: [],
+    kanbanAggregateOperationViewUniversalIdentifiers: [],
+    calendarViewUniversalIdentifiers: [],
+    mainGroupByFieldMetadataViewUniversalIdentifiers: [],
+    fieldPermissionUniversalIdentifiers: [],
+    universalSettings: null,
+    viewSortUniversalIdentifiers: [],
+    searchFieldMetadataUniversalIdentifiers: [],
+  };
+};

packages/twenty-server/src/engine/metadata-modules/object-metadata/utils/build-reserved-system-flat-field-metadatas-for-custom-object.util.ts

@@ -0,0 +1,100 @@
+import { getFieldUniversalIdentifier } from 'twenty-shared/application';
+
+import { PARTIAL_SYSTEM_FLAT_FIELD_METADATAS } from 'src/engine/metadata-modules/object-metadata/constants/partial-system-flat-field-metadatas.constant';
+import { type UniversalFlatFieldMetadata } from 'src/engine/workspace-manager/workspace-migration/universal-flat-entity/types/universal-flat-field-metadata.type';
+import { type UniversalFlatObjectMetadata } from 'src/engine/workspace-manager/workspace-migration/universal-flat-entity/types/universal-flat-object-metadata.type';
+
+type BuildReservedSystemFlatFieldMetadatasForCustomObjectArgs = {
+  flatObjectMetadata: Pick<
+    UniversalFlatObjectMetadata,
+    'universalIdentifier' | 'applicationUniversalIdentifier'
+  >;
+};
+
+export const buildReservedSystemFlatFieldMetadatasForCustomObject = ({
+  flatObjectMetadata: {
+    applicationUniversalIdentifier,
+    universalIdentifier: objectMetadataUniversalIdentifier,
+  },
+}: BuildReservedSystemFlatFieldMetadatasForCustomObjectArgs): Record<
+  string,
+  UniversalFlatFieldMetadata
+> => {
+  const now = new Date().toISOString();
+
+  const {
+    createdAt,
+    createdBy,
+    deletedAt,
+    id,
+    position,
+    updatedAt,
+    updatedBy,
+  } = PARTIAL_SYSTEM_FLAT_FIELD_METADATAS;
+
+  const computeFieldUniversalIdentifier = (name: string) =>
+    getFieldUniversalIdentifier({
+      applicationUniversalIdentifier,
+      objectUniversalIdentifier: objectMetadataUniversalIdentifier,
+      name,
+    });
+
+  return {
+    id: {
+      ...id,
+      universalIdentifier: computeFieldUniversalIdentifier(id.name),
+      applicationUniversalIdentifier,
+      objectMetadataUniversalIdentifier,
+      createdAt: now,
+      updatedAt: now,
+    },
+    createdAt: {
+      ...createdAt,
+      universalIdentifier: computeFieldUniversalIdentifier(createdAt.name),
+      applicationUniversalIdentifier,
+      objectMetadataUniversalIdentifier,
+      createdAt: now,
+      updatedAt: now,
+    },
+    createdBy: {
+      ...createdBy,
+      universalIdentifier: computeFieldUniversalIdentifier(createdBy.name),
+      applicationUniversalIdentifier,
+      objectMetadataUniversalIdentifier,
+      createdAt: now,
+      updatedAt: now,
+    },
+    deletedAt: {
+      ...deletedAt,
+      universalIdentifier: computeFieldUniversalIdentifier(deletedAt.name),
+      applicationUniversalIdentifier,
+      objectMetadataUniversalIdentifier,
+      createdAt: now,
+      updatedAt: now,
+    },
+    position: {
+      ...position,
+      universalIdentifier: computeFieldUniversalIdentifier(position.name),
+      applicationUniversalIdentifier,
+      objectMetadataUniversalIdentifier,
+      createdAt: now,
+      updatedAt: now,
+    },
+    updatedAt: {
+      ...updatedAt,
+      universalIdentifier: computeFieldUniversalIdentifier(updatedAt.name),
+      applicationUniversalIdentifier,
+      objectMetadataUniversalIdentifier,
+      createdAt: now,
+      updatedAt: now,
+    },
+    updatedBy: {
+      ...updatedBy,
+      universalIdentifier: computeFieldUniversalIdentifier(updatedBy.name),
+      applicationUniversalIdentifier,
+      objectMetadataUniversalIdentifier,
+      createdAt: now,
+      updatedAt: now,
+    },
+  } as const satisfies Record<string, UniversalFlatFieldMetadata>;
+};

packages/twenty-server/src/engine/metadata-modules/object-metadata/utils/build-search-vector-flat-field-metadata-for-custom-object.util.ts

@@ -0,0 +1,38 @@
+import { getFieldUniversalIdentifier } from 'twenty-shared/application';
+import { type FieldMetadataType } from 'twenty-shared/types';
+
+import { PARTIAL_SYSTEM_FLAT_FIELD_METADATAS } from 'src/engine/metadata-modules/object-metadata/constants/partial-system-flat-field-metadatas.constant';
+import { type UniversalFlatFieldMetadata } from 'src/engine/workspace-manager/workspace-migration/universal-flat-entity/types/universal-flat-field-metadata.type';
+import { type UniversalFlatObjectMetadata } from 'src/engine/workspace-manager/workspace-migration/universal-flat-entity/types/universal-flat-object-metadata.type';
+
+type BuildSearchVectorFlatFieldMetadataForCustomObjectArgs = {
+  flatObjectMetadata: Pick<
+    UniversalFlatObjectMetadata,
+    'universalIdentifier' | 'applicationUniversalIdentifier'
+  >;
+};
+
+export const buildSearchVectorFlatFieldMetadataForCustomObject = ({
+  flatObjectMetadata: {
+    applicationUniversalIdentifier,
+    universalIdentifier: objectMetadataUniversalIdentifier,
+  },
+}: BuildSearchVectorFlatFieldMetadataForCustomObjectArgs): UniversalFlatFieldMetadata<FieldMetadataType.TS_VECTOR> => {
+  const now = new Date().toISOString();
+
+  const { searchVector } = PARTIAL_SYSTEM_FLAT_FIELD_METADATAS;
+
+  return {
+    ...searchVector,
+    universalIdentifier: getFieldUniversalIdentifier({
+      applicationUniversalIdentifier,
+      objectUniversalIdentifier: objectMetadataUniversalIdentifier,
+      name: searchVector.name,
+    }),
+    applicationUniversalIdentifier,
+    objectMetadataUniversalIdentifier,
+    createdAt: now,
+    updatedAt: now,
+    universalSettings: null,
+  };
+};

packages/twenty-server/src/engine/metadata-modules/object-metadata/utils/build-search-vector-gin-index-for-custom-object.util.ts

@@ -0,0 +1,42 @@
+import { IndexType } from 'src/engine/metadata-modules/index-metadata/types/indexType.types';
+import { generateDeterministicFlatIndexMetadataOrThrow } from 'src/engine/metadata-modules/index-metadata/utils/generate-deterministic-flat-index.util';
+import { type UniversalFlatFieldMetadata } from 'src/engine/workspace-manager/workspace-migration/universal-flat-entity/types/universal-flat-field-metadata.type';
+import { type UniversalFlatIndexMetadata } from 'src/engine/workspace-manager/workspace-migration/universal-flat-entity/types/universal-flat-index-metadata.type';
+import { type UniversalFlatObjectMetadata } from 'src/engine/workspace-manager/workspace-migration/universal-flat-entity/types/universal-flat-object-metadata.type';
+
+export const buildSearchVectorGinIndexForCustomObject = ({
+  flatObjectMetadata,
+  searchVectorFlatFieldMetadata,
+}: {
+  flatObjectMetadata: UniversalFlatObjectMetadata;
+  searchVectorFlatFieldMetadata: UniversalFlatFieldMetadata;
+}): UniversalFlatIndexMetadata => {
+  const createdAt = new Date().toISOString();
+
+  return generateDeterministicFlatIndexMetadataOrThrow({
+    objectFlatFieldMetadatas: [searchVectorFlatFieldMetadata],
+    flatIndex: {
+      createdAt,
+      universalFlatIndexFieldMetadatas: [
+        {
+          createdAt,
+          fieldMetadataUniversalIdentifier:
+            searchVectorFlatFieldMetadata.universalIdentifier,
+          order: 0,
+          subFieldName: null,
+          updatedAt: createdAt,
+        },
+      ],
+      indexType: IndexType.GIN,
+      indexWhereClause: null,
+      isCustom: false,
+      isUnique: false,
+      isSystemSideEffect: true,
+      objectMetadataUniversalIdentifier: flatObjectMetadata.universalIdentifier,
+      updatedAt: createdAt,
+      applicationUniversalIdentifier:
+        flatObjectMetadata.applicationUniversalIdentifier,
+    },
+    flatObjectMetadata,
+  });
+};

packages/twenty-server/src/engine/metadata-modules/object-metadata/utils/object-metadata-exception-code-to-http-status.util.ts

@@ -14,8 +14,6 @@ export const objectMetadataExceptionCodeToHttpStatus = (
     case ObjectMetadataExceptionCode.NAME_CONFLICT:
       return 403;
     case ObjectMetadataExceptionCode.INVALID_OBJECT_INPUT:
-    case ObjectMetadataExceptionCode.MISSING_SYSTEM_FIELD:
-    case ObjectMetadataExceptionCode.INVALID_SYSTEM_FIELD:
     case ObjectMetadataExceptionCode.MISSING_CUSTOM_OBJECT_DEFAULT_LABEL_IDENTIFIER_FIELD:
     case ObjectMetadataExceptionCode.APPLICATION_NOT_FOUND:
       return 400;

packages/twenty-server/src/engine/metadata-modules/object-metadata/utils/object-metadata-graphql-api-exception-handler.util.ts

@@ -41,9 +41,6 @@ export const objectMetadataGraphqlApiExceptionHandler = (error: Error) => {
       case ObjectMetadataExceptionCode.MISSING_CUSTOM_OBJECT_DEFAULT_LABEL_IDENTIFIER_FIELD:
       case ObjectMetadataExceptionCode.APPLICATION_NOT_FOUND:
         throw error;
-      case ObjectMetadataExceptionCode.MISSING_SYSTEM_FIELD:
-      case ObjectMetadataExceptionCode.INVALID_SYSTEM_FIELD:
-        throw new UserInputError(error);
       default: {
         return assertUnreachable(error.code);
       }