Wire up search field metadata (#21964)

## Part 1 - Exact scope of the current PR (#21964)

close https://github.com/twentyhq/core-team-issues/issues/2586

This PR introduces `searchFieldMetadata` as a first-class flat metadata
entity and migrates the existing search surface onto it, with **no
change to which records are searchable** (ISO with `main`).

In scope (what the PR does):
- New flat entity `searchFieldMetadata` (universalIdentifier,
applicationId, **`position`**, maps, conversions), registered in the
central flat-entity constants and the migration build orchestrator.
- `searchVector.asExpression` is **derived server-side** from
`searchFieldMetadata` rows (validated by `isSafeTsVectorExpression`);
never trusted from client input.
- **Derivation order is deterministic, driven by each row's `position`**
([compute-search-vector-as-expression-from-search-field-metadatas.util.ts](packages/twenty-server/src/engine/metadata-modules/flat-search-field-metadata/utils/compute-search-vector-as-expression-from-search-field-metadatas.util.ts)),
replacing the previous non-deterministic `(createdAt, id)` sort. That
sort collapsed to random UUIDs for standard fields (same `createdAt`),
so any rename/relabel rewrote the `STORED` generated column to a
logically-identical-but-textually-different expression and produced a
permanent per-workspace diff vs the standard definition. Ordering now
equals provisioning order; ties break on `universalIdentifier`.
- Provisioning at object creation mirrors the existing surface exactly
**and seeds `position`**:
- custom objects -> the `name` field only, at `position: 0`
([build-default-search-field-metadatas-for-custom-object.util.ts](packages/twenty-server/src/engine/metadata-modules/object-metadata/utils/build-default-search-field-metadatas-for-custom-object.util.ts))
- standard objects -> their curated `SEARCH_FIELDS_FOR_*` sets,
`position` = the curated index
- Backfill (instance + workspace commands in `2-16`) provisions rows for
existing workspaces with the same surface **and the same positions**
(standard from the curated standard maps, custom `name` = `0`), scoped
to the workspace's own custom application
([build-search-field-metadata-backfill-operations.util.ts](packages/twenty-server/src/database/commands/upgrade-version-command/2-16/utils/build-search-field-metadata-backfill-operations.util.ts)).
The `position` column is added in the same `2-16` fast instance command
as `universalIdentifier`/`applicationId`.
- Field rename of an already-indexed field recomputes `asExpression`
(positions preserved, so order is stable)
([recompute-search-vector-on-field-rename.util.ts](packages/twenty-server/src/engine/metadata-modules/flat-field-metadata/utils/recompute-search-vector-on-field-rename.util.ts)).
- Field delete drops the matching row(s) and recomputes; remaining rows
keep their relative order (no renumber)
([from-delete-field-input-to-flat-field-metadatas-to-delete.util.ts](packages/twenty-server/src/engine/metadata-modules/flat-field-metadata/utils/from-delete-field-input-to-flat-field-metadatas-to-delete.util.ts)).
- Object relabel is **additive** and ISO/regression-fix only: it indexes
the new label identifier **appended last (`position = max(existing) +
1`)** without dropping `name`
([recompute-search-vector-on-label-identifier-update.util.ts](packages/twenty-server/src/engine/metadata-modules/flat-object-metadata/utils/recompute-search-vector-on-label-identifier-update.util.ts)).
This is a deliberate, temporary bridge.

Explicitly OUT of scope (deferred):
- No API to edit `searchFieldMetadata` (no user-facing search-field
configuration, including `position` — it is internal and only written by
provisioning/backfill/recompute).
- No auto-indexing of arbitrary searchable fields. Creating a custom
TEXT/EMAILS/etc. field does NOT add it to search (the
`computeSearchFieldMetadataCreationForFields` behavior was removed in
`e6820ad`).
- No field-type-transition handling (field type is immutable - not in
`FLAT_FIELD_METADATA_EDITABLE_PROPERTIES`, so that path was dead code).
- No `position` validation (uniqueness/range) and no multi-vector /
per-field `weight` config — deferred to the configurable-search
follow-up (#1428).

Net: `searchFieldMetadata` becomes the source of truth for the *same*
surface as `main`. The only intentional divergences from `main` are
"relabel preserves `name`" (additive) and the deterministic
`position`-ordered `asExpression` (a correctness/perf fix that is
byte-identical to provisioning order, so it does not change the
searchable surface).

---------

Co-authored-by: Félix Malfait <felix@twenty.com>

packages/twenty-server/src/engine/metadata-modules/object-metadata/object-metadata.entity.ts

@@ -12,6 +12,7 @@ import {
 import { type WorkspaceEntityDuplicateCriteria } from 'src/engine/api/graphql/workspace-query-builder/types/workspace-entity-duplicate-criteria.type';
 import { FieldMetadataEntity } from 'src/engine/metadata-modules/field-metadata/field-metadata.entity';
 import { IndexMetadataEntity } from 'src/engine/metadata-modules/index-metadata/index-metadata.entity';
+import { SearchFieldMetadataEntity } from 'src/engine/metadata-modules/search-field-metadata/search-field-metadata.entity';
 import { type ObjectStandardOverridesDTO } from 'src/engine/metadata-modules/object-metadata/dtos/object-standard-overrides.dto';
 import { FieldPermissionEntity } from 'src/engine/metadata-modules/object-permission/field-permission/field-permission.entity';
 import { ObjectPermissionEntity } from 'src/engine/metadata-modules/object-permission/object-permission.entity';
@@ -142,6 +143,15 @@ export class ObjectMetadataEntity
   })
   indexMetadatas: Relation<IndexMetadataEntity[]>;
 
+  @OneToMany(
+    () => SearchFieldMetadataEntity,
+    (searchFieldMetadata) => searchFieldMetadata.objectMetadata,
+    {
+      cascade: true,
+    },
+  )
+  searchFieldMetadatas: Relation<SearchFieldMetadataEntity[]>;
+
   @CreateDateColumn({ type: 'timestamptz' })
   createdAt: Date;
 

packages/twenty-server/src/engine/metadata-modules/object-metadata/object-metadata.service.ts

@@ -89,6 +89,7 @@ export class ObjectMetadataService extends TypeOrmQueryService<ObjectMetadataEnt
       flatViewFieldMaps: existingFlatViewFieldMaps,
       flatViewMaps: existingFlatViewMaps,
       flatCommandMenuItemMaps: existingFlatCommandMenuItemMaps,
+      flatSearchFieldMetadataMaps: existingFlatSearchFieldMetadataMaps,
     } = await this.flatEntityMapsCacheService.getOrRecomputeManyOrAllFlatEntityMaps(
       {
         workspaceId,
@@ -99,6 +100,7 @@ export class ObjectMetadataService extends TypeOrmQueryService<ObjectMetadataEnt
           'flatViewFieldMaps',
           'flatViewMaps',
           'flatCommandMenuItemMaps',
+          'flatSearchFieldMetadataMaps',
         ],
       },
     );
@@ -110,6 +112,7 @@ export class ObjectMetadataService extends TypeOrmQueryService<ObjectMetadataEnt
       flatIndexMetadatasToUpdate,
       flatViewFieldsToCreate,
       flatViewFieldsToUpdate,
+      searchFieldMetadatasToCreate,
     } = fromUpdateObjectInputToFlatObjectMetadataAndRelatedFlatEntities({
       flatFieldMetadataMaps: existingFlatFieldMetadataMaps,
       flatObjectMetadataMaps: existingFlatObjectMetadataMaps,
@@ -117,6 +120,7 @@ export class ObjectMetadataService extends TypeOrmQueryService<ObjectMetadataEnt
       flatIndexMaps: existingFlatIndexMaps,
       flatViewFieldMaps: existingFlatViewFieldMaps,
       flatViewMaps: existingFlatViewMaps,
+      flatSearchFieldMetadataMaps: existingFlatSearchFieldMetadataMaps,
     });
 
     const existingFlatObjectMetadata = findFlatEntityByIdInFlatEntityMaps({
@@ -177,6 +181,11 @@ export class ObjectMetadataService extends TypeOrmQueryService<ObjectMetadataEnt
               flatEntityToDelete: [],
               flatEntityToUpdate: flatViewFieldsToUpdate,
             },
+            searchFieldMetadata: {
+              flatEntityToCreate: searchFieldMetadatasToCreate,
+              flatEntityToDelete: [],
+              flatEntityToUpdate: [],
+            },
             commandMenuItem: {
               flatEntityToCreate: [],
               flatEntityToDelete: [],
@@ -485,6 +494,7 @@ export class ObjectMetadataService extends TypeOrmQueryService<ObjectMetadataEnt
     const {
       flatObjectMetadataToCreate,
       flatIndexMetadataToCreate,
+      flatSearchFieldMetadataToCreate,
       flatFieldMetadataToCreateOnObject,
       relationTargetFlatFieldMetadataToCreate,
     } = fromCreateObjectInputToFlatObjectMetadataAndFlatFieldMetadatasToCreate({
@@ -620,6 +630,11 @@ export class ObjectMetadataService extends TypeOrmQueryService<ObjectMetadataEnt
               flatEntityToDelete: [],
               flatEntityToUpdate: [],
             },
+            searchFieldMetadata: {
+              flatEntityToCreate: flatSearchFieldMetadataToCreate,
+              flatEntityToDelete: [],
+              flatEntityToUpdate: [],
+            },
             ...(isDefined(flatNavigationMenuItemToCreate)
               ? {
                   navigationMenuItem: {

packages/twenty-server/src/engine/metadata-modules/object-metadata/utils/build-default-search-field-metadatas-for-custom-object.util.ts

@@ -0,0 +1,36 @@
+import { isDefined } from 'twenty-shared/utils';
+
+import { buildFlatSearchFieldMetadataForField } from 'src/engine/metadata-modules/flat-search-field-metadata/utils/build-flat-search-field-metadata-for-field.util';
+import { type DefaultFlatFieldForCustomObjectMaps } from 'src/engine/metadata-modules/object-metadata/utils/build-default-flat-field-metadatas-for-custom-object.util';
+import { type UniversalFlatObjectMetadata } from 'src/engine/workspace-manager/workspace-migration/universal-flat-entity/types/universal-flat-object-metadata.type';
+import { type UniversalFlatSearchFieldMetadata } from 'src/engine/workspace-manager/workspace-migration/universal-flat-entity/types/universal-flat-search-field-metadata.type';
+
+// Mirrors the custom-object searchVector, which indexes the name field only. Junction
+// objects (skipNameField) have no name field and therefore get no row.
+export const buildDefaultSearchFieldMetadatasForCustomObject = ({
+  flatObjectMetadata,
+  defaultFlatFieldForCustomObjectMaps,
+}: {
+  flatObjectMetadata: UniversalFlatObjectMetadata & { id: string };
+  defaultFlatFieldForCustomObjectMaps: DefaultFlatFieldForCustomObjectMaps;
+}): {
+  searchFieldMetadatas: UniversalFlatSearchFieldMetadata[];
+} => {
+  const nameField = defaultFlatFieldForCustomObjectMaps.fields.nameField;
+
+  if (!isDefined(nameField)) {
+    return {
+      searchFieldMetadatas: [],
+    };
+  }
+
+  const nameSearchFieldMetadata = buildFlatSearchFieldMetadataForField({
+    flatObjectMetadata,
+    flatFieldMetadata: nameField,
+    position: 0,
+  });
+
+  return {
+    searchFieldMetadatas: [nameSearchFieldMetadata],
+  };
+};