Deprecate asExpression from field metadata search_vector (#22287)

## Summary

Fully deprecates the cached `asExpression` / `generatedType` settings on
`TS_VECTOR` (searchVector) fields. Previously the generated-column
expression was stored in `FieldMetadataSettings` and kept in sync via
imperative recompute side-effects. It is now **derived at DDL time**
from the `searchFieldMetadata` rows that describe which fields feed the
search vector, making `searchFieldMetadata` the single source of truth
and removing a whole class of cache-drift bugs.

This is delivered across the milestones tracked in #2587 and coordinates
with the frontend migration (#1428).

## Why

- The searchVector expression lived in two places (stored
`settings.asExpression` + the actual generated column), kept consistent
by bespoke side-effects (`recompute-search-vector-on-field-rename`,
label-identifier recompute, etc.).
- The frontend reconstructed the searchable-fields list by
**regex-parsing** the stored `asExpression`.
- Both are brittle. Deriving the expression from `searchFieldMetadata`
rows at build/run time removes the cache and the parsing.

## What changed

### Server - data model & derivation
- Introduce the `tsVectorFieldMetadata` relation on
`searchFieldMetadata` (`tsVectorFieldMetadataId` / universal identifier)
linking each searchable-field row to its target `TS_VECTOR` field.
- New runtime derivation
`deriveSearchVectorAsExpressionForTsVectorField`
(`flat-search-field-metadata/utils/...`) used by the create-object and
update-field handlers to generate the column expression from
`searchFieldMetadata` rows.
- Remove `asExpression` / `generatedType` from stored settings:
`FieldMetadataSettings.TS_VECTOR` is now `null`; the column builder
(`generate-column-definitions.util.ts`) hardcodes `generatedType:
'STORED'` and requires the derived expression.
- Delete the imperative recompute side-effects and the
`compute-search-vector-universal-settings-from-object-manifest` path;
drop the `settings` block from all 28 standard
`compute-*-standard-flat-field-metadata` utils.

### Server - migration runner
- New `rebuildSearchVector` marker on `update-field` actions: the
orchestrator synthesizes targeted column rebuilds
(`compute-search-vector-rebuild-target-universal-identifiers.util.ts` +
the deprioritize aggregator) only when a searchFieldMetadata change or
indexed-field rename actually requires it - instead of rebuilding on
every settings touch.
- Deferrable FKs + in-flight ID resolution so a `searchFieldMetadata`
row and its `TS_VECTOR` field can be created in the same transaction
(deterministic UUIDs).

### Frontend (contract change, #1428)
- New `SearchFieldMetadataDTO` + dataloader exposing
`searchFieldMetadataList` on object metadata.
- `SettingsObjectSearchSection` now reads
`objectMetadataItem.searchFieldMetadatas` instead of parsing
`asExpression`; new `SearchFieldMetadataItem` type, fragment, and
mapping updates.

### Upgrade commands (2.18)
-
`2-18-instance-command-fast-...-add-ts-vector-field-metadata-id-to-search-field-metadata`
-
`2-18-instance-command-fast-...-make-search-field-metadata-fks-deferrable`
-
`2-18-instance-command-slow-...-backfill-ts-vector-field-metadata-id-on-search-field-metadata`

(These were relocated from 2.16 to 2.18 and re-timestamped into an
ordered block - add column -> make FK deferrable -> backfill data -
since 2.16/2.17 are released.)

### Tests
- Updated search-vector side-effect integration specs to assert behavior
(search works) rather than the now-removed `asExpression`; removed the
obsolete expression-validation specs; refreshed the application-sync
snapshot (`universalSettings: null`).

## Upgrade / compatibility notes
- Existing workspaces keep their stored `settings` until a later
cleanup; nothing reads it anymore. The new derivation drives all DDL
going forward.
- Schema changes are gated behind the 2.18 instance commands above.

## Known follow-up (separate PR)
https://github.com/twentyhq/core-team-issues/issues/2620
- The column rebuild (`DROP`/`ADD` of the `searchVector` STORED column)
cascade-drops its GIN index and does not recreate it - a pre-existing
regression on `main` inherited here. A follow-up PR will fix the rebuild
handler to recreate the GIN index and add a 2.18 workspace command to
recompute every search vector + strip the deprecated settings.
(Planned.)

## Test plan
- [ ] `npx nx typecheck twenty-server` / `twenty-front`
- [ ] `npx nx lint:diff-with-main twenty-server` / `twenty-front`
- [ ] Server integration: create/update/delete field, rename indexed
field, update object - search returns expected records
- [ ] Run the 2.18 instance commands on a seeded DB; verify
`tsVectorFieldMetadataId` backfilled and FKs deferrable
- [ ] Frontend: object Search settings tab lists the correct searchable
fields (no `asExpression` parsing)

close https://github.com/twentyhq/core-team-issues/issues/2587

<!-- This is an auto-generated description by cubic. -->
<a
href="https://cubic.dev/pr/twentyhq/twenty/pull/22287?utm_source=github"
target="_blank" rel="noopener noreferrer"
data-no-image-dialog="true"><picture><source
media="(prefers-color-scheme: dark)"
srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source
media="(prefers-color-scheme: light)"
srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img
alt="Review in cubic"
src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a>
<!-- End of auto-generated description by cubic. -->

packages/twenty-server/src/engine/metadata-modules/object-metadata/constants/partial-system-flat-field-metadatas.constant.ts

@@ -40,6 +40,7 @@ const PARTIAL_ID_FIELD = {
   mainGroupByFieldMetadataViewUniversalIdentifiers: [],
   fieldPermissionUniversalIdentifiers: [],
   viewSortUniversalIdentifiers: [],
+  searchFieldMetadataUniversalIdentifiers: [],
 } as const satisfies PartialSystemFlatFieldMetadata<FieldMetadataType.UUID>;
 
 const PARTIAL_CREATED_AT_FIELD = {
@@ -69,6 +70,7 @@ const PARTIAL_CREATED_AT_FIELD = {
   mainGroupByFieldMetadataViewUniversalIdentifiers: [],
   fieldPermissionUniversalIdentifiers: [],
   viewSortUniversalIdentifiers: [],
+  searchFieldMetadataUniversalIdentifiers: [],
 } as const satisfies PartialSystemFlatFieldMetadata<FieldMetadataType.DATE_TIME>;
 
 const PARTIAL_UPDATED_AT_FIELD = {
@@ -98,6 +100,7 @@ const PARTIAL_UPDATED_AT_FIELD = {
   mainGroupByFieldMetadataViewUniversalIdentifiers: [],
   fieldPermissionUniversalIdentifiers: [],
   viewSortUniversalIdentifiers: [],
+  searchFieldMetadataUniversalIdentifiers: [],
 } as const satisfies PartialSystemFlatFieldMetadata<FieldMetadataType.DATE_TIME>;
 
 const PARTIAL_DELETED_AT_FIELD = {
@@ -127,6 +130,7 @@ const PARTIAL_DELETED_AT_FIELD = {
   mainGroupByFieldMetadataViewUniversalIdentifiers: [],
   fieldPermissionUniversalIdentifiers: [],
   viewSortUniversalIdentifiers: [],
+  searchFieldMetadataUniversalIdentifiers: [],
 } as const satisfies PartialSystemFlatFieldMetadata<FieldMetadataType.DATE_TIME>;
 
 const PARTIAL_CREATED_BY_FIELD = {
@@ -156,6 +160,7 @@ const PARTIAL_CREATED_BY_FIELD = {
   mainGroupByFieldMetadataViewUniversalIdentifiers: [],
   fieldPermissionUniversalIdentifiers: [],
   viewSortUniversalIdentifiers: [],
+  searchFieldMetadataUniversalIdentifiers: [],
 } as const satisfies PartialSystemFlatFieldMetadata<FieldMetadataType.ACTOR>;
 
 const PARTIAL_UPDATED_BY_FIELD = {
@@ -185,6 +190,7 @@ const PARTIAL_UPDATED_BY_FIELD = {
   mainGroupByFieldMetadataViewUniversalIdentifiers: [],
   fieldPermissionUniversalIdentifiers: [],
   viewSortUniversalIdentifiers: [],
+  searchFieldMetadataUniversalIdentifiers: [],
 } as const satisfies PartialSystemFlatFieldMetadata<FieldMetadataType.ACTOR>;
 
 const PARTIAL_POSITION_FIELD = {
@@ -214,6 +220,7 @@ const PARTIAL_POSITION_FIELD = {
   mainGroupByFieldMetadataViewUniversalIdentifiers: [],
   fieldPermissionUniversalIdentifiers: [],
   viewSortUniversalIdentifiers: [],
+  searchFieldMetadataUniversalIdentifiers: [],
 } as const satisfies PartialSystemFlatFieldMetadata<FieldMetadataType.POSITION>;
 
 const PARTIAL_SEARCH_VECTOR_FIELD = {
@@ -245,6 +252,7 @@ const PARTIAL_SEARCH_VECTOR_FIELD = {
   mainGroupByFieldMetadataViewUniversalIdentifiers: [],
   fieldPermissionUniversalIdentifiers: [],
   viewSortUniversalIdentifiers: [],
+  searchFieldMetadataUniversalIdentifiers: [],
 } as const satisfies PartialSystemFlatFieldMetadata<FieldMetadataType.TS_VECTOR>;
 
 export const PARTIAL_SYSTEM_FLAT_FIELD_METADATAS = {

packages/twenty-server/src/engine/metadata-modules/object-metadata/object-metadata.resolver.ts

@@ -30,6 +30,7 @@ import { ObjectRecordCountDTO } from 'src/engine/metadata-modules/object-metadat
 import { UpdateOneObjectInput } from 'src/engine/metadata-modules/object-metadata/dtos/update-object.input';
 import { ObjectMetadataService } from 'src/engine/metadata-modules/object-metadata/object-metadata.service';
 import { ObjectRecordCountService } from 'src/engine/metadata-modules/object-metadata/object-record-count.service';
+import { SearchFieldMetadataDTO } from 'src/engine/metadata-modules/search-field-metadata/dtos/search-field-metadata.dto';
 import { objectMetadataGraphqlApiExceptionHandler } from 'src/engine/metadata-modules/object-metadata/utils/object-metadata-graphql-api-exception-handler.util';
 import { resolveObjectMetadataStandardOverride } from 'src/engine/metadata-modules/object-metadata/utils/resolve-object-metadata-standard-override.util';
 import { PermissionsGraphqlApiExceptionFilter } from 'src/engine/metadata-modules/permissions/utils/permissions-graphql-api-exception.filter';
@@ -273,4 +274,25 @@ export class ObjectMetadataResolver {
       return [];
     }
   }
+
+  @ResolveField(() => [SearchFieldMetadataDTO], { nullable: false })
+  async searchFieldMetadataList(
+    @AuthWorkspace() workspace: WorkspaceEntity,
+    @Parent() objectMetadata: ObjectMetadataDTO,
+    @Context() context: { loaders: IDataloaders },
+  ): Promise<SearchFieldMetadataDTO[]> {
+    try {
+      const searchFieldMetadataItems =
+        await context.loaders.searchFieldMetadataLoader.load({
+          objectMetadata,
+          workspaceId: workspace.id,
+        });
+
+      return searchFieldMetadataItems;
+    } catch (error) {
+      objectMetadataGraphqlApiExceptionHandler(error);
+
+      return [];
+    }
+  }
 }

packages/twenty-server/src/engine/metadata-modules/object-metadata/object-metadata.service.ts

@@ -107,7 +107,6 @@ export class ObjectMetadataService extends TypeOrmQueryService<ObjectMetadataEnt
 
     const {
       otherObjectFlatFieldMetadatasToUpdate,
-      sameObjectFlatFieldMetadatasToUpdate,
       flatObjectMetadataToUpdate,
       flatIndexMetadatasToUpdate,
       flatViewFieldsToCreate,
@@ -171,10 +170,7 @@ export class ObjectMetadataService extends TypeOrmQueryService<ObjectMetadataEnt
             fieldMetadata: {
               flatEntityToCreate: [],
               flatEntityToDelete: [],
-              flatEntityToUpdate: [
-                ...otherObjectFlatFieldMetadatasToUpdate,
-                ...sameObjectFlatFieldMetadatasToUpdate,
-              ],
+              flatEntityToUpdate: [...otherObjectFlatFieldMetadatasToUpdate],
             },
             viewField: {
               flatEntityToCreate: flatViewFieldsToCreate,

packages/twenty-server/src/engine/metadata-modules/object-metadata/utils/build-default-flat-field-metadatas-for-custom-object.util.ts

@@ -2,7 +2,6 @@ import { FieldMetadataType } from 'twenty-shared/types';
 import { v4 } from 'uuid';
 
 import { PARTIAL_SYSTEM_FLAT_FIELD_METADATAS } from 'src/engine/metadata-modules/object-metadata/constants/partial-system-flat-field-metadatas.constant';
-import { getTsVectorColumnExpressionFromFields } from 'src/engine/workspace-manager/utils/get-ts-vector-column-expression.util';
 import { type UniversalFlatFieldMetadata } from 'src/engine/workspace-manager/workspace-migration/universal-flat-entity/types/universal-flat-field-metadata.type';
 import { type UniversalFlatObjectMetadata } from 'src/engine/workspace-manager/workspace-migration/universal-flat-entity/types/universal-flat-object-metadata.type';
 
@@ -154,15 +153,11 @@ export const buildDefaultFlatFieldMetadatasForCustomObject = ({
           fieldPermissionUniversalIdentifiers: [],
           universalSettings: null,
           viewSortUniversalIdentifiers: [],
+          searchFieldMetadataUniversalIdentifiers: [],
         };
 
   const searchVectorUniversalSettings: UniversalFlatFieldMetadata<FieldMetadataType.TS_VECTOR>['universalSettings'] =
-    {
-      asExpression: getTsVectorColumnExpressionFromFields(
-        nameField ? [nameField] : [],
-      ),
-      generatedType: 'STORED',
-    };
+    null;
 
   return {
     fields: {

packages/twenty-server/src/engine/metadata-modules/object-metadata/utils/build-default-search-field-metadatas-for-custom-object.util.ts

@@ -27,6 +27,8 @@ export const buildDefaultSearchFieldMetadatasForCustomObject = ({
   const nameSearchFieldMetadata = buildFlatSearchFieldMetadataForField({
     flatObjectMetadata,
     flatFieldMetadata: nameField,
+    tsVectorFlatFieldMetadata:
+      defaultFlatFieldForCustomObjectMaps.fields.searchVector,
     position: 0,
   });