Wire up search field metadata (#21964)

## Part 1 - Exact scope of the current PR (#21964)

close https://github.com/twentyhq/core-team-issues/issues/2586

This PR introduces `searchFieldMetadata` as a first-class flat metadata
entity and migrates the existing search surface onto it, with **no
change to which records are searchable** (ISO with `main`).

In scope (what the PR does):
- New flat entity `searchFieldMetadata` (universalIdentifier,
applicationId, **`position`**, maps, conversions), registered in the
central flat-entity constants and the migration build orchestrator.
- `searchVector.asExpression` is **derived server-side** from
`searchFieldMetadata` rows (validated by `isSafeTsVectorExpression`);
never trusted from client input.
- **Derivation order is deterministic, driven by each row's `position`**
([compute-search-vector-as-expression-from-search-field-metadatas.util.ts](packages/twenty-server/src/engine/metadata-modules/flat-search-field-metadata/utils/compute-search-vector-as-expression-from-search-field-metadatas.util.ts)),
replacing the previous non-deterministic `(createdAt, id)` sort. That
sort collapsed to random UUIDs for standard fields (same `createdAt`),
so any rename/relabel rewrote the `STORED` generated column to a
logically-identical-but-textually-different expression and produced a
permanent per-workspace diff vs the standard definition. Ordering now
equals provisioning order; ties break on `universalIdentifier`.
- Provisioning at object creation mirrors the existing surface exactly
**and seeds `position`**:
- custom objects -> the `name` field only, at `position: 0`
([build-default-search-field-metadatas-for-custom-object.util.ts](packages/twenty-server/src/engine/metadata-modules/object-metadata/utils/build-default-search-field-metadatas-for-custom-object.util.ts))
- standard objects -> their curated `SEARCH_FIELDS_FOR_*` sets,
`position` = the curated index
- Backfill (instance + workspace commands in `2-16`) provisions rows for
existing workspaces with the same surface **and the same positions**
(standard from the curated standard maps, custom `name` = `0`), scoped
to the workspace's own custom application
([build-search-field-metadata-backfill-operations.util.ts](packages/twenty-server/src/database/commands/upgrade-version-command/2-16/utils/build-search-field-metadata-backfill-operations.util.ts)).
The `position` column is added in the same `2-16` fast instance command
as `universalIdentifier`/`applicationId`.
- Field rename of an already-indexed field recomputes `asExpression`
(positions preserved, so order is stable)
([recompute-search-vector-on-field-rename.util.ts](packages/twenty-server/src/engine/metadata-modules/flat-field-metadata/utils/recompute-search-vector-on-field-rename.util.ts)).
- Field delete drops the matching row(s) and recomputes; remaining rows
keep their relative order (no renumber)
([from-delete-field-input-to-flat-field-metadatas-to-delete.util.ts](packages/twenty-server/src/engine/metadata-modules/flat-field-metadata/utils/from-delete-field-input-to-flat-field-metadatas-to-delete.util.ts)).
- Object relabel is **additive** and ISO/regression-fix only: it indexes
the new label identifier **appended last (`position = max(existing) +
1`)** without dropping `name`
([recompute-search-vector-on-label-identifier-update.util.ts](packages/twenty-server/src/engine/metadata-modules/flat-object-metadata/utils/recompute-search-vector-on-label-identifier-update.util.ts)).
This is a deliberate, temporary bridge.

Explicitly OUT of scope (deferred):
- No API to edit `searchFieldMetadata` (no user-facing search-field
configuration, including `position` — it is internal and only written by
provisioning/backfill/recompute).
- No auto-indexing of arbitrary searchable fields. Creating a custom
TEXT/EMAILS/etc. field does NOT add it to search (the
`computeSearchFieldMetadataCreationForFields` behavior was removed in
`e6820ad`).
- No field-type-transition handling (field type is immutable - not in
`FLAT_FIELD_METADATA_EDITABLE_PROPERTIES`, so that path was dead code).
- No `position` validation (uniqueness/range) and no multi-vector /
per-field `weight` config — deferred to the configurable-search
follow-up (#1428).

Net: `searchFieldMetadata` becomes the source of truth for the *same*
surface as `main`. The only intentional divergences from `main` are
"relabel preserves `name`" (additive) and the deterministic
`position`-ordered `asExpression` (a correctness/perf fix that is
byte-identical to provisioning order, so it does not change the
searchable surface).

---------

Co-authored-by: Félix Malfait <felix@twenty.com>

packages/twenty-server/src/database/commands/upgrade-version-command/instance-commands.constant.ts

@@ -29,8 +29,11 @@ import { CreateUnsubscribeTopicCoreTableFastInstanceCommand } from 'src/database
 import { RenameIsUiReadOnlyToIsUiEditableFastInstanceCommand } from 'src/database/commands/upgrade-version-command/2-13/2-13-instance-command-fast-1781277453604-rename-is-ui-read-only-to-is-ui-editable';
 import { BackfillNonUiCreatableStandardSystemObjectsSlowInstanceCommand } from 'src/database/commands/upgrade-version-command/2-13/2-13-instance-command-slow-1781277480000-backfill-non-ui-creatable-standard-system-objects';
 import { SetTableWidgetViewsVisibilityToWorkspaceSlowInstanceCommand } from 'src/database/commands/upgrade-version-command/2-14/2-14-instance-command-slow-1781515653781-set-table-widget-views-visibility-to-workspace';
+import { AddHasPaymentMethodToBillingCustomerFastInstanceCommand } from 'src/database/commands/upgrade-version-command/2-15/2-15-instance-command-fast-1781280240009-add-has-payment-method-to-billing-customer';
 import { AddIsSystemSideEffectFastInstanceCommand } from 'src/database/commands/upgrade-version-command/2-15/2-15-instance-command-fast-1781600000000-add-is-system-side-effect';
 import { BackfillConnectionSecuritySlowInstanceCommand } from 'src/database/commands/upgrade-version-command/2-15/2-15-instance-command-slow-1781461753981-backfill-connection-security';
+import { AddChannelWebhookSubscriptionFieldsFastInstanceCommand } from 'src/database/commands/upgrade-version-command/2-16/2-16-instance-command-fast-1782152096938-add-channel-webhook-subscription-fields';
+import { AddUniversalIdentifierAndApplicationIdToSearchFieldMetadataFastInstanceCommand } from 'src/database/commands/upgrade-version-command/2-16/2-16-instance-command-fast-1782200000000-add-universal-identifier-and-application-id-to-search-field-metadata';
 import { AddCacheTokensToAgentChatThreadFastInstanceCommand } from 'src/database/commands/upgrade-version-command/2-2/2-2-instance-command-fast-1777455269302-add-cache-tokens-to-agent-chat-thread';
 import { AddLogoToApplicationFastInstanceCommand } from 'src/database/commands/upgrade-version-command/2-2/2-2-instance-command-fast-1777539664664-add-logo-to-application';
 import { AddSubFieldNameToViewSortEarlyFastInstanceCommand } from 'src/database/commands/upgrade-version-command/2-3/2-3-instance-command-fast-1747234200000-add-sub-field-name-to-view-sort';
@@ -72,10 +75,8 @@ import { EmailingDomainTenantStatusAndGlobalUniquenessFastInstanceCommand } from
 import { AddLogicFunctionExecutionModeFastInstanceCommand } from 'src/database/commands/upgrade-version-command/2-9/2-9-instance-command-fast-1799000030000-add-logic-function-execution-mode';
 import { EncryptNonSecretApplicationVariableSlowInstanceCommand } from 'src/database/commands/upgrade-version-command/2-9/2-9-instance-command-slow-1798400000000-encrypt-non-secret-application-variable';
 import { MigrateAiModelPreferencesSlowInstanceCommand } from 'src/database/commands/upgrade-version-command/2-9/2-9-instance-command-slow-1799000010000-migrate-ai-model-preferences';
-import { AddHasPaymentMethodToBillingCustomerFastInstanceCommand } from 'src/database/commands/upgrade-version-command/2-15/2-15-instance-command-fast-1781280240009-add-has-payment-method-to-billing-customer';
 import { AddFolderImportToMessageFolderPendingSyncActionFastInstanceCommand } from './2-15/2-15-instance-command-fast-1781714499016-add-folder-import-to-message-folder-pending-sync-action';
 import { AddViewKanbanColumnWidthFastInstanceCommand } from './2-15/2-15-instance-command-fast-1781900000000-add-view-kanban-column-width';
-import { AddChannelWebhookSubscriptionFieldsFastInstanceCommand } from 'src/database/commands/upgrade-version-command/2-16/2-16-instance-command-fast-1782152096938-add-channel-webhook-subscription-fields';
 
 export const INSTANCE_COMMANDS = [
   AddViewFieldGroupIdIndexOnViewFieldFastInstanceCommand,
@@ -154,4 +155,5 @@ export const INSTANCE_COMMANDS = [
   AddFolderImportToMessageFolderPendingSyncActionFastInstanceCommand,
   AddViewKanbanColumnWidthFastInstanceCommand,
   AddChannelWebhookSubscriptionFieldsFastInstanceCommand,
+  AddUniversalIdentifierAndApplicationIdToSearchFieldMetadataFastInstanceCommand,
 ];