Wire up search field metadata (#21964)
## Part 1 - Exact scope of the current PR (#21964) close https://github.com/twentyhq/core-team-issues/issues/2586 This PR introduces `searchFieldMetadata` as a first-class flat metadata entity and migrates the existing search surface onto it, with **no change to which records are searchable** (ISO with `main`). In scope (what the PR does): - New flat entity `searchFieldMetadata` (universalIdentifier, applicationId, **`position`**, maps, conversions), registered in the central flat-entity constants and the migration build orchestrator. - `searchVector.asExpression` is **derived server-side** from `searchFieldMetadata` rows (validated by `isSafeTsVectorExpression`); never trusted from client input. - **Derivation order is deterministic, driven by each row's `position`** ([compute-search-vector-as-expression-from-search-field-metadatas.util.ts](packages/twenty-server/src/engine/metadata-modules/flat-search-field-metadata/utils/compute-search-vector-as-expression-from-search-field-metadatas.util.ts)), replacing the previous non-deterministic `(createdAt, id)` sort. That sort collapsed to random UUIDs for standard fields (same `createdAt`), so any rename/relabel rewrote the `STORED` generated column to a logically-identical-but-textually-different expression and produced a permanent per-workspace diff vs the standard definition. Ordering now equals provisioning order; ties break on `universalIdentifier`. - Provisioning at object creation mirrors the existing surface exactly **and seeds `position`**: - custom objects -> the `name` field only, at `position: 0` ([build-default-search-field-metadatas-for-custom-object.util.ts](packages/twenty-server/src/engine/metadata-modules/object-metadata/utils/build-default-search-field-metadatas-for-custom-object.util.ts)) - standard objects -> their curated `SEARCH_FIELDS_FOR_*` sets, `position` = the curated index - Backfill (instance + workspace commands in `2-16`) provisions rows for existing workspaces with the same surface **and the same positions** (standard from the curated standard maps, custom `name` = `0`), scoped to the workspace's own custom application ([build-search-field-metadata-backfill-operations.util.ts](packages/twenty-server/src/database/commands/upgrade-version-command/2-16/utils/build-search-field-metadata-backfill-operations.util.ts)). The `position` column is added in the same `2-16` fast instance command as `universalIdentifier`/`applicationId`. - Field rename of an already-indexed field recomputes `asExpression` (positions preserved, so order is stable) ([recompute-search-vector-on-field-rename.util.ts](packages/twenty-server/src/engine/metadata-modules/flat-field-metadata/utils/recompute-search-vector-on-field-rename.util.ts)). - Field delete drops the matching row(s) and recomputes; remaining rows keep their relative order (no renumber) ([from-delete-field-input-to-flat-field-metadatas-to-delete.util.ts](packages/twenty-server/src/engine/metadata-modules/flat-field-metadata/utils/from-delete-field-input-to-flat-field-metadatas-to-delete.util.ts)). - Object relabel is **additive** and ISO/regression-fix only: it indexes the new label identifier **appended last (`position = max(existing) + 1`)** without dropping `name` ([recompute-search-vector-on-label-identifier-update.util.ts](packages/twenty-server/src/engine/metadata-modules/flat-object-metadata/utils/recompute-search-vector-on-label-identifier-update.util.ts)). This is a deliberate, temporary bridge. Explicitly OUT of scope (deferred): - No API to edit `searchFieldMetadata` (no user-facing search-field configuration, including `position` — it is internal and only written by provisioning/backfill/recompute). - No auto-indexing of arbitrary searchable fields. Creating a custom TEXT/EMAILS/etc. field does NOT add it to search (the `computeSearchFieldMetadataCreationForFields` behavior was removed in `e6820ad`). - No field-type-transition handling (field type is immutable - not in `FLAT_FIELD_METADATA_EDITABLE_PROPERTIES`, so that path was dead code). - No `position` validation (uniqueness/range) and no multi-vector / per-field `weight` config — deferred to the configurable-search follow-up (#1428). Net: `searchFieldMetadata` becomes the source of truth for the *same* surface as `main`. The only intentional divergences from `main` are "relabel preserves `name`" (additive) and the deterministic `position`-ordered `asExpression` (a correctness/perf fix that is byte-identical to provisioning order, so it does not change the searchable surface). --------- Co-authored-by: Félix Malfait <felix@twenty.com>
This commit is contained in:
Paul Rastoin
+14
@@ -14,6 +14,7 @@ import { fromObjectMetadataEntityToFlatObjectMetadata } from 'src/engine/metadat
|
||||
import { IndexMetadataEntity } from 'src/engine/metadata-modules/index-metadata/index-metadata.entity';
|
||||
import { ObjectMetadataEntity } from 'src/engine/metadata-modules/object-metadata/object-metadata.entity';
|
||||
import { ObjectPermissionEntity } from 'src/engine/metadata-modules/object-permission/object-permission.entity';
|
||||
import { SearchFieldMetadataEntity } from 'src/engine/metadata-modules/search-field-metadata/search-field-metadata.entity';
|
||||
import { ViewEntity } from 'src/engine/metadata-modules/view/entities/view.entity';
|
||||
import { InjectWorkspaceScopedRepository } from 'src/engine/twenty-orm/workspace-scoped-repository/inject-workspace-scoped-repository.decorator';
|
||||
import { WorkspaceScopedRepository } from 'src/engine/twenty-orm/workspace-scoped-repository/workspace-scoped-repository';
|
||||
@@ -40,6 +41,8 @@ export class WorkspaceFlatObjectMetadataMapCacheService extends WorkspaceCachePr
|
||||
private readonly viewRepository: WorkspaceScopedRepository<ViewEntity>,
|
||||
@InjectWorkspaceScopedRepository(ObjectPermissionEntity)
|
||||
private readonly objectPermissionRepository: WorkspaceScopedRepository<ObjectPermissionEntity>,
|
||||
@InjectWorkspaceScopedRepository(SearchFieldMetadataEntity)
|
||||
private readonly searchFieldMetadataRepository: WorkspaceScopedRepository<SearchFieldMetadataEntity>,
|
||||
) {
|
||||
super();
|
||||
}
|
||||
@@ -54,6 +57,7 @@ export class WorkspaceFlatObjectMetadataMapCacheService extends WorkspaceCachePr
|
||||
indexMetadatas,
|
||||
views,
|
||||
objectPermissions,
|
||||
searchFieldMetadatas,
|
||||
] = await Promise.all([
|
||||
this.objectMetadataRepository.find({
|
||||
where: { workspaceId },
|
||||
@@ -81,6 +85,9 @@ export class WorkspaceFlatObjectMetadataMapCacheService extends WorkspaceCachePr
|
||||
select: ['id', 'universalIdentifier', 'objectMetadataId'],
|
||||
withDeleted: true,
|
||||
}),
|
||||
this.searchFieldMetadataRepository.find(workspaceId, {
|
||||
select: ['id', 'universalIdentifier', 'objectMetadataId'],
|
||||
}),
|
||||
]);
|
||||
|
||||
const [
|
||||
@@ -88,6 +95,7 @@ export class WorkspaceFlatObjectMetadataMapCacheService extends WorkspaceCachePr
|
||||
indexesByObjectId,
|
||||
viewsByObjectId,
|
||||
objectPermissionsByObjectId,
|
||||
searchFieldMetadatasByObjectId,
|
||||
] = (
|
||||
[
|
||||
{
|
||||
@@ -106,6 +114,10 @@ export class WorkspaceFlatObjectMetadataMapCacheService extends WorkspaceCachePr
|
||||
entities: objectPermissions,
|
||||
foreignKey: 'objectMetadataId',
|
||||
},
|
||||
{
|
||||
entities: searchFieldMetadatas,
|
||||
foreignKey: 'objectMetadataId',
|
||||
},
|
||||
] as const
|
||||
).map(regroupEntitiesByRelatedEntityId);
|
||||
|
||||
@@ -125,6 +137,8 @@ export class WorkspaceFlatObjectMetadataMapCacheService extends WorkspaceCachePr
|
||||
views: viewsByObjectId.get(objectMetadataEntity.id) || [],
|
||||
objectPermissions:
|
||||
objectPermissionsByObjectId.get(objectMetadataEntity.id) || [],
|
||||
searchFieldMetadatas:
|
||||
searchFieldMetadatasByObjectId.get(objectMetadataEntity.id) || [],
|
||||
},
|
||||
applicationIdToUniversalIdentifierMap,
|
||||
fieldMetadataIdToUniversalIdentifierMap,
|
||||
|
||||
Reference in New Issue
Block a user