Wire up search field metadata (#21964)
## Part 1 - Exact scope of the current PR (#21964) close https://github.com/twentyhq/core-team-issues/issues/2586 This PR introduces `searchFieldMetadata` as a first-class flat metadata entity and migrates the existing search surface onto it, with **no change to which records are searchable** (ISO with `main`). In scope (what the PR does): - New flat entity `searchFieldMetadata` (universalIdentifier, applicationId, **`position`**, maps, conversions), registered in the central flat-entity constants and the migration build orchestrator. - `searchVector.asExpression` is **derived server-side** from `searchFieldMetadata` rows (validated by `isSafeTsVectorExpression`); never trusted from client input. - **Derivation order is deterministic, driven by each row's `position`** ([compute-search-vector-as-expression-from-search-field-metadatas.util.ts](packages/twenty-server/src/engine/metadata-modules/flat-search-field-metadata/utils/compute-search-vector-as-expression-from-search-field-metadatas.util.ts)), replacing the previous non-deterministic `(createdAt, id)` sort. That sort collapsed to random UUIDs for standard fields (same `createdAt`), so any rename/relabel rewrote the `STORED` generated column to a logically-identical-but-textually-different expression and produced a permanent per-workspace diff vs the standard definition. Ordering now equals provisioning order; ties break on `universalIdentifier`. - Provisioning at object creation mirrors the existing surface exactly **and seeds `position`**: - custom objects -> the `name` field only, at `position: 0` ([build-default-search-field-metadatas-for-custom-object.util.ts](packages/twenty-server/src/engine/metadata-modules/object-metadata/utils/build-default-search-field-metadatas-for-custom-object.util.ts)) - standard objects -> their curated `SEARCH_FIELDS_FOR_*` sets, `position` = the curated index - Backfill (instance + workspace commands in `2-16`) provisions rows for existing workspaces with the same surface **and the same positions** (standard from the curated standard maps, custom `name` = `0`), scoped to the workspace's own custom application ([build-search-field-metadata-backfill-operations.util.ts](packages/twenty-server/src/database/commands/upgrade-version-command/2-16/utils/build-search-field-metadata-backfill-operations.util.ts)). The `position` column is added in the same `2-16` fast instance command as `universalIdentifier`/`applicationId`. - Field rename of an already-indexed field recomputes `asExpression` (positions preserved, so order is stable) ([recompute-search-vector-on-field-rename.util.ts](packages/twenty-server/src/engine/metadata-modules/flat-field-metadata/utils/recompute-search-vector-on-field-rename.util.ts)). - Field delete drops the matching row(s) and recomputes; remaining rows keep their relative order (no renumber) ([from-delete-field-input-to-flat-field-metadatas-to-delete.util.ts](packages/twenty-server/src/engine/metadata-modules/flat-field-metadata/utils/from-delete-field-input-to-flat-field-metadatas-to-delete.util.ts)). - Object relabel is **additive** and ISO/regression-fix only: it indexes the new label identifier **appended last (`position = max(existing) + 1`)** without dropping `name` ([recompute-search-vector-on-label-identifier-update.util.ts](packages/twenty-server/src/engine/metadata-modules/flat-object-metadata/utils/recompute-search-vector-on-label-identifier-update.util.ts)). This is a deliberate, temporary bridge. Explicitly OUT of scope (deferred): - No API to edit `searchFieldMetadata` (no user-facing search-field configuration, including `position` — it is internal and only written by provisioning/backfill/recompute). - No auto-indexing of arbitrary searchable fields. Creating a custom TEXT/EMAILS/etc. field does NOT add it to search (the `computeSearchFieldMetadataCreationForFields` behavior was removed in `e6820ad`). - No field-type-transition handling (field type is immutable - not in `FLAT_FIELD_METADATA_EDITABLE_PROPERTIES`, so that path was dead code). - No `position` validation (uniqueness/range) and no multi-vector / per-field `weight` config — deferred to the configurable-search follow-up (#1428). Net: `searchFieldMetadata` becomes the source of truth for the *same* surface as `main`. The only intentional divergences from `main` are "relabel preserves `name`" (additive) and the deterministic `position`-ordered `asExpression` (a correctness/perf fix that is byte-identical to provisioning order, so it does not change the searchable surface). --------- Co-authored-by: Félix Malfait <felix@twenty.com>
This commit is contained in:
+2
@@ -20,6 +20,7 @@ export const getFlatObjectMetadataMock = (
|
||||
return {
|
||||
viewIds: [],
|
||||
indexMetadataIds: [],
|
||||
searchFieldMetadataIds: [],
|
||||
objectPermissionIds: [],
|
||||
fieldPermissionIds: [],
|
||||
fieldIds: [],
|
||||
@@ -55,6 +56,7 @@ export const getFlatObjectMetadataMock = (
|
||||
fieldPermissionUniversalIdentifiers: [],
|
||||
viewUniversalIdentifiers: [],
|
||||
indexMetadataUniversalIdentifiers: [],
|
||||
searchFieldMetadataUniversalIdentifiers: [],
|
||||
labelIdentifierFieldMetadataUniversalIdentifier:
|
||||
labelIdentifierFieldMetadataId,
|
||||
imageIdentifierFieldMetadataUniversalIdentifier:
|
||||
|
||||
+14
@@ -14,6 +14,7 @@ import { fromObjectMetadataEntityToFlatObjectMetadata } from 'src/engine/metadat
|
||||
import { IndexMetadataEntity } from 'src/engine/metadata-modules/index-metadata/index-metadata.entity';
|
||||
import { ObjectMetadataEntity } from 'src/engine/metadata-modules/object-metadata/object-metadata.entity';
|
||||
import { ObjectPermissionEntity } from 'src/engine/metadata-modules/object-permission/object-permission.entity';
|
||||
import { SearchFieldMetadataEntity } from 'src/engine/metadata-modules/search-field-metadata/search-field-metadata.entity';
|
||||
import { ViewEntity } from 'src/engine/metadata-modules/view/entities/view.entity';
|
||||
import { InjectWorkspaceScopedRepository } from 'src/engine/twenty-orm/workspace-scoped-repository/inject-workspace-scoped-repository.decorator';
|
||||
import { WorkspaceScopedRepository } from 'src/engine/twenty-orm/workspace-scoped-repository/workspace-scoped-repository';
|
||||
@@ -40,6 +41,8 @@ export class WorkspaceFlatObjectMetadataMapCacheService extends WorkspaceCachePr
|
||||
private readonly viewRepository: WorkspaceScopedRepository<ViewEntity>,
|
||||
@InjectWorkspaceScopedRepository(ObjectPermissionEntity)
|
||||
private readonly objectPermissionRepository: WorkspaceScopedRepository<ObjectPermissionEntity>,
|
||||
@InjectWorkspaceScopedRepository(SearchFieldMetadataEntity)
|
||||
private readonly searchFieldMetadataRepository: WorkspaceScopedRepository<SearchFieldMetadataEntity>,
|
||||
) {
|
||||
super();
|
||||
}
|
||||
@@ -54,6 +57,7 @@ export class WorkspaceFlatObjectMetadataMapCacheService extends WorkspaceCachePr
|
||||
indexMetadatas,
|
||||
views,
|
||||
objectPermissions,
|
||||
searchFieldMetadatas,
|
||||
] = await Promise.all([
|
||||
this.objectMetadataRepository.find({
|
||||
where: { workspaceId },
|
||||
@@ -81,6 +85,9 @@ export class WorkspaceFlatObjectMetadataMapCacheService extends WorkspaceCachePr
|
||||
select: ['id', 'universalIdentifier', 'objectMetadataId'],
|
||||
withDeleted: true,
|
||||
}),
|
||||
this.searchFieldMetadataRepository.find(workspaceId, {
|
||||
select: ['id', 'universalIdentifier', 'objectMetadataId'],
|
||||
}),
|
||||
]);
|
||||
|
||||
const [
|
||||
@@ -88,6 +95,7 @@ export class WorkspaceFlatObjectMetadataMapCacheService extends WorkspaceCachePr
|
||||
indexesByObjectId,
|
||||
viewsByObjectId,
|
||||
objectPermissionsByObjectId,
|
||||
searchFieldMetadatasByObjectId,
|
||||
] = (
|
||||
[
|
||||
{
|
||||
@@ -106,6 +114,10 @@ export class WorkspaceFlatObjectMetadataMapCacheService extends WorkspaceCachePr
|
||||
entities: objectPermissions,
|
||||
foreignKey: 'objectMetadataId',
|
||||
},
|
||||
{
|
||||
entities: searchFieldMetadatas,
|
||||
foreignKey: 'objectMetadataId',
|
||||
},
|
||||
] as const
|
||||
).map(regroupEntitiesByRelatedEntityId);
|
||||
|
||||
@@ -125,6 +137,8 @@ export class WorkspaceFlatObjectMetadataMapCacheService extends WorkspaceCachePr
|
||||
views: viewsByObjectId.get(objectMetadataEntity.id) || [],
|
||||
objectPermissions:
|
||||
objectPermissionsByObjectId.get(objectMetadataEntity.id) || [],
|
||||
searchFieldMetadatas:
|
||||
searchFieldMetadatasByObjectId.get(objectMetadataEntity.id) || [],
|
||||
},
|
||||
applicationIdToUniversalIdentifierMap,
|
||||
fieldMetadataIdToUniversalIdentifierMap,
|
||||
|
||||
+12
@@ -10,9 +10,11 @@ import { type CreateObjectInput } from 'src/engine/metadata-modules/object-metad
|
||||
import { buildDefaultFlatFieldMetadatasForCustomObject } from 'src/engine/metadata-modules/object-metadata/utils/build-default-flat-field-metadatas-for-custom-object.util';
|
||||
import { buildDefaultIndexesForCustomObject } from 'src/engine/metadata-modules/object-metadata/utils/build-default-index-for-custom-object.util';
|
||||
import { buildDefaultRelationFlatFieldMetadatasForCustomObject } from 'src/engine/metadata-modules/object-metadata/utils/build-default-relation-flat-field-metadatas-for-custom-object.util';
|
||||
import { buildDefaultSearchFieldMetadatasForCustomObject } from 'src/engine/metadata-modules/object-metadata/utils/build-default-search-field-metadatas-for-custom-object.util';
|
||||
import { type UniversalFlatFieldMetadata } from 'src/engine/workspace-manager/workspace-migration/universal-flat-entity/types/universal-flat-field-metadata.type';
|
||||
import { type UniversalFlatIndexMetadata } from 'src/engine/workspace-manager/workspace-migration/universal-flat-entity/types/universal-flat-index-metadata.type';
|
||||
import { type UniversalFlatObjectMetadata } from 'src/engine/workspace-manager/workspace-migration/universal-flat-entity/types/universal-flat-object-metadata.type';
|
||||
import { type UniversalFlatSearchFieldMetadata } from 'src/engine/workspace-manager/workspace-migration/universal-flat-entity/types/universal-flat-search-field-metadata.type';
|
||||
|
||||
type FromCreateObjectInputToFlatObjectMetadataAndFlatFieldMetadatasToCreateArgs =
|
||||
{
|
||||
@@ -29,6 +31,7 @@ export const fromCreateObjectInputToFlatObjectMetadataAndFlatFieldMetadatasToCre
|
||||
relationTargetFlatFieldMetadataToCreate: UniversalFlatFieldMetadata[];
|
||||
flatFieldMetadataToCreateOnObject: UniversalFlatFieldMetadata[];
|
||||
flatIndexMetadataToCreate: UniversalFlatIndexMetadata[];
|
||||
flatSearchFieldMetadataToCreate: UniversalFlatSearchFieldMetadata[];
|
||||
} => {
|
||||
const createObjectInput =
|
||||
trimAndRemoveDuplicatedWhitespacesFromObjectStringProperties(
|
||||
@@ -94,6 +97,7 @@ export const fromCreateObjectInputToFlatObjectMetadataAndFlatFieldMetadatasToCre
|
||||
fieldPermissionUniversalIdentifiers: [],
|
||||
viewUniversalIdentifiers: [],
|
||||
indexMetadataUniversalIdentifiers: [],
|
||||
searchFieldMetadataUniversalIdentifiers: [],
|
||||
labelIdentifierFieldMetadataUniversalIdentifier,
|
||||
imageIdentifierFieldMetadataUniversalIdentifier: null,
|
||||
};
|
||||
@@ -119,12 +123,20 @@ export const fromCreateObjectInputToFlatObjectMetadataAndFlatFieldMetadatasToCre
|
||||
flatObjectMetadata: universalFlatObjectMetadataToCreate,
|
||||
});
|
||||
|
||||
const defaultSearchFieldMetadatasForCustomObject =
|
||||
buildDefaultSearchFieldMetadatasForCustomObject({
|
||||
defaultFlatFieldForCustomObjectMaps,
|
||||
flatObjectMetadata: universalFlatObjectMetadataToCreate,
|
||||
});
|
||||
|
||||
return {
|
||||
flatObjectMetadataToCreate: universalFlatObjectMetadataToCreate,
|
||||
flatIndexMetadataToCreate: [
|
||||
...Object.values(defaultIndexesForCustomObject.indexes),
|
||||
...standardTargetFlatIndexMetadatas,
|
||||
],
|
||||
flatSearchFieldMetadataToCreate:
|
||||
defaultSearchFieldMetadatasForCustomObject.searchFieldMetadatas,
|
||||
relationTargetFlatFieldMetadataToCreate: standardTargetFlatFieldMetadatas,
|
||||
flatFieldMetadataToCreateOnObject: objectFlatFieldMetadatas,
|
||||
};
|
||||
|
||||
+7
@@ -77,6 +77,9 @@ export const fromObjectMetadataEntityToFlatObjectMetadata = ({
|
||||
updatedAt: objectMetadataEntity.updatedAt.toISOString(),
|
||||
viewIds: objectMetadataEntity.views.map(({ id }) => id),
|
||||
indexMetadataIds: objectMetadataEntity.indexMetadatas.map(({ id }) => id),
|
||||
searchFieldMetadataIds: objectMetadataEntity.searchFieldMetadatas.map(
|
||||
({ id }) => id,
|
||||
),
|
||||
fieldIds: objectMetadataEntity.fields.map(({ id }) => id),
|
||||
objectPermissionIds: objectMetadataEntity.objectPermissions.map(
|
||||
({ id }) => id,
|
||||
@@ -92,6 +95,10 @@ export const fromObjectMetadataEntityToFlatObjectMetadata = ({
|
||||
indexMetadataUniversalIdentifiers: objectMetadataEntity.indexMetadatas.map(
|
||||
({ universalIdentifier }) => universalIdentifier,
|
||||
),
|
||||
searchFieldMetadataUniversalIdentifiers:
|
||||
objectMetadataEntity.searchFieldMetadatas?.map(
|
||||
({ universalIdentifier }) => universalIdentifier,
|
||||
) ?? [],
|
||||
viewUniversalIdentifiers: objectMetadataEntity.views.map(
|
||||
({ universalIdentifier }) => universalIdentifier,
|
||||
),
|
||||
|
||||
+5
@@ -30,6 +30,7 @@ type FromUpdateObjectInputToFlatObjectMetadataArgs = {
|
||||
| 'flatFieldMetadataMaps'
|
||||
| 'flatViewFieldMaps'
|
||||
| 'flatViewMaps'
|
||||
| 'flatSearchFieldMetadataMaps'
|
||||
>;
|
||||
|
||||
export const fromUpdateObjectInputToFlatObjectMetadataAndRelatedFlatEntities =
|
||||
@@ -40,6 +41,7 @@ export const fromUpdateObjectInputToFlatObjectMetadataAndRelatedFlatEntities =
|
||||
flatFieldMetadataMaps,
|
||||
flatViewFieldMaps,
|
||||
flatViewMaps,
|
||||
flatSearchFieldMetadataMaps,
|
||||
}: FromUpdateObjectInputToFlatObjectMetadataArgs): FlatObjectMetadataUpdateSideEffects & {
|
||||
flatObjectMetadataToUpdate: UniversalFlatObjectMetadata;
|
||||
} => {
|
||||
@@ -101,6 +103,7 @@ export const fromUpdateObjectInputToFlatObjectMetadataAndRelatedFlatEntities =
|
||||
flatViewFieldsToUpdate,
|
||||
otherObjectFlatFieldMetadatasToUpdate,
|
||||
sameObjectFlatFieldMetadatasToUpdate,
|
||||
searchFieldMetadatasToCreate,
|
||||
} = handleFlatObjectMetadataUpdateSideEffect({
|
||||
fromFlatObjectMetadata: existingFlatObjectMetadata,
|
||||
toFlatObjectMetadata,
|
||||
@@ -109,6 +112,7 @@ export const fromUpdateObjectInputToFlatObjectMetadataAndRelatedFlatEntities =
|
||||
flatIndexMaps,
|
||||
flatViewFieldMaps,
|
||||
flatViewMaps,
|
||||
flatSearchFieldMetadataMaps,
|
||||
});
|
||||
|
||||
return {
|
||||
@@ -118,5 +122,6 @@ export const fromUpdateObjectInputToFlatObjectMetadataAndRelatedFlatEntities =
|
||||
flatViewFieldsToUpdate,
|
||||
otherObjectFlatFieldMetadatasToUpdate,
|
||||
sameObjectFlatFieldMetadatasToUpdate,
|
||||
searchFieldMetadatasToCreate,
|
||||
};
|
||||
};
|
||||
|
||||
+15
-19
@@ -5,11 +5,12 @@ import { type AllFlatEntityMaps } from 'src/engine/metadata-modules/flat-entity/
|
||||
import { type FlatFieldMetadata } from 'src/engine/metadata-modules/flat-field-metadata/types/flat-field-metadata.type';
|
||||
import { type FlatObjectMetadata } from 'src/engine/metadata-modules/flat-object-metadata/types/flat-object-metadata.type';
|
||||
import { recomputeIndexAfterFlatObjectMetadataSingularNameUpdate } from 'src/engine/metadata-modules/flat-object-metadata/utils/recompute-index-after-flat-object-metadata-singular-name-update.util';
|
||||
import { recomputeSearchVectorFieldAfterLabelIdentifierUpdate } from 'src/engine/metadata-modules/flat-object-metadata/utils/recompute-search-vector-field-after-label-identifier-update.util';
|
||||
import { recomputeSearchVectorOnLabelIdentifierUpdate } from 'src/engine/metadata-modules/flat-object-metadata/utils/recompute-search-vector-on-label-identifier-update.util';
|
||||
import { recomputeViewFieldIdentifierAfterFlatObjectIdentifierUpdate } from 'src/engine/metadata-modules/flat-object-metadata/utils/recompute-view-field-identifier-after-flat-object-identifier-update.util';
|
||||
import { renameRelatedMorphFieldOnObjectNamesUpdate } from 'src/engine/metadata-modules/flat-object-metadata/utils/rename-related-morph-field-on-object-names-update.util';
|
||||
import { type UniversalFlatFieldMetadata } from 'src/engine/workspace-manager/workspace-migration/universal-flat-entity/types/universal-flat-field-metadata.type';
|
||||
import { type UniversalFlatIndexMetadata } from 'src/engine/workspace-manager/workspace-migration/universal-flat-entity/types/universal-flat-index-metadata.type';
|
||||
import { type UniversalFlatSearchFieldMetadata } from 'src/engine/workspace-manager/workspace-migration/universal-flat-entity/types/universal-flat-search-field-metadata.type';
|
||||
import { type UniversalFlatViewField } from 'src/engine/workspace-manager/workspace-migration/universal-flat-entity/types/universal-flat-view-field.type';
|
||||
|
||||
export type FlatObjectMetadataUpdateSideEffects = {
|
||||
@@ -18,6 +19,7 @@ export type FlatObjectMetadataUpdateSideEffects = {
|
||||
flatViewFieldsToUpdate: UniversalFlatViewField[];
|
||||
flatViewFieldsToCreate: UniversalFlatViewField[];
|
||||
flatIndexMetadatasToUpdate: UniversalFlatIndexMetadata[];
|
||||
searchFieldMetadatasToCreate: UniversalFlatSearchFieldMetadata[];
|
||||
};
|
||||
|
||||
type HandleFlatObjectMetadataUpdateSideEffectArgs = FromTo<
|
||||
@@ -31,6 +33,7 @@ type HandleFlatObjectMetadataUpdateSideEffectArgs = FromTo<
|
||||
| 'flatViewFieldMaps'
|
||||
| 'flatIndexMaps'
|
||||
| 'flatViewMaps'
|
||||
| 'flatSearchFieldMetadataMaps'
|
||||
>;
|
||||
|
||||
export const handleFlatObjectMetadataUpdateSideEffect = ({
|
||||
@@ -39,6 +42,7 @@ export const handleFlatObjectMetadataUpdateSideEffect = ({
|
||||
flatObjectMetadataMaps,
|
||||
flatViewFieldMaps,
|
||||
flatViewMaps,
|
||||
flatSearchFieldMetadataMaps,
|
||||
fromFlatObjectMetadata,
|
||||
toFlatObjectMetadata,
|
||||
}: HandleFlatObjectMetadataUpdateSideEffectArgs): FlatObjectMetadataUpdateSideEffects => {
|
||||
@@ -87,25 +91,16 @@ export const handleFlatObjectMetadataUpdateSideEffect = ({
|
||||
|
||||
const sameObjectFlatFieldMetadatasToUpdate: FlatFieldMetadata[] = [];
|
||||
|
||||
if (
|
||||
toFlatObjectMetadata.isSearchable &&
|
||||
isDefined(
|
||||
toFlatObjectMetadata.labelIdentifierFieldMetadataUniversalIdentifier,
|
||||
) &&
|
||||
fromFlatObjectMetadata.labelIdentifierFieldMetadataUniversalIdentifier !==
|
||||
toFlatObjectMetadata.labelIdentifierFieldMetadataUniversalIdentifier
|
||||
) {
|
||||
const updatedSearchVectorField =
|
||||
recomputeSearchVectorFieldAfterLabelIdentifierUpdate({
|
||||
existingFlatObjectMetadata: fromFlatObjectMetadata,
|
||||
flatFieldMetadataMaps,
|
||||
labelIdentifierFieldMetadataUniversalIdentifier:
|
||||
toFlatObjectMetadata.labelIdentifierFieldMetadataUniversalIdentifier,
|
||||
});
|
||||
const { flatSearchVectorFieldToUpdate, searchFieldMetadatasToCreate } =
|
||||
recomputeSearchVectorOnLabelIdentifierUpdate({
|
||||
fromFlatObjectMetadata,
|
||||
toFlatObjectMetadata,
|
||||
flatFieldMetadataMaps,
|
||||
flatSearchFieldMetadataMaps,
|
||||
});
|
||||
|
||||
if (isDefined(updatedSearchVectorField)) {
|
||||
sameObjectFlatFieldMetadatasToUpdate.push(updatedSearchVectorField);
|
||||
}
|
||||
if (isDefined(flatSearchVectorFieldToUpdate)) {
|
||||
sameObjectFlatFieldMetadatasToUpdate.push(flatSearchVectorFieldToUpdate);
|
||||
}
|
||||
|
||||
return {
|
||||
@@ -117,5 +112,6 @@ export const handleFlatObjectMetadataUpdateSideEffect = ({
|
||||
flatViewFieldsToUpdate,
|
||||
otherObjectFlatFieldMetadatasToUpdate: morphFlatFieldMetadatasToUpdate,
|
||||
sameObjectFlatFieldMetadatasToUpdate,
|
||||
searchFieldMetadatasToCreate,
|
||||
};
|
||||
};
|
||||
|
||||
-77
@@ -1,77 +0,0 @@
|
||||
import { type FieldMetadataType } from 'twenty-shared/types';
|
||||
import {
|
||||
findOrThrow,
|
||||
isDefined,
|
||||
type SearchableFieldType,
|
||||
} from 'twenty-shared/utils';
|
||||
|
||||
import { type AllFlatEntityMaps } from 'src/engine/metadata-modules/flat-entity/types/all-flat-entity-maps.type';
|
||||
import { findFlatEntityByUniversalIdentifier } from 'src/engine/metadata-modules/flat-entity/utils/find-flat-entity-by-universal-identifier.util';
|
||||
import { findManyFlatEntityByIdInFlatEntityMapsOrThrow } from 'src/engine/metadata-modules/flat-entity/utils/find-many-flat-entity-by-id-in-flat-entity-maps-or-throw.util';
|
||||
import { type FlatFieldMetadata } from 'src/engine/metadata-modules/flat-field-metadata/types/flat-field-metadata.type';
|
||||
import { type FlatObjectMetadata } from 'src/engine/metadata-modules/flat-object-metadata/types/flat-object-metadata.type';
|
||||
import {
|
||||
ObjectMetadataException,
|
||||
ObjectMetadataExceptionCode,
|
||||
} from 'src/engine/metadata-modules/object-metadata/object-metadata.exception';
|
||||
import { SEARCH_VECTOR_FIELD } from 'src/engine/metadata-modules/search-field-metadata/constants/search-vector-field.constants';
|
||||
import { getTsVectorColumnExpressionFromFields } from 'src/engine/workspace-manager/utils/get-ts-vector-column-expression.util';
|
||||
|
||||
type RecomputeSearchVectorFieldAfterLabelIdentifierUpdateArgs = {
|
||||
existingFlatObjectMetadata: FlatObjectMetadata;
|
||||
labelIdentifierFieldMetadataUniversalIdentifier: string;
|
||||
} & Pick<AllFlatEntityMaps, 'flatFieldMetadataMaps'>;
|
||||
|
||||
export const recomputeSearchVectorFieldAfterLabelIdentifierUpdate = ({
|
||||
existingFlatObjectMetadata,
|
||||
flatFieldMetadataMaps,
|
||||
labelIdentifierFieldMetadataUniversalIdentifier,
|
||||
}: RecomputeSearchVectorFieldAfterLabelIdentifierUpdateArgs):
|
||||
| FlatFieldMetadata<FieldMetadataType.TS_VECTOR>
|
||||
| undefined => {
|
||||
const objectFlatFieldMetadatas =
|
||||
findManyFlatEntityByIdInFlatEntityMapsOrThrow({
|
||||
flatEntityMaps: flatFieldMetadataMaps,
|
||||
flatEntityIds: existingFlatObjectMetadata.fieldIds,
|
||||
});
|
||||
|
||||
const searchVectorField = findOrThrow(
|
||||
objectFlatFieldMetadatas,
|
||||
(field) => field.name === SEARCH_VECTOR_FIELD.name,
|
||||
) as FlatFieldMetadata<FieldMetadataType.TS_VECTOR>;
|
||||
|
||||
const newLabelIdentifierField = findFlatEntityByUniversalIdentifier({
|
||||
flatEntityMaps: flatFieldMetadataMaps,
|
||||
universalIdentifier: labelIdentifierFieldMetadataUniversalIdentifier,
|
||||
});
|
||||
|
||||
if (!isDefined(newLabelIdentifierField)) {
|
||||
throw new ObjectMetadataException(
|
||||
`New label identifier field not found for object metadata`,
|
||||
ObjectMetadataExceptionCode.INVALID_OBJECT_INPUT,
|
||||
);
|
||||
}
|
||||
|
||||
try {
|
||||
const newAsExpression = getTsVectorColumnExpressionFromFields([
|
||||
{
|
||||
name: newLabelIdentifierField.name,
|
||||
type: newLabelIdentifierField.type as SearchableFieldType,
|
||||
},
|
||||
]);
|
||||
|
||||
return {
|
||||
...searchVectorField,
|
||||
universalSettings: {
|
||||
...searchVectorField.universalSettings,
|
||||
asExpression: newAsExpression,
|
||||
generatedType: 'STORED',
|
||||
},
|
||||
};
|
||||
} catch {
|
||||
throw new ObjectMetadataException(
|
||||
`Failed to compute search vector column expression for field ${newLabelIdentifierField.name}`,
|
||||
ObjectMetadataExceptionCode.INVALID_OBJECT_INPUT,
|
||||
);
|
||||
}
|
||||
};
|
||||
+112
@@ -0,0 +1,112 @@
|
||||
import { FieldMetadataType } from 'twenty-shared/types';
|
||||
import { isDefined } from 'twenty-shared/utils';
|
||||
|
||||
import { type AllFlatEntityMaps } from 'src/engine/metadata-modules/flat-entity/types/all-flat-entity-maps.type';
|
||||
import { findFlatEntityByIdInFlatEntityMaps } from 'src/engine/metadata-modules/flat-entity/utils/find-flat-entity-by-id-in-flat-entity-maps.util';
|
||||
import { findManyFlatEntityByIdInFlatEntityMapsOrThrow } from 'src/engine/metadata-modules/flat-entity/utils/find-many-flat-entity-by-id-in-flat-entity-maps-or-throw.util';
|
||||
import { type FlatFieldMetadata } from 'src/engine/metadata-modules/flat-field-metadata/types/flat-field-metadata.type';
|
||||
import { isFlatFieldMetadataOfType } from 'src/engine/metadata-modules/flat-field-metadata/utils/is-flat-field-metadata-of-type.util';
|
||||
import { type FlatObjectMetadata } from 'src/engine/metadata-modules/flat-object-metadata/types/flat-object-metadata.type';
|
||||
import { type FlatSearchFieldMetadata } from 'src/engine/metadata-modules/flat-search-field-metadata/types/flat-search-field-metadata.type';
|
||||
import {
|
||||
ObjectMetadataException,
|
||||
ObjectMetadataExceptionCode,
|
||||
} from 'src/engine/metadata-modules/object-metadata/object-metadata.exception';
|
||||
import {
|
||||
buildSearchVectorTargetField,
|
||||
computeSearchVectorAsExpressionFromSearchFieldMetadatas,
|
||||
} from 'src/engine/metadata-modules/flat-search-field-metadata/utils/compute-search-vector-as-expression-from-search-field-metadatas.util';
|
||||
import { SEARCH_VECTOR_FIELD } from 'src/engine/metadata-modules/search-field-metadata/constants/search-vector-field.constants';
|
||||
|
||||
type RecomputeSearchVectorFieldFromSearchFieldMetadatasArgs = {
|
||||
flatObjectMetadata: FlatObjectMetadata;
|
||||
// Post-change searchFieldMetadata rows for the object, carrying the field id and the
|
||||
// per-object position that drives the deterministic asExpression order.
|
||||
searchFieldMetadatas: Pick<
|
||||
FlatSearchFieldMetadata,
|
||||
'fieldMetadataId' | 'position' | 'universalIdentifier'
|
||||
>[];
|
||||
// Takes precedence over the flat maps when resolving an id — for a field created or
|
||||
// renamed in the same migration, whose maps entry is absent or stale.
|
||||
overrideFlatFieldMetadataById?: Map<string, FlatFieldMetadata>;
|
||||
} & Pick<AllFlatEntityMaps, 'flatFieldMetadataMaps'>;
|
||||
|
||||
export const recomputeSearchVectorFieldFromSearchFieldMetadatas = ({
|
||||
flatObjectMetadata,
|
||||
flatFieldMetadataMaps,
|
||||
searchFieldMetadatas,
|
||||
overrideFlatFieldMetadataById,
|
||||
}: RecomputeSearchVectorFieldFromSearchFieldMetadatasArgs):
|
||||
| FlatFieldMetadata<FieldMetadataType.TS_VECTOR>
|
||||
| undefined => {
|
||||
const objectFlatFieldMetadatas =
|
||||
findManyFlatEntityByIdInFlatEntityMapsOrThrow({
|
||||
flatEntityMaps: flatFieldMetadataMaps,
|
||||
flatEntityIds: flatObjectMetadata.fieldIds,
|
||||
});
|
||||
|
||||
const searchVectorField = objectFlatFieldMetadatas.find(
|
||||
(field) => field.name === SEARCH_VECTOR_FIELD.name,
|
||||
);
|
||||
|
||||
if (
|
||||
!isDefined(searchVectorField) ||
|
||||
!isFlatFieldMetadataOfType(searchVectorField, FieldMetadataType.TS_VECTOR)
|
||||
) {
|
||||
throw new ObjectMetadataException(
|
||||
`Search vector field not found for object metadata ${flatObjectMetadata.id}`,
|
||||
ObjectMetadataExceptionCode.INVALID_OBJECT_INPUT,
|
||||
);
|
||||
}
|
||||
|
||||
const targetSearchableFields = searchFieldMetadatas.map(
|
||||
(searchFieldMetadata) => {
|
||||
const { fieldMetadataId, position, universalIdentifier } =
|
||||
searchFieldMetadata;
|
||||
|
||||
const overriddenFlatFieldMetadata =
|
||||
overrideFlatFieldMetadataById?.get(fieldMetadataId);
|
||||
|
||||
const flatFieldMetadata =
|
||||
overriddenFlatFieldMetadata ??
|
||||
findFlatEntityByIdInFlatEntityMaps({
|
||||
flatEntityId: fieldMetadataId,
|
||||
flatEntityMaps: flatFieldMetadataMaps,
|
||||
});
|
||||
|
||||
if (!isDefined(flatFieldMetadata)) {
|
||||
throw new ObjectMetadataException(
|
||||
`Field metadata ${fieldMetadataId} referenced by searchFieldMetadata not found for object metadata ${flatObjectMetadata.id}`,
|
||||
ObjectMetadataExceptionCode.INVALID_OBJECT_INPUT,
|
||||
);
|
||||
}
|
||||
|
||||
return buildSearchVectorTargetField(
|
||||
flatFieldMetadata,
|
||||
position,
|
||||
universalIdentifier,
|
||||
);
|
||||
},
|
||||
);
|
||||
|
||||
try {
|
||||
const newAsExpression =
|
||||
computeSearchVectorAsExpressionFromSearchFieldMetadatas(
|
||||
targetSearchableFields,
|
||||
);
|
||||
|
||||
return {
|
||||
...searchVectorField,
|
||||
universalSettings: {
|
||||
...searchVectorField.universalSettings,
|
||||
asExpression: newAsExpression,
|
||||
generatedType: 'STORED',
|
||||
},
|
||||
};
|
||||
} catch (error) {
|
||||
throw new ObjectMetadataException(
|
||||
`Failed to compute search vector column expression for object metadata ${flatObjectMetadata.id}: ${error instanceof Error ? error.message : String(error)}`,
|
||||
ObjectMetadataExceptionCode.INVALID_OBJECT_INPUT,
|
||||
);
|
||||
}
|
||||
};
|
||||
+113
@@ -0,0 +1,113 @@
|
||||
import { type FieldMetadataType, type FromTo } from 'twenty-shared/types';
|
||||
import { isDefined, isSearchableFieldType } from 'twenty-shared/utils';
|
||||
|
||||
import { type AllFlatEntityMaps } from 'src/engine/metadata-modules/flat-entity/types/all-flat-entity-maps.type';
|
||||
import { findFlatEntityByIdInFlatEntityMaps } from 'src/engine/metadata-modules/flat-entity/utils/find-flat-entity-by-id-in-flat-entity-maps.util';
|
||||
import { findManyFlatEntityByIdInFlatEntityMapsOrThrow } from 'src/engine/metadata-modules/flat-entity/utils/find-many-flat-entity-by-id-in-flat-entity-maps-or-throw.util';
|
||||
import { type FlatFieldMetadata } from 'src/engine/metadata-modules/flat-field-metadata/types/flat-field-metadata.type';
|
||||
import { type FlatObjectMetadata } from 'src/engine/metadata-modules/flat-object-metadata/types/flat-object-metadata.type';
|
||||
import { recomputeSearchVectorFieldFromSearchFieldMetadatas } from 'src/engine/metadata-modules/flat-object-metadata/utils/recompute-search-vector-field-from-search-field-metadatas.util';
|
||||
import { type FlatSearchFieldMetadata } from 'src/engine/metadata-modules/flat-search-field-metadata/types/flat-search-field-metadata.type';
|
||||
import { buildFlatSearchFieldMetadataForField } from 'src/engine/metadata-modules/flat-search-field-metadata/utils/build-flat-search-field-metadata-for-field.util';
|
||||
import { type UniversalFlatSearchFieldMetadata } from 'src/engine/workspace-manager/workspace-migration/universal-flat-entity/types/universal-flat-search-field-metadata.type';
|
||||
|
||||
export type LabelIdentifierUpdateSearchVectorSideEffect = {
|
||||
flatSearchVectorFieldToUpdate:
|
||||
| FlatFieldMetadata<FieldMetadataType.TS_VECTOR>
|
||||
| undefined;
|
||||
searchFieldMetadatasToCreate: UniversalFlatSearchFieldMetadata[];
|
||||
};
|
||||
|
||||
type RecomputeSearchVectorOnLabelIdentifierUpdateArgs = FromTo<
|
||||
FlatObjectMetadata,
|
||||
'flatObjectMetadata'
|
||||
> &
|
||||
Pick<
|
||||
AllFlatEntityMaps,
|
||||
'flatFieldMetadataMaps' | 'flatSearchFieldMetadataMaps'
|
||||
>;
|
||||
|
||||
const EMPTY_SIDE_EFFECT: LabelIdentifierUpdateSearchVectorSideEffect = {
|
||||
flatSearchVectorFieldToUpdate: undefined,
|
||||
searchFieldMetadatasToCreate: [],
|
||||
};
|
||||
|
||||
// Relabeling is additive: it indexes the new label identifier field while preserving the
|
||||
// existing search surface (e.g. the provisioned name row). We never drop a row here so the
|
||||
// previous label identifier stays searchable. Removal of fields from the search surface is
|
||||
// deferred to a follow-up PR.
|
||||
export const recomputeSearchVectorOnLabelIdentifierUpdate = ({
|
||||
fromFlatObjectMetadata,
|
||||
toFlatObjectMetadata,
|
||||
flatFieldMetadataMaps,
|
||||
flatSearchFieldMetadataMaps,
|
||||
}: RecomputeSearchVectorOnLabelIdentifierUpdateArgs): LabelIdentifierUpdateSearchVectorSideEffect => {
|
||||
const newLabelIdentifierFieldMetadataId =
|
||||
toFlatObjectMetadata.labelIdentifierFieldMetadataId;
|
||||
|
||||
if (
|
||||
!toFlatObjectMetadata.isSearchable ||
|
||||
!isDefined(newLabelIdentifierFieldMetadataId) ||
|
||||
fromFlatObjectMetadata.labelIdentifierFieldMetadataId ===
|
||||
newLabelIdentifierFieldMetadataId
|
||||
) {
|
||||
return EMPTY_SIDE_EFFECT;
|
||||
}
|
||||
|
||||
const existingSearchFieldMetadatas =
|
||||
findManyFlatEntityByIdInFlatEntityMapsOrThrow<FlatSearchFieldMetadata>({
|
||||
flatEntityMaps: flatSearchFieldMetadataMaps,
|
||||
flatEntityIds: fromFlatObjectMetadata.searchFieldMetadataIds,
|
||||
});
|
||||
|
||||
const newLabelIdentifierAlreadyIndexed = existingSearchFieldMetadatas.some(
|
||||
(searchFieldMetadata) =>
|
||||
searchFieldMetadata.fieldMetadataId === newLabelIdentifierFieldMetadataId,
|
||||
);
|
||||
|
||||
const newLabelIdentifierField = findFlatEntityByIdInFlatEntityMaps({
|
||||
flatEntityMaps: flatFieldMetadataMaps,
|
||||
flatEntityId: newLabelIdentifierFieldMetadataId,
|
||||
});
|
||||
|
||||
const shouldIndexNewLabelIdentifier =
|
||||
isDefined(newLabelIdentifierField) &&
|
||||
isSearchableFieldType(newLabelIdentifierField.type) &&
|
||||
!newLabelIdentifierAlreadyIndexed;
|
||||
|
||||
if (!shouldIndexNewLabelIdentifier) {
|
||||
return EMPTY_SIDE_EFFECT;
|
||||
}
|
||||
|
||||
const newLabelIdentifierPosition =
|
||||
existingSearchFieldMetadatas.reduce(
|
||||
(maxPosition, searchFieldMetadata) =>
|
||||
Math.max(maxPosition, searchFieldMetadata.position),
|
||||
-1,
|
||||
) + 1;
|
||||
|
||||
const newSearchFieldMetadata = buildFlatSearchFieldMetadataForField({
|
||||
flatObjectMetadata: fromFlatObjectMetadata,
|
||||
flatFieldMetadata: newLabelIdentifierField,
|
||||
position: newLabelIdentifierPosition,
|
||||
});
|
||||
|
||||
const nextSearchFieldMetadatas = [
|
||||
...existingSearchFieldMetadatas,
|
||||
{
|
||||
fieldMetadataId: newLabelIdentifierFieldMetadataId,
|
||||
position: newLabelIdentifierPosition,
|
||||
universalIdentifier: newSearchFieldMetadata.universalIdentifier,
|
||||
},
|
||||
];
|
||||
|
||||
return {
|
||||
flatSearchVectorFieldToUpdate:
|
||||
recomputeSearchVectorFieldFromSearchFieldMetadatas({
|
||||
flatObjectMetadata: fromFlatObjectMetadata,
|
||||
flatFieldMetadataMaps,
|
||||
searchFieldMetadatas: nextSearchFieldMetadatas,
|
||||
}),
|
||||
searchFieldMetadatasToCreate: [newSearchFieldMetadata],
|
||||
};
|
||||
};
|
||||
Reference in New Issue
Block a user