Wire up search field metadata (#21964)

## Part 1 - Exact scope of the current PR (#21964)

close https://github.com/twentyhq/core-team-issues/issues/2586

This PR introduces `searchFieldMetadata` as a first-class flat metadata
entity and migrates the existing search surface onto it, with **no
change to which records are searchable** (ISO with `main`).

In scope (what the PR does):
- New flat entity `searchFieldMetadata` (universalIdentifier,
applicationId, **`position`**, maps, conversions), registered in the
central flat-entity constants and the migration build orchestrator.
- `searchVector.asExpression` is **derived server-side** from
`searchFieldMetadata` rows (validated by `isSafeTsVectorExpression`);
never trusted from client input.
- **Derivation order is deterministic, driven by each row's `position`**
([compute-search-vector-as-expression-from-search-field-metadatas.util.ts](packages/twenty-server/src/engine/metadata-modules/flat-search-field-metadata/utils/compute-search-vector-as-expression-from-search-field-metadatas.util.ts)),
replacing the previous non-deterministic `(createdAt, id)` sort. That
sort collapsed to random UUIDs for standard fields (same `createdAt`),
so any rename/relabel rewrote the `STORED` generated column to a
logically-identical-but-textually-different expression and produced a
permanent per-workspace diff vs the standard definition. Ordering now
equals provisioning order; ties break on `universalIdentifier`.
- Provisioning at object creation mirrors the existing surface exactly
**and seeds `position`**:
- custom objects -> the `name` field only, at `position: 0`
([build-default-search-field-metadatas-for-custom-object.util.ts](packages/twenty-server/src/engine/metadata-modules/object-metadata/utils/build-default-search-field-metadatas-for-custom-object.util.ts))
- standard objects -> their curated `SEARCH_FIELDS_FOR_*` sets,
`position` = the curated index
- Backfill (instance + workspace commands in `2-16`) provisions rows for
existing workspaces with the same surface **and the same positions**
(standard from the curated standard maps, custom `name` = `0`), scoped
to the workspace's own custom application
([build-search-field-metadata-backfill-operations.util.ts](packages/twenty-server/src/database/commands/upgrade-version-command/2-16/utils/build-search-field-metadata-backfill-operations.util.ts)).
The `position` column is added in the same `2-16` fast instance command
as `universalIdentifier`/`applicationId`.
- Field rename of an already-indexed field recomputes `asExpression`
(positions preserved, so order is stable)
([recompute-search-vector-on-field-rename.util.ts](packages/twenty-server/src/engine/metadata-modules/flat-field-metadata/utils/recompute-search-vector-on-field-rename.util.ts)).
- Field delete drops the matching row(s) and recomputes; remaining rows
keep their relative order (no renumber)
([from-delete-field-input-to-flat-field-metadatas-to-delete.util.ts](packages/twenty-server/src/engine/metadata-modules/flat-field-metadata/utils/from-delete-field-input-to-flat-field-metadatas-to-delete.util.ts)).
- Object relabel is **additive** and ISO/regression-fix only: it indexes
the new label identifier **appended last (`position = max(existing) +
1`)** without dropping `name`
([recompute-search-vector-on-label-identifier-update.util.ts](packages/twenty-server/src/engine/metadata-modules/flat-object-metadata/utils/recompute-search-vector-on-label-identifier-update.util.ts)).
This is a deliberate, temporary bridge.

Explicitly OUT of scope (deferred):
- No API to edit `searchFieldMetadata` (no user-facing search-field
configuration, including `position` — it is internal and only written by
provisioning/backfill/recompute).
- No auto-indexing of arbitrary searchable fields. Creating a custom
TEXT/EMAILS/etc. field does NOT add it to search (the
`computeSearchFieldMetadataCreationForFields` behavior was removed in
`e6820ad`).
- No field-type-transition handling (field type is immutable - not in
`FLAT_FIELD_METADATA_EDITABLE_PROPERTIES`, so that path was dead code).
- No `position` validation (uniqueness/range) and no multi-vector /
per-field `weight` config — deferred to the configurable-search
follow-up (#1428).

Net: `searchFieldMetadata` becomes the source of truth for the *same*
surface as `main`. The only intentional divergences from `main` are
"relabel preserves `name`" (additive) and the deterministic
`position`-ordered `asExpression` (a correctness/perf fix that is
byte-identical to provisioning order, so it does not change the
searchable surface).

---------

Co-authored-by: Félix Malfait <felix@twenty.com>
This commit is contained in:
Paul Rastoin
2026-06-23 16:27:13 +02:00
committed by GitHub
parent 99e7c2cde0
commit e9d5d71cd3
102 changed files with 3850 additions and 366 deletions
@@ -95,6 +95,7 @@ export class FieldMetadataService extends TypeOrmQueryService<FieldMetadataEntit
flatIndexMaps: existingFlatIndexMaps,
flatFieldMetadataMaps: existingFlatFieldMetadataMaps,
flatPageLayoutWidgetMaps: existingFlatPageLayoutWidgetMaps,
flatSearchFieldMetadataMaps: existingFlatSearchFieldMetadataMaps,
} = await this.flatEntityMapsCacheService.getOrRecomputeManyOrAllFlatEntityMaps(
{
workspaceId,
@@ -103,6 +104,7 @@ export class FieldMetadataService extends TypeOrmQueryService<FieldMetadataEntit
'flatIndexMaps',
'flatFieldMetadataMaps',
'flatPageLayoutWidgetMaps',
'flatSearchFieldMetadataMaps',
],
},
);
@@ -111,11 +113,14 @@ export class FieldMetadataService extends TypeOrmQueryService<FieldMetadataEntit
flatFieldMetadatasToDelete,
flatIndexesToDelete,
flatIndexesToUpdate,
searchFieldMetadatasToDelete,
flatSearchVectorFieldsToUpdate,
} = fromDeleteFieldInputToFlatFieldMetadatasToDelete({
deleteOneFieldInput,
flatFieldMetadataMaps: existingFlatFieldMetadataMaps,
flatIndexMaps: existingFlatIndexMaps,
flatObjectMetadataMaps: existingFlatObjectMetadataMaps,
flatSearchFieldMetadataMaps: existingFlatSearchFieldMetadataMaps,
});
const deletedFlatFieldMetadata = findFlatEntityByUniversalIdentifierOrThrow(
@@ -157,13 +162,18 @@ export class FieldMetadataService extends TypeOrmQueryService<FieldMetadataEntit
fieldMetadata: {
flatEntityToCreate: [],
flatEntityToDelete: flatFieldMetadatasToDelete,
flatEntityToUpdate: [],
flatEntityToUpdate: flatSearchVectorFieldsToUpdate,
},
index: {
flatEntityToCreate: [],
flatEntityToDelete: flatIndexesToDelete,
flatEntityToUpdate: flatIndexesToUpdate,
},
searchFieldMetadata: {
flatEntityToCreate: [],
flatEntityToDelete: searchFieldMetadatasToDelete,
flatEntityToUpdate: [],
},
...(flatPageLayoutWidgetsToDelete.length > 0
? {
pageLayoutWidget: {
@@ -218,6 +228,7 @@ export class FieldMetadataService extends TypeOrmQueryService<FieldMetadataEntit
flatViewGroupMaps: existingFlatViewGroupMaps,
flatViewMaps: existingFlatViewMaps,
flatViewFieldMaps: existingFlatViewFieldMaps,
flatSearchFieldMetadataMaps: existingFlatSearchFieldMetadataMaps,
} = await this.flatEntityMapsCacheService.getOrRecomputeManyOrAllFlatEntityMaps(
{
workspaceId,
@@ -229,6 +240,7 @@ export class FieldMetadataService extends TypeOrmQueryService<FieldMetadataEntit
'flatViewGroupMaps',
'flatViewMaps',
'flatViewFieldMaps',
'flatSearchFieldMetadataMaps',
],
},
);
@@ -242,6 +254,7 @@ export class FieldMetadataService extends TypeOrmQueryService<FieldMetadataEntit
flatViewGroupMaps: existingFlatViewGroupMaps,
flatViewMaps: existingFlatViewMaps,
flatViewFieldMaps: existingFlatViewFieldMaps,
flatSearchFieldMetadataMaps: existingFlatSearchFieldMetadataMaps,
flatApplication: resolvedOwnerFlatApplication,
isSystemBuild,
});
@@ -303,6 +303,12 @@ exports[`ALL_UNIVERSAL_FLAT_ENTITY_PROPERTIES_TO_COMPARE_AND_STRINGIFY should ma
],
"propertiesToStringify": [],
},
"searchFieldMetadata": {
"propertiesToCompare": [
"position",
],
"propertiesToStringify": [],
},
"skill": {
"propertiesToCompare": [
"name",
@@ -1769,6 +1769,33 @@ export const ALL_ENTITY_PROPERTIES_CONFIGURATION_BY_METADATA_NAME = {
universalProperty: undefined,
},
},
searchFieldMetadata: {
objectMetadataId: {
toCompare: false,
toStringify: false,
universalProperty: 'objectMetadataUniversalIdentifier',
},
fieldMetadataId: {
toCompare: false,
toStringify: false,
universalProperty: 'fieldMetadataUniversalIdentifier',
},
position: {
toCompare: true,
toStringify: false,
universalProperty: undefined,
},
createdAt: {
toCompare: false,
toStringify: false,
universalProperty: undefined,
},
updatedAt: {
toCompare: false,
toStringify: false,
universalProperty: undefined,
},
},
} as const satisfies {
[P in AllMetadataName]: MetadataEntityPropertyConfiguration<P>;
};
@@ -303,6 +303,16 @@ export const ALL_MANY_TO_ONE_METADATA_FOREIGN_KEY = {
workspace: null,
application: null,
},
searchFieldMetadata: {
workspace: null,
application: null,
objectMetadata: {
foreignKey: 'objectMetadataId',
},
fieldMetadata: {
foreignKey: 'fieldMetadataId',
},
},
} as const satisfies ManyToOneMetadataRelationsProperties;
// satisfies with complex mapped types involving nested generics doesn't always catch missing required keys
@@ -531,6 +531,24 @@ export const ALL_MANY_TO_ONE_METADATA_RELATIONS = {
workspace: null,
application: null,
},
searchFieldMetadata: {
workspace: null,
application: null,
objectMetadata: {
metadataName: 'objectMetadata',
foreignKey: 'objectMetadataId',
inverseOneToManyProperty: 'searchFieldMetadatas',
isNullable: false,
universalForeignKey: 'objectMetadataUniversalIdentifier',
},
fieldMetadata: {
metadataName: 'fieldMetadata',
foreignKey: 'fieldMetadataId',
inverseOneToManyProperty: null,
isNullable: false,
universalForeignKey: 'fieldMetadataUniversalIdentifier',
},
},
} as const satisfies ManyToOneMetadataRelationsProperties;
// satisfies with complex mapped types involving nested generics doesn't always catch missing required keys
@@ -31,6 +31,7 @@ import { ViewFilterEntity } from 'src/engine/metadata-modules/view-filter/entiti
import { ViewGroupEntity } from 'src/engine/metadata-modules/view-group/entities/view-group.entity';
import { ViewEntity } from 'src/engine/metadata-modules/view/entities/view.entity';
import { ViewSortEntity } from 'src/engine/metadata-modules/view-sort/entities/view-sort.entity';
import { SearchFieldMetadataEntity } from 'src/engine/metadata-modules/search-field-metadata/search-field-metadata.entity';
export const ALL_METADATA_ENTITY_BY_METADATA_NAME = {
viewField: ViewFieldEntity,
@@ -63,4 +64,5 @@ export const ALL_METADATA_ENTITY_BY_METADATA_NAME = {
applicationVariable: ApplicationVariableEntity,
viewSort: ViewSortEntity,
connectionProvider: ConnectionProviderEntity,
searchFieldMetadata: SearchFieldMetadataEntity,
} as const satisfies Record<AllMetadataName, EntityTarget<ObjectLiteral>>;
@@ -118,4 +118,8 @@ export const ALL_METADATA_REQUIRED_METADATA_FOR_VALIDATION = {
webhook: {},
applicationVariable: {},
connectionProvider: {},
searchFieldMetadata: {
objectMetadata: true,
fieldMetadata: true,
},
} as const satisfies MetadataRequiredForValidation;
@@ -53,6 +53,7 @@ export const ALL_METADATA_SERIALIZED_RELATION = {
webhook: {},
applicationVariable: {},
connectionProvider: {},
searchFieldMetadata: {},
} as const satisfies MetadataSerializedRelationProperties;
// satisfies with complex mapped types involving nested generics doesn't always catch missing required keys
@@ -107,6 +107,12 @@ export const ALL_ONE_TO_MANY_METADATA_RELATIONS = {
flatEntityForeignKeyAggregator: 'viewIds',
universalFlatEntityForeignKeyAggregator: 'viewUniversalIdentifiers',
},
searchFieldMetadatas: {
metadataName: 'searchFieldMetadata',
flatEntityForeignKeyAggregator: 'searchFieldMetadataIds',
universalFlatEntityForeignKeyAggregator:
'searchFieldMetadataUniversalIdentifiers',
},
},
view: {
viewFields: {
@@ -254,6 +260,7 @@ export const ALL_ONE_TO_MANY_METADATA_RELATIONS = {
applicationVariable: {},
viewSort: {},
connectionProvider: {},
searchFieldMetadata: {},
} as const satisfies OneToManyMetadataRelationsProperties;
// satisfies with complex mapped types involving nested generics doesn't always catch missing required keys
@@ -21,6 +21,7 @@ import { WorkspaceFlatViewFieldMapCacheService } from 'src/engine/metadata-modul
import { WorkspaceFlatViewFilterGroupMapCacheService } from 'src/engine/metadata-modules/flat-view-filter-group/services/workspace-flat-view-filter-group-map-cache.service';
import { WorkspaceFlatViewFilterMapCacheService } from 'src/engine/metadata-modules/flat-view-filter/services/workspace-flat-view-filter-map-cache.service';
import { WorkspaceFlatViewGroupMapCacheService } from 'src/engine/metadata-modules/flat-view-group/services/workspace-flat-view-group-map-cache.service';
import { WorkspaceFlatSearchFieldMetadataMapCacheService } from 'src/engine/metadata-modules/flat-search-field-metadata/services/workspace-flat-search-field-metadata-map-cache.service';
import { WorkspaceFlatViewSortMapCacheService } from 'src/engine/metadata-modules/flat-view-sort/services/workspace-flat-view-sort-map-cache.service';
import { WorkspaceFlatViewMapCacheService } from 'src/engine/metadata-modules/flat-view/services/workspace-flat-view-map-cache.service';
import { FrontComponentEntity } from 'src/engine/metadata-modules/front-component/entities/front-component.entity';
@@ -37,6 +38,7 @@ import { RolePermissionFlagEntity } from 'src/engine/metadata-modules/role-permi
import { RoleEntity } from 'src/engine/metadata-modules/role/role.entity';
import { RowLevelPermissionPredicateGroupEntity } from 'src/engine/metadata-modules/row-level-permission-predicate/entities/row-level-permission-predicate-group.entity';
import { RowLevelPermissionPredicateEntity } from 'src/engine/metadata-modules/row-level-permission-predicate/entities/row-level-permission-predicate.entity';
import { SearchFieldMetadataEntity } from 'src/engine/metadata-modules/search-field-metadata/search-field-metadata.entity';
import { ViewFieldGroupEntity } from 'src/engine/metadata-modules/view-field-group/entities/view-field-group.entity';
import { ViewFieldEntity } from 'src/engine/metadata-modules/view-field/entities/view-field.entity';
import { ViewFilterGroupEntity } from 'src/engine/metadata-modules/view-filter-group/entities/view-filter-group.entity';
@@ -74,6 +76,7 @@ import { WorkspaceCacheModule } from 'src/engine/workspace-cache/workspace-cache
ApplicationEntity,
RoleEntity,
FrontComponentEntity,
SearchFieldMetadataEntity,
]),
],
providers: [
@@ -92,12 +95,14 @@ import { WorkspaceCacheModule } from 'src/engine/workspace-cache/workspace-cache
WorkspaceFlatRolePermissionFlagMapCacheService,
WorkspaceFlatPermissionFlagMapCacheService,
WorkspaceFlatViewSortMapCacheService,
WorkspaceFlatSearchFieldMetadataMapCacheService,
WorkspaceFlatPageLayoutMapCacheService,
WorkspaceFlatPageLayoutTabMapCacheService,
WorkspaceFlatPageLayoutWidgetMapCacheService,
WorkspaceFlatRowLevelPermissionPredicateMapCacheService,
WorkspaceFlatRowLevelPermissionPredicateGroupMapCacheService,
provideWorkspaceScopedRepository(PermissionFlagEntity),
provideWorkspaceScopedRepository(SearchFieldMetadataEntity),
provideWorkspaceScopedRepository(IndexMetadataEntity),
provideWorkspaceScopedRepository(ViewEntity),
provideWorkspaceScopedRepository(ViewFieldEntity),
@@ -131,6 +136,7 @@ import { WorkspaceCacheModule } from 'src/engine/workspace-cache/workspace-cache
WorkspaceFlatRolePermissionFlagMapCacheService,
WorkspaceFlatPermissionFlagMapCacheService,
WorkspaceFlatViewSortMapCacheService,
WorkspaceFlatSearchFieldMetadataMapCacheService,
WorkspaceFlatPageLayoutMapCacheService,
WorkspaceFlatPageLayoutTabMapCacheService,
WorkspaceFlatPageLayoutWidgetMapCacheService,
@@ -45,6 +45,8 @@ import { type FlatViewGroupMaps } from 'src/engine/metadata-modules/flat-view-gr
import { type FlatViewGroup } from 'src/engine/metadata-modules/flat-view-group/types/flat-view-group.type';
import { type FlatViewSortMaps } from 'src/engine/metadata-modules/flat-view-sort/types/flat-view-sort-maps.type';
import { type FlatViewSort } from 'src/engine/metadata-modules/flat-view-sort/types/flat-view-sort.type';
import { type FlatSearchFieldMetadataMaps } from 'src/engine/metadata-modules/flat-search-field-metadata/types/flat-search-field-metadata-maps.type';
import { type FlatSearchFieldMetadata } from 'src/engine/metadata-modules/flat-search-field-metadata/types/flat-search-field-metadata.type';
import { type FlatViewMaps } from 'src/engine/metadata-modules/flat-view/types/flat-view-maps.type';
import { type FlatView } from 'src/engine/metadata-modules/flat-view/types/flat-view.type';
import { type FlatApplicationVariableMaps } from 'src/engine/metadata-modules/flat-application-variable/types/flat-application-variable-maps.type';
@@ -84,6 +86,7 @@ import { type UniversalFlatViewFilterGroup } from 'src/engine/workspace-manager/
import { type UniversalFlatViewFilter } from 'src/engine/workspace-manager/workspace-migration/universal-flat-entity/types/universal-flat-view-filter.type';
import { type UniversalFlatViewGroup } from 'src/engine/workspace-manager/workspace-migration/universal-flat-entity/types/universal-flat-view-group.type';
import { type UniversalFlatViewSort } from 'src/engine/workspace-manager/workspace-migration/universal-flat-entity/types/universal-flat-view-sort.type';
import { type UniversalFlatSearchFieldMetadata } from 'src/engine/workspace-manager/workspace-migration/universal-flat-entity/types/universal-flat-search-field-metadata.type';
import { type UniversalFlatView } from 'src/engine/workspace-manager/workspace-migration/universal-flat-entity/types/universal-flat-view.type';
import { type UniversalFlatApplicationVariable } from 'src/engine/workspace-manager/workspace-migration/universal-flat-entity/types/universal-flat-application-variable.type';
import { type UniversalFlatWebhook } from 'src/engine/workspace-manager/workspace-migration/universal-flat-entity/types/universal-flat-webhook.type';
@@ -327,6 +330,14 @@ import {
type UniversalDeleteWebhookAction,
type UniversalUpdateWebhookAction,
} from 'src/engine/workspace-manager/workspace-migration/workspace-migration-builder/builders/webhook/types/workspace-migration-webhook-action.type';
import {
type FlatCreateSearchFieldMetadataAction,
type FlatDeleteSearchFieldMetadataAction,
type FlatUpdateSearchFieldMetadataAction,
type UniversalCreateSearchFieldMetadataAction,
type UniversalDeleteSearchFieldMetadataAction,
type UniversalUpdateSearchFieldMetadataAction,
} from 'src/engine/workspace-manager/workspace-migration/workspace-migration-builder/builders/search-field-metadata/types/workspace-migration-search-field-metadata-action.type';
export type AllFlatEntityTypesByMetadataName = {
fieldMetadata: {
@@ -812,4 +823,20 @@ export type AllFlatEntityTypesByMetadataName = {
universalFlatEntity: UniversalFlatConnectionProvider;
entity: MetadataEntity<'connectionProvider'>;
};
searchFieldMetadata: {
flatEntityMaps: FlatSearchFieldMetadataMaps;
universalActions: {
create: UniversalCreateSearchFieldMetadataAction;
update: UniversalUpdateSearchFieldMetadataAction;
delete: UniversalDeleteSearchFieldMetadataAction;
};
flatActions: {
create: FlatCreateSearchFieldMetadataAction;
update: FlatUpdateSearchFieldMetadataAction;
delete: FlatDeleteSearchFieldMetadataAction;
};
flatEntity: FlatSearchFieldMetadata;
universalFlatEntity: UniversalFlatSearchFieldMetadata;
entity: MetadataEntity<'searchFieldMetadata'>;
};
};
@@ -60,6 +60,7 @@ exports[`getMetadataRelatedMetadataNames should return related metadata names fo
"objectPermission",
"fieldPermission",
"view",
"searchFieldMetadata",
]
`;
@@ -139,6 +140,13 @@ exports[`getMetadataRelatedMetadataNames should return related metadata names fo
]
`;
exports[`getMetadataRelatedMetadataNames should return related metadata names for searchFieldMetadata 1`] = `
[
"objectMetadata",
"fieldMetadata",
]
`;
exports[`getMetadataRelatedMetadataNames should return related metadata names for skill 1`] = `[]`;
exports[`getMetadataRelatedMetadataNames should return related metadata names for view 1`] = `
@@ -11,6 +11,7 @@ exports[`sortMetadataNamesChildrenFirst should return metadata names sorted with
"objectPermission",
"pageLayoutWidget",
"rolePermissionFlag",
"searchFieldMetadata",
"viewSort",
"index",
"pageLayout",
@@ -46,6 +46,7 @@ describe('fromDeleteFieldInputToFlatFieldMetadatasToDelete', () => {
flatFieldMetadataMaps: buildFlatFieldMetadataMaps([]),
flatObjectMetadataMaps: buildFlatObjectMetadataMaps('obj-1'),
flatIndexMaps: createEmptyFlatEntityMaps(),
flatSearchFieldMetadataMaps: createEmptyFlatEntityMaps(),
}),
).toThrow(
expect.objectContaining({
@@ -69,6 +70,7 @@ describe('fromDeleteFieldInputToFlatFieldMetadatasToDelete', () => {
flatFieldMetadataMaps: buildFlatFieldMetadataMaps([standardField]),
flatObjectMetadataMaps: buildFlatObjectMetadataMaps(objectId),
flatIndexMaps: createEmptyFlatEntityMaps(),
flatSearchFieldMetadataMaps: createEmptyFlatEntityMaps(),
}),
).toThrow(
expect.objectContaining({
@@ -92,6 +94,7 @@ describe('fromDeleteFieldInputToFlatFieldMetadatasToDelete', () => {
flatFieldMetadataMaps: buildFlatFieldMetadataMaps([standardField]),
flatObjectMetadataMaps: buildFlatObjectMetadataMaps(objectId),
flatIndexMaps: createEmptyFlatEntityMaps(),
flatSearchFieldMetadataMaps: createEmptyFlatEntityMaps(),
}),
).toThrow(new RegExp('Cannot delete standard field "city"'));
});
@@ -110,6 +113,7 @@ describe('fromDeleteFieldInputToFlatFieldMetadatasToDelete', () => {
flatFieldMetadataMaps: buildFlatFieldMetadataMaps([customField]),
flatObjectMetadataMaps: buildFlatObjectMetadataMaps(objectId),
flatIndexMaps: createEmptyFlatEntityMaps(),
flatSearchFieldMetadataMaps: createEmptyFlatEntityMaps(),
});
expect(result.flatFieldMetadatasToDelete).toContainEqual(
@@ -132,6 +136,7 @@ describe('fromDeleteFieldInputToFlatFieldMetadatasToDelete', () => {
flatFieldMetadataMaps: buildFlatFieldMetadataMaps([standardField]),
flatObjectMetadataMaps: buildFlatObjectMetadataMaps(objectId),
flatIndexMaps: createEmptyFlatEntityMaps(),
flatSearchFieldMetadataMaps: createEmptyFlatEntityMaps(),
}),
).toThrow(FieldMetadataException);
});
@@ -155,6 +160,7 @@ describe('fromDeleteFieldInputToFlatFieldMetadatasToDelete', () => {
flatFieldMetadataMaps: buildFlatFieldMetadataMaps([standardAppField]),
flatObjectMetadataMaps: buildFlatObjectMetadataMaps(objectId),
flatIndexMaps: createEmptyFlatEntityMaps(),
flatSearchFieldMetadataMaps: createEmptyFlatEntityMaps(),
}),
).toThrow(
expect.objectContaining({
@@ -0,0 +1,89 @@
import { type FieldMetadataType } from 'twenty-shared/types';
import { isDefined } from 'twenty-shared/utils';
import { type AllFlatEntityMaps } from 'src/engine/metadata-modules/flat-entity/types/all-flat-entity-maps.type';
import { findFlatEntityByIdInFlatEntityMaps } from 'src/engine/metadata-modules/flat-entity/utils/find-flat-entity-by-id-in-flat-entity-maps.util';
import { findManyFlatEntityByIdInFlatEntityMapsOrThrow } from 'src/engine/metadata-modules/flat-entity/utils/find-many-flat-entity-by-id-in-flat-entity-maps-or-throw.util';
import { type FlatFieldMetadata } from 'src/engine/metadata-modules/flat-field-metadata/types/flat-field-metadata.type';
import { recomputeSearchVectorFieldFromSearchFieldMetadatas } from 'src/engine/metadata-modules/flat-object-metadata/utils/recompute-search-vector-field-from-search-field-metadatas.util';
import { type FlatSearchFieldMetadata } from 'src/engine/metadata-modules/flat-search-field-metadata/types/flat-search-field-metadata.type';
import { type UniversalFlatSearchFieldMetadata } from 'src/engine/workspace-manager/workspace-migration/universal-flat-entity/types/universal-flat-search-field-metadata.type';
export const computeSearchFieldMetadataDeletionForDeletedFields = ({
flatFieldMetadatasToDelete,
flatObjectMetadataMaps,
flatFieldMetadataMaps,
flatSearchFieldMetadataMaps,
}: {
flatFieldMetadatasToDelete: FlatFieldMetadata[];
} & Pick<
AllFlatEntityMaps,
| 'flatObjectMetadataMaps'
| 'flatFieldMetadataMaps'
| 'flatSearchFieldMetadataMaps'
>): {
searchFieldMetadatasToDelete: UniversalFlatSearchFieldMetadata[];
flatSearchVectorFieldsToUpdate: FlatFieldMetadata<FieldMetadataType.TS_VECTOR>[];
} => {
const deletedFieldIds = new Set(
flatFieldMetadatasToDelete.map((flatFieldMetadata) => flatFieldMetadata.id),
);
const affectedObjectMetadataIds = new Set(
flatFieldMetadatasToDelete.map(
(flatFieldMetadata) => flatFieldMetadata.objectMetadataId,
),
);
const searchFieldMetadatasToDelete: UniversalFlatSearchFieldMetadata[] = [];
const flatSearchVectorFieldsToUpdate: FlatFieldMetadata<FieldMetadataType.TS_VECTOR>[] =
[];
for (const objectMetadataId of affectedObjectMetadataIds) {
const flatObjectMetadata = findFlatEntityByIdInFlatEntityMaps({
flatEntityId: objectMetadataId,
flatEntityMaps: flatObjectMetadataMaps,
});
if (!isDefined(flatObjectMetadata) || !flatObjectMetadata.isSearchable) {
continue;
}
const objectSearchFieldMetadatas =
findManyFlatEntityByIdInFlatEntityMapsOrThrow<FlatSearchFieldMetadata>({
flatEntityMaps: flatSearchFieldMetadataMaps,
flatEntityIds: flatObjectMetadata.searchFieldMetadataIds,
});
const rowsToDelete = objectSearchFieldMetadatas.filter(
(searchFieldMetadata) =>
deletedFieldIds.has(searchFieldMetadata.fieldMetadataId),
);
if (rowsToDelete.length === 0) {
continue;
}
searchFieldMetadatasToDelete.push(...rowsToDelete);
const remainingSearchFieldMetadatas = objectSearchFieldMetadatas.filter(
(searchFieldMetadata) =>
!deletedFieldIds.has(searchFieldMetadata.fieldMetadataId),
);
const flatSearchVectorFieldToUpdate =
recomputeSearchVectorFieldFromSearchFieldMetadatas({
flatObjectMetadata,
flatFieldMetadataMaps,
searchFieldMetadatas: remainingSearchFieldMetadatas,
});
if (isDefined(flatSearchVectorFieldToUpdate)) {
flatSearchVectorFieldsToUpdate.push(flatSearchVectorFieldToUpdate);
}
}
return {
searchFieldMetadatasToDelete,
flatSearchVectorFieldsToUpdate,
};
};
@@ -1,3 +1,4 @@
import { type FieldMetadataType } from 'twenty-shared/types';
import {
isDefined,
trimAndRemoveDuplicatedWhitespacesFromObjectStringProperties,
@@ -12,18 +13,24 @@ import { type AllFlatEntityMaps } from 'src/engine/metadata-modules/flat-entity/
import { findFlatEntityByIdInFlatEntityMaps } from 'src/engine/metadata-modules/flat-entity/utils/find-flat-entity-by-id-in-flat-entity-maps.util';
import { findFlatEntityByUniversalIdentifierOrThrow } from 'src/engine/metadata-modules/flat-entity/utils/find-flat-entity-by-universal-identifier-or-throw.util';
import { findManyFlatEntityByUniversalIdentifierInUniversalFlatEntityMapsOrThrow } from 'src/engine/metadata-modules/flat-entity/utils/find-many-flat-entity-by-universal-identifier-in-universal-flat-entity-maps-or-throw.util';
import { type FlatFieldMetadata } from 'src/engine/metadata-modules/flat-field-metadata/types/flat-field-metadata.type';
import { computeFlatFieldMetadataRelatedFlatFieldMetadata } from 'src/engine/metadata-modules/flat-field-metadata/utils/compute-flat-field-metadata-related-flat-field-metadata.util';
import { computeSearchFieldMetadataDeletionForDeletedFields } from 'src/engine/metadata-modules/flat-field-metadata/utils/compute-search-field-metadata-deletion-for-deleted-fields.util';
import { type FlatIndexMetadata } from 'src/engine/metadata-modules/flat-index-metadata/types/flat-index-metadata.type';
import { generateFlatIndexMetadataWithNameOrThrow } from 'src/engine/metadata-modules/index-metadata/utils/generate-flat-index.util';
import { belongsToTwentyStandardApp } from 'src/engine/metadata-modules/utils/belongs-to-twenty-standard-app.util';
import { type UniversalFlatFieldMetadata } from 'src/engine/workspace-manager/workspace-migration/universal-flat-entity/types/universal-flat-field-metadata.type';
import { type UniversalFlatIndexMetadata } from 'src/engine/workspace-manager/workspace-migration/universal-flat-entity/types/universal-flat-index-metadata.type';
import { type UniversalFlatSearchFieldMetadata } from 'src/engine/workspace-manager/workspace-migration/universal-flat-entity/types/universal-flat-search-field-metadata.type';
type FromDeleteFieldInputToFlatFieldMetadatasToDeleteArgs = {
deleteOneFieldInput: DeleteOneFieldInput;
} & Pick<
AllFlatEntityMaps,
'flatFieldMetadataMaps' | 'flatIndexMaps' | 'flatObjectMetadataMaps'
| 'flatFieldMetadataMaps'
| 'flatIndexMaps'
| 'flatObjectMetadataMaps'
| 'flatSearchFieldMetadataMaps'
>;
// TODO refactor as a side effect service
export const fromDeleteFieldInputToFlatFieldMetadatasToDelete = ({
@@ -31,10 +38,13 @@ export const fromDeleteFieldInputToFlatFieldMetadatasToDelete = ({
deleteOneFieldInput: rawDeleteOneInput,
flatIndexMaps: existingFlatIndexMaps,
flatFieldMetadataMaps: existingFlatFieldMetadataMaps,
flatSearchFieldMetadataMaps: existingFlatSearchFieldMetadataMaps,
}: FromDeleteFieldInputToFlatFieldMetadatasToDeleteArgs): {
flatFieldMetadatasToDelete: UniversalFlatFieldMetadata[];
flatIndexesToUpdate: UniversalFlatIndexMetadata[];
flatIndexesToDelete: UniversalFlatIndexMetadata[];
searchFieldMetadatasToDelete: UniversalFlatSearchFieldMetadata[];
flatSearchVectorFieldsToUpdate: FlatFieldMetadata<FieldMetadataType.TS_VECTOR>[];
} => {
const { id: fieldMetadataToDeleteId } =
trimAndRemoveDuplicatedWhitespacesFromObjectStringProperties(
@@ -177,9 +187,19 @@ export const fromDeleteFieldInputToFlatFieldMetadatasToDelete = ({
},
);
const { searchFieldMetadatasToDelete, flatSearchVectorFieldsToUpdate } =
computeSearchFieldMetadataDeletionForDeletedFields({
flatFieldMetadatasToDelete,
flatObjectMetadataMaps: existingFlatObjectMetadataMaps,
flatFieldMetadataMaps: existingFlatFieldMetadataMaps,
flatSearchFieldMetadataMaps: existingFlatSearchFieldMetadataMaps,
});
return {
flatFieldMetadatasToDelete,
flatIndexesToDelete,
flatIndexesToUpdate,
searchFieldMetadatasToDelete,
flatSearchVectorFieldsToUpdate,
};
};
@@ -35,6 +35,7 @@ type FromUpdateFieldInputToFlatFieldMetadataArgs = {
| 'flatViewGroupMaps'
| 'flatViewMaps'
| 'flatViewFieldMaps'
| 'flatSearchFieldMetadataMaps'
>;
type FlatFieldMetadataAndIndexToUpdate = {
@@ -51,6 +52,7 @@ export const fromUpdateFieldInputToFlatFieldMetadata = ({
flatViewGroupMaps,
flatViewMaps,
flatViewFieldMaps,
flatSearchFieldMetadataMaps,
isSystemBuild,
}: FromUpdateFieldInputToFlatFieldMetadataArgs): FieldInputTranspilationResult<FlatFieldMetadataAndIndexToUpdate> => {
const updateFieldInputInformalProperties =
@@ -150,6 +152,7 @@ export const fromUpdateFieldInputToFlatFieldMetadata = ({
toFlatFieldMetadata,
flatViewMaps,
flatViewFieldMaps,
flatSearchFieldMetadataMaps,
flatApplication,
});
@@ -15,8 +15,8 @@ import {
type FieldMetadataUpdateIndexSideEffect,
handleIndexChangesDuringFieldUpdate,
} from 'src/engine/metadata-modules/flat-field-metadata/utils/handle-index-changes-during-field-update.util';
import { handleLabelIdentifierChangesDuringFieldUpdate } from 'src/engine/metadata-modules/flat-field-metadata/utils/handle-label-identifier-changes-during-field-update.util';
import { isEnumFlatFieldMetadata } from 'src/engine/metadata-modules/flat-field-metadata/utils/is-enum-flat-field-metadata.util';
import { recomputeSearchVectorOnFieldRename } from 'src/engine/metadata-modules/flat-field-metadata/utils/recompute-search-vector-on-field-rename.util';
import { type FlatViewFiltersToDeleteAndUpdate } from 'src/engine/metadata-modules/flat-field-metadata/utils/recompute-view-filters-on-flat-field-metadata-options-update.util';
import { type FlatViewGroupsToDeleteUpdateAndCreate } from 'src/engine/metadata-modules/flat-field-metadata/utils/recompute-view-groups-on-flat-field-metadata-options-update.util';
@@ -41,6 +41,7 @@ type HandleFlatFieldMetadataUpdateSideEffectArgs = FromTo<
| 'flatViewGroupMaps'
| 'flatViewMaps'
| 'flatViewFieldMaps'
| 'flatSearchFieldMetadataMaps'
> & {
flatApplication: FlatApplication;
};
@@ -71,6 +72,7 @@ export const handleFlatFieldMetadataUpdateSideEffect = ({
flatViewGroupMaps,
flatViewMaps,
flatViewFieldMaps,
flatSearchFieldMetadataMaps,
flatApplication,
}: HandleFlatFieldMetadataUpdateSideEffectArgs): FieldInputTranspilationResult<FlatFieldMetadataUpdateSideEffects> => {
const sideEffectResult = structuredClone(
@@ -143,24 +145,18 @@ export const handleFlatFieldMetadataUpdateSideEffect = ({
flatEntityId: fromFlatFieldMetadata.objectMetadataId,
});
const isLabelIdentifierFieldMetadata =
flatObjectMetadata.labelIdentifierFieldMetadataId ===
toFlatFieldMetadata.id;
const { flatSearchVectorFieldToUpdate } = recomputeSearchVectorOnFieldRename({
fromFlatFieldMetadata,
toFlatFieldMetadata,
flatObjectMetadata,
flatFieldMetadataMaps,
flatSearchFieldMetadataMaps,
});
if (isLabelIdentifierFieldMetadata) {
const flatSearchVectorFieldToUpdate =
handleLabelIdentifierChangesDuringFieldUpdate({
fromFlatFieldMetadata,
toFlatFieldMetadata,
flatObjectMetadata,
flatFieldMetadataMaps,
});
if (isDefined(flatSearchVectorFieldToUpdate)) {
sideEffectResult.flatFieldMetadatasToUpdate.push(
flatSearchVectorFieldToUpdate,
);
}
if (isDefined(flatSearchVectorFieldToUpdate)) {
sideEffectResult.flatFieldMetadatasToUpdate.push(
flatSearchVectorFieldToUpdate,
);
}
const {
@@ -1,71 +0,0 @@
import { type FieldMetadataType, type FromTo } from 'twenty-shared/types';
import { findOrThrow, type SearchableFieldType } from 'twenty-shared/utils';
import {
FieldMetadataException,
FieldMetadataExceptionCode,
} from 'src/engine/metadata-modules/field-metadata/field-metadata.exception';
import { type AllFlatEntityMaps } from 'src/engine/metadata-modules/flat-entity/types/all-flat-entity-maps.type';
import { findManyFlatEntityByIdInFlatEntityMapsOrThrow } from 'src/engine/metadata-modules/flat-entity/utils/find-many-flat-entity-by-id-in-flat-entity-maps-or-throw.util';
import { type FlatFieldMetadata } from 'src/engine/metadata-modules/flat-field-metadata/types/flat-field-metadata.type';
import { type FlatObjectMetadata } from 'src/engine/metadata-modules/flat-object-metadata/types/flat-object-metadata.type';
import { SEARCH_VECTOR_FIELD } from 'src/engine/metadata-modules/search-field-metadata/constants/search-vector-field.constants';
import { getTsVectorColumnExpressionFromFields } from 'src/engine/workspace-manager/utils/get-ts-vector-column-expression.util';
type HandleLabelIdentifierChangesDuringFieldUpdateArgs = {
flatObjectMetadata: FlatObjectMetadata;
} & FromTo<FlatFieldMetadata, 'flatFieldMetadata'> &
Pick<AllFlatEntityMaps, 'flatFieldMetadataMaps'>;
export const handleLabelIdentifierChangesDuringFieldUpdate = ({
fromFlatFieldMetadata,
toFlatFieldMetadata,
flatObjectMetadata,
flatFieldMetadataMaps,
}: HandleLabelIdentifierChangesDuringFieldUpdateArgs):
| FlatFieldMetadata<FieldMetadataType.TS_VECTOR>
| undefined => {
const hasNameChanged =
fromFlatFieldMetadata.name !== toFlatFieldMetadata.name;
if (!hasNameChanged) {
return undefined;
}
const objectFlatFieldMetadatas =
findManyFlatEntityByIdInFlatEntityMapsOrThrow({
flatEntityMaps: flatFieldMetadataMaps,
flatEntityIds: flatObjectMetadata.fieldIds,
});
const searchVectorField = findOrThrow(
objectFlatFieldMetadatas,
(field) => field.name === SEARCH_VECTOR_FIELD.name,
new FieldMetadataException(
`Search vector field not found for object metadata ${flatObjectMetadata.id}`,
FieldMetadataExceptionCode.FIELD_METADATA_NOT_FOUND,
),
) as FlatFieldMetadata<FieldMetadataType.TS_VECTOR>;
try {
const newAsExpression = getTsVectorColumnExpressionFromFields([
{
name: toFlatFieldMetadata.name,
type: toFlatFieldMetadata.type as SearchableFieldType,
},
]);
return {
...searchVectorField,
universalSettings: {
...searchVectorField.universalSettings,
asExpression: newAsExpression,
},
};
} catch {
throw new FieldMetadataException(
`Failed to compute search vector column expression for field ${toFlatFieldMetadata.name}`,
FieldMetadataExceptionCode.INVALID_FIELD_INPUT,
);
}
};
@@ -0,0 +1,68 @@
import { type FieldMetadataType, type FromTo } from 'twenty-shared/types';
import { type AllFlatEntityMaps } from 'src/engine/metadata-modules/flat-entity/types/all-flat-entity-maps.type';
import { findManyFlatEntityByIdInFlatEntityMapsOrThrow } from 'src/engine/metadata-modules/flat-entity/utils/find-many-flat-entity-by-id-in-flat-entity-maps-or-throw.util';
import { type FlatFieldMetadata } from 'src/engine/metadata-modules/flat-field-metadata/types/flat-field-metadata.type';
import { type FlatObjectMetadata } from 'src/engine/metadata-modules/flat-object-metadata/types/flat-object-metadata.type';
import { recomputeSearchVectorFieldFromSearchFieldMetadatas } from 'src/engine/metadata-modules/flat-object-metadata/utils/recompute-search-vector-field-from-search-field-metadatas.util';
import { type FlatSearchFieldMetadata } from 'src/engine/metadata-modules/flat-search-field-metadata/types/flat-search-field-metadata.type';
export type FieldMetadataUpdateSearchVectorSideEffect = {
flatSearchVectorFieldToUpdate:
| FlatFieldMetadata<FieldMetadataType.TS_VECTOR>
| undefined;
};
type RecomputeSearchVectorOnFieldRenameArgs = {
flatObjectMetadata: FlatObjectMetadata;
} & FromTo<FlatFieldMetadata, 'flatFieldMetadata'> &
Pick<
AllFlatEntityMaps,
'flatFieldMetadataMaps' | 'flatSearchFieldMetadataMaps'
>;
const EMPTY_SIDE_EFFECT: FieldMetadataUpdateSearchVectorSideEffect = {
flatSearchVectorFieldToUpdate: undefined,
};
export const recomputeSearchVectorOnFieldRename = ({
fromFlatFieldMetadata,
toFlatFieldMetadata,
flatObjectMetadata,
flatFieldMetadataMaps,
flatSearchFieldMetadataMaps,
}: RecomputeSearchVectorOnFieldRenameArgs): FieldMetadataUpdateSearchVectorSideEffect => {
if (!flatObjectMetadata.isSearchable) {
return EMPTY_SIDE_EFFECT;
}
const objectSearchFieldMetadatas =
findManyFlatEntityByIdInFlatEntityMapsOrThrow<FlatSearchFieldMetadata>({
flatEntityMaps: flatSearchFieldMetadataMaps,
flatEntityIds: flatObjectMetadata.searchFieldMetadataIds,
});
const isIndexedField = objectSearchFieldMetadatas.some(
(searchFieldMetadata) =>
searchFieldMetadata.fieldMetadataId === toFlatFieldMetadata.id,
);
const hasNameChanged =
fromFlatFieldMetadata.name !== toFlatFieldMetadata.name;
if (!isIndexedField || !hasNameChanged) {
return EMPTY_SIDE_EFFECT;
}
return {
flatSearchVectorFieldToUpdate:
recomputeSearchVectorFieldFromSearchFieldMetadatas({
flatObjectMetadata,
flatFieldMetadataMaps,
searchFieldMetadatas: objectSearchFieldMetadatas,
overrideFlatFieldMetadataById: new Map([
[toFlatFieldMetadata.id, toFlatFieldMetadata],
]),
}),
};
};
@@ -20,6 +20,7 @@ export const getFlatObjectMetadataMock = (
return {
viewIds: [],
indexMetadataIds: [],
searchFieldMetadataIds: [],
objectPermissionIds: [],
fieldPermissionIds: [],
fieldIds: [],
@@ -55,6 +56,7 @@ export const getFlatObjectMetadataMock = (
fieldPermissionUniversalIdentifiers: [],
viewUniversalIdentifiers: [],
indexMetadataUniversalIdentifiers: [],
searchFieldMetadataUniversalIdentifiers: [],
labelIdentifierFieldMetadataUniversalIdentifier:
labelIdentifierFieldMetadataId,
imageIdentifierFieldMetadataUniversalIdentifier:
@@ -14,6 +14,7 @@ import { fromObjectMetadataEntityToFlatObjectMetadata } from 'src/engine/metadat
import { IndexMetadataEntity } from 'src/engine/metadata-modules/index-metadata/index-metadata.entity';
import { ObjectMetadataEntity } from 'src/engine/metadata-modules/object-metadata/object-metadata.entity';
import { ObjectPermissionEntity } from 'src/engine/metadata-modules/object-permission/object-permission.entity';
import { SearchFieldMetadataEntity } from 'src/engine/metadata-modules/search-field-metadata/search-field-metadata.entity';
import { ViewEntity } from 'src/engine/metadata-modules/view/entities/view.entity';
import { InjectWorkspaceScopedRepository } from 'src/engine/twenty-orm/workspace-scoped-repository/inject-workspace-scoped-repository.decorator';
import { WorkspaceScopedRepository } from 'src/engine/twenty-orm/workspace-scoped-repository/workspace-scoped-repository';
@@ -40,6 +41,8 @@ export class WorkspaceFlatObjectMetadataMapCacheService extends WorkspaceCachePr
private readonly viewRepository: WorkspaceScopedRepository<ViewEntity>,
@InjectWorkspaceScopedRepository(ObjectPermissionEntity)
private readonly objectPermissionRepository: WorkspaceScopedRepository<ObjectPermissionEntity>,
@InjectWorkspaceScopedRepository(SearchFieldMetadataEntity)
private readonly searchFieldMetadataRepository: WorkspaceScopedRepository<SearchFieldMetadataEntity>,
) {
super();
}
@@ -54,6 +57,7 @@ export class WorkspaceFlatObjectMetadataMapCacheService extends WorkspaceCachePr
indexMetadatas,
views,
objectPermissions,
searchFieldMetadatas,
] = await Promise.all([
this.objectMetadataRepository.find({
where: { workspaceId },
@@ -81,6 +85,9 @@ export class WorkspaceFlatObjectMetadataMapCacheService extends WorkspaceCachePr
select: ['id', 'universalIdentifier', 'objectMetadataId'],
withDeleted: true,
}),
this.searchFieldMetadataRepository.find(workspaceId, {
select: ['id', 'universalIdentifier', 'objectMetadataId'],
}),
]);
const [
@@ -88,6 +95,7 @@ export class WorkspaceFlatObjectMetadataMapCacheService extends WorkspaceCachePr
indexesByObjectId,
viewsByObjectId,
objectPermissionsByObjectId,
searchFieldMetadatasByObjectId,
] = (
[
{
@@ -106,6 +114,10 @@ export class WorkspaceFlatObjectMetadataMapCacheService extends WorkspaceCachePr
entities: objectPermissions,
foreignKey: 'objectMetadataId',
},
{
entities: searchFieldMetadatas,
foreignKey: 'objectMetadataId',
},
] as const
).map(regroupEntitiesByRelatedEntityId);
@@ -125,6 +137,8 @@ export class WorkspaceFlatObjectMetadataMapCacheService extends WorkspaceCachePr
views: viewsByObjectId.get(objectMetadataEntity.id) || [],
objectPermissions:
objectPermissionsByObjectId.get(objectMetadataEntity.id) || [],
searchFieldMetadatas:
searchFieldMetadatasByObjectId.get(objectMetadataEntity.id) || [],
},
applicationIdToUniversalIdentifierMap,
fieldMetadataIdToUniversalIdentifierMap,
@@ -10,9 +10,11 @@ import { type CreateObjectInput } from 'src/engine/metadata-modules/object-metad
import { buildDefaultFlatFieldMetadatasForCustomObject } from 'src/engine/metadata-modules/object-metadata/utils/build-default-flat-field-metadatas-for-custom-object.util';
import { buildDefaultIndexesForCustomObject } from 'src/engine/metadata-modules/object-metadata/utils/build-default-index-for-custom-object.util';
import { buildDefaultRelationFlatFieldMetadatasForCustomObject } from 'src/engine/metadata-modules/object-metadata/utils/build-default-relation-flat-field-metadatas-for-custom-object.util';
import { buildDefaultSearchFieldMetadatasForCustomObject } from 'src/engine/metadata-modules/object-metadata/utils/build-default-search-field-metadatas-for-custom-object.util';
import { type UniversalFlatFieldMetadata } from 'src/engine/workspace-manager/workspace-migration/universal-flat-entity/types/universal-flat-field-metadata.type';
import { type UniversalFlatIndexMetadata } from 'src/engine/workspace-manager/workspace-migration/universal-flat-entity/types/universal-flat-index-metadata.type';
import { type UniversalFlatObjectMetadata } from 'src/engine/workspace-manager/workspace-migration/universal-flat-entity/types/universal-flat-object-metadata.type';
import { type UniversalFlatSearchFieldMetadata } from 'src/engine/workspace-manager/workspace-migration/universal-flat-entity/types/universal-flat-search-field-metadata.type';
type FromCreateObjectInputToFlatObjectMetadataAndFlatFieldMetadatasToCreateArgs =
{
@@ -29,6 +31,7 @@ export const fromCreateObjectInputToFlatObjectMetadataAndFlatFieldMetadatasToCre
relationTargetFlatFieldMetadataToCreate: UniversalFlatFieldMetadata[];
flatFieldMetadataToCreateOnObject: UniversalFlatFieldMetadata[];
flatIndexMetadataToCreate: UniversalFlatIndexMetadata[];
flatSearchFieldMetadataToCreate: UniversalFlatSearchFieldMetadata[];
} => {
const createObjectInput =
trimAndRemoveDuplicatedWhitespacesFromObjectStringProperties(
@@ -94,6 +97,7 @@ export const fromCreateObjectInputToFlatObjectMetadataAndFlatFieldMetadatasToCre
fieldPermissionUniversalIdentifiers: [],
viewUniversalIdentifiers: [],
indexMetadataUniversalIdentifiers: [],
searchFieldMetadataUniversalIdentifiers: [],
labelIdentifierFieldMetadataUniversalIdentifier,
imageIdentifierFieldMetadataUniversalIdentifier: null,
};
@@ -119,12 +123,20 @@ export const fromCreateObjectInputToFlatObjectMetadataAndFlatFieldMetadatasToCre
flatObjectMetadata: universalFlatObjectMetadataToCreate,
});
const defaultSearchFieldMetadatasForCustomObject =
buildDefaultSearchFieldMetadatasForCustomObject({
defaultFlatFieldForCustomObjectMaps,
flatObjectMetadata: universalFlatObjectMetadataToCreate,
});
return {
flatObjectMetadataToCreate: universalFlatObjectMetadataToCreate,
flatIndexMetadataToCreate: [
...Object.values(defaultIndexesForCustomObject.indexes),
...standardTargetFlatIndexMetadatas,
],
flatSearchFieldMetadataToCreate:
defaultSearchFieldMetadatasForCustomObject.searchFieldMetadatas,
relationTargetFlatFieldMetadataToCreate: standardTargetFlatFieldMetadatas,
flatFieldMetadataToCreateOnObject: objectFlatFieldMetadatas,
};
@@ -77,6 +77,9 @@ export const fromObjectMetadataEntityToFlatObjectMetadata = ({
updatedAt: objectMetadataEntity.updatedAt.toISOString(),
viewIds: objectMetadataEntity.views.map(({ id }) => id),
indexMetadataIds: objectMetadataEntity.indexMetadatas.map(({ id }) => id),
searchFieldMetadataIds: objectMetadataEntity.searchFieldMetadatas.map(
({ id }) => id,
),
fieldIds: objectMetadataEntity.fields.map(({ id }) => id),
objectPermissionIds: objectMetadataEntity.objectPermissions.map(
({ id }) => id,
@@ -92,6 +95,10 @@ export const fromObjectMetadataEntityToFlatObjectMetadata = ({
indexMetadataUniversalIdentifiers: objectMetadataEntity.indexMetadatas.map(
({ universalIdentifier }) => universalIdentifier,
),
searchFieldMetadataUniversalIdentifiers:
objectMetadataEntity.searchFieldMetadatas?.map(
({ universalIdentifier }) => universalIdentifier,
) ?? [],
viewUniversalIdentifiers: objectMetadataEntity.views.map(
({ universalIdentifier }) => universalIdentifier,
),
@@ -30,6 +30,7 @@ type FromUpdateObjectInputToFlatObjectMetadataArgs = {
| 'flatFieldMetadataMaps'
| 'flatViewFieldMaps'
| 'flatViewMaps'
| 'flatSearchFieldMetadataMaps'
>;
export const fromUpdateObjectInputToFlatObjectMetadataAndRelatedFlatEntities =
@@ -40,6 +41,7 @@ export const fromUpdateObjectInputToFlatObjectMetadataAndRelatedFlatEntities =
flatFieldMetadataMaps,
flatViewFieldMaps,
flatViewMaps,
flatSearchFieldMetadataMaps,
}: FromUpdateObjectInputToFlatObjectMetadataArgs): FlatObjectMetadataUpdateSideEffects & {
flatObjectMetadataToUpdate: UniversalFlatObjectMetadata;
} => {
@@ -101,6 +103,7 @@ export const fromUpdateObjectInputToFlatObjectMetadataAndRelatedFlatEntities =
flatViewFieldsToUpdate,
otherObjectFlatFieldMetadatasToUpdate,
sameObjectFlatFieldMetadatasToUpdate,
searchFieldMetadatasToCreate,
} = handleFlatObjectMetadataUpdateSideEffect({
fromFlatObjectMetadata: existingFlatObjectMetadata,
toFlatObjectMetadata,
@@ -109,6 +112,7 @@ export const fromUpdateObjectInputToFlatObjectMetadataAndRelatedFlatEntities =
flatIndexMaps,
flatViewFieldMaps,
flatViewMaps,
flatSearchFieldMetadataMaps,
});
return {
@@ -118,5 +122,6 @@ export const fromUpdateObjectInputToFlatObjectMetadataAndRelatedFlatEntities =
flatViewFieldsToUpdate,
otherObjectFlatFieldMetadatasToUpdate,
sameObjectFlatFieldMetadatasToUpdate,
searchFieldMetadatasToCreate,
};
};
@@ -5,11 +5,12 @@ import { type AllFlatEntityMaps } from 'src/engine/metadata-modules/flat-entity/
import { type FlatFieldMetadata } from 'src/engine/metadata-modules/flat-field-metadata/types/flat-field-metadata.type';
import { type FlatObjectMetadata } from 'src/engine/metadata-modules/flat-object-metadata/types/flat-object-metadata.type';
import { recomputeIndexAfterFlatObjectMetadataSingularNameUpdate } from 'src/engine/metadata-modules/flat-object-metadata/utils/recompute-index-after-flat-object-metadata-singular-name-update.util';
import { recomputeSearchVectorFieldAfterLabelIdentifierUpdate } from 'src/engine/metadata-modules/flat-object-metadata/utils/recompute-search-vector-field-after-label-identifier-update.util';
import { recomputeSearchVectorOnLabelIdentifierUpdate } from 'src/engine/metadata-modules/flat-object-metadata/utils/recompute-search-vector-on-label-identifier-update.util';
import { recomputeViewFieldIdentifierAfterFlatObjectIdentifierUpdate } from 'src/engine/metadata-modules/flat-object-metadata/utils/recompute-view-field-identifier-after-flat-object-identifier-update.util';
import { renameRelatedMorphFieldOnObjectNamesUpdate } from 'src/engine/metadata-modules/flat-object-metadata/utils/rename-related-morph-field-on-object-names-update.util';
import { type UniversalFlatFieldMetadata } from 'src/engine/workspace-manager/workspace-migration/universal-flat-entity/types/universal-flat-field-metadata.type';
import { type UniversalFlatIndexMetadata } from 'src/engine/workspace-manager/workspace-migration/universal-flat-entity/types/universal-flat-index-metadata.type';
import { type UniversalFlatSearchFieldMetadata } from 'src/engine/workspace-manager/workspace-migration/universal-flat-entity/types/universal-flat-search-field-metadata.type';
import { type UniversalFlatViewField } from 'src/engine/workspace-manager/workspace-migration/universal-flat-entity/types/universal-flat-view-field.type';
export type FlatObjectMetadataUpdateSideEffects = {
@@ -18,6 +19,7 @@ export type FlatObjectMetadataUpdateSideEffects = {
flatViewFieldsToUpdate: UniversalFlatViewField[];
flatViewFieldsToCreate: UniversalFlatViewField[];
flatIndexMetadatasToUpdate: UniversalFlatIndexMetadata[];
searchFieldMetadatasToCreate: UniversalFlatSearchFieldMetadata[];
};
type HandleFlatObjectMetadataUpdateSideEffectArgs = FromTo<
@@ -31,6 +33,7 @@ type HandleFlatObjectMetadataUpdateSideEffectArgs = FromTo<
| 'flatViewFieldMaps'
| 'flatIndexMaps'
| 'flatViewMaps'
| 'flatSearchFieldMetadataMaps'
>;
export const handleFlatObjectMetadataUpdateSideEffect = ({
@@ -39,6 +42,7 @@ export const handleFlatObjectMetadataUpdateSideEffect = ({
flatObjectMetadataMaps,
flatViewFieldMaps,
flatViewMaps,
flatSearchFieldMetadataMaps,
fromFlatObjectMetadata,
toFlatObjectMetadata,
}: HandleFlatObjectMetadataUpdateSideEffectArgs): FlatObjectMetadataUpdateSideEffects => {
@@ -87,25 +91,16 @@ export const handleFlatObjectMetadataUpdateSideEffect = ({
const sameObjectFlatFieldMetadatasToUpdate: FlatFieldMetadata[] = [];
if (
toFlatObjectMetadata.isSearchable &&
isDefined(
toFlatObjectMetadata.labelIdentifierFieldMetadataUniversalIdentifier,
) &&
fromFlatObjectMetadata.labelIdentifierFieldMetadataUniversalIdentifier !==
toFlatObjectMetadata.labelIdentifierFieldMetadataUniversalIdentifier
) {
const updatedSearchVectorField =
recomputeSearchVectorFieldAfterLabelIdentifierUpdate({
existingFlatObjectMetadata: fromFlatObjectMetadata,
flatFieldMetadataMaps,
labelIdentifierFieldMetadataUniversalIdentifier:
toFlatObjectMetadata.labelIdentifierFieldMetadataUniversalIdentifier,
});
const { flatSearchVectorFieldToUpdate, searchFieldMetadatasToCreate } =
recomputeSearchVectorOnLabelIdentifierUpdate({
fromFlatObjectMetadata,
toFlatObjectMetadata,
flatFieldMetadataMaps,
flatSearchFieldMetadataMaps,
});
if (isDefined(updatedSearchVectorField)) {
sameObjectFlatFieldMetadatasToUpdate.push(updatedSearchVectorField);
}
if (isDefined(flatSearchVectorFieldToUpdate)) {
sameObjectFlatFieldMetadatasToUpdate.push(flatSearchVectorFieldToUpdate);
}
return {
@@ -117,5 +112,6 @@ export const handleFlatObjectMetadataUpdateSideEffect = ({
flatViewFieldsToUpdate,
otherObjectFlatFieldMetadatasToUpdate: morphFlatFieldMetadatasToUpdate,
sameObjectFlatFieldMetadatasToUpdate,
searchFieldMetadatasToCreate,
};
};
@@ -1,77 +0,0 @@
import { type FieldMetadataType } from 'twenty-shared/types';
import {
findOrThrow,
isDefined,
type SearchableFieldType,
} from 'twenty-shared/utils';
import { type AllFlatEntityMaps } from 'src/engine/metadata-modules/flat-entity/types/all-flat-entity-maps.type';
import { findFlatEntityByUniversalIdentifier } from 'src/engine/metadata-modules/flat-entity/utils/find-flat-entity-by-universal-identifier.util';
import { findManyFlatEntityByIdInFlatEntityMapsOrThrow } from 'src/engine/metadata-modules/flat-entity/utils/find-many-flat-entity-by-id-in-flat-entity-maps-or-throw.util';
import { type FlatFieldMetadata } from 'src/engine/metadata-modules/flat-field-metadata/types/flat-field-metadata.type';
import { type FlatObjectMetadata } from 'src/engine/metadata-modules/flat-object-metadata/types/flat-object-metadata.type';
import {
ObjectMetadataException,
ObjectMetadataExceptionCode,
} from 'src/engine/metadata-modules/object-metadata/object-metadata.exception';
import { SEARCH_VECTOR_FIELD } from 'src/engine/metadata-modules/search-field-metadata/constants/search-vector-field.constants';
import { getTsVectorColumnExpressionFromFields } from 'src/engine/workspace-manager/utils/get-ts-vector-column-expression.util';
type RecomputeSearchVectorFieldAfterLabelIdentifierUpdateArgs = {
existingFlatObjectMetadata: FlatObjectMetadata;
labelIdentifierFieldMetadataUniversalIdentifier: string;
} & Pick<AllFlatEntityMaps, 'flatFieldMetadataMaps'>;
export const recomputeSearchVectorFieldAfterLabelIdentifierUpdate = ({
existingFlatObjectMetadata,
flatFieldMetadataMaps,
labelIdentifierFieldMetadataUniversalIdentifier,
}: RecomputeSearchVectorFieldAfterLabelIdentifierUpdateArgs):
| FlatFieldMetadata<FieldMetadataType.TS_VECTOR>
| undefined => {
const objectFlatFieldMetadatas =
findManyFlatEntityByIdInFlatEntityMapsOrThrow({
flatEntityMaps: flatFieldMetadataMaps,
flatEntityIds: existingFlatObjectMetadata.fieldIds,
});
const searchVectorField = findOrThrow(
objectFlatFieldMetadatas,
(field) => field.name === SEARCH_VECTOR_FIELD.name,
) as FlatFieldMetadata<FieldMetadataType.TS_VECTOR>;
const newLabelIdentifierField = findFlatEntityByUniversalIdentifier({
flatEntityMaps: flatFieldMetadataMaps,
universalIdentifier: labelIdentifierFieldMetadataUniversalIdentifier,
});
if (!isDefined(newLabelIdentifierField)) {
throw new ObjectMetadataException(
`New label identifier field not found for object metadata`,
ObjectMetadataExceptionCode.INVALID_OBJECT_INPUT,
);
}
try {
const newAsExpression = getTsVectorColumnExpressionFromFields([
{
name: newLabelIdentifierField.name,
type: newLabelIdentifierField.type as SearchableFieldType,
},
]);
return {
...searchVectorField,
universalSettings: {
...searchVectorField.universalSettings,
asExpression: newAsExpression,
generatedType: 'STORED',
},
};
} catch {
throw new ObjectMetadataException(
`Failed to compute search vector column expression for field ${newLabelIdentifierField.name}`,
ObjectMetadataExceptionCode.INVALID_OBJECT_INPUT,
);
}
};
@@ -0,0 +1,112 @@
import { FieldMetadataType } from 'twenty-shared/types';
import { isDefined } from 'twenty-shared/utils';
import { type AllFlatEntityMaps } from 'src/engine/metadata-modules/flat-entity/types/all-flat-entity-maps.type';
import { findFlatEntityByIdInFlatEntityMaps } from 'src/engine/metadata-modules/flat-entity/utils/find-flat-entity-by-id-in-flat-entity-maps.util';
import { findManyFlatEntityByIdInFlatEntityMapsOrThrow } from 'src/engine/metadata-modules/flat-entity/utils/find-many-flat-entity-by-id-in-flat-entity-maps-or-throw.util';
import { type FlatFieldMetadata } from 'src/engine/metadata-modules/flat-field-metadata/types/flat-field-metadata.type';
import { isFlatFieldMetadataOfType } from 'src/engine/metadata-modules/flat-field-metadata/utils/is-flat-field-metadata-of-type.util';
import { type FlatObjectMetadata } from 'src/engine/metadata-modules/flat-object-metadata/types/flat-object-metadata.type';
import { type FlatSearchFieldMetadata } from 'src/engine/metadata-modules/flat-search-field-metadata/types/flat-search-field-metadata.type';
import {
ObjectMetadataException,
ObjectMetadataExceptionCode,
} from 'src/engine/metadata-modules/object-metadata/object-metadata.exception';
import {
buildSearchVectorTargetField,
computeSearchVectorAsExpressionFromSearchFieldMetadatas,
} from 'src/engine/metadata-modules/flat-search-field-metadata/utils/compute-search-vector-as-expression-from-search-field-metadatas.util';
import { SEARCH_VECTOR_FIELD } from 'src/engine/metadata-modules/search-field-metadata/constants/search-vector-field.constants';
type RecomputeSearchVectorFieldFromSearchFieldMetadatasArgs = {
flatObjectMetadata: FlatObjectMetadata;
// Post-change searchFieldMetadata rows for the object, carrying the field id and the
// per-object position that drives the deterministic asExpression order.
searchFieldMetadatas: Pick<
FlatSearchFieldMetadata,
'fieldMetadataId' | 'position' | 'universalIdentifier'
>[];
// Takes precedence over the flat maps when resolving an id — for a field created or
// renamed in the same migration, whose maps entry is absent or stale.
overrideFlatFieldMetadataById?: Map<string, FlatFieldMetadata>;
} & Pick<AllFlatEntityMaps, 'flatFieldMetadataMaps'>;
export const recomputeSearchVectorFieldFromSearchFieldMetadatas = ({
flatObjectMetadata,
flatFieldMetadataMaps,
searchFieldMetadatas,
overrideFlatFieldMetadataById,
}: RecomputeSearchVectorFieldFromSearchFieldMetadatasArgs):
| FlatFieldMetadata<FieldMetadataType.TS_VECTOR>
| undefined => {
const objectFlatFieldMetadatas =
findManyFlatEntityByIdInFlatEntityMapsOrThrow({
flatEntityMaps: flatFieldMetadataMaps,
flatEntityIds: flatObjectMetadata.fieldIds,
});
const searchVectorField = objectFlatFieldMetadatas.find(
(field) => field.name === SEARCH_VECTOR_FIELD.name,
);
if (
!isDefined(searchVectorField) ||
!isFlatFieldMetadataOfType(searchVectorField, FieldMetadataType.TS_VECTOR)
) {
throw new ObjectMetadataException(
`Search vector field not found for object metadata ${flatObjectMetadata.id}`,
ObjectMetadataExceptionCode.INVALID_OBJECT_INPUT,
);
}
const targetSearchableFields = searchFieldMetadatas.map(
(searchFieldMetadata) => {
const { fieldMetadataId, position, universalIdentifier } =
searchFieldMetadata;
const overriddenFlatFieldMetadata =
overrideFlatFieldMetadataById?.get(fieldMetadataId);
const flatFieldMetadata =
overriddenFlatFieldMetadata ??
findFlatEntityByIdInFlatEntityMaps({
flatEntityId: fieldMetadataId,
flatEntityMaps: flatFieldMetadataMaps,
});
if (!isDefined(flatFieldMetadata)) {
throw new ObjectMetadataException(
`Field metadata ${fieldMetadataId} referenced by searchFieldMetadata not found for object metadata ${flatObjectMetadata.id}`,
ObjectMetadataExceptionCode.INVALID_OBJECT_INPUT,
);
}
return buildSearchVectorTargetField(
flatFieldMetadata,
position,
universalIdentifier,
);
},
);
try {
const newAsExpression =
computeSearchVectorAsExpressionFromSearchFieldMetadatas(
targetSearchableFields,
);
return {
...searchVectorField,
universalSettings: {
...searchVectorField.universalSettings,
asExpression: newAsExpression,
generatedType: 'STORED',
},
};
} catch (error) {
throw new ObjectMetadataException(
`Failed to compute search vector column expression for object metadata ${flatObjectMetadata.id}: ${error instanceof Error ? error.message : String(error)}`,
ObjectMetadataExceptionCode.INVALID_OBJECT_INPUT,
);
}
};
@@ -0,0 +1,113 @@
import { type FieldMetadataType, type FromTo } from 'twenty-shared/types';
import { isDefined, isSearchableFieldType } from 'twenty-shared/utils';
import { type AllFlatEntityMaps } from 'src/engine/metadata-modules/flat-entity/types/all-flat-entity-maps.type';
import { findFlatEntityByIdInFlatEntityMaps } from 'src/engine/metadata-modules/flat-entity/utils/find-flat-entity-by-id-in-flat-entity-maps.util';
import { findManyFlatEntityByIdInFlatEntityMapsOrThrow } from 'src/engine/metadata-modules/flat-entity/utils/find-many-flat-entity-by-id-in-flat-entity-maps-or-throw.util';
import { type FlatFieldMetadata } from 'src/engine/metadata-modules/flat-field-metadata/types/flat-field-metadata.type';
import { type FlatObjectMetadata } from 'src/engine/metadata-modules/flat-object-metadata/types/flat-object-metadata.type';
import { recomputeSearchVectorFieldFromSearchFieldMetadatas } from 'src/engine/metadata-modules/flat-object-metadata/utils/recompute-search-vector-field-from-search-field-metadatas.util';
import { type FlatSearchFieldMetadata } from 'src/engine/metadata-modules/flat-search-field-metadata/types/flat-search-field-metadata.type';
import { buildFlatSearchFieldMetadataForField } from 'src/engine/metadata-modules/flat-search-field-metadata/utils/build-flat-search-field-metadata-for-field.util';
import { type UniversalFlatSearchFieldMetadata } from 'src/engine/workspace-manager/workspace-migration/universal-flat-entity/types/universal-flat-search-field-metadata.type';
export type LabelIdentifierUpdateSearchVectorSideEffect = {
flatSearchVectorFieldToUpdate:
| FlatFieldMetadata<FieldMetadataType.TS_VECTOR>
| undefined;
searchFieldMetadatasToCreate: UniversalFlatSearchFieldMetadata[];
};
type RecomputeSearchVectorOnLabelIdentifierUpdateArgs = FromTo<
FlatObjectMetadata,
'flatObjectMetadata'
> &
Pick<
AllFlatEntityMaps,
'flatFieldMetadataMaps' | 'flatSearchFieldMetadataMaps'
>;
const EMPTY_SIDE_EFFECT: LabelIdentifierUpdateSearchVectorSideEffect = {
flatSearchVectorFieldToUpdate: undefined,
searchFieldMetadatasToCreate: [],
};
// Relabeling is additive: it indexes the new label identifier field while preserving the
// existing search surface (e.g. the provisioned name row). We never drop a row here so the
// previous label identifier stays searchable. Removal of fields from the search surface is
// deferred to a follow-up PR.
export const recomputeSearchVectorOnLabelIdentifierUpdate = ({
fromFlatObjectMetadata,
toFlatObjectMetadata,
flatFieldMetadataMaps,
flatSearchFieldMetadataMaps,
}: RecomputeSearchVectorOnLabelIdentifierUpdateArgs): LabelIdentifierUpdateSearchVectorSideEffect => {
const newLabelIdentifierFieldMetadataId =
toFlatObjectMetadata.labelIdentifierFieldMetadataId;
if (
!toFlatObjectMetadata.isSearchable ||
!isDefined(newLabelIdentifierFieldMetadataId) ||
fromFlatObjectMetadata.labelIdentifierFieldMetadataId ===
newLabelIdentifierFieldMetadataId
) {
return EMPTY_SIDE_EFFECT;
}
const existingSearchFieldMetadatas =
findManyFlatEntityByIdInFlatEntityMapsOrThrow<FlatSearchFieldMetadata>({
flatEntityMaps: flatSearchFieldMetadataMaps,
flatEntityIds: fromFlatObjectMetadata.searchFieldMetadataIds,
});
const newLabelIdentifierAlreadyIndexed = existingSearchFieldMetadatas.some(
(searchFieldMetadata) =>
searchFieldMetadata.fieldMetadataId === newLabelIdentifierFieldMetadataId,
);
const newLabelIdentifierField = findFlatEntityByIdInFlatEntityMaps({
flatEntityMaps: flatFieldMetadataMaps,
flatEntityId: newLabelIdentifierFieldMetadataId,
});
const shouldIndexNewLabelIdentifier =
isDefined(newLabelIdentifierField) &&
isSearchableFieldType(newLabelIdentifierField.type) &&
!newLabelIdentifierAlreadyIndexed;
if (!shouldIndexNewLabelIdentifier) {
return EMPTY_SIDE_EFFECT;
}
const newLabelIdentifierPosition =
existingSearchFieldMetadatas.reduce(
(maxPosition, searchFieldMetadata) =>
Math.max(maxPosition, searchFieldMetadata.position),
-1,
) + 1;
const newSearchFieldMetadata = buildFlatSearchFieldMetadataForField({
flatObjectMetadata: fromFlatObjectMetadata,
flatFieldMetadata: newLabelIdentifierField,
position: newLabelIdentifierPosition,
});
const nextSearchFieldMetadatas = [
...existingSearchFieldMetadatas,
{
fieldMetadataId: newLabelIdentifierFieldMetadataId,
position: newLabelIdentifierPosition,
universalIdentifier: newSearchFieldMetadata.universalIdentifier,
},
];
return {
flatSearchVectorFieldToUpdate:
recomputeSearchVectorFieldFromSearchFieldMetadatas({
flatObjectMetadata: fromFlatObjectMetadata,
flatFieldMetadataMaps,
searchFieldMetadatas: nextSearchFieldMetadatas,
}),
searchFieldMetadatasToCreate: [newSearchFieldMetadata],
};
};
@@ -0,0 +1,90 @@
import { Injectable } from '@nestjs/common';
import { InjectRepository } from '@nestjs/typeorm';
import { Repository } from 'typeorm';
import { WorkspaceCacheProvider } from 'src/engine/workspace-cache/interfaces/workspace-cache-provider.service';
import { ApplicationEntity } from 'src/engine/core-modules/application/application.entity';
import { FieldMetadataEntity } from 'src/engine/metadata-modules/field-metadata/field-metadata.entity';
import { createEmptyFlatEntityMaps } from 'src/engine/metadata-modules/flat-entity/constant/create-empty-flat-entity-maps.constant';
import { FlatSearchFieldMetadataMaps } from 'src/engine/metadata-modules/flat-search-field-metadata/types/flat-search-field-metadata-maps.type';
import { fromSearchFieldMetadataEntityToFlatSearchFieldMetadata } from 'src/engine/metadata-modules/flat-search-field-metadata/utils/from-search-field-metadata-entity-to-flat-search-field-metadata.util';
import { ObjectMetadataEntity } from 'src/engine/metadata-modules/object-metadata/object-metadata.entity';
import { SearchFieldMetadataEntity } from 'src/engine/metadata-modules/search-field-metadata/search-field-metadata.entity';
import { InjectWorkspaceScopedRepository } from 'src/engine/twenty-orm/workspace-scoped-repository/inject-workspace-scoped-repository.decorator';
import { WorkspaceScopedRepository } from 'src/engine/twenty-orm/workspace-scoped-repository/workspace-scoped-repository';
import { WorkspaceCache } from 'src/engine/workspace-cache/decorators/workspace-cache.decorator';
import { createIdToUniversalIdentifierMap } from 'src/engine/workspace-cache/utils/create-id-to-universal-identifier-map.util';
import { addFlatEntityToFlatEntityMapsThroughMutationOrThrow } from 'src/engine/workspace-manager/workspace-migration/utils/add-flat-entity-to-flat-entity-maps-through-mutation-or-throw.util';
@Injectable()
@WorkspaceCache('flatSearchFieldMetadataMaps')
export class WorkspaceFlatSearchFieldMetadataMapCacheService extends WorkspaceCacheProvider<FlatSearchFieldMetadataMaps> {
constructor(
@InjectWorkspaceScopedRepository(SearchFieldMetadataEntity)
private readonly searchFieldMetadataRepository: WorkspaceScopedRepository<SearchFieldMetadataEntity>,
@InjectRepository(ApplicationEntity)
private readonly applicationRepository: Repository<ApplicationEntity>,
@InjectRepository(ObjectMetadataEntity)
private readonly objectMetadataRepository: Repository<ObjectMetadataEntity>,
@InjectRepository(FieldMetadataEntity)
private readonly fieldMetadataRepository: Repository<FieldMetadataEntity>,
) {
super();
}
async computeForCache(
workspaceId: string,
): Promise<FlatSearchFieldMetadataMaps> {
const [
existingSearchFieldMetadatas,
applications,
objectMetadatas,
fieldMetadatas,
] = await Promise.all([
this.searchFieldMetadataRepository.find(workspaceId),
this.applicationRepository.find({
where: { workspaceId },
select: ['id', 'universalIdentifier'],
withDeleted: true,
}),
this.objectMetadataRepository.find({
where: { workspaceId },
select: ['id', 'universalIdentifier'],
withDeleted: true,
}),
this.fieldMetadataRepository.find({
where: { workspaceId },
select: ['id', 'universalIdentifier'],
withDeleted: true,
}),
]);
const applicationIdToUniversalIdentifierMap =
createIdToUniversalIdentifierMap(applications);
const objectMetadataIdToUniversalIdentifierMap =
createIdToUniversalIdentifierMap(objectMetadatas);
const fieldMetadataIdToUniversalIdentifierMap =
createIdToUniversalIdentifierMap(fieldMetadatas);
const flatSearchFieldMetadataMaps = createEmptyFlatEntityMaps();
for (const searchFieldMetadata of existingSearchFieldMetadatas) {
const flatSearchFieldMetadata =
fromSearchFieldMetadataEntityToFlatSearchFieldMetadata({
entity: searchFieldMetadata,
applicationIdToUniversalIdentifierMap,
objectMetadataIdToUniversalIdentifierMap,
fieldMetadataIdToUniversalIdentifierMap,
});
addFlatEntityToFlatEntityMapsThroughMutationOrThrow({
flatEntity: flatSearchFieldMetadata,
flatEntityMapsToMutate: flatSearchFieldMetadataMaps,
});
}
return flatSearchFieldMetadataMaps;
}
}
@@ -0,0 +1,5 @@
import { type FlatEntityMaps } from 'src/engine/metadata-modules/flat-entity/types/flat-entity-maps.type';
import { type FlatSearchFieldMetadata } from 'src/engine/metadata-modules/flat-search-field-metadata/types/flat-search-field-metadata.type';
export type FlatSearchFieldMetadataMaps =
FlatEntityMaps<FlatSearchFieldMetadata>;
@@ -0,0 +1,4 @@
import { type FlatEntityFrom } from 'src/engine/metadata-modules/flat-entity/types/flat-entity-from.type';
import { type SearchFieldMetadataEntity } from 'src/engine/metadata-modules/search-field-metadata/search-field-metadata.entity';
export type FlatSearchFieldMetadata = FlatEntityFrom<SearchFieldMetadataEntity>;
@@ -0,0 +1,30 @@
import { v4 } from 'uuid';
import { type FlatObjectMetadata } from 'src/engine/metadata-modules/flat-object-metadata/types/flat-object-metadata.type';
import { type UniversalFlatSearchFieldMetadata } from 'src/engine/workspace-manager/workspace-migration/universal-flat-entity/types/universal-flat-search-field-metadata.type';
export const buildFlatSearchFieldMetadataForField = ({
flatObjectMetadata,
flatFieldMetadata,
position,
}: {
flatObjectMetadata: Pick<
FlatObjectMetadata,
'applicationUniversalIdentifier' | 'universalIdentifier'
>;
flatFieldMetadata: { universalIdentifier: string };
position: number;
}): UniversalFlatSearchFieldMetadata => {
const createdAt = new Date().toISOString();
return {
universalIdentifier: v4(),
createdAt,
updatedAt: createdAt,
position,
applicationUniversalIdentifier:
flatObjectMetadata.applicationUniversalIdentifier,
objectMetadataUniversalIdentifier: flatObjectMetadata.universalIdentifier,
fieldMetadataUniversalIdentifier: flatFieldMetadata.universalIdentifier,
};
};
@@ -0,0 +1,53 @@
import { type FieldMetadataType } from 'twenty-shared/types';
import { isSearchableFieldType } from 'twenty-shared/utils';
import {
type FieldTypeAndNameMetadata,
getTsVectorColumnExpressionFromFields,
} from 'src/engine/workspace-manager/utils/get-ts-vector-column-expression.util';
export type SearchVectorTargetField = {
name: string;
type: FieldMetadataType;
// Per-object ordinal from the searchFieldMetadata row, driving deterministic order.
position: number;
// Tie-break for rows sharing a position (searchFieldMetadata universalIdentifier).
sortKey: string;
};
export const buildSearchVectorTargetField = (
field: { name: string; type: FieldMetadataType },
position: number,
sortKey: string,
): SearchVectorTargetField => ({
name: field.name,
type: field.type,
position,
sortKey,
});
// Builds the searchVector to_tsvector asExpression from an object's targeted fields.
// Callers MUST pass the POST-change field set (account for rows added/removed in the
// same operation), not re-read stale maps. Ordering is deterministic only to minimize
// asExpression churn — tsvector matching is order-insensitive.
export const computeSearchVectorAsExpressionFromSearchFieldMetadatas = (
targetSearchableFields: SearchVectorTargetField[],
): string => {
const orderedSearchableFields: FieldTypeAndNameMetadata[] = [
...targetSearchableFields,
]
.sort((a, b) => {
if (a.position !== b.position) {
return a.position - b.position;
}
return a.sortKey < b.sortKey ? -1 : a.sortKey > b.sortKey ? 1 : 0;
})
.flatMap((targetField) =>
isSearchableFieldType(targetField.type)
? [{ name: targetField.name, type: targetField.type }]
: [],
);
return getTsVectorColumnExpressionFromFields(orderedSearchableFields);
};
@@ -0,0 +1,68 @@
import { isDefined, removePropertiesFromRecord } from 'twenty-shared/utils';
import {
FlatEntityMapsException,
FlatEntityMapsExceptionCode,
} from 'src/engine/metadata-modules/flat-entity/exceptions/flat-entity-maps.exception';
import { getMetadataEntityRelationProperties } from 'src/engine/metadata-modules/flat-entity/utils/get-metadata-entity-relation-properties.util';
import { type FlatSearchFieldMetadata } from 'src/engine/metadata-modules/flat-search-field-metadata/types/flat-search-field-metadata.type';
import type { FromEntityToFlatEntityArgs } from 'src/engine/workspace-cache/types/from-entity-to-flat-entity-args.type';
export const fromSearchFieldMetadataEntityToFlatSearchFieldMetadata = ({
entity: searchFieldMetadataEntity,
applicationIdToUniversalIdentifierMap,
objectMetadataIdToUniversalIdentifierMap,
fieldMetadataIdToUniversalIdentifierMap,
}: FromEntityToFlatEntityArgs<'searchFieldMetadata'>): FlatSearchFieldMetadata => {
const searchFieldMetadataEntityWithoutRelations = removePropertiesFromRecord(
searchFieldMetadataEntity,
getMetadataEntityRelationProperties('searchFieldMetadata'),
);
const applicationUniversalIdentifier =
applicationIdToUniversalIdentifierMap.get(
searchFieldMetadataEntity.applicationId,
);
if (!isDefined(applicationUniversalIdentifier)) {
throw new FlatEntityMapsException(
`Application with id ${searchFieldMetadataEntity.applicationId} not found for searchFieldMetadata ${searchFieldMetadataEntity.id}`,
FlatEntityMapsExceptionCode.ENTITY_NOT_FOUND,
);
}
const objectMetadataUniversalIdentifier =
objectMetadataIdToUniversalIdentifierMap.get(
searchFieldMetadataEntity.objectMetadataId,
);
if (!isDefined(objectMetadataUniversalIdentifier)) {
throw new FlatEntityMapsException(
`Object metadata with id ${searchFieldMetadataEntity.objectMetadataId} not found for searchFieldMetadata ${searchFieldMetadataEntity.id}`,
FlatEntityMapsExceptionCode.ENTITY_NOT_FOUND,
);
}
const fieldMetadataUniversalIdentifier =
fieldMetadataIdToUniversalIdentifierMap.get(
searchFieldMetadataEntity.fieldMetadataId,
);
if (!isDefined(fieldMetadataUniversalIdentifier)) {
throw new FlatEntityMapsException(
`Field metadata with id ${searchFieldMetadataEntity.fieldMetadataId} not found for searchFieldMetadata ${searchFieldMetadataEntity.id}`,
FlatEntityMapsExceptionCode.ENTITY_NOT_FOUND,
);
}
return {
...searchFieldMetadataEntityWithoutRelations,
createdAt: searchFieldMetadataEntity.createdAt.toISOString(),
updatedAt: searchFieldMetadataEntity.updatedAt.toISOString(),
universalIdentifier:
searchFieldMetadataEntityWithoutRelations.universalIdentifier,
applicationUniversalIdentifier,
objectMetadataUniversalIdentifier,
fieldMetadataUniversalIdentifier,
};
};
@@ -12,6 +12,7 @@ import {
import { type WorkspaceEntityDuplicateCriteria } from 'src/engine/api/graphql/workspace-query-builder/types/workspace-entity-duplicate-criteria.type';
import { FieldMetadataEntity } from 'src/engine/metadata-modules/field-metadata/field-metadata.entity';
import { IndexMetadataEntity } from 'src/engine/metadata-modules/index-metadata/index-metadata.entity';
import { SearchFieldMetadataEntity } from 'src/engine/metadata-modules/search-field-metadata/search-field-metadata.entity';
import { type ObjectStandardOverridesDTO } from 'src/engine/metadata-modules/object-metadata/dtos/object-standard-overrides.dto';
import { FieldPermissionEntity } from 'src/engine/metadata-modules/object-permission/field-permission/field-permission.entity';
import { ObjectPermissionEntity } from 'src/engine/metadata-modules/object-permission/object-permission.entity';
@@ -142,6 +143,15 @@ export class ObjectMetadataEntity
})
indexMetadatas: Relation<IndexMetadataEntity[]>;
@OneToMany(
() => SearchFieldMetadataEntity,
(searchFieldMetadata) => searchFieldMetadata.objectMetadata,
{
cascade: true,
},
)
searchFieldMetadatas: Relation<SearchFieldMetadataEntity[]>;
@CreateDateColumn({ type: 'timestamptz' })
createdAt: Date;
@@ -89,6 +89,7 @@ export class ObjectMetadataService extends TypeOrmQueryService<ObjectMetadataEnt
flatViewFieldMaps: existingFlatViewFieldMaps,
flatViewMaps: existingFlatViewMaps,
flatCommandMenuItemMaps: existingFlatCommandMenuItemMaps,
flatSearchFieldMetadataMaps: existingFlatSearchFieldMetadataMaps,
} = await this.flatEntityMapsCacheService.getOrRecomputeManyOrAllFlatEntityMaps(
{
workspaceId,
@@ -99,6 +100,7 @@ export class ObjectMetadataService extends TypeOrmQueryService<ObjectMetadataEnt
'flatViewFieldMaps',
'flatViewMaps',
'flatCommandMenuItemMaps',
'flatSearchFieldMetadataMaps',
],
},
);
@@ -110,6 +112,7 @@ export class ObjectMetadataService extends TypeOrmQueryService<ObjectMetadataEnt
flatIndexMetadatasToUpdate,
flatViewFieldsToCreate,
flatViewFieldsToUpdate,
searchFieldMetadatasToCreate,
} = fromUpdateObjectInputToFlatObjectMetadataAndRelatedFlatEntities({
flatFieldMetadataMaps: existingFlatFieldMetadataMaps,
flatObjectMetadataMaps: existingFlatObjectMetadataMaps,
@@ -117,6 +120,7 @@ export class ObjectMetadataService extends TypeOrmQueryService<ObjectMetadataEnt
flatIndexMaps: existingFlatIndexMaps,
flatViewFieldMaps: existingFlatViewFieldMaps,
flatViewMaps: existingFlatViewMaps,
flatSearchFieldMetadataMaps: existingFlatSearchFieldMetadataMaps,
});
const existingFlatObjectMetadata = findFlatEntityByIdInFlatEntityMaps({
@@ -177,6 +181,11 @@ export class ObjectMetadataService extends TypeOrmQueryService<ObjectMetadataEnt
flatEntityToDelete: [],
flatEntityToUpdate: flatViewFieldsToUpdate,
},
searchFieldMetadata: {
flatEntityToCreate: searchFieldMetadatasToCreate,
flatEntityToDelete: [],
flatEntityToUpdate: [],
},
commandMenuItem: {
flatEntityToCreate: [],
flatEntityToDelete: [],
@@ -485,6 +494,7 @@ export class ObjectMetadataService extends TypeOrmQueryService<ObjectMetadataEnt
const {
flatObjectMetadataToCreate,
flatIndexMetadataToCreate,
flatSearchFieldMetadataToCreate,
flatFieldMetadataToCreateOnObject,
relationTargetFlatFieldMetadataToCreate,
} = fromCreateObjectInputToFlatObjectMetadataAndFlatFieldMetadatasToCreate({
@@ -620,6 +630,11 @@ export class ObjectMetadataService extends TypeOrmQueryService<ObjectMetadataEnt
flatEntityToDelete: [],
flatEntityToUpdate: [],
},
searchFieldMetadata: {
flatEntityToCreate: flatSearchFieldMetadataToCreate,
flatEntityToDelete: [],
flatEntityToUpdate: [],
},
...(isDefined(flatNavigationMenuItemToCreate)
? {
navigationMenuItem: {
@@ -0,0 +1,36 @@
import { isDefined } from 'twenty-shared/utils';
import { buildFlatSearchFieldMetadataForField } from 'src/engine/metadata-modules/flat-search-field-metadata/utils/build-flat-search-field-metadata-for-field.util';
import { type DefaultFlatFieldForCustomObjectMaps } from 'src/engine/metadata-modules/object-metadata/utils/build-default-flat-field-metadatas-for-custom-object.util';
import { type UniversalFlatObjectMetadata } from 'src/engine/workspace-manager/workspace-migration/universal-flat-entity/types/universal-flat-object-metadata.type';
import { type UniversalFlatSearchFieldMetadata } from 'src/engine/workspace-manager/workspace-migration/universal-flat-entity/types/universal-flat-search-field-metadata.type';
// Mirrors the custom-object searchVector, which indexes the name field only. Junction
// objects (skipNameField) have no name field and therefore get no row.
export const buildDefaultSearchFieldMetadatasForCustomObject = ({
flatObjectMetadata,
defaultFlatFieldForCustomObjectMaps,
}: {
flatObjectMetadata: UniversalFlatObjectMetadata & { id: string };
defaultFlatFieldForCustomObjectMaps: DefaultFlatFieldForCustomObjectMaps;
}): {
searchFieldMetadatas: UniversalFlatSearchFieldMetadata[];
} => {
const nameField = defaultFlatFieldForCustomObjectMaps.fields.nameField;
if (!isDefined(nameField)) {
return {
searchFieldMetadatas: [],
};
}
const nameSearchFieldMetadata = buildFlatSearchFieldMetadataForField({
flatObjectMetadata,
flatFieldMetadata: nameField,
position: 0,
});
return {
searchFieldMetadatas: [nameSearchFieldMetadata],
};
};
@@ -0,0 +1,6 @@
export enum SearchFieldMetadataExceptionCode {
SEARCH_FIELD_METADATA_NOT_FOUND = 'SEARCH_FIELD_METADATA_NOT_FOUND',
INVALID_SEARCH_FIELD_METADATA_DATA = 'INVALID_SEARCH_FIELD_METADATA_DATA',
OBJECT_METADATA_NOT_FOUND = 'OBJECT_METADATA_NOT_FOUND',
FIELD_METADATA_NOT_FOUND = 'FIELD_METADATA_NOT_FOUND',
}
@@ -13,23 +13,27 @@ import {
import { FieldMetadataEntity } from 'src/engine/metadata-modules/field-metadata/field-metadata.entity';
import { ObjectMetadataEntity } from 'src/engine/metadata-modules/object-metadata/object-metadata.entity';
import { WorkspaceRelatedEntity } from 'src/engine/workspace-manager/types/workspace-related-entity';
import { SyncableEntity } from 'src/engine/workspace-manager/types/syncable-entity.interface';
@Entity('searchFieldMetadata')
@Entity({ name: 'searchFieldMetadata', schema: 'core' })
@Unique('IDX_SEARCH_FIELD_METADATA_OBJECT_FIELD_UNIQUE', [
'objectMetadataId',
'fieldMetadataId',
])
@Index('IDX_SEARCH_FIELD_METADATA_WORKSPACE_ID', ['workspaceId'])
@Index('IDX_SEARCH_FIELD_METADATA_OBJECT_METADATA_ID', ['objectMetadataId'])
export class SearchFieldMetadataEntity extends WorkspaceRelatedEntity {
export class SearchFieldMetadataEntity extends SyncableEntity {
@PrimaryGeneratedColumn('uuid')
id: string;
@Column({ nullable: false, type: 'uuid' })
objectMetadataId: string;
@ManyToOne(() => ObjectMetadataEntity, { onDelete: 'CASCADE' })
@ManyToOne(
() => ObjectMetadataEntity,
(objectMetadata) => objectMetadata.searchFieldMetadatas,
{ onDelete: 'CASCADE' },
)
@JoinColumn({ name: 'objectMetadataId' })
objectMetadata: Relation<ObjectMetadataEntity>;
@@ -40,6 +44,9 @@ export class SearchFieldMetadataEntity extends WorkspaceRelatedEntity {
@JoinColumn({ name: 'fieldMetadataId' })
fieldMetadata: Relation<FieldMetadataEntity>;
@Column({ nullable: false, type: 'float' })
position: number;
@CreateDateColumn({ type: 'timestamptz' })
createdAt: Date;