Wire up search field metadata (#21964)
## Part 1 - Exact scope of the current PR (#21964) close https://github.com/twentyhq/core-team-issues/issues/2586 This PR introduces `searchFieldMetadata` as a first-class flat metadata entity and migrates the existing search surface onto it, with **no change to which records are searchable** (ISO with `main`). In scope (what the PR does): - New flat entity `searchFieldMetadata` (universalIdentifier, applicationId, **`position`**, maps, conversions), registered in the central flat-entity constants and the migration build orchestrator. - `searchVector.asExpression` is **derived server-side** from `searchFieldMetadata` rows (validated by `isSafeTsVectorExpression`); never trusted from client input. - **Derivation order is deterministic, driven by each row's `position`** ([compute-search-vector-as-expression-from-search-field-metadatas.util.ts](packages/twenty-server/src/engine/metadata-modules/flat-search-field-metadata/utils/compute-search-vector-as-expression-from-search-field-metadatas.util.ts)), replacing the previous non-deterministic `(createdAt, id)` sort. That sort collapsed to random UUIDs for standard fields (same `createdAt`), so any rename/relabel rewrote the `STORED` generated column to a logically-identical-but-textually-different expression and produced a permanent per-workspace diff vs the standard definition. Ordering now equals provisioning order; ties break on `universalIdentifier`. - Provisioning at object creation mirrors the existing surface exactly **and seeds `position`**: - custom objects -> the `name` field only, at `position: 0` ([build-default-search-field-metadatas-for-custom-object.util.ts](packages/twenty-server/src/engine/metadata-modules/object-metadata/utils/build-default-search-field-metadatas-for-custom-object.util.ts)) - standard objects -> their curated `SEARCH_FIELDS_FOR_*` sets, `position` = the curated index - Backfill (instance + workspace commands in `2-16`) provisions rows for existing workspaces with the same surface **and the same positions** (standard from the curated standard maps, custom `name` = `0`), scoped to the workspace's own custom application ([build-search-field-metadata-backfill-operations.util.ts](packages/twenty-server/src/database/commands/upgrade-version-command/2-16/utils/build-search-field-metadata-backfill-operations.util.ts)). The `position` column is added in the same `2-16` fast instance command as `universalIdentifier`/`applicationId`. - Field rename of an already-indexed field recomputes `asExpression` (positions preserved, so order is stable) ([recompute-search-vector-on-field-rename.util.ts](packages/twenty-server/src/engine/metadata-modules/flat-field-metadata/utils/recompute-search-vector-on-field-rename.util.ts)). - Field delete drops the matching row(s) and recomputes; remaining rows keep their relative order (no renumber) ([from-delete-field-input-to-flat-field-metadatas-to-delete.util.ts](packages/twenty-server/src/engine/metadata-modules/flat-field-metadata/utils/from-delete-field-input-to-flat-field-metadatas-to-delete.util.ts)). - Object relabel is **additive** and ISO/regression-fix only: it indexes the new label identifier **appended last (`position = max(existing) + 1`)** without dropping `name` ([recompute-search-vector-on-label-identifier-update.util.ts](packages/twenty-server/src/engine/metadata-modules/flat-object-metadata/utils/recompute-search-vector-on-label-identifier-update.util.ts)). This is a deliberate, temporary bridge. Explicitly OUT of scope (deferred): - No API to edit `searchFieldMetadata` (no user-facing search-field configuration, including `position` — it is internal and only written by provisioning/backfill/recompute). - No auto-indexing of arbitrary searchable fields. Creating a custom TEXT/EMAILS/etc. field does NOT add it to search (the `computeSearchFieldMetadataCreationForFields` behavior was removed in `e6820ad`). - No field-type-transition handling (field type is immutable - not in `FLAT_FIELD_METADATA_EDITABLE_PROPERTIES`, so that path was dead code). - No `position` validation (uniqueness/range) and no multi-vector / per-field `weight` config — deferred to the configurable-search follow-up (#1428). Net: `searchFieldMetadata` becomes the source of truth for the *same* surface as `main`. The only intentional divergences from `main` are "relabel preserves `name`" (additive) and the deterministic `position`-ordered `asExpression` (a correctness/perf fix that is byte-identical to provisioning order, so it does not change the searchable surface). --------- Co-authored-by: Félix Malfait <felix@twenty.com>
This commit is contained in:
+25
@@ -0,0 +1,25 @@
|
||||
import { QueryRunner } from 'typeorm';
|
||||
|
||||
import { RegisteredInstanceCommand } from 'src/engine/core-modules/upgrade/decorators/registered-instance-command.decorator';
|
||||
import { FastInstanceCommand } from 'src/engine/core-modules/upgrade/interfaces/fast-instance-command.interface';
|
||||
|
||||
@RegisteredInstanceCommand('2.16.0', 1782200000000)
|
||||
export class AddUniversalIdentifierAndApplicationIdToSearchFieldMetadataFastInstanceCommand implements FastInstanceCommand {
|
||||
public async up(queryRunner: QueryRunner): Promise<void> {
|
||||
await queryRunner.query('ALTER TABLE "core"."searchFieldMetadata" ADD "universalIdentifier" uuid NOT NULL');
|
||||
await queryRunner.query('ALTER TABLE "core"."searchFieldMetadata" ADD "applicationId" uuid NOT NULL');
|
||||
// The column is NOT NULL but added without a default: searchFieldMetadata is dormant/empty
|
||||
// at instance-command time and the 2-16 backfill workspace command populates positions.
|
||||
await queryRunner.query('ALTER TABLE "core"."searchFieldMetadata" ADD "position" double precision NOT NULL');
|
||||
await queryRunner.query('CREATE UNIQUE INDEX "IDX_c2e441c901b45221a70d325349" ON "core"."searchFieldMetadata" ("workspaceId", "universalIdentifier") ');
|
||||
await queryRunner.query('ALTER TABLE "core"."searchFieldMetadata" ADD CONSTRAINT "FK_927b6101a5d9562a558a18ed412" FOREIGN KEY ("applicationId") REFERENCES "core"."application"("id") ON DELETE CASCADE ON UPDATE NO ACTION');
|
||||
}
|
||||
|
||||
public async down(queryRunner: QueryRunner): Promise<void> {
|
||||
await queryRunner.query('ALTER TABLE "core"."searchFieldMetadata" DROP CONSTRAINT "FK_927b6101a5d9562a558a18ed412"');
|
||||
await queryRunner.query('DROP INDEX "core"."IDX_c2e441c901b45221a70d325349"');
|
||||
await queryRunner.query('ALTER TABLE "core"."searchFieldMetadata" DROP COLUMN "position"');
|
||||
await queryRunner.query('ALTER TABLE "core"."searchFieldMetadata" DROP COLUMN "applicationId"');
|
||||
await queryRunner.query('ALTER TABLE "core"."searchFieldMetadata" DROP COLUMN "universalIdentifier"');
|
||||
}
|
||||
}
|
||||
+18
@@ -0,0 +1,18 @@
|
||||
import { Module } from '@nestjs/common';
|
||||
|
||||
import { WorkspaceIteratorModule } from 'src/database/commands/command-runners/workspace-iterator.module';
|
||||
import { BackfillSearchFieldMetadataCommand } from 'src/database/commands/upgrade-version-command/2-16/2-16-workspace-command-1799100000000-backfill-search-field-metadata.command';
|
||||
import { ApplicationModule } from 'src/engine/core-modules/application/application.module';
|
||||
import { WorkspaceCacheModule } from 'src/engine/workspace-cache/workspace-cache.module';
|
||||
import { WorkspaceMigrationModule } from 'src/engine/workspace-manager/workspace-migration/workspace-migration.module';
|
||||
|
||||
@Module({
|
||||
imports: [
|
||||
ApplicationModule,
|
||||
WorkspaceCacheModule,
|
||||
WorkspaceIteratorModule,
|
||||
WorkspaceMigrationModule,
|
||||
],
|
||||
providers: [BackfillSearchFieldMetadataCommand],
|
||||
})
|
||||
export class V2_16_UpgradeVersionCommandModule {}
|
||||
+151
@@ -0,0 +1,151 @@
|
||||
import { Command } from 'nest-commander';
|
||||
import { isDefined } from 'twenty-shared/utils';
|
||||
|
||||
import { ActiveOrSuspendedWorkspaceCommandRunner } from 'src/database/commands/command-runners/active-or-suspended-workspace.command-runner';
|
||||
import { WorkspaceIteratorService } from 'src/database/commands/command-runners/workspace-iterator.service';
|
||||
import { type RunOnWorkspaceArgs } from 'src/database/commands/command-runners/workspace.command-runner';
|
||||
import { buildSearchFieldMetadataBackfillOperations } from 'src/database/commands/upgrade-version-command/2-16/utils/build-search-field-metadata-backfill-operations.util';
|
||||
import { ApplicationService } from 'src/engine/core-modules/application/application.service';
|
||||
import { RegisteredWorkspaceCommand } from 'src/engine/core-modules/upgrade/decorators/registered-workspace-command.decorator';
|
||||
import { computeTwentyStandardApplicationAllFlatEntityMaps } from 'src/engine/workspace-manager/twenty-standard-application/utils/twenty-standard-application-all-flat-entity-maps.constant';
|
||||
import { WorkspaceCacheService } from 'src/engine/workspace-cache/services/workspace-cache.service';
|
||||
import { WorkspaceMigrationValidateBuildAndRunService } from 'src/engine/workspace-manager/workspace-migration/services/workspace-migration-validate-build-and-run-service';
|
||||
|
||||
@RegisteredWorkspaceCommand('2.16.0', 1799100000000)
|
||||
@Command({
|
||||
name: 'upgrade:2-16:backfill-search-field-metadata',
|
||||
description:
|
||||
'Backfill searchFieldMetadata rows for each searchable object. Standard objects mirror their SEARCH_FIELDS_FOR_* set; custom objects get their label-identifier field. Idempotent: existing rows are skipped.',
|
||||
})
|
||||
export class BackfillSearchFieldMetadataCommand extends ActiveOrSuspendedWorkspaceCommandRunner {
|
||||
constructor(
|
||||
protected readonly workspaceIteratorService: WorkspaceIteratorService,
|
||||
private readonly applicationService: ApplicationService,
|
||||
private readonly workspaceMigrationValidateBuildAndRunService: WorkspaceMigrationValidateBuildAndRunService,
|
||||
private readonly workspaceCacheService: WorkspaceCacheService,
|
||||
) {
|
||||
super(workspaceIteratorService);
|
||||
}
|
||||
|
||||
override async runOnWorkspace({
|
||||
workspaceId,
|
||||
options,
|
||||
}: RunOnWorkspaceArgs): Promise<void> {
|
||||
const isDryRun = options.dryRun ?? false;
|
||||
|
||||
const {
|
||||
flatObjectMetadataMaps,
|
||||
flatFieldMetadataMaps,
|
||||
flatSearchFieldMetadataMaps,
|
||||
} = await this.workspaceCacheService.getOrRecompute(workspaceId, [
|
||||
'flatObjectMetadataMaps',
|
||||
'flatFieldMetadataMaps',
|
||||
'flatSearchFieldMetadataMaps',
|
||||
]);
|
||||
|
||||
const { twentyStandardFlatApplication, workspaceCustomFlatApplication } =
|
||||
await this.applicationService.findWorkspaceTwentyStandardAndCustomApplicationOrThrow(
|
||||
{ workspaceId },
|
||||
);
|
||||
|
||||
// The standard-application sync does not run during upgrades, so standard objects'
|
||||
// rows are backfilled from the same definition provisioning uses
|
||||
// (SEARCH_FIELDS_FOR_*), not by parsing the searchVector asExpression.
|
||||
const { allFlatEntityMaps: standardAllFlatEntityMaps } =
|
||||
computeTwentyStandardApplicationAllFlatEntityMaps({
|
||||
now: new Date().toISOString(),
|
||||
workspaceId,
|
||||
twentyStandardApplicationId: twentyStandardFlatApplication.id,
|
||||
});
|
||||
|
||||
const {
|
||||
flatSearchFieldMetadatasToCreateByApplicationUniversalIdentifier,
|
||||
} = buildSearchFieldMetadataBackfillOperations({
|
||||
flatObjectMetadataMaps,
|
||||
flatFieldMetadataMaps,
|
||||
flatSearchFieldMetadataMaps,
|
||||
standardFlatSearchFieldMetadataMaps:
|
||||
standardAllFlatEntityMaps.flatSearchFieldMetadataMaps,
|
||||
customApplicationId: workspaceCustomFlatApplication.id,
|
||||
});
|
||||
|
||||
const applicationUniversalIdentifiers = Object.keys(
|
||||
flatSearchFieldMetadatasToCreateByApplicationUniversalIdentifier,
|
||||
);
|
||||
|
||||
const totalRowsToCreate = applicationUniversalIdentifiers.reduce(
|
||||
(total, applicationUniversalIdentifier) =>
|
||||
total +
|
||||
(flatSearchFieldMetadatasToCreateByApplicationUniversalIdentifier[
|
||||
applicationUniversalIdentifier
|
||||
]?.length ?? 0),
|
||||
0,
|
||||
);
|
||||
|
||||
if (totalRowsToCreate === 0) {
|
||||
this.logger.log(
|
||||
`No missing searchFieldMetadata rows for workspace ${workspaceId}, skipping`,
|
||||
);
|
||||
|
||||
return;
|
||||
}
|
||||
|
||||
this.logger.log(
|
||||
`${isDryRun ? '[DRY RUN] ' : ''}Found ${totalRowsToCreate} missing searchFieldMetadata row(s) for workspace ${workspaceId} across ${applicationUniversalIdentifiers.length} application(s)`,
|
||||
);
|
||||
|
||||
if (isDryRun) {
|
||||
return;
|
||||
}
|
||||
|
||||
// One migration per application: the runner assigns applicationId from the single
|
||||
// application passed here, keeping custom-object rows tied to the custom application.
|
||||
for (const applicationUniversalIdentifier of applicationUniversalIdentifiers) {
|
||||
const flatSearchFieldMetadataToCreate =
|
||||
flatSearchFieldMetadatasToCreateByApplicationUniversalIdentifier[
|
||||
applicationUniversalIdentifier
|
||||
];
|
||||
|
||||
if (
|
||||
!isDefined(flatSearchFieldMetadataToCreate) ||
|
||||
flatSearchFieldMetadataToCreate.length === 0
|
||||
) {
|
||||
continue;
|
||||
}
|
||||
|
||||
const validateAndBuildResult =
|
||||
await this.workspaceMigrationValidateBuildAndRunService.validateBuildAndRunWorkspaceMigration(
|
||||
{
|
||||
isSystemBuild: true,
|
||||
allFlatEntityOperationByMetadataName: {
|
||||
searchFieldMetadata: {
|
||||
flatEntityToCreate: flatSearchFieldMetadataToCreate,
|
||||
flatEntityToDelete: [],
|
||||
flatEntityToUpdate: [],
|
||||
},
|
||||
},
|
||||
workspaceId,
|
||||
applicationUniversalIdentifier,
|
||||
},
|
||||
);
|
||||
|
||||
if (validateAndBuildResult.status === 'fail') {
|
||||
this.logger.error(
|
||||
`Failed to persist searchFieldMetadata rows for application ${applicationUniversalIdentifier}:\n${JSON.stringify(
|
||||
validateAndBuildResult,
|
||||
null,
|
||||
2,
|
||||
)}`,
|
||||
);
|
||||
|
||||
throw new Error(
|
||||
`Failed to persist searchFieldMetadata rows for workspace ${workspaceId}`,
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
this.logger.log(
|
||||
`Successfully backfilled ${totalRowsToCreate} searchFieldMetadata row(s) for workspace ${workspaceId}`,
|
||||
);
|
||||
}
|
||||
}
|
||||
+646
@@ -0,0 +1,646 @@
|
||||
import { TWENTY_STANDARD_APPLICATION_UNIVERSAL_IDENTIFIER } from 'twenty-shared/application';
|
||||
import { FieldMetadataType } from 'twenty-shared/types';
|
||||
|
||||
import { buildSearchFieldMetadataBackfillOperations } from 'src/database/commands/upgrade-version-command/2-16/utils/build-search-field-metadata-backfill-operations.util';
|
||||
import { type FlatEntityMaps } from 'src/engine/metadata-modules/flat-entity/types/flat-entity-maps.type';
|
||||
import { getFlatFieldMetadataMock } from 'src/engine/metadata-modules/flat-field-metadata/__mocks__/get-flat-field-metadata.mock';
|
||||
import { type FlatFieldMetadata } from 'src/engine/metadata-modules/flat-field-metadata/types/flat-field-metadata.type';
|
||||
import {
|
||||
getFlatObjectMetadataMock,
|
||||
getStandardFlatObjectMetadataMock,
|
||||
} from 'src/engine/metadata-modules/flat-object-metadata/__mocks__/get-flat-object-metadata.mock';
|
||||
import { type FlatObjectMetadata } from 'src/engine/metadata-modules/flat-object-metadata/types/flat-object-metadata.type';
|
||||
import { type FlatSearchFieldMetadata } from 'src/engine/metadata-modules/flat-search-field-metadata/types/flat-search-field-metadata.type';
|
||||
|
||||
const CUSTOM_APPLICATION_UNIVERSAL_IDENTIFIER = 'custom-application-uid';
|
||||
const CUSTOM_APPLICATION_ID = 'custom-application-id';
|
||||
|
||||
const buildUniversalIdentifiersByApplicationId = (
|
||||
flatObjectMetadatas: FlatObjectMetadata[],
|
||||
): Record<string, string[]> =>
|
||||
flatObjectMetadatas.reduce<Record<string, string[]>>(
|
||||
(universalIdentifiersByApplicationId, flatObjectMetadata) => {
|
||||
const applicationUniversalIdentifiers =
|
||||
universalIdentifiersByApplicationId[flatObjectMetadata.applicationId] ??
|
||||
[];
|
||||
|
||||
applicationUniversalIdentifiers.push(
|
||||
flatObjectMetadata.universalIdentifier,
|
||||
);
|
||||
|
||||
universalIdentifiersByApplicationId[flatObjectMetadata.applicationId] =
|
||||
applicationUniversalIdentifiers;
|
||||
|
||||
return universalIdentifiersByApplicationId;
|
||||
},
|
||||
{},
|
||||
);
|
||||
|
||||
const buildFlatObjectMetadataMaps = (
|
||||
flatObjectMetadatas: FlatObjectMetadata[],
|
||||
): FlatEntityMaps<FlatObjectMetadata> => ({
|
||||
byUniversalIdentifier: Object.fromEntries(
|
||||
flatObjectMetadatas.map((flatObjectMetadata) => [
|
||||
flatObjectMetadata.universalIdentifier,
|
||||
flatObjectMetadata,
|
||||
]),
|
||||
),
|
||||
universalIdentifierById: Object.fromEntries(
|
||||
flatObjectMetadatas.map((flatObjectMetadata) => [
|
||||
flatObjectMetadata.id,
|
||||
flatObjectMetadata.universalIdentifier,
|
||||
]),
|
||||
),
|
||||
universalIdentifiersByApplicationId:
|
||||
buildUniversalIdentifiersByApplicationId(flatObjectMetadatas),
|
||||
});
|
||||
|
||||
const buildFlatFieldMetadataMaps = (
|
||||
flatFieldMetadatas: FlatFieldMetadata[],
|
||||
): FlatEntityMaps<FlatFieldMetadata> => ({
|
||||
byUniversalIdentifier: Object.fromEntries(
|
||||
flatFieldMetadatas.map((flatFieldMetadata) => [
|
||||
flatFieldMetadata.universalIdentifier,
|
||||
flatFieldMetadata,
|
||||
]),
|
||||
),
|
||||
universalIdentifierById: Object.fromEntries(
|
||||
flatFieldMetadatas.map((flatFieldMetadata) => [
|
||||
flatFieldMetadata.id,
|
||||
flatFieldMetadata.universalIdentifier,
|
||||
]),
|
||||
),
|
||||
universalIdentifiersByApplicationId: {},
|
||||
});
|
||||
|
||||
const buildFlatSearchFieldMetadataMaps = (
|
||||
flatSearchFieldMetadatas: FlatSearchFieldMetadata[],
|
||||
): FlatEntityMaps<FlatSearchFieldMetadata> => ({
|
||||
byUniversalIdentifier: Object.fromEntries(
|
||||
flatSearchFieldMetadatas.map((flatSearchFieldMetadata) => [
|
||||
flatSearchFieldMetadata.universalIdentifier,
|
||||
flatSearchFieldMetadata,
|
||||
]),
|
||||
),
|
||||
universalIdentifierById: Object.fromEntries(
|
||||
flatSearchFieldMetadatas.map((flatSearchFieldMetadata) => [
|
||||
flatSearchFieldMetadata.id,
|
||||
flatSearchFieldMetadata.universalIdentifier,
|
||||
]),
|
||||
),
|
||||
universalIdentifiersByApplicationId: {},
|
||||
});
|
||||
|
||||
const buildSearchFieldMetadata = ({
|
||||
id,
|
||||
universalIdentifier,
|
||||
objectMetadataId,
|
||||
fieldMetadataId,
|
||||
objectMetadataUniversalIdentifier,
|
||||
fieldMetadataUniversalIdentifier,
|
||||
applicationUniversalIdentifier,
|
||||
position = 0,
|
||||
}: {
|
||||
id: string;
|
||||
universalIdentifier: string;
|
||||
objectMetadataId: string;
|
||||
fieldMetadataId: string;
|
||||
objectMetadataUniversalIdentifier: string;
|
||||
fieldMetadataUniversalIdentifier: string;
|
||||
applicationUniversalIdentifier: string;
|
||||
position?: number;
|
||||
}): FlatSearchFieldMetadata => {
|
||||
const createdAt = '2024-01-01T00:00:00.000Z';
|
||||
|
||||
return {
|
||||
id,
|
||||
universalIdentifier,
|
||||
objectMetadataId,
|
||||
fieldMetadataId,
|
||||
objectMetadataUniversalIdentifier,
|
||||
fieldMetadataUniversalIdentifier,
|
||||
applicationId: 'unused-application-id',
|
||||
applicationUniversalIdentifier,
|
||||
position,
|
||||
workspaceId: 'workspace-id',
|
||||
createdAt,
|
||||
updatedAt: createdAt,
|
||||
};
|
||||
};
|
||||
|
||||
// Custom object with a searchable `name` field plus a TEXT field whose name
|
||||
// overlaps it by prefix (name / nameDescription). Only the `name` field should
|
||||
// produce a row; the exact-name match makes the prefix overlap irrelevant. This
|
||||
// mirrors pre-2.15 provisioning, which indexes the custom object's `name` field only.
|
||||
const buildCustomObjectFixture = () => {
|
||||
const nameFieldId = 'name-field-id';
|
||||
const nameDescriptionFieldId = 'name-description-field-id';
|
||||
const customObjectUniversalIdentifier = 'custom-object-uid';
|
||||
const customObjectId = 'custom-object-id';
|
||||
|
||||
const nameField = getFlatFieldMetadataMock({
|
||||
id: nameFieldId,
|
||||
universalIdentifier: 'name-field-uid',
|
||||
objectMetadataId: customObjectId,
|
||||
objectMetadataUniversalIdentifier: customObjectUniversalIdentifier,
|
||||
type: FieldMetadataType.TEXT,
|
||||
name: 'name',
|
||||
applicationUniversalIdentifier: CUSTOM_APPLICATION_UNIVERSAL_IDENTIFIER,
|
||||
});
|
||||
|
||||
const nameDescriptionField = getFlatFieldMetadataMock({
|
||||
id: nameDescriptionFieldId,
|
||||
universalIdentifier: 'name-description-field-uid',
|
||||
objectMetadataId: customObjectId,
|
||||
objectMetadataUniversalIdentifier: customObjectUniversalIdentifier,
|
||||
type: FieldMetadataType.TEXT,
|
||||
name: 'nameDescription',
|
||||
applicationUniversalIdentifier: CUSTOM_APPLICATION_UNIVERSAL_IDENTIFIER,
|
||||
});
|
||||
|
||||
const customObject = getFlatObjectMetadataMock({
|
||||
id: customObjectId,
|
||||
universalIdentifier: customObjectUniversalIdentifier,
|
||||
isSearchable: true,
|
||||
applicationId: CUSTOM_APPLICATION_ID,
|
||||
applicationUniversalIdentifier: CUSTOM_APPLICATION_UNIVERSAL_IDENTIFIER,
|
||||
labelIdentifierFieldMetadataId: nameFieldId,
|
||||
fieldUniversalIdentifiers: [
|
||||
nameField.universalIdentifier,
|
||||
nameDescriptionField.universalIdentifier,
|
||||
],
|
||||
});
|
||||
|
||||
return { customObject, nameField, nameDescriptionField };
|
||||
};
|
||||
|
||||
describe('buildSearchFieldMetadataBackfillOperations', () => {
|
||||
it('selects only the name field for a custom object, ignoring a prefix-overlapping field name', () => {
|
||||
const { customObject, nameField, nameDescriptionField } =
|
||||
buildCustomObjectFixture();
|
||||
|
||||
const { flatSearchFieldMetadatasToCreateByApplicationUniversalIdentifier } =
|
||||
buildSearchFieldMetadataBackfillOperations({
|
||||
flatObjectMetadataMaps: buildFlatObjectMetadataMaps([customObject]),
|
||||
flatFieldMetadataMaps: buildFlatFieldMetadataMaps([
|
||||
nameField,
|
||||
nameDescriptionField,
|
||||
]),
|
||||
flatSearchFieldMetadataMaps: buildFlatSearchFieldMetadataMaps([]),
|
||||
standardFlatSearchFieldMetadataMaps:
|
||||
buildFlatSearchFieldMetadataMaps([]),
|
||||
customApplicationId: CUSTOM_APPLICATION_ID,
|
||||
});
|
||||
|
||||
const customApplicationRows =
|
||||
flatSearchFieldMetadatasToCreateByApplicationUniversalIdentifier[
|
||||
CUSTOM_APPLICATION_UNIVERSAL_IDENTIFIER
|
||||
];
|
||||
|
||||
expect(customApplicationRows).toHaveLength(1);
|
||||
expect(customApplicationRows?.[0].fieldMetadataUniversalIdentifier).toBe(
|
||||
nameField.universalIdentifier,
|
||||
);
|
||||
expect(customApplicationRows?.[0].objectMetadataUniversalIdentifier).toBe(
|
||||
customObject.universalIdentifier,
|
||||
);
|
||||
// Custom object name field is seeded at position 0.
|
||||
expect(customApplicationRows?.[0].position).toBe(0);
|
||||
// No spurious row for the prefix-overlapping `nameDescription` field.
|
||||
expect(
|
||||
customApplicationRows?.some(
|
||||
(row) =>
|
||||
row.fieldMetadataUniversalIdentifier ===
|
||||
nameDescriptionField.universalIdentifier,
|
||||
),
|
||||
).toBe(false);
|
||||
});
|
||||
|
||||
it('creates no row for a searchable custom object that has no name field (junction object whose label identifier is a UUID id field)', () => {
|
||||
const junctionObjectId = 'junction-object-id';
|
||||
const junctionObjectUniversalIdentifier = 'junction-object-uid';
|
||||
const idFieldId = 'junction-id-field-id';
|
||||
|
||||
// No name field: the label identifier falls back to the UUID `id` field.
|
||||
// UUID is a searchable type, so a label-identifier-based derivation would
|
||||
// wrongly emit a row — but pre-2.15 indexes nothing here, so backfill must
|
||||
// skip it. Regression guard for the junction-object over-creation bug.
|
||||
const idField = getFlatFieldMetadataMock({
|
||||
id: idFieldId,
|
||||
universalIdentifier: 'junction-id-field-uid',
|
||||
objectMetadataId: junctionObjectId,
|
||||
objectMetadataUniversalIdentifier: junctionObjectUniversalIdentifier,
|
||||
type: FieldMetadataType.UUID,
|
||||
name: 'id',
|
||||
applicationUniversalIdentifier: CUSTOM_APPLICATION_UNIVERSAL_IDENTIFIER,
|
||||
});
|
||||
|
||||
const junctionObject = getFlatObjectMetadataMock({
|
||||
id: junctionObjectId,
|
||||
universalIdentifier: junctionObjectUniversalIdentifier,
|
||||
isSearchable: true,
|
||||
applicationId: CUSTOM_APPLICATION_ID,
|
||||
applicationUniversalIdentifier: CUSTOM_APPLICATION_UNIVERSAL_IDENTIFIER,
|
||||
labelIdentifierFieldMetadataId: idFieldId,
|
||||
fieldUniversalIdentifiers: [idField.universalIdentifier],
|
||||
});
|
||||
|
||||
const { flatSearchFieldMetadatasToCreateByApplicationUniversalIdentifier } =
|
||||
buildSearchFieldMetadataBackfillOperations({
|
||||
flatObjectMetadataMaps: buildFlatObjectMetadataMaps([junctionObject]),
|
||||
flatFieldMetadataMaps: buildFlatFieldMetadataMaps([idField]),
|
||||
flatSearchFieldMetadataMaps: buildFlatSearchFieldMetadataMaps([]),
|
||||
standardFlatSearchFieldMetadataMaps:
|
||||
buildFlatSearchFieldMetadataMaps([]),
|
||||
customApplicationId: CUSTOM_APPLICATION_ID,
|
||||
});
|
||||
|
||||
expect(
|
||||
Object.keys(
|
||||
flatSearchFieldMetadatasToCreateByApplicationUniversalIdentifier,
|
||||
),
|
||||
).toHaveLength(0);
|
||||
});
|
||||
|
||||
it('groups rows per application: standard-object rows under the standard application, custom-object rows under the custom application', () => {
|
||||
const { customObject, nameField, nameDescriptionField } =
|
||||
buildCustomObjectFixture();
|
||||
|
||||
const standardObjectId = 'standard-object-id';
|
||||
const standardObjectUniversalIdentifier = 'standard-object-uid';
|
||||
const standardFieldId = 'standard-field-id';
|
||||
const standardFieldUniversalIdentifier = 'standard-field-uid';
|
||||
|
||||
const standardField = getFlatFieldMetadataMock({
|
||||
id: standardFieldId,
|
||||
universalIdentifier: standardFieldUniversalIdentifier,
|
||||
objectMetadataId: standardObjectId,
|
||||
objectMetadataUniversalIdentifier: standardObjectUniversalIdentifier,
|
||||
type: FieldMetadataType.TEXT,
|
||||
name: 'name',
|
||||
applicationUniversalIdentifier:
|
||||
TWENTY_STANDARD_APPLICATION_UNIVERSAL_IDENTIFIER,
|
||||
});
|
||||
|
||||
const standardObject = getStandardFlatObjectMetadataMock({
|
||||
id: standardObjectId,
|
||||
universalIdentifier: standardObjectUniversalIdentifier,
|
||||
isSearchable: true,
|
||||
labelIdentifierFieldMetadataId: standardFieldId,
|
||||
fieldIds: [standardFieldId],
|
||||
});
|
||||
|
||||
const standardSearchFieldMetadata = buildSearchFieldMetadata({
|
||||
id: 'standard-search-field-id',
|
||||
universalIdentifier: 'standard-search-field-uid',
|
||||
objectMetadataId: standardObjectId,
|
||||
fieldMetadataId: standardFieldId,
|
||||
objectMetadataUniversalIdentifier: standardObjectUniversalIdentifier,
|
||||
fieldMetadataUniversalIdentifier: standardFieldUniversalIdentifier,
|
||||
applicationUniversalIdentifier:
|
||||
TWENTY_STANDARD_APPLICATION_UNIVERSAL_IDENTIFIER,
|
||||
});
|
||||
|
||||
const { flatSearchFieldMetadatasToCreateByApplicationUniversalIdentifier } =
|
||||
buildSearchFieldMetadataBackfillOperations({
|
||||
flatObjectMetadataMaps: buildFlatObjectMetadataMaps([
|
||||
customObject,
|
||||
standardObject,
|
||||
]),
|
||||
flatFieldMetadataMaps: buildFlatFieldMetadataMaps([
|
||||
nameField,
|
||||
nameDescriptionField,
|
||||
standardField,
|
||||
]),
|
||||
flatSearchFieldMetadataMaps: buildFlatSearchFieldMetadataMaps([]),
|
||||
standardFlatSearchFieldMetadataMaps: buildFlatSearchFieldMetadataMaps([
|
||||
standardSearchFieldMetadata,
|
||||
]),
|
||||
customApplicationId: CUSTOM_APPLICATION_ID,
|
||||
});
|
||||
|
||||
expect(
|
||||
Object.keys(
|
||||
flatSearchFieldMetadatasToCreateByApplicationUniversalIdentifier,
|
||||
).sort(),
|
||||
).toEqual(
|
||||
[
|
||||
CUSTOM_APPLICATION_UNIVERSAL_IDENTIFIER,
|
||||
TWENTY_STANDARD_APPLICATION_UNIVERSAL_IDENTIFIER,
|
||||
].sort(),
|
||||
);
|
||||
|
||||
const standardApplicationRows =
|
||||
flatSearchFieldMetadatasToCreateByApplicationUniversalIdentifier[
|
||||
TWENTY_STANDARD_APPLICATION_UNIVERSAL_IDENTIFIER
|
||||
];
|
||||
|
||||
expect(standardApplicationRows).toHaveLength(1);
|
||||
expect(standardApplicationRows?.[0].fieldMetadataUniversalIdentifier).toBe(
|
||||
standardFieldUniversalIdentifier,
|
||||
);
|
||||
|
||||
const customApplicationRows =
|
||||
flatSearchFieldMetadatasToCreateByApplicationUniversalIdentifier[
|
||||
CUSTOM_APPLICATION_UNIVERSAL_IDENTIFIER
|
||||
];
|
||||
|
||||
expect(customApplicationRows).toHaveLength(1);
|
||||
expect(customApplicationRows?.[0].fieldMetadataUniversalIdentifier).toBe(
|
||||
nameField.universalIdentifier,
|
||||
);
|
||||
});
|
||||
|
||||
it('selects only the standard-set field on a standard object, never a prefix-overlapping sibling (phone vs phoneNumber)', () => {
|
||||
const standardObjectId = 'standard-object-id';
|
||||
const standardObjectUniversalIdentifier = 'standard-object-uid';
|
||||
|
||||
// Two fields whose names overlap by prefix. Only `phone` is part of the
|
||||
// standard search set; `phoneNumber` must never be pulled in. The old
|
||||
// asExpression-token heuristic could mis-bind these by name; the id-based
|
||||
// derivation cannot.
|
||||
const phoneFieldId = 'phone-field-id';
|
||||
const phoneFieldUniversalIdentifier = 'phone-field-uid';
|
||||
const phoneNumberFieldId = 'phone-number-field-id';
|
||||
const phoneNumberFieldUniversalIdentifier = 'phone-number-field-uid';
|
||||
|
||||
const phoneField = getFlatFieldMetadataMock({
|
||||
id: phoneFieldId,
|
||||
universalIdentifier: phoneFieldUniversalIdentifier,
|
||||
objectMetadataId: standardObjectId,
|
||||
objectMetadataUniversalIdentifier: standardObjectUniversalIdentifier,
|
||||
type: FieldMetadataType.TEXT,
|
||||
name: 'phone',
|
||||
applicationUniversalIdentifier:
|
||||
TWENTY_STANDARD_APPLICATION_UNIVERSAL_IDENTIFIER,
|
||||
});
|
||||
|
||||
const phoneNumberField = getFlatFieldMetadataMock({
|
||||
id: phoneNumberFieldId,
|
||||
universalIdentifier: phoneNumberFieldUniversalIdentifier,
|
||||
objectMetadataId: standardObjectId,
|
||||
objectMetadataUniversalIdentifier: standardObjectUniversalIdentifier,
|
||||
type: FieldMetadataType.TEXT,
|
||||
name: 'phoneNumber',
|
||||
applicationUniversalIdentifier:
|
||||
TWENTY_STANDARD_APPLICATION_UNIVERSAL_IDENTIFIER,
|
||||
});
|
||||
|
||||
const standardObject = getStandardFlatObjectMetadataMock({
|
||||
id: standardObjectId,
|
||||
universalIdentifier: standardObjectUniversalIdentifier,
|
||||
isSearchable: true,
|
||||
labelIdentifierFieldMetadataId: phoneFieldId,
|
||||
fieldIds: [phoneFieldId, phoneNumberFieldId],
|
||||
});
|
||||
|
||||
const phoneSearchFieldMetadata = buildSearchFieldMetadata({
|
||||
id: 'phone-search-field-id',
|
||||
universalIdentifier: 'phone-search-field-uid',
|
||||
objectMetadataId: standardObjectId,
|
||||
fieldMetadataId: phoneFieldId,
|
||||
objectMetadataUniversalIdentifier: standardObjectUniversalIdentifier,
|
||||
fieldMetadataUniversalIdentifier: phoneFieldUniversalIdentifier,
|
||||
applicationUniversalIdentifier:
|
||||
TWENTY_STANDARD_APPLICATION_UNIVERSAL_IDENTIFIER,
|
||||
});
|
||||
|
||||
const { flatSearchFieldMetadatasToCreateByApplicationUniversalIdentifier } =
|
||||
buildSearchFieldMetadataBackfillOperations({
|
||||
flatObjectMetadataMaps: buildFlatObjectMetadataMaps([standardObject]),
|
||||
flatFieldMetadataMaps: buildFlatFieldMetadataMaps([
|
||||
phoneField,
|
||||
phoneNumberField,
|
||||
]),
|
||||
flatSearchFieldMetadataMaps: buildFlatSearchFieldMetadataMaps([]),
|
||||
standardFlatSearchFieldMetadataMaps: buildFlatSearchFieldMetadataMaps([
|
||||
phoneSearchFieldMetadata,
|
||||
]),
|
||||
customApplicationId: CUSTOM_APPLICATION_ID,
|
||||
});
|
||||
|
||||
const standardApplicationRows =
|
||||
flatSearchFieldMetadatasToCreateByApplicationUniversalIdentifier[
|
||||
TWENTY_STANDARD_APPLICATION_UNIVERSAL_IDENTIFIER
|
||||
];
|
||||
|
||||
expect(standardApplicationRows).toHaveLength(1);
|
||||
expect(standardApplicationRows?.[0].fieldMetadataUniversalIdentifier).toBe(
|
||||
phoneFieldUniversalIdentifier,
|
||||
);
|
||||
// The prefix-overlapping `phoneNumber` field is never selected.
|
||||
expect(
|
||||
standardApplicationRows?.some(
|
||||
(row) =>
|
||||
row.fieldMetadataUniversalIdentifier ===
|
||||
phoneNumberFieldUniversalIdentifier,
|
||||
),
|
||||
).toBe(false);
|
||||
});
|
||||
|
||||
it('carries the position from the standard maps row onto the backfilled standard row', () => {
|
||||
const standardObjectId = 'standard-object-id';
|
||||
const standardObjectUniversalIdentifier = 'standard-object-uid';
|
||||
const emailsFieldId = 'emails-field-id';
|
||||
const emailsFieldUniversalIdentifier = 'emails-field-uid';
|
||||
|
||||
const emailsField = getFlatFieldMetadataMock({
|
||||
id: emailsFieldId,
|
||||
universalIdentifier: emailsFieldUniversalIdentifier,
|
||||
objectMetadataId: standardObjectId,
|
||||
objectMetadataUniversalIdentifier: standardObjectUniversalIdentifier,
|
||||
type: FieldMetadataType.TEXT,
|
||||
name: 'emails',
|
||||
applicationUniversalIdentifier:
|
||||
TWENTY_STANDARD_APPLICATION_UNIVERSAL_IDENTIFIER,
|
||||
});
|
||||
|
||||
const standardObject = getStandardFlatObjectMetadataMock({
|
||||
id: standardObjectId,
|
||||
universalIdentifier: standardObjectUniversalIdentifier,
|
||||
isSearchable: true,
|
||||
labelIdentifierFieldMetadataId: emailsFieldId,
|
||||
fieldIds: [emailsFieldId],
|
||||
});
|
||||
|
||||
// The standard maps row sits at position 3 (e.g. SEARCH_FIELDS_FOR_PERSON order);
|
||||
// backfill must replicate that ordinal, not reset it to 0.
|
||||
const standardSearchFieldMetadata = buildSearchFieldMetadata({
|
||||
id: 'emails-search-field-id',
|
||||
universalIdentifier: 'emails-search-field-uid',
|
||||
objectMetadataId: standardObjectId,
|
||||
fieldMetadataId: emailsFieldId,
|
||||
objectMetadataUniversalIdentifier: standardObjectUniversalIdentifier,
|
||||
fieldMetadataUniversalIdentifier: emailsFieldUniversalIdentifier,
|
||||
applicationUniversalIdentifier:
|
||||
TWENTY_STANDARD_APPLICATION_UNIVERSAL_IDENTIFIER,
|
||||
position: 3,
|
||||
});
|
||||
|
||||
const { flatSearchFieldMetadatasToCreateByApplicationUniversalIdentifier } =
|
||||
buildSearchFieldMetadataBackfillOperations({
|
||||
flatObjectMetadataMaps: buildFlatObjectMetadataMaps([standardObject]),
|
||||
flatFieldMetadataMaps: buildFlatFieldMetadataMaps([emailsField]),
|
||||
flatSearchFieldMetadataMaps: buildFlatSearchFieldMetadataMaps([]),
|
||||
standardFlatSearchFieldMetadataMaps: buildFlatSearchFieldMetadataMaps([
|
||||
standardSearchFieldMetadata,
|
||||
]),
|
||||
customApplicationId: CUSTOM_APPLICATION_ID,
|
||||
});
|
||||
|
||||
const standardApplicationRows =
|
||||
flatSearchFieldMetadatasToCreateByApplicationUniversalIdentifier[
|
||||
TWENTY_STANDARD_APPLICATION_UNIVERSAL_IDENTIFIER
|
||||
];
|
||||
|
||||
expect(standardApplicationRows).toHaveLength(1);
|
||||
expect(standardApplicationRows?.[0].position).toBe(3);
|
||||
});
|
||||
|
||||
it('is a no-op when the searchFieldMetadata rows already exist (idempotent)', () => {
|
||||
const { customObject, nameField, nameDescriptionField } =
|
||||
buildCustomObjectFixture();
|
||||
|
||||
const existingSearchFieldMetadata = buildSearchFieldMetadata({
|
||||
id: 'existing-search-field-id',
|
||||
universalIdentifier: 'existing-search-field-uid',
|
||||
objectMetadataId: customObject.id,
|
||||
fieldMetadataId: nameField.id,
|
||||
objectMetadataUniversalIdentifier: customObject.universalIdentifier,
|
||||
fieldMetadataUniversalIdentifier: nameField.universalIdentifier,
|
||||
applicationUniversalIdentifier:
|
||||
CUSTOM_APPLICATION_UNIVERSAL_IDENTIFIER,
|
||||
});
|
||||
|
||||
const { flatSearchFieldMetadatasToCreateByApplicationUniversalIdentifier } =
|
||||
buildSearchFieldMetadataBackfillOperations({
|
||||
flatObjectMetadataMaps: buildFlatObjectMetadataMaps([customObject]),
|
||||
flatFieldMetadataMaps: buildFlatFieldMetadataMaps([
|
||||
nameField,
|
||||
nameDescriptionField,
|
||||
]),
|
||||
flatSearchFieldMetadataMaps: buildFlatSearchFieldMetadataMaps([
|
||||
existingSearchFieldMetadata,
|
||||
]),
|
||||
standardFlatSearchFieldMetadataMaps:
|
||||
buildFlatSearchFieldMetadataMaps([]),
|
||||
customApplicationId: CUSTOM_APPLICATION_ID,
|
||||
});
|
||||
|
||||
expect(
|
||||
Object.keys(
|
||||
flatSearchFieldMetadatasToCreateByApplicationUniversalIdentifier,
|
||||
),
|
||||
).toHaveLength(0);
|
||||
});
|
||||
|
||||
it('skips a custom object whose name field is not a searchable type', () => {
|
||||
const customObjectId = 'relation-name-object-id';
|
||||
const customObjectUniversalIdentifier = 'relation-name-object-uid';
|
||||
const relationNameFieldId = 'relation-name-field-id';
|
||||
|
||||
// A `name` field that is a RELATION (non-searchable) type: pre-2.15 would not
|
||||
// include it in the tsvector and the recompute filters it out, so no row.
|
||||
const relationNameField = getFlatFieldMetadataMock({
|
||||
id: relationNameFieldId,
|
||||
universalIdentifier: 'relation-name-field-uid',
|
||||
objectMetadataId: customObjectId,
|
||||
objectMetadataUniversalIdentifier: customObjectUniversalIdentifier,
|
||||
type: FieldMetadataType.RELATION,
|
||||
name: 'name',
|
||||
applicationUniversalIdentifier: CUSTOM_APPLICATION_UNIVERSAL_IDENTIFIER,
|
||||
});
|
||||
|
||||
const customObject = getFlatObjectMetadataMock({
|
||||
id: customObjectId,
|
||||
universalIdentifier: customObjectUniversalIdentifier,
|
||||
isSearchable: true,
|
||||
applicationId: CUSTOM_APPLICATION_ID,
|
||||
applicationUniversalIdentifier: CUSTOM_APPLICATION_UNIVERSAL_IDENTIFIER,
|
||||
labelIdentifierFieldMetadataId: relationNameFieldId,
|
||||
fieldUniversalIdentifiers: [relationNameField.universalIdentifier],
|
||||
});
|
||||
|
||||
const { flatSearchFieldMetadatasToCreateByApplicationUniversalIdentifier } =
|
||||
buildSearchFieldMetadataBackfillOperations({
|
||||
flatObjectMetadataMaps: buildFlatObjectMetadataMaps([customObject]),
|
||||
flatFieldMetadataMaps: buildFlatFieldMetadataMaps([relationNameField]),
|
||||
flatSearchFieldMetadataMaps: buildFlatSearchFieldMetadataMaps([]),
|
||||
standardFlatSearchFieldMetadataMaps:
|
||||
buildFlatSearchFieldMetadataMaps([]),
|
||||
customApplicationId: CUSTOM_APPLICATION_ID,
|
||||
});
|
||||
|
||||
expect(
|
||||
Object.keys(
|
||||
flatSearchFieldMetadatasToCreateByApplicationUniversalIdentifier,
|
||||
),
|
||||
).toHaveLength(0);
|
||||
});
|
||||
|
||||
it('skips a searchable object owned by a third-party application (only the workspace custom application is backfilled)', () => {
|
||||
const thirdPartyApplicationId = 'third-party-application-id';
|
||||
const thirdPartyApplicationUniversalIdentifier =
|
||||
'third-party-application-uid';
|
||||
const { customObject, nameField, nameDescriptionField } =
|
||||
buildCustomObjectFixture();
|
||||
|
||||
// Same fixture but owned by a different (third-party) application: each application
|
||||
// owns its own searchFieldMetadata, so this backfill must not fabricate a row for it.
|
||||
const thirdPartyObject: FlatObjectMetadata = {
|
||||
...customObject,
|
||||
applicationId: thirdPartyApplicationId,
|
||||
applicationUniversalIdentifier: thirdPartyApplicationUniversalIdentifier,
|
||||
};
|
||||
|
||||
const { flatSearchFieldMetadatasToCreateByApplicationUniversalIdentifier } =
|
||||
buildSearchFieldMetadataBackfillOperations({
|
||||
flatObjectMetadataMaps: buildFlatObjectMetadataMaps([thirdPartyObject]),
|
||||
flatFieldMetadataMaps: buildFlatFieldMetadataMaps([
|
||||
nameField,
|
||||
nameDescriptionField,
|
||||
]),
|
||||
flatSearchFieldMetadataMaps: buildFlatSearchFieldMetadataMaps([]),
|
||||
standardFlatSearchFieldMetadataMaps:
|
||||
buildFlatSearchFieldMetadataMaps([]),
|
||||
customApplicationId: CUSTOM_APPLICATION_ID,
|
||||
});
|
||||
|
||||
expect(
|
||||
Object.keys(
|
||||
flatSearchFieldMetadatasToCreateByApplicationUniversalIdentifier,
|
||||
),
|
||||
).toHaveLength(0);
|
||||
});
|
||||
|
||||
it('skips non-searchable objects', () => {
|
||||
const { customObject, nameField, nameDescriptionField } =
|
||||
buildCustomObjectFixture();
|
||||
|
||||
const nonSearchableObject: FlatObjectMetadata = {
|
||||
...customObject,
|
||||
isSearchable: false,
|
||||
};
|
||||
|
||||
const { flatSearchFieldMetadatasToCreateByApplicationUniversalIdentifier } =
|
||||
buildSearchFieldMetadataBackfillOperations({
|
||||
flatObjectMetadataMaps: buildFlatObjectMetadataMaps([
|
||||
nonSearchableObject,
|
||||
]),
|
||||
flatFieldMetadataMaps: buildFlatFieldMetadataMaps([
|
||||
nameField,
|
||||
nameDescriptionField,
|
||||
]),
|
||||
flatSearchFieldMetadataMaps: buildFlatSearchFieldMetadataMaps([]),
|
||||
standardFlatSearchFieldMetadataMaps:
|
||||
buildFlatSearchFieldMetadataMaps([]),
|
||||
customApplicationId: CUSTOM_APPLICATION_ID,
|
||||
});
|
||||
|
||||
expect(
|
||||
Object.keys(
|
||||
flatSearchFieldMetadatasToCreateByApplicationUniversalIdentifier,
|
||||
),
|
||||
).toHaveLength(0);
|
||||
});
|
||||
});
|
||||
+159
@@ -0,0 +1,159 @@
|
||||
import {
|
||||
fromArrayToValuesByKeyRecord,
|
||||
isDefined,
|
||||
isSearchableFieldType,
|
||||
} from 'twenty-shared/utils';
|
||||
|
||||
import { type FlatEntityMaps } from 'src/engine/metadata-modules/flat-entity/types/flat-entity-maps.type';
|
||||
import { findFlatEntitiesByApplicationId } from 'src/engine/metadata-modules/flat-entity/utils/find-flat-entities-by-application-id.util';
|
||||
import { type FlatFieldMetadata } from 'src/engine/metadata-modules/flat-field-metadata/types/flat-field-metadata.type';
|
||||
import { type FlatObjectMetadata } from 'src/engine/metadata-modules/flat-object-metadata/types/flat-object-metadata.type';
|
||||
import { type FlatSearchFieldMetadata } from 'src/engine/metadata-modules/flat-search-field-metadata/types/flat-search-field-metadata.type';
|
||||
import { buildFlatSearchFieldMetadataForField } from 'src/engine/metadata-modules/flat-search-field-metadata/utils/build-flat-search-field-metadata-for-field.util';
|
||||
import { DEFAULT_LABEL_IDENTIFIER_FIELD_NAME } from 'src/engine/metadata-modules/object-metadata/constants/object-metadata.constants';
|
||||
import { type UniversalFlatSearchFieldMetadata } from 'src/engine/workspace-manager/workspace-migration/universal-flat-entity/types/universal-flat-search-field-metadata.type';
|
||||
|
||||
type SearchFieldMetadataBackfillOperationsArgs = {
|
||||
flatObjectMetadataMaps: FlatEntityMaps<FlatObjectMetadata>;
|
||||
flatFieldMetadataMaps: FlatEntityMaps<FlatFieldMetadata>;
|
||||
flatSearchFieldMetadataMaps: FlatEntityMaps<FlatSearchFieldMetadata>;
|
||||
standardFlatSearchFieldMetadataMaps: FlatEntityMaps<FlatSearchFieldMetadata>;
|
||||
customApplicationId: string;
|
||||
};
|
||||
|
||||
type BuildSearchFieldMetadataBackfillOperationsReturnType = {
|
||||
flatSearchFieldMetadatasToCreateByApplicationUniversalIdentifier: Record<
|
||||
string,
|
||||
UniversalFlatSearchFieldMetadata[]
|
||||
>;
|
||||
};
|
||||
|
||||
// Groups rows by application so each group runs through the migration runner under
|
||||
// the matching application (the runner assigns applicationId from that single app).
|
||||
export const buildSearchFieldMetadataBackfillOperations = ({
|
||||
flatObjectMetadataMaps,
|
||||
flatFieldMetadataMaps,
|
||||
flatSearchFieldMetadataMaps,
|
||||
standardFlatSearchFieldMetadataMaps,
|
||||
customApplicationId,
|
||||
}: SearchFieldMetadataBackfillOperationsArgs): BuildSearchFieldMetadataBackfillOperationsReturnType => {
|
||||
const existingSearchFieldMetadataKeys = new Set(
|
||||
Object.values(flatSearchFieldMetadataMaps.byUniversalIdentifier)
|
||||
.filter(isDefined)
|
||||
.map(
|
||||
(searchFieldMetadata) =>
|
||||
`${searchFieldMetadata.objectMetadataId}:${searchFieldMetadata.fieldMetadataId}`,
|
||||
),
|
||||
);
|
||||
|
||||
const flatSearchFieldMetadatasToCreate: UniversalFlatSearchFieldMetadata[] =
|
||||
[];
|
||||
// Dedupe (object, field) within a run: standard and custom derivations can overlap.
|
||||
const candidateSearchFieldMetadataKeys = new Set<string>();
|
||||
|
||||
const pushCandidateIfMissing = ({
|
||||
objectMetadataUniversalIdentifier,
|
||||
fieldMetadataUniversalIdentifier,
|
||||
position,
|
||||
}: {
|
||||
objectMetadataUniversalIdentifier: string;
|
||||
fieldMetadataUniversalIdentifier: string;
|
||||
position: number;
|
||||
}): void => {
|
||||
const flatObjectMetadata =
|
||||
flatObjectMetadataMaps.byUniversalIdentifier[
|
||||
objectMetadataUniversalIdentifier
|
||||
];
|
||||
const flatFieldMetadata =
|
||||
flatFieldMetadataMaps.byUniversalIdentifier[
|
||||
fieldMetadataUniversalIdentifier
|
||||
];
|
||||
|
||||
// Skip rows whose object/field isn't provisioned in this workspace yet (keeps
|
||||
// the backfill idempotent and lets each sync own its own rows).
|
||||
if (!isDefined(flatObjectMetadata) || !isDefined(flatFieldMetadata)) {
|
||||
return;
|
||||
}
|
||||
|
||||
const searchFieldMetadataKey = `${flatObjectMetadata.id}:${flatFieldMetadata.id}`;
|
||||
|
||||
if (
|
||||
existingSearchFieldMetadataKeys.has(searchFieldMetadataKey) ||
|
||||
candidateSearchFieldMetadataKeys.has(searchFieldMetadataKey)
|
||||
) {
|
||||
return;
|
||||
}
|
||||
|
||||
candidateSearchFieldMetadataKeys.add(searchFieldMetadataKey);
|
||||
|
||||
flatSearchFieldMetadatasToCreate.push(
|
||||
buildFlatSearchFieldMetadataForField({
|
||||
flatObjectMetadata,
|
||||
flatFieldMetadata,
|
||||
position,
|
||||
}),
|
||||
);
|
||||
};
|
||||
|
||||
// Standard objects: mirror exactly what provisioning/standard-sync creates.
|
||||
for (const standardSearchFieldMetadata of Object.values(
|
||||
standardFlatSearchFieldMetadataMaps.byUniversalIdentifier,
|
||||
).filter(isDefined)) {
|
||||
pushCandidateIfMissing({
|
||||
objectMetadataUniversalIdentifier:
|
||||
standardSearchFieldMetadata.objectMetadataUniversalIdentifier,
|
||||
fieldMetadataUniversalIdentifier:
|
||||
standardSearchFieldMetadata.fieldMetadataUniversalIdentifier,
|
||||
position: standardSearchFieldMetadata.position,
|
||||
});
|
||||
}
|
||||
|
||||
const customApplicationFlatObjectMetadatas = findFlatEntitiesByApplicationId({
|
||||
flatEntityMaps: flatObjectMetadataMaps,
|
||||
applicationId: customApplicationId,
|
||||
});
|
||||
|
||||
// Custom objects index only the field named 'name' (SEARCH_FIELDS_FOR_CUSTOM_OBJECT).
|
||||
// Resolve it by exact name, not the label identifier: junction objects (skipNameField)
|
||||
// have no name field and must stay unsearchable — their label identifier is the UUID id.
|
||||
for (const flatObjectMetadata of customApplicationFlatObjectMetadatas) {
|
||||
if (!flatObjectMetadata.isSearchable) {
|
||||
continue;
|
||||
}
|
||||
|
||||
const nameFieldMetadata = flatObjectMetadata.fieldUniversalIdentifiers
|
||||
.map(
|
||||
(fieldUniversalIdentifier) =>
|
||||
flatFieldMetadataMaps.byUniversalIdentifier[fieldUniversalIdentifier],
|
||||
)
|
||||
.find(
|
||||
(flatFieldMetadata) =>
|
||||
isDefined(flatFieldMetadata) &&
|
||||
flatFieldMetadata.name === DEFAULT_LABEL_IDENTIFIER_FIELD_NAME,
|
||||
);
|
||||
|
||||
// Aligns with the recompute, which drops non-searchable-type fields.
|
||||
if (
|
||||
!isDefined(nameFieldMetadata) ||
|
||||
!isSearchableFieldType(nameFieldMetadata.type)
|
||||
) {
|
||||
continue;
|
||||
}
|
||||
|
||||
pushCandidateIfMissing({
|
||||
objectMetadataUniversalIdentifier: flatObjectMetadata.universalIdentifier,
|
||||
fieldMetadataUniversalIdentifier: nameFieldMetadata.universalIdentifier,
|
||||
position: 0,
|
||||
});
|
||||
}
|
||||
|
||||
const flatSearchFieldMetadatasToCreateByApplicationUniversalIdentifier =
|
||||
fromArrayToValuesByKeyRecord({
|
||||
array: flatSearchFieldMetadatasToCreate,
|
||||
key: 'applicationUniversalIdentifier',
|
||||
});
|
||||
|
||||
return {
|
||||
flatSearchFieldMetadatasToCreateByApplicationUniversalIdentifier,
|
||||
};
|
||||
};
|
||||
+4
-2
@@ -29,8 +29,11 @@ import { CreateUnsubscribeTopicCoreTableFastInstanceCommand } from 'src/database
|
||||
import { RenameIsUiReadOnlyToIsUiEditableFastInstanceCommand } from 'src/database/commands/upgrade-version-command/2-13/2-13-instance-command-fast-1781277453604-rename-is-ui-read-only-to-is-ui-editable';
|
||||
import { BackfillNonUiCreatableStandardSystemObjectsSlowInstanceCommand } from 'src/database/commands/upgrade-version-command/2-13/2-13-instance-command-slow-1781277480000-backfill-non-ui-creatable-standard-system-objects';
|
||||
import { SetTableWidgetViewsVisibilityToWorkspaceSlowInstanceCommand } from 'src/database/commands/upgrade-version-command/2-14/2-14-instance-command-slow-1781515653781-set-table-widget-views-visibility-to-workspace';
|
||||
import { AddHasPaymentMethodToBillingCustomerFastInstanceCommand } from 'src/database/commands/upgrade-version-command/2-15/2-15-instance-command-fast-1781280240009-add-has-payment-method-to-billing-customer';
|
||||
import { AddIsSystemSideEffectFastInstanceCommand } from 'src/database/commands/upgrade-version-command/2-15/2-15-instance-command-fast-1781600000000-add-is-system-side-effect';
|
||||
import { BackfillConnectionSecuritySlowInstanceCommand } from 'src/database/commands/upgrade-version-command/2-15/2-15-instance-command-slow-1781461753981-backfill-connection-security';
|
||||
import { AddChannelWebhookSubscriptionFieldsFastInstanceCommand } from 'src/database/commands/upgrade-version-command/2-16/2-16-instance-command-fast-1782152096938-add-channel-webhook-subscription-fields';
|
||||
import { AddUniversalIdentifierAndApplicationIdToSearchFieldMetadataFastInstanceCommand } from 'src/database/commands/upgrade-version-command/2-16/2-16-instance-command-fast-1782200000000-add-universal-identifier-and-application-id-to-search-field-metadata';
|
||||
import { AddCacheTokensToAgentChatThreadFastInstanceCommand } from 'src/database/commands/upgrade-version-command/2-2/2-2-instance-command-fast-1777455269302-add-cache-tokens-to-agent-chat-thread';
|
||||
import { AddLogoToApplicationFastInstanceCommand } from 'src/database/commands/upgrade-version-command/2-2/2-2-instance-command-fast-1777539664664-add-logo-to-application';
|
||||
import { AddSubFieldNameToViewSortEarlyFastInstanceCommand } from 'src/database/commands/upgrade-version-command/2-3/2-3-instance-command-fast-1747234200000-add-sub-field-name-to-view-sort';
|
||||
@@ -72,10 +75,8 @@ import { EmailingDomainTenantStatusAndGlobalUniquenessFastInstanceCommand } from
|
||||
import { AddLogicFunctionExecutionModeFastInstanceCommand } from 'src/database/commands/upgrade-version-command/2-9/2-9-instance-command-fast-1799000030000-add-logic-function-execution-mode';
|
||||
import { EncryptNonSecretApplicationVariableSlowInstanceCommand } from 'src/database/commands/upgrade-version-command/2-9/2-9-instance-command-slow-1798400000000-encrypt-non-secret-application-variable';
|
||||
import { MigrateAiModelPreferencesSlowInstanceCommand } from 'src/database/commands/upgrade-version-command/2-9/2-9-instance-command-slow-1799000010000-migrate-ai-model-preferences';
|
||||
import { AddHasPaymentMethodToBillingCustomerFastInstanceCommand } from 'src/database/commands/upgrade-version-command/2-15/2-15-instance-command-fast-1781280240009-add-has-payment-method-to-billing-customer';
|
||||
import { AddFolderImportToMessageFolderPendingSyncActionFastInstanceCommand } from './2-15/2-15-instance-command-fast-1781714499016-add-folder-import-to-message-folder-pending-sync-action';
|
||||
import { AddViewKanbanColumnWidthFastInstanceCommand } from './2-15/2-15-instance-command-fast-1781900000000-add-view-kanban-column-width';
|
||||
import { AddChannelWebhookSubscriptionFieldsFastInstanceCommand } from 'src/database/commands/upgrade-version-command/2-16/2-16-instance-command-fast-1782152096938-add-channel-webhook-subscription-fields';
|
||||
|
||||
export const INSTANCE_COMMANDS = [
|
||||
AddViewFieldGroupIdIndexOnViewFieldFastInstanceCommand,
|
||||
@@ -154,4 +155,5 @@ export const INSTANCE_COMMANDS = [
|
||||
AddFolderImportToMessageFolderPendingSyncActionFastInstanceCommand,
|
||||
AddViewKanbanColumnWidthFastInstanceCommand,
|
||||
AddChannelWebhookSubscriptionFieldsFastInstanceCommand,
|
||||
AddUniversalIdentifierAndApplicationIdToSearchFieldMetadataFastInstanceCommand,
|
||||
];
|
||||
|
||||
+2
@@ -16,6 +16,7 @@ import { V2_10_UpgradeVersionCommandModule } from 'src/database/commands/upgrade
|
||||
import { V2_13_UpgradeVersionCommandModule } from 'src/database/commands/upgrade-version-command/2-13/2-13-upgrade-version-command.module';
|
||||
import { V2_14_UpgradeVersionCommandModule } from 'src/database/commands/upgrade-version-command/2-14/2-14-upgrade-version-command.module';
|
||||
import { V2_15_UpgradeVersionCommandModule } from 'src/database/commands/upgrade-version-command/2-15/2-15-upgrade-version-command.module';
|
||||
import { V2_16_UpgradeVersionCommandModule } from 'src/database/commands/upgrade-version-command/2-16/2-16-upgrade-version-command.module';
|
||||
|
||||
@Module({
|
||||
imports: [
|
||||
@@ -35,6 +36,7 @@ import { V2_15_UpgradeVersionCommandModule } from 'src/database/commands/upgrade
|
||||
V2_13_UpgradeVersionCommandModule,
|
||||
V2_14_UpgradeVersionCommandModule,
|
||||
V2_15_UpgradeVersionCommandModule,
|
||||
V2_16_UpgradeVersionCommandModule,
|
||||
],
|
||||
})
|
||||
export class WorkspaceCommandProviderModule {}
|
||||
|
||||
Reference in New Issue
Block a user