Deprecate asExpression from field metadata search_vector (#22287)
## Summary Fully deprecates the cached `asExpression` / `generatedType` settings on `TS_VECTOR` (searchVector) fields. Previously the generated-column expression was stored in `FieldMetadataSettings` and kept in sync via imperative recompute side-effects. It is now **derived at DDL time** from the `searchFieldMetadata` rows that describe which fields feed the search vector, making `searchFieldMetadata` the single source of truth and removing a whole class of cache-drift bugs. This is delivered across the milestones tracked in #2587 and coordinates with the frontend migration (#1428). ## Why - The searchVector expression lived in two places (stored `settings.asExpression` + the actual generated column), kept consistent by bespoke side-effects (`recompute-search-vector-on-field-rename`, label-identifier recompute, etc.). - The frontend reconstructed the searchable-fields list by **regex-parsing** the stored `asExpression`. - Both are brittle. Deriving the expression from `searchFieldMetadata` rows at build/run time removes the cache and the parsing. ## What changed ### Server - data model & derivation - Introduce the `tsVectorFieldMetadata` relation on `searchFieldMetadata` (`tsVectorFieldMetadataId` / universal identifier) linking each searchable-field row to its target `TS_VECTOR` field. - New runtime derivation `deriveSearchVectorAsExpressionForTsVectorField` (`flat-search-field-metadata/utils/...`) used by the create-object and update-field handlers to generate the column expression from `searchFieldMetadata` rows. - Remove `asExpression` / `generatedType` from stored settings: `FieldMetadataSettings.TS_VECTOR` is now `null`; the column builder (`generate-column-definitions.util.ts`) hardcodes `generatedType: 'STORED'` and requires the derived expression. - Delete the imperative recompute side-effects and the `compute-search-vector-universal-settings-from-object-manifest` path; drop the `settings` block from all 28 standard `compute-*-standard-flat-field-metadata` utils. ### Server - migration runner - New `rebuildSearchVector` marker on `update-field` actions: the orchestrator synthesizes targeted column rebuilds (`compute-search-vector-rebuild-target-universal-identifiers.util.ts` + the deprioritize aggregator) only when a searchFieldMetadata change or indexed-field rename actually requires it - instead of rebuilding on every settings touch. - Deferrable FKs + in-flight ID resolution so a `searchFieldMetadata` row and its `TS_VECTOR` field can be created in the same transaction (deterministic UUIDs). ### Frontend (contract change, #1428) - New `SearchFieldMetadataDTO` + dataloader exposing `searchFieldMetadataList` on object metadata. - `SettingsObjectSearchSection` now reads `objectMetadataItem.searchFieldMetadatas` instead of parsing `asExpression`; new `SearchFieldMetadataItem` type, fragment, and mapping updates. ### Upgrade commands (2.18) - `2-18-instance-command-fast-...-add-ts-vector-field-metadata-id-to-search-field-metadata` - `2-18-instance-command-fast-...-make-search-field-metadata-fks-deferrable` - `2-18-instance-command-slow-...-backfill-ts-vector-field-metadata-id-on-search-field-metadata` (These were relocated from 2.16 to 2.18 and re-timestamped into an ordered block - add column -> make FK deferrable -> backfill data - since 2.16/2.17 are released.) ### Tests - Updated search-vector side-effect integration specs to assert behavior (search works) rather than the now-removed `asExpression`; removed the obsolete expression-validation specs; refreshed the application-sync snapshot (`universalSettings: null`). ## Upgrade / compatibility notes - Existing workspaces keep their stored `settings` until a later cleanup; nothing reads it anymore. The new derivation drives all DDL going forward. - Schema changes are gated behind the 2.18 instance commands above. ## Known follow-up (separate PR) https://github.com/twentyhq/core-team-issues/issues/2620 - The column rebuild (`DROP`/`ADD` of the `searchVector` STORED column) cascade-drops its GIN index and does not recreate it - a pre-existing regression on `main` inherited here. A follow-up PR will fix the rebuild handler to recreate the GIN index and add a 2.18 workspace command to recompute every search vector + strip the deprecated settings. (Planned.) ## Test plan - [ ] `npx nx typecheck twenty-server` / `twenty-front` - [ ] `npx nx lint:diff-with-main twenty-server` / `twenty-front` - [ ] Server integration: create/update/delete field, rename indexed field, update object - search returns expected records - [ ] Run the 2.18 instance commands on a seeded DB; verify `tsVectorFieldMetadataId` backfilled and FKs deferrable - [ ] Frontend: object Search settings tab lists the correct searchable fields (no `asExpression` parsing) close https://github.com/twentyhq/core-team-issues/issues/2587 <!-- This is an auto-generated description by cubic. --> <a href="https://cubic.dev/pr/twentyhq/twenty/pull/22287?utm_source=github" target="_blank" rel="noopener noreferrer" data-no-image-dialog="true"><picture><source media="(prefers-color-scheme: dark)" srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source media="(prefers-color-scheme: light)" srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img alt="Review in cubic" src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a> <!-- End of auto-generated description by cubic. -->
This commit is contained in:
Paul Rastoin
+2
@@ -178,6 +178,8 @@ const buildLegacyCalendarEventRecordingPreferenceFieldMetadata = ({
|
||||
mainGroupByFieldMetadataViewUniversalIdentifiers: [],
|
||||
viewSortIds: [],
|
||||
viewSortUniversalIdentifiers: [],
|
||||
searchFieldMetadataIds: [],
|
||||
searchFieldMetadataUniversalIdentifiers: [],
|
||||
universalSettings: null,
|
||||
});
|
||||
|
||||
|
||||
+88
-4
@@ -5,6 +5,7 @@ import { buildSearchFieldMetadataBackfillOperations } from 'src/database/command
|
||||
import { type FlatEntityMaps } from 'src/engine/metadata-modules/flat-entity/types/flat-entity-maps.type';
|
||||
import { getFlatFieldMetadataMock } from 'src/engine/metadata-modules/flat-field-metadata/__mocks__/get-flat-field-metadata.mock';
|
||||
import { type FlatFieldMetadata } from 'src/engine/metadata-modules/flat-field-metadata/types/flat-field-metadata.type';
|
||||
import { SEARCH_VECTOR_FIELD } from 'src/engine/metadata-modules/search-field-metadata/constants/search-vector-field.constants';
|
||||
import {
|
||||
getFlatObjectMetadataMock,
|
||||
getStandardFlatObjectMetadataMock,
|
||||
@@ -100,6 +101,8 @@ const buildSearchFieldMetadata = ({
|
||||
fieldMetadataUniversalIdentifier,
|
||||
applicationUniversalIdentifier,
|
||||
position = 0,
|
||||
tsVectorFieldMetadataId = 'ts-vector-field-metadata-id',
|
||||
tsVectorFieldMetadataUniversalIdentifier = 'ts-vector-field-metadata-universal-identifier',
|
||||
}: {
|
||||
id: string;
|
||||
universalIdentifier: string;
|
||||
@@ -109,6 +112,8 @@ const buildSearchFieldMetadata = ({
|
||||
fieldMetadataUniversalIdentifier: string;
|
||||
applicationUniversalIdentifier: string;
|
||||
position?: number;
|
||||
tsVectorFieldMetadataId?: string;
|
||||
tsVectorFieldMetadataUniversalIdentifier?: string;
|
||||
}): FlatSearchFieldMetadata => {
|
||||
const createdAt = '2024-01-01T00:00:00.000Z';
|
||||
|
||||
@@ -119,6 +124,8 @@ const buildSearchFieldMetadata = ({
|
||||
fieldMetadataId,
|
||||
objectMetadataUniversalIdentifier,
|
||||
fieldMetadataUniversalIdentifier,
|
||||
tsVectorFieldMetadataId,
|
||||
tsVectorFieldMetadataUniversalIdentifier,
|
||||
applicationId: 'unused-application-id',
|
||||
applicationUniversalIdentifier,
|
||||
position,
|
||||
@@ -128,6 +135,25 @@ const buildSearchFieldMetadata = ({
|
||||
};
|
||||
};
|
||||
|
||||
const buildSearchVectorField = ({
|
||||
objectMetadataId,
|
||||
objectMetadataUniversalIdentifier,
|
||||
applicationUniversalIdentifier,
|
||||
}: {
|
||||
objectMetadataId: string;
|
||||
objectMetadataUniversalIdentifier: string;
|
||||
applicationUniversalIdentifier: string;
|
||||
}): FlatFieldMetadata =>
|
||||
getFlatFieldMetadataMock({
|
||||
id: `${objectMetadataId}-search-vector-field-id`,
|
||||
universalIdentifier: `${objectMetadataUniversalIdentifier}-search-vector-field-uid`,
|
||||
objectMetadataId,
|
||||
objectMetadataUniversalIdentifier,
|
||||
type: FieldMetadataType.TS_VECTOR,
|
||||
name: SEARCH_VECTOR_FIELD.name,
|
||||
applicationUniversalIdentifier,
|
||||
});
|
||||
|
||||
// Custom object with a searchable `name` field plus a TEXT field whose name
|
||||
// overlaps it by prefix (name / nameDescription). Only the `name` field should
|
||||
// produce a row; the exact-name match makes the prefix overlap irrelevant. This
|
||||
@@ -158,6 +184,12 @@ const buildCustomObjectFixture = () => {
|
||||
applicationUniversalIdentifier: CUSTOM_APPLICATION_UNIVERSAL_IDENTIFIER,
|
||||
});
|
||||
|
||||
const searchVectorField = buildSearchVectorField({
|
||||
objectMetadataId: customObjectId,
|
||||
objectMetadataUniversalIdentifier: customObjectUniversalIdentifier,
|
||||
applicationUniversalIdentifier: CUSTOM_APPLICATION_UNIVERSAL_IDENTIFIER,
|
||||
});
|
||||
|
||||
const customObject = getFlatObjectMetadataMock({
|
||||
id: customObjectId,
|
||||
universalIdentifier: customObjectUniversalIdentifier,
|
||||
@@ -168,15 +200,16 @@ const buildCustomObjectFixture = () => {
|
||||
fieldUniversalIdentifiers: [
|
||||
nameField.universalIdentifier,
|
||||
nameDescriptionField.universalIdentifier,
|
||||
searchVectorField.universalIdentifier,
|
||||
],
|
||||
});
|
||||
|
||||
return { customObject, nameField, nameDescriptionField };
|
||||
return { customObject, nameField, nameDescriptionField, searchVectorField };
|
||||
};
|
||||
|
||||
describe('buildSearchFieldMetadataBackfillOperations', () => {
|
||||
it('selects only the name field for a custom object, ignoring a prefix-overlapping field name', () => {
|
||||
const { customObject, nameField, nameDescriptionField } =
|
||||
const { customObject, nameField, nameDescriptionField, searchVectorField } =
|
||||
buildCustomObjectFixture();
|
||||
|
||||
const { flatSearchFieldMetadatasToCreateByApplicationUniversalIdentifier } =
|
||||
@@ -185,6 +218,7 @@ describe('buildSearchFieldMetadataBackfillOperations', () => {
|
||||
flatFieldMetadataMaps: buildFlatFieldMetadataMaps([
|
||||
nameField,
|
||||
nameDescriptionField,
|
||||
searchVectorField,
|
||||
]),
|
||||
flatSearchFieldMetadataMaps: buildFlatSearchFieldMetadataMaps([]),
|
||||
standardFlatSearchFieldMetadataMaps:
|
||||
@@ -204,6 +238,10 @@ describe('buildSearchFieldMetadataBackfillOperations', () => {
|
||||
expect(customApplicationRows?.[0].objectMetadataUniversalIdentifier).toBe(
|
||||
customObject.universalIdentifier,
|
||||
);
|
||||
// The row points at the object's searchVector field via the new FK.
|
||||
expect(
|
||||
customApplicationRows?.[0].tsVectorFieldMetadataUniversalIdentifier,
|
||||
).toBe(searchVectorField.universalIdentifier);
|
||||
// Custom object name field is seeded at position 0.
|
||||
expect(customApplicationRows?.[0].position).toBe(0);
|
||||
// No spurious row for the prefix-overlapping `nameDescription` field.
|
||||
@@ -263,7 +301,7 @@ describe('buildSearchFieldMetadataBackfillOperations', () => {
|
||||
});
|
||||
|
||||
it('groups rows per application: standard-object rows under the standard application, custom-object rows under the custom application', () => {
|
||||
const { customObject, nameField, nameDescriptionField } =
|
||||
const { customObject, nameField, nameDescriptionField, searchVectorField } =
|
||||
buildCustomObjectFixture();
|
||||
|
||||
const standardObjectId = 'standard-object-id';
|
||||
@@ -282,12 +320,23 @@ describe('buildSearchFieldMetadataBackfillOperations', () => {
|
||||
TWENTY_STANDARD_APPLICATION_UNIVERSAL_IDENTIFIER,
|
||||
});
|
||||
|
||||
const standardSearchVectorField = buildSearchVectorField({
|
||||
objectMetadataId: standardObjectId,
|
||||
objectMetadataUniversalIdentifier: standardObjectUniversalIdentifier,
|
||||
applicationUniversalIdentifier:
|
||||
TWENTY_STANDARD_APPLICATION_UNIVERSAL_IDENTIFIER,
|
||||
});
|
||||
|
||||
const standardObject = getStandardFlatObjectMetadataMock({
|
||||
id: standardObjectId,
|
||||
universalIdentifier: standardObjectUniversalIdentifier,
|
||||
isSearchable: true,
|
||||
labelIdentifierFieldMetadataId: standardFieldId,
|
||||
fieldIds: [standardFieldId],
|
||||
fieldUniversalIdentifiers: [
|
||||
standardFieldUniversalIdentifier,
|
||||
standardSearchVectorField.universalIdentifier,
|
||||
],
|
||||
});
|
||||
|
||||
const standardSearchFieldMetadata = buildSearchFieldMetadata({
|
||||
@@ -310,7 +359,9 @@ describe('buildSearchFieldMetadataBackfillOperations', () => {
|
||||
flatFieldMetadataMaps: buildFlatFieldMetadataMaps([
|
||||
nameField,
|
||||
nameDescriptionField,
|
||||
searchVectorField,
|
||||
standardField,
|
||||
standardSearchVectorField,
|
||||
]),
|
||||
flatSearchFieldMetadataMaps: buildFlatSearchFieldMetadataMaps([]),
|
||||
standardFlatSearchFieldMetadataMaps: buildFlatSearchFieldMetadataMaps([
|
||||
@@ -339,6 +390,9 @@ describe('buildSearchFieldMetadataBackfillOperations', () => {
|
||||
expect(standardApplicationRows?.[0].fieldMetadataUniversalIdentifier).toBe(
|
||||
standardFieldUniversalIdentifier,
|
||||
);
|
||||
expect(
|
||||
standardApplicationRows?.[0].tsVectorFieldMetadataUniversalIdentifier,
|
||||
).toBe(standardSearchVectorField.universalIdentifier);
|
||||
|
||||
const customApplicationRows =
|
||||
flatSearchFieldMetadatasToCreateByApplicationUniversalIdentifier[
|
||||
@@ -349,6 +403,9 @@ describe('buildSearchFieldMetadataBackfillOperations', () => {
|
||||
expect(customApplicationRows?.[0].fieldMetadataUniversalIdentifier).toBe(
|
||||
nameField.universalIdentifier,
|
||||
);
|
||||
expect(
|
||||
customApplicationRows?.[0].tsVectorFieldMetadataUniversalIdentifier,
|
||||
).toBe(searchVectorField.universalIdentifier);
|
||||
});
|
||||
|
||||
it('selects only the standard-set field on a standard object, never a prefix-overlapping sibling (phone vs phoneNumber)', () => {
|
||||
@@ -386,12 +443,24 @@ describe('buildSearchFieldMetadataBackfillOperations', () => {
|
||||
TWENTY_STANDARD_APPLICATION_UNIVERSAL_IDENTIFIER,
|
||||
});
|
||||
|
||||
const standardSearchVectorField = buildSearchVectorField({
|
||||
objectMetadataId: standardObjectId,
|
||||
objectMetadataUniversalIdentifier: standardObjectUniversalIdentifier,
|
||||
applicationUniversalIdentifier:
|
||||
TWENTY_STANDARD_APPLICATION_UNIVERSAL_IDENTIFIER,
|
||||
});
|
||||
|
||||
const standardObject = getStandardFlatObjectMetadataMock({
|
||||
id: standardObjectId,
|
||||
universalIdentifier: standardObjectUniversalIdentifier,
|
||||
isSearchable: true,
|
||||
labelIdentifierFieldMetadataId: phoneFieldId,
|
||||
fieldIds: [phoneFieldId, phoneNumberFieldId],
|
||||
fieldUniversalIdentifiers: [
|
||||
phoneFieldUniversalIdentifier,
|
||||
phoneNumberFieldUniversalIdentifier,
|
||||
standardSearchVectorField.universalIdentifier,
|
||||
],
|
||||
});
|
||||
|
||||
const phoneSearchFieldMetadata = buildSearchFieldMetadata({
|
||||
@@ -411,6 +480,7 @@ describe('buildSearchFieldMetadataBackfillOperations', () => {
|
||||
flatFieldMetadataMaps: buildFlatFieldMetadataMaps([
|
||||
phoneField,
|
||||
phoneNumberField,
|
||||
standardSearchVectorField,
|
||||
]),
|
||||
flatSearchFieldMetadataMaps: buildFlatSearchFieldMetadataMaps([]),
|
||||
standardFlatSearchFieldMetadataMaps: buildFlatSearchFieldMetadataMaps([
|
||||
@@ -455,12 +525,23 @@ describe('buildSearchFieldMetadataBackfillOperations', () => {
|
||||
TWENTY_STANDARD_APPLICATION_UNIVERSAL_IDENTIFIER,
|
||||
});
|
||||
|
||||
const standardSearchVectorField = buildSearchVectorField({
|
||||
objectMetadataId: standardObjectId,
|
||||
objectMetadataUniversalIdentifier: standardObjectUniversalIdentifier,
|
||||
applicationUniversalIdentifier:
|
||||
TWENTY_STANDARD_APPLICATION_UNIVERSAL_IDENTIFIER,
|
||||
});
|
||||
|
||||
const standardObject = getStandardFlatObjectMetadataMock({
|
||||
id: standardObjectId,
|
||||
universalIdentifier: standardObjectUniversalIdentifier,
|
||||
isSearchable: true,
|
||||
labelIdentifierFieldMetadataId: emailsFieldId,
|
||||
fieldIds: [emailsFieldId],
|
||||
fieldUniversalIdentifiers: [
|
||||
emailsFieldUniversalIdentifier,
|
||||
standardSearchVectorField.universalIdentifier,
|
||||
],
|
||||
});
|
||||
|
||||
// The standard maps row sits at position 3 (e.g. SEARCH_FIELDS_FOR_PERSON order);
|
||||
@@ -480,7 +561,10 @@ describe('buildSearchFieldMetadataBackfillOperations', () => {
|
||||
const { flatSearchFieldMetadatasToCreateByApplicationUniversalIdentifier } =
|
||||
buildSearchFieldMetadataBackfillOperations({
|
||||
flatObjectMetadataMaps: buildFlatObjectMetadataMaps([standardObject]),
|
||||
flatFieldMetadataMaps: buildFlatFieldMetadataMaps([emailsField]),
|
||||
flatFieldMetadataMaps: buildFlatFieldMetadataMaps([
|
||||
emailsField,
|
||||
standardSearchVectorField,
|
||||
]),
|
||||
flatSearchFieldMetadataMaps: buildFlatSearchFieldMetadataMaps([]),
|
||||
standardFlatSearchFieldMetadataMaps: buildFlatSearchFieldMetadataMaps([
|
||||
standardSearchFieldMetadata,
|
||||
|
||||
+11
@@ -10,6 +10,7 @@ import { type FlatFieldMetadata } from 'src/engine/metadata-modules/flat-field-m
|
||||
import { type FlatObjectMetadata } from 'src/engine/metadata-modules/flat-object-metadata/types/flat-object-metadata.type';
|
||||
import { type FlatSearchFieldMetadata } from 'src/engine/metadata-modules/flat-search-field-metadata/types/flat-search-field-metadata.type';
|
||||
import { buildFlatSearchFieldMetadataForField } from 'src/engine/metadata-modules/flat-search-field-metadata/utils/build-flat-search-field-metadata-for-field.util';
|
||||
import { findTsVectorFlatFieldMetadataForObject } from 'src/engine/metadata-modules/flat-search-field-metadata/utils/find-ts-vector-flat-field-metadata-for-object.util';
|
||||
import { DEFAULT_LABEL_IDENTIFIER_FIELD_NAME } from 'src/engine/metadata-modules/object-metadata/constants/object-metadata.constants';
|
||||
import { type UniversalFlatSearchFieldMetadata } from 'src/engine/workspace-manager/workspace-migration/universal-flat-entity/types/universal-flat-search-field-metadata.type';
|
||||
|
||||
@@ -84,12 +85,22 @@ export const buildSearchFieldMetadataBackfillOperations = ({
|
||||
return;
|
||||
}
|
||||
|
||||
const tsVectorFlatFieldMetadata = findTsVectorFlatFieldMetadataForObject({
|
||||
fieldUniversalIdentifiers: flatObjectMetadata.fieldUniversalIdentifiers,
|
||||
flatFieldMetadataMaps,
|
||||
});
|
||||
|
||||
if (!isDefined(tsVectorFlatFieldMetadata)) {
|
||||
return;
|
||||
}
|
||||
|
||||
candidateSearchFieldMetadataKeys.add(searchFieldMetadataKey);
|
||||
|
||||
flatSearchFieldMetadatasToCreate.push(
|
||||
buildFlatSearchFieldMetadataForField({
|
||||
flatObjectMetadata,
|
||||
flatFieldMetadata,
|
||||
tsVectorFlatFieldMetadata,
|
||||
position,
|
||||
}),
|
||||
);
|
||||
|
||||
+53
@@ -0,0 +1,53 @@
|
||||
import { QueryRunner } from 'typeorm';
|
||||
|
||||
import { RegisteredInstanceCommand } from 'src/engine/core-modules/upgrade/decorators/registered-instance-command.decorator';
|
||||
import { FastInstanceCommand } from 'src/engine/core-modules/upgrade/interfaces/fast-instance-command.interface';
|
||||
|
||||
@RegisteredInstanceCommand('2.18.0', 1810000001000)
|
||||
export class AddTsVectorFieldMetadataIdToSearchFieldMetadataFastInstanceCommand implements FastInstanceCommand {
|
||||
public async up(queryRunner: QueryRunner): Promise<void> {
|
||||
await queryRunner.query(
|
||||
'ALTER TABLE "core"."searchFieldMetadata" ADD "tsVectorFieldMetadataId" uuid',
|
||||
);
|
||||
await queryRunner.query(
|
||||
'ALTER TABLE "core"."searchFieldMetadata" ADD CONSTRAINT "FK_31102b31f7338b6a298fc22996a" FOREIGN KEY ("tsVectorFieldMetadataId") REFERENCES "core"."fieldMetadata"("id") ON DELETE CASCADE ON UPDATE NO ACTION DEFERRABLE INITIALLY DEFERRED',
|
||||
);
|
||||
|
||||
await queryRunner.query(
|
||||
'ALTER TABLE "core"."searchFieldMetadata" DROP CONSTRAINT IF EXISTS "FK_1b78544eb06f82059a2a01013a3"',
|
||||
);
|
||||
await queryRunner.query(
|
||||
'ALTER TABLE "core"."searchFieldMetadata" ADD CONSTRAINT "FK_1b78544eb06f82059a2a01013a3" FOREIGN KEY ("objectMetadataId") REFERENCES "core"."objectMetadata"("id") ON DELETE CASCADE ON UPDATE NO ACTION DEFERRABLE INITIALLY DEFERRED',
|
||||
);
|
||||
|
||||
await queryRunner.query(
|
||||
'ALTER TABLE "core"."searchFieldMetadata" DROP CONSTRAINT IF EXISTS "FK_6d5c6922bfd1578b1eff2abb9d6"',
|
||||
);
|
||||
await queryRunner.query(
|
||||
'ALTER TABLE "core"."searchFieldMetadata" ADD CONSTRAINT "FK_6d5c6922bfd1578b1eff2abb9d6" FOREIGN KEY ("fieldMetadataId") REFERENCES "core"."fieldMetadata"("id") ON DELETE CASCADE ON UPDATE NO ACTION DEFERRABLE INITIALLY DEFERRED',
|
||||
);
|
||||
}
|
||||
|
||||
public async down(queryRunner: QueryRunner): Promise<void> {
|
||||
await queryRunner.query(
|
||||
'ALTER TABLE "core"."searchFieldMetadata" DROP CONSTRAINT IF EXISTS "FK_6d5c6922bfd1578b1eff2abb9d6"',
|
||||
);
|
||||
await queryRunner.query(
|
||||
'ALTER TABLE "core"."searchFieldMetadata" ADD CONSTRAINT "FK_6d5c6922bfd1578b1eff2abb9d6" FOREIGN KEY ("fieldMetadataId") REFERENCES "core"."fieldMetadata"("id") ON DELETE CASCADE ON UPDATE NO ACTION',
|
||||
);
|
||||
|
||||
await queryRunner.query(
|
||||
'ALTER TABLE "core"."searchFieldMetadata" DROP CONSTRAINT IF EXISTS "FK_1b78544eb06f82059a2a01013a3"',
|
||||
);
|
||||
await queryRunner.query(
|
||||
'ALTER TABLE "core"."searchFieldMetadata" ADD CONSTRAINT "FK_1b78544eb06f82059a2a01013a3" FOREIGN KEY ("objectMetadataId") REFERENCES "core"."objectMetadata"("id") ON DELETE CASCADE ON UPDATE NO ACTION',
|
||||
);
|
||||
|
||||
await queryRunner.query(
|
||||
'ALTER TABLE "core"."searchFieldMetadata" DROP CONSTRAINT "FK_31102b31f7338b6a298fc22996a"',
|
||||
);
|
||||
await queryRunner.query(
|
||||
'ALTER TABLE "core"."searchFieldMetadata" DROP COLUMN "tsVectorFieldMetadataId"',
|
||||
);
|
||||
}
|
||||
}
|
||||
+51
@@ -0,0 +1,51 @@
|
||||
import { DataSource, QueryRunner } from 'typeorm';
|
||||
|
||||
import { RegisteredInstanceCommand } from 'src/engine/core-modules/upgrade/decorators/registered-instance-command.decorator';
|
||||
import { SlowInstanceCommand } from 'src/engine/core-modules/upgrade/interfaces/slow-instance-command.interface';
|
||||
|
||||
// Backfills the tsVectorFieldMetadataId FK by pointing every existing
|
||||
// searchFieldMetadata row at its object's system searchVector (TS_VECTOR) field,
|
||||
// then enforces NOT NULL. Runs after the fast command that adds the column (and makes
|
||||
// the FKs deferrable). Any row left without a resolvable searchVector
|
||||
// aborts the upgrade rather than silently relaxing the NOT NULL guarantee.
|
||||
@RegisteredInstanceCommand('2.18.0', 1810000003000, { type: 'slow' })
|
||||
export class BackfillTsVectorFieldMetadataIdOnSearchFieldMetadataSlowInstanceCommand
|
||||
implements SlowInstanceCommand
|
||||
{
|
||||
async runDataMigration(dataSource: DataSource): Promise<void> {
|
||||
await dataSource.query(
|
||||
`UPDATE "core"."searchFieldMetadata" "searchFieldMetadata"
|
||||
SET "tsVectorFieldMetadataId" = "fieldMetadata"."id"
|
||||
FROM "core"."fieldMetadata" "fieldMetadata"
|
||||
WHERE "fieldMetadata"."objectMetadataId" = "searchFieldMetadata"."objectMetadataId"
|
||||
AND "fieldMetadata"."type" = 'TS_VECTOR'
|
||||
AND "fieldMetadata"."name" = 'searchVector'
|
||||
AND "searchFieldMetadata"."tsVectorFieldMetadataId" IS NULL`,
|
||||
);
|
||||
|
||||
const unresolvedRows: { count: string }[] = await dataSource.query(
|
||||
`SELECT COUNT(*) AS "count" FROM "core"."searchFieldMetadata"
|
||||
WHERE "tsVectorFieldMetadataId" IS NULL`,
|
||||
);
|
||||
|
||||
const unresolvedCount = Number(unresolvedRows[0]?.count ?? 0);
|
||||
|
||||
if (unresolvedCount > 0) {
|
||||
throw new Error(
|
||||
`Cannot enforce searchFieldMetadata.tsVectorFieldMetadataId NOT NULL: ${unresolvedCount} row(s) reference an object without a searchVector (TS_VECTOR) field`,
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
public async up(queryRunner: QueryRunner): Promise<void> {
|
||||
await queryRunner.query(
|
||||
'ALTER TABLE "core"."searchFieldMetadata" ALTER COLUMN "tsVectorFieldMetadataId" SET NOT NULL',
|
||||
);
|
||||
}
|
||||
|
||||
public async down(queryRunner: QueryRunner): Promise<void> {
|
||||
await queryRunner.query(
|
||||
'ALTER TABLE "core"."searchFieldMetadata" ALTER COLUMN "tsVectorFieldMetadataId" DROP NOT NULL',
|
||||
);
|
||||
}
|
||||
}
|
||||
+2
@@ -0,0 +1,2 @@
|
||||
export const ADD_TS_VECTOR_FIELD_METADATA_ID_TO_SEARCH_FIELD_METADATA_UPGRADE_COMMAND_NAME =
|
||||
'2.18.0_AddTsVectorFieldMetadataIdToSearchFieldMetadataFastInstanceCommand_1810000001000';
|
||||
+9
-5
@@ -34,6 +34,13 @@ import { AddIsSystemSideEffectFastInstanceCommand } from 'src/database/commands/
|
||||
import { BackfillConnectionSecuritySlowInstanceCommand } from 'src/database/commands/upgrade-version-command/2-15/2-15-instance-command-slow-1781461753981-backfill-connection-security';
|
||||
import { AddChannelWebhookSubscriptionFieldsFastInstanceCommand } from 'src/database/commands/upgrade-version-command/2-16/2-16-instance-command-fast-1782152096938-add-channel-webhook-subscription-fields';
|
||||
import { AddUniversalIdentifierAndApplicationIdToSearchFieldMetadataFastInstanceCommand } from 'src/database/commands/upgrade-version-command/2-16/2-16-instance-command-fast-1782200000000-add-universal-identifier-and-application-id-to-search-field-metadata';
|
||||
import { AddServerTriggerSettingsToLogicFunctionFastInstanceCommand } from 'src/database/commands/upgrade-version-command/2-16/2-16-instance-command-fast-1782211913427-add-server-trigger-settings-to-logic-function';
|
||||
import { AddPrimaryPublicDomainToApplicationFastInstanceCommand } from 'src/database/commands/upgrade-version-command/2-16/2-16-instance-command-fast-1782281874768-add-primary-public-domain-to-application';
|
||||
import { MakePublicDomainApplicationIdNotNullSlowInstanceCommand } from 'src/database/commands/upgrade-version-command/2-16/2-16-instance-command-slow-1782281874769-make-public-domain-application-id-not-null';
|
||||
import { CreateDpaAgreementCoreTableFastInstanceCommand } from 'src/database/commands/upgrade-version-command/2-17/2-17-instance-command-fast-1801000020000-create-dpa-agreement-core-table';
|
||||
import { CreateApplicationTranslationCoreTableFastInstanceCommand } from 'src/database/commands/upgrade-version-command/2-17/2-17-instance-command-fast-1801000100000-create-application-translation-core-table';
|
||||
import { AddTsVectorFieldMetadataIdToSearchFieldMetadataFastInstanceCommand } from 'src/database/commands/upgrade-version-command/2-18/2-18-instance-command-fast-1810000001000-add-ts-vector-field-metadata-id-to-search-field-metadata';
|
||||
import { BackfillTsVectorFieldMetadataIdOnSearchFieldMetadataSlowInstanceCommand } from 'src/database/commands/upgrade-version-command/2-18/2-18-instance-command-slow-1810000003000-backfill-ts-vector-field-metadata-id-on-search-field-metadata';
|
||||
import { AddCacheTokensToAgentChatThreadFastInstanceCommand } from 'src/database/commands/upgrade-version-command/2-2/2-2-instance-command-fast-1777455269302-add-cache-tokens-to-agent-chat-thread';
|
||||
import { AddLogoToApplicationFastInstanceCommand } from 'src/database/commands/upgrade-version-command/2-2/2-2-instance-command-fast-1777539664664-add-logo-to-application';
|
||||
import { AddSubFieldNameToViewSortEarlyFastInstanceCommand } from 'src/database/commands/upgrade-version-command/2-3/2-3-instance-command-fast-1747234200000-add-sub-field-name-to-view-sort';
|
||||
@@ -77,11 +84,6 @@ import { EncryptNonSecretApplicationVariableSlowInstanceCommand } from 'src/data
|
||||
import { MigrateAiModelPreferencesSlowInstanceCommand } from 'src/database/commands/upgrade-version-command/2-9/2-9-instance-command-slow-1799000010000-migrate-ai-model-preferences';
|
||||
import { AddFolderImportToMessageFolderPendingSyncActionFastInstanceCommand } from './2-15/2-15-instance-command-fast-1781714499016-add-folder-import-to-message-folder-pending-sync-action';
|
||||
import { AddViewKanbanColumnWidthFastInstanceCommand } from './2-15/2-15-instance-command-fast-1781900000000-add-view-kanban-column-width';
|
||||
import { AddPrimaryPublicDomainToApplicationFastInstanceCommand } from 'src/database/commands/upgrade-version-command/2-16/2-16-instance-command-fast-1782281874768-add-primary-public-domain-to-application';
|
||||
import { MakePublicDomainApplicationIdNotNullSlowInstanceCommand } from 'src/database/commands/upgrade-version-command/2-16/2-16-instance-command-slow-1782281874769-make-public-domain-application-id-not-null';
|
||||
import { AddServerTriggerSettingsToLogicFunctionFastInstanceCommand } from 'src/database/commands/upgrade-version-command/2-16/2-16-instance-command-fast-1782211913427-add-server-trigger-settings-to-logic-function';
|
||||
import { CreateDpaAgreementCoreTableFastInstanceCommand } from 'src/database/commands/upgrade-version-command/2-17/2-17-instance-command-fast-1801000020000-create-dpa-agreement-core-table';
|
||||
import { CreateApplicationTranslationCoreTableFastInstanceCommand } from 'src/database/commands/upgrade-version-command/2-17/2-17-instance-command-fast-1801000100000-create-application-translation-core-table';
|
||||
|
||||
export const INSTANCE_COMMANDS = [
|
||||
AddViewFieldGroupIdIndexOnViewFieldFastInstanceCommand,
|
||||
@@ -166,4 +168,6 @@ export const INSTANCE_COMMANDS = [
|
||||
MakePublicDomainApplicationIdNotNullSlowInstanceCommand,
|
||||
CreateDpaAgreementCoreTableFastInstanceCommand,
|
||||
CreateApplicationTranslationCoreTableFastInstanceCommand,
|
||||
AddTsVectorFieldMetadataIdToSearchFieldMetadataFastInstanceCommand,
|
||||
BackfillTsVectorFieldMetadataIdOnSearchFieldMetadataSlowInstanceCommand,
|
||||
];
|
||||
|
||||
+114
@@ -0,0 +1,114 @@
|
||||
import { FieldMetadataType } from 'twenty-shared/types';
|
||||
|
||||
import { type FieldMetadataEntity } from 'src/engine/metadata-modules/field-metadata/field-metadata.entity';
|
||||
import { getFlatFieldMetadataMock } from 'src/engine/metadata-modules/flat-field-metadata/__mocks__/get-flat-field-metadata.mock';
|
||||
import { getFlatObjectMetadataMock } from 'src/engine/metadata-modules/flat-object-metadata/__mocks__/get-flat-object-metadata.mock';
|
||||
import { type ObjectMetadataEntity } from 'src/engine/metadata-modules/object-metadata/object-metadata.entity';
|
||||
import { type SearchFieldMetadataEntity } from 'src/engine/metadata-modules/search-field-metadata/search-field-metadata.entity';
|
||||
import { generateWorkspaceSchemaDdl } from 'src/database/commands/workspace-export/utils/generate-workspace-schema-ddl.util';
|
||||
|
||||
describe('generateWorkspaceSchemaDdl', () => {
|
||||
const workspaceId = '20202020-1c25-4d02-bf25-6aeccf7ea419';
|
||||
const objectMetadataId = '20202020-object-id';
|
||||
const nameFieldId = '20202020-name-field-id';
|
||||
const searchVectorFieldId = '20202020-search-vector-field-id';
|
||||
|
||||
const objectMetadata = {
|
||||
...getFlatObjectMetadataMock({
|
||||
universalIdentifier: 'person',
|
||||
id: objectMetadataId,
|
||||
nameSingular: 'person',
|
||||
namePlural: 'persons',
|
||||
}),
|
||||
isActive: true,
|
||||
} as unknown as ObjectMetadataEntity;
|
||||
|
||||
const nameField = getFlatFieldMetadataMock({
|
||||
universalIdentifier: 'name',
|
||||
id: nameFieldId,
|
||||
objectMetadataId,
|
||||
type: FieldMetadataType.TEXT,
|
||||
name: 'name',
|
||||
}) as unknown as FieldMetadataEntity;
|
||||
|
||||
const searchVectorField = getFlatFieldMetadataMock({
|
||||
universalIdentifier: 'searchVector',
|
||||
id: searchVectorFieldId,
|
||||
objectMetadataId,
|
||||
type: FieldMetadataType.TS_VECTOR,
|
||||
name: 'searchVector',
|
||||
}) as unknown as FieldMetadataEntity;
|
||||
|
||||
const fieldsByObjectId = new Map<string, FieldMetadataEntity[]>([
|
||||
[objectMetadataId, [nameField, searchVectorField]],
|
||||
]);
|
||||
|
||||
const buildSearchFieldMetadata = (
|
||||
overrides: Partial<SearchFieldMetadataEntity> = {},
|
||||
): SearchFieldMetadataEntity =>
|
||||
({
|
||||
universalIdentifier: 'search-name',
|
||||
objectMetadataId,
|
||||
fieldMetadataId: nameFieldId,
|
||||
tsVectorFieldMetadataId: searchVectorFieldId,
|
||||
position: 0,
|
||||
...overrides,
|
||||
}) as SearchFieldMetadataEntity;
|
||||
|
||||
const generateSearchVectorColumnSql = (
|
||||
searchFieldMetadatasByObjectId: Map<string, SearchFieldMetadataEntity[]>,
|
||||
): string => {
|
||||
const statements = generateWorkspaceSchemaDdl(
|
||||
workspaceId,
|
||||
'workspace_schema',
|
||||
[objectMetadata],
|
||||
fieldsByObjectId,
|
||||
searchFieldMetadatasByObjectId,
|
||||
);
|
||||
|
||||
const createTableStatement = statements.find((statement) =>
|
||||
statement.includes('"searchVector"'),
|
||||
);
|
||||
|
||||
expect(createTableStatement).toBeDefined();
|
||||
|
||||
return createTableStatement as string;
|
||||
};
|
||||
|
||||
it('should emit the searchVector column as a STORED generated column derived from searchFieldMetadata', () => {
|
||||
const createTableStatement = generateSearchVectorColumnSql(
|
||||
new Map([[objectMetadataId, [buildSearchFieldMetadata()]]]),
|
||||
);
|
||||
|
||||
expect(createTableStatement).toContain(
|
||||
`"searchVector" tsvector GENERATED ALWAYS AS (to_tsvector('simple', COALESCE(public.unaccent_immutable("name"), ''))) STORED`,
|
||||
);
|
||||
});
|
||||
|
||||
it('should tolerate a legacy NULL tsVectorFieldMetadataId row (pre-2.18 backfill)', () => {
|
||||
const createTableStatement = generateSearchVectorColumnSql(
|
||||
new Map([
|
||||
[
|
||||
objectMetadataId,
|
||||
[
|
||||
buildSearchFieldMetadata({
|
||||
tsVectorFieldMetadataId: null as never,
|
||||
}),
|
||||
],
|
||||
],
|
||||
]),
|
||||
);
|
||||
|
||||
expect(createTableStatement).toContain(
|
||||
`GENERATED ALWAYS AS (to_tsvector('simple', COALESCE(public.unaccent_immutable("name"), ''))) STORED`,
|
||||
);
|
||||
});
|
||||
|
||||
it('should still emit a valid STORED generated column when no searchFieldMetadata rows exist', () => {
|
||||
const createTableStatement = generateSearchVectorColumnSql(new Map());
|
||||
|
||||
expect(createTableStatement).toContain(
|
||||
`"searchVector" tsvector GENERATED ALWAYS AS (to_tsvector('simple', NULL)) STORED`,
|
||||
);
|
||||
});
|
||||
});
|
||||
+74
-1
@@ -1,26 +1,82 @@
|
||||
import { FieldMetadataType } from 'twenty-shared/types';
|
||||
|
||||
import { type FieldMetadataEntity } from 'src/engine/metadata-modules/field-metadata/field-metadata.entity';
|
||||
import { type FlatEntityMaps } from 'src/engine/metadata-modules/flat-entity/types/flat-entity-maps.type';
|
||||
import { type FlatFieldMetadata } from 'src/engine/metadata-modules/flat-field-metadata/types/flat-field-metadata.type';
|
||||
import { isFlatFieldMetadataOfType } from 'src/engine/metadata-modules/flat-field-metadata/utils/is-flat-field-metadata-of-type.util';
|
||||
import { type FlatObjectMetadata } from 'src/engine/metadata-modules/flat-object-metadata/types/flat-object-metadata.type';
|
||||
import { type FlatSearchFieldMetadata } from 'src/engine/metadata-modules/flat-search-field-metadata/types/flat-search-field-metadata.type';
|
||||
import { deriveSearchVectorAsExpressionForTsVectorField } from 'src/engine/metadata-modules/flat-search-field-metadata/utils/derive-search-vector-as-expression-for-ts-vector-field.util';
|
||||
import { type ObjectMetadataEntity } from 'src/engine/metadata-modules/object-metadata/object-metadata.entity';
|
||||
import { SEARCH_VECTOR_FIELD } from 'src/engine/metadata-modules/search-field-metadata/constants/search-vector-field.constants';
|
||||
import { type SearchFieldMetadataEntity } from 'src/engine/metadata-modules/search-field-metadata/search-field-metadata.entity';
|
||||
import { buildSqlColumnDefinition } from 'src/engine/twenty-orm/workspace-schema-manager/utils/build-sql-column-definition.util';
|
||||
import { computeTableName } from 'src/engine/utils/compute-table-name.util';
|
||||
import { TWENTY_STANDARD_APPLICATION } from 'src/engine/workspace-manager/twenty-standard-application/constants/twenty-standard-applications';
|
||||
import {
|
||||
escapeIdentifier,
|
||||
escapeLiteral,
|
||||
} from 'src/engine/workspace-manager/workspace-migration/utils/remove-sql-injection.util';
|
||||
import { generateColumnDefinitions } from 'src/engine/workspace-manager/workspace-migration/workspace-migration-runner/utils/generate-column-definitions.util';
|
||||
import { TWENTY_STANDARD_APPLICATION } from 'src/engine/workspace-manager/twenty-standard-application/constants/twenty-standard-applications';
|
||||
import {
|
||||
type CreateEnumOperationSpec,
|
||||
EnumOperation,
|
||||
collectEnumOperationsForObject,
|
||||
} from 'src/engine/workspace-manager/workspace-migration/workspace-migration-runner/utils/workspace-schema-enum-operations.util';
|
||||
|
||||
const buildSearchFieldMetadataDerivationInputs = ({
|
||||
fieldMetadatas,
|
||||
objectSearchFieldMetadatas,
|
||||
}: {
|
||||
fieldMetadatas: FieldMetadataEntity[];
|
||||
objectSearchFieldMetadatas: SearchFieldMetadataEntity[];
|
||||
}): {
|
||||
indexedFieldById: ReadonlyMap<
|
||||
string,
|
||||
{ name: string; type: FieldMetadataType }
|
||||
>;
|
||||
flatSearchFieldMetadataMaps: FlatEntityMaps<FlatSearchFieldMetadata>;
|
||||
} => {
|
||||
const indexedFieldById = new Map(
|
||||
fieldMetadatas.map((fieldMetadata) => [
|
||||
fieldMetadata.id,
|
||||
{ name: fieldMetadata.name, type: fieldMetadata.type },
|
||||
]),
|
||||
);
|
||||
|
||||
const searchVectorFieldId = fieldMetadatas.find(
|
||||
(fieldMetadata) =>
|
||||
fieldMetadata.type === FieldMetadataType.TS_VECTOR &&
|
||||
fieldMetadata.name === SEARCH_VECTOR_FIELD.name,
|
||||
)?.id;
|
||||
|
||||
const byUniversalIdentifier = Object.fromEntries(
|
||||
objectSearchFieldMetadatas.map((searchFieldMetadata) => [
|
||||
searchFieldMetadata.universalIdentifier,
|
||||
{
|
||||
...searchFieldMetadata,
|
||||
tsVectorFieldMetadataId:
|
||||
searchFieldMetadata.tsVectorFieldMetadataId ?? searchVectorFieldId,
|
||||
} as unknown as FlatSearchFieldMetadata,
|
||||
]),
|
||||
);
|
||||
|
||||
return {
|
||||
indexedFieldById,
|
||||
flatSearchFieldMetadataMaps: {
|
||||
byUniversalIdentifier,
|
||||
universalIdentifierById: {},
|
||||
universalIdentifiersByApplicationId: {},
|
||||
},
|
||||
};
|
||||
};
|
||||
|
||||
export const generateWorkspaceSchemaDdl = (
|
||||
workspaceId: string,
|
||||
schemaName: string,
|
||||
objectMetadatas: ObjectMetadataEntity[],
|
||||
fieldsByObjectId: Map<string, FieldMetadataEntity[]>,
|
||||
searchFieldMetadatasByObjectId: Map<string, SearchFieldMetadataEntity[]>,
|
||||
): string[] => {
|
||||
const statements: string[] = [];
|
||||
|
||||
@@ -37,6 +93,13 @@ export const generateWorkspaceSchemaDdl = (
|
||||
const flatFieldMetadatas = fieldMetadatas as unknown as FlatFieldMetadata[];
|
||||
const flatObjectMetadata = objectMetadata as unknown as FlatObjectMetadata;
|
||||
|
||||
const { indexedFieldById, flatSearchFieldMetadataMaps } =
|
||||
buildSearchFieldMetadataDerivationInputs({
|
||||
fieldMetadatas,
|
||||
objectSearchFieldMetadatas:
|
||||
searchFieldMetadatasByObjectId.get(objectMetadata.id) ?? [],
|
||||
});
|
||||
|
||||
const enumOperations = collectEnumOperationsForObject({
|
||||
tableName,
|
||||
operation: EnumOperation.CREATE,
|
||||
@@ -57,6 +120,16 @@ export const generateWorkspaceSchemaDdl = (
|
||||
flatFieldMetadata,
|
||||
flatObjectMetadata,
|
||||
workspaceId,
|
||||
searchVectorAsExpression: isFlatFieldMetadataOfType(
|
||||
flatFieldMetadata,
|
||||
FieldMetadataType.TS_VECTOR,
|
||||
)
|
||||
? deriveSearchVectorAsExpressionForTsVectorField({
|
||||
tsVectorFieldMetadataId: flatFieldMetadata.id,
|
||||
flatSearchFieldMetadataMaps,
|
||||
indexedFieldById,
|
||||
})
|
||||
: undefined,
|
||||
}),
|
||||
);
|
||||
|
||||
|
||||
+6
-1
@@ -5,10 +5,15 @@ import { WorkspaceExportCommand } from 'src/database/commands/workspace-export/w
|
||||
import { WorkspaceExportService } from 'src/database/commands/workspace-export/workspace-export.service';
|
||||
import { FieldMetadataEntity } from 'src/engine/metadata-modules/field-metadata/field-metadata.entity';
|
||||
import { ObjectMetadataEntity } from 'src/engine/metadata-modules/object-metadata/object-metadata.entity';
|
||||
import { SearchFieldMetadataEntity } from 'src/engine/metadata-modules/search-field-metadata/search-field-metadata.entity';
|
||||
|
||||
@Module({
|
||||
imports: [
|
||||
TypeOrmModule.forFeature([ObjectMetadataEntity, FieldMetadataEntity]),
|
||||
TypeOrmModule.forFeature([
|
||||
ObjectMetadataEntity,
|
||||
FieldMetadataEntity,
|
||||
SearchFieldMetadataEntity,
|
||||
]),
|
||||
],
|
||||
providers: [WorkspaceExportCommand, WorkspaceExportService],
|
||||
})
|
||||
|
||||
+39
-10
@@ -11,20 +11,21 @@ import {
|
||||
Repository,
|
||||
} from 'typeorm';
|
||||
|
||||
import { FieldMetadataEntity } from 'src/engine/metadata-modules/field-metadata/field-metadata.entity';
|
||||
import { ObjectMetadataEntity } from 'src/engine/metadata-modules/object-metadata/object-metadata.entity';
|
||||
import { WorkspaceEntity } from 'src/engine/core-modules/workspace/workspace.entity';
|
||||
import { TWENTY_STANDARD_APPLICATION } from 'src/engine/workspace-manager/twenty-standard-application/constants/twenty-standard-applications';
|
||||
import { getWorkspaceSchemaName } from 'src/engine/workspace-datasource/utils/get-workspace-schema-name.util';
|
||||
import { computeTableName } from 'src/engine/utils/compute-table-name.util';
|
||||
import { escapeIdentifier } from 'src/engine/workspace-manager/workspace-migration/utils/remove-sql-injection.util';
|
||||
import { getCoreEntityMetadatasWithWorkspaceId } from 'src/database/commands/workspace-export/utils/get-core-entity-metadatas-with-workspace-id.util';
|
||||
import { generateWorkspaceSchemaDdl } from 'src/database/commands/workspace-export/utils/generate-workspace-schema-ddl.util';
|
||||
import { buildInsertPrefix } from 'src/database/commands/workspace-export/utils/build-insert-prefix.util';
|
||||
import { buildWorkspaceTableColumnSets } from 'src/database/commands/workspace-export/utils/build-workspace-table-column-sets.util';
|
||||
import { formatSqlValue } from 'src/database/commands/workspace-export/utils/format-sql-value.util';
|
||||
import { formatPgCopyField } from './utils/format-pg-copy-value.util';
|
||||
import { generateWorkspaceSchemaDdl } from 'src/database/commands/workspace-export/utils/generate-workspace-schema-ddl.util';
|
||||
import { getCoreEntityMetadatasWithWorkspaceId } from 'src/database/commands/workspace-export/utils/get-core-entity-metadatas-with-workspace-id.util';
|
||||
import { WorkspaceEntity } from 'src/engine/core-modules/workspace/workspace.entity';
|
||||
import { FieldMetadataEntity } from 'src/engine/metadata-modules/field-metadata/field-metadata.entity';
|
||||
import { ObjectMetadataEntity } from 'src/engine/metadata-modules/object-metadata/object-metadata.entity';
|
||||
import { SearchFieldMetadataEntity } from 'src/engine/metadata-modules/search-field-metadata/search-field-metadata.entity';
|
||||
import { computeTableName } from 'src/engine/utils/compute-table-name.util';
|
||||
import { getWorkspaceSchemaName } from 'src/engine/workspace-datasource/utils/get-workspace-schema-name.util';
|
||||
import { TWENTY_STANDARD_APPLICATION } from 'src/engine/workspace-manager/twenty-standard-application/constants/twenty-standard-applications';
|
||||
import { escapeIdentifier } from 'src/engine/workspace-manager/workspace-migration/utils/remove-sql-injection.util';
|
||||
import { isNonEmptyArray } from 'twenty-shared/utils';
|
||||
import { formatPgCopyField } from './utils/format-pg-copy-value.util';
|
||||
|
||||
const BATCH_SIZE = 10_000;
|
||||
|
||||
@@ -57,6 +58,9 @@ export class WorkspaceExportService {
|
||||
private readonly objectMetadataRepository: Repository<ObjectMetadataEntity>,
|
||||
@InjectRepository(FieldMetadataEntity)
|
||||
private readonly fieldMetadataRepository: Repository<FieldMetadataEntity>,
|
||||
// eslint-disable-next-line twenty/prefer-workspace-scoped-repository -- Ignored
|
||||
@InjectRepository(SearchFieldMetadataEntity)
|
||||
private readonly searchFieldMetadataRepository: Repository<SearchFieldMetadataEntity>,
|
||||
) {}
|
||||
|
||||
async exportWorkspace({
|
||||
@@ -95,6 +99,28 @@ export class WorkspaceExportService {
|
||||
fieldsByObjectId.set(fieldMetadata.objectMetadataId, objectFields);
|
||||
}
|
||||
|
||||
const searchFieldMetadatas = await this.searchFieldMetadataRepository.find({
|
||||
where: { workspaceId },
|
||||
});
|
||||
|
||||
const searchFieldMetadatasByObjectId = new Map<
|
||||
string,
|
||||
SearchFieldMetadataEntity[]
|
||||
>();
|
||||
|
||||
for (const searchFieldMetadata of searchFieldMetadatas) {
|
||||
const objectSearchFieldMetadatas =
|
||||
searchFieldMetadatasByObjectId.get(
|
||||
searchFieldMetadata.objectMetadataId,
|
||||
) ?? [];
|
||||
|
||||
objectSearchFieldMetadatas.push(searchFieldMetadata);
|
||||
searchFieldMetadatasByObjectId.set(
|
||||
searchFieldMetadata.objectMetadataId,
|
||||
objectSearchFieldMetadatas,
|
||||
);
|
||||
}
|
||||
|
||||
mkdirSync(outputPath, { recursive: true });
|
||||
|
||||
const timestamp = new Date().toISOString().replace(/[:.]/g, '-');
|
||||
@@ -117,6 +143,7 @@ export class WorkspaceExportService {
|
||||
schemaName,
|
||||
objectMetadatas,
|
||||
fieldsByObjectId,
|
||||
searchFieldMetadatasByObjectId,
|
||||
stream,
|
||||
);
|
||||
|
||||
@@ -327,6 +354,7 @@ export class WorkspaceExportService {
|
||||
schemaName: string,
|
||||
objectMetadatas: ObjectMetadataEntity[],
|
||||
fieldsByObjectId: Map<string, FieldMetadataEntity[]>,
|
||||
searchFieldMetadatasByObjectId: Map<string, SearchFieldMetadataEntity[]>,
|
||||
stream: WriteStream,
|
||||
): void {
|
||||
this.logger.log('Generating workspace schema DDL from metadata...');
|
||||
@@ -336,6 +364,7 @@ export class WorkspaceExportService {
|
||||
schemaName,
|
||||
objectMetadatas,
|
||||
fieldsByObjectId,
|
||||
searchFieldMetadatasByObjectId,
|
||||
);
|
||||
|
||||
this.logger.log(` ${ddlStatements.length} DDL statements`);
|
||||
|
||||
Reference in New Issue
Block a user