[BREAKING-CHANGE] Centralize system View viewField side effect (#23081)
# Introduction Closes https://github.com/twentyhq/core-team-issues/issues/2669 Part of the `isSystemSideEffect` engine-ownership effort. Until now, a custom object's default **INDEX** table view (`All {objectLabelPlural}`) and its view fields were built imperatively in `ObjectMetadataService` with random `v4()` identifiers, while `twenty-standard` authored its own copies with hardcoded literals. The two never converged, an object rename could drift the view, and nothing marked these rows as engine-owned. This PR makes the metadata side-effect engine the **single owner** of the INDEX view and its view fields, on name-free deterministic identifiers, for custom and standard objects alike. ## Core design - **Name-free deterministic identity.** The INDEX view identifier derives from `object identifier + ViewKey.INDEX` (`getSystemViewUniversalIdentifier`); each view-field identifier derives from `view identifier + field identifier` (`getViewFieldUniversalIdentifier`). An object rename (with a pinned object identifier) keeps the same view, losslessly. - **`isSystemSideEffect: true` is provenance.** Every INDEX view / view field the engine emits is flagged system-owned, so manifest deletion inference never drops it. The flag follows the view: a view field inherits its parent view's flag. - **The engine is the sole owner of the INDEX view.** It always emits it; a caller providing one with the same derived identifier is a genuine conflict surfaced by the engine's reserved-identifier collision, not silently deferred. ## Changes ### Shared (`twenty-shared`) - `getIndexViewUniversalIdentifier` → `getSystemViewUniversalIdentifier`, now taking a `viewKey` (generalizes to any singleton engine-owned view). - Standard field identifiers extracted into a new `STANDARD_OBJECT_FIELDS` constant, so both an object's `fields` and its INDEX view read the same field identifiers. - `buildStandardObjectIndexView` derives the standard INDEX view + view-field identifiers from `STANDARD_OBJECT_FIELDS`, replacing the hardcoded literals in `standard-object.constant.ts`. ### Metadata side-effect engine (custom objects) - **`objectSystemFieldsAndIndexViewOnCreate`** (replaces `objectSystemFieldsOnCreate`): on object creation, provisions the 7 reserved system fields **and** the INDEX view with one view field per displayable system field, all `isSystemSideEffect: true`. - **`fieldIndexViewFieldOnCreate`** (new): on field creation, provisions the field's INDEX view field. Object created in the same batch → visible, positioned before the system view fields; pre-existing object → hidden, appended (preserving the historical `createOneField` behavior). Both branches resolve the INDEX view by its derived identifier (single map access, never a scan). - **`fieldSystemViewFieldsOnDelete`** (new): on field deletion, cascade-deletes every engine-owned view field displaying it. - **`objectSystemSideEffectsOnDelete`** (extended): now also cascade-deletes the object's engine-owned views and their view fields (in addition to system fields, indexes, searchFieldMetadata). Every lookup walks a foreign-key aggregator down from the deleted object, so the work is proportional to what the object owns, never to workspace size. - Object-create and field-create positions are derived from the same caller-input field list, so the INDEX view layout is contiguous with no handler-ordering dependency. - `view` / `viewField` added to the side-effect companion metadata names for `fieldMetadata` and `objectMetadata`. ### Reserved-identifier invariant A caller can never define an entity whose identifier collides with one a system side effect produces: caller inputs are forced `isSystemSideEffect: false` at every entry point (API and app-manifest transpilers), and the engine raises `RESERVED_SYSTEM_UNIVERSAL_IDENTIFIER`, aborting the operation, when a system emission lands on a caller-claimed identifier. Covered by a new engine-level test. ### Caller-side provisioning removed The imperative INDEX view + view-field provisioning is removed from `ObjectMetadataService.createOneObject`. The record-page `FIELDS_WIDGET` view is intentionally left caller-side and deferred to the follow-up (see below). ### `twenty-standard` convergence Standard INDEX views and their view fields converge on the same derived-identifier + `isSystemSideEffect: true` scheme as the engine. `twenty-standard` syncs through the from/to migration path (which never runs the side-effect engine), so it authors this INDEX surface itself, matching what the engine produces for custom objects. ## Rollout Two `2.26.0` workspace commands, running after the `2.25` messageCampaign commands: - `upgrade:2-26:reconcile-index-view-universal-identifier` re-owns the INDEX views of the **twenty-standard and workspace-custom applications** and all their view fields to the derived identifiers with `isSystemSideEffect: true`, in a single per-workspace transaction. Each view field identifier is keyed on the application of the **displayed field** (an app or user column on a standard INDEX view converges too). Soft-deleted views and view fields are skipped: one can coexist with an active successor on the same derivation inputs and both would derive the same identifier. Children reference the view by primary key, so the re-own is lossless. - `upgrade:2-26:demote-and-backfill-application-index-view` handles **manifest-installed applications**, which never had their INDEX view auto-provisioned: every caller-authored INDEX view of another application is demoted to `key: null` (a plain additional view under its manifest identifier), then every application object gets the engine-owned INDEX view and its full view-field layout backfilled through the migration pipeline's legacy path (no side-effect expansion), views committed before view fields across applications since a view field belongs to the application owning its field. Idempotent and retry-safe: engine-owned INDEX views are neither demoted nor re-backfilled, and view creation and view-field creation are gated independently, so a retry after a partial failure still backfills the missing view fields of an already-committed view. Both support `--dry-run` and invalidate the full flat-maps closure (parents aggregate the re-owned identifiers, children resolve them as universal foreign keys, and page-layout widget universal configurations resolve view PKs at cache-build time). The `2.25` `upgrade:2-25:add-message-campaign-name-field` command is adapted to resolve the campaign INDEX view by its INDEX key on the object instead of by universal identifier: it now runs before the reconcile, on workspaces still holding legacy identifiers. ## ⚠️ Breaking change This PR **mutates 187 previously hardcoded universal identifiers** — the standard objects' INDEX views and their view fields (the literals removed from `standard-object.constant.ts`), now derived. - **Handled by the `2.26` commands above** for all existing workspaces. - **The INDEX key is now engine-reserved.** The flat view validator rejects caller-created INDEX views (API and manifest inputs are forced `isSystemSideEffect: false`) and enforces a single non-deleted INDEX view per object; `view.key` is no longer a comparable/updatable property, so no writer can promote or demote a view after creation. `ViewManifest.key` is deprecated and ignored (manifest views are always additional views, so old apps keep syncing and demoted views are not promoted back); the REST/GraphQL create path now rejects `key: INDEX`. In-repo example apps (`hello-world`, `document-generator`) no longer declare it. - **12 declared-but-never-seeded standard INDEX view field identifiers deleted** (the former `preservedViewFields` on `timelineActivity`, `workflowRun` and `workspaceMember`): after the reconcile, no workspace row references them. - **`computeFlatViewFieldsToCreate` now derives view field identifiers** instead of drawing `v4()` ones, which also changes what the committed `1-23` record-page backfill produces going forward (deliberate, documented in-code). - **Record-page views and view fields are not affected** (identifiers unchanged). - **In-repo apps: `twenty-last-contact` updated.** It was the only app declaring explicit INDEX view fields (10 columns across `allPeople` / `allCompanies` / `allOpportunities`) through manifest `viewFields`. Those target identifiers are now engine-owned and derived, so the manifest inputs no longer resolve and install failed with `View not found`. The app now declares only its fields; the engine's `fieldIndexViewFieldOnCreate` provisions the matching INDEX view field automatically. No other app under `packages/twenty-apps` references any of the 187 mutated identifiers, and apps that target standard views point at record-page views (e.g. `real-estate` → `opportunityRecordPageFields`) or their own objects (`twenty-partners`), all unchanged. ### Loss of granularity for app maintainers The engine now owns the INDEX view field of every field a caller adds to an object, so app maintainers lose direct control over those columns. Previously an app could target the engine-owned INDEX view with an explicit manifest `viewField` and set its `position` and `isVisible`. Now `fieldIndexViewFieldOnCreate` appends a **hidden** view field in caller-input order on field creation, so: - Columns an app previously showed at a **dedicated position** and **visible** (e.g. `twenty-last-contact`'s last-contact columns) become **hidden** and **appended in input order** after install. - There is currently **no manifest way to override** the engine-provisioned INDEX view field's position, visibility, or size. This is a deliberate regression accepted for the sake of single-ownership, and app maintainers should expect their INDEX columns to move/hide after upgrading. A follow-up override API will let maintainers reclaim per-field control over the engine-provisioned INDEX view field. ## Testing - Unit specs for each handler: object create (system fields + INDEX view/view fields, override, position offset), field create (same-batch vs existing-object, non-displayable noop, no-INDEX-view noop), field delete, object delete (fields/indexes/searchFieldMetadata/views/view fields cascade, reverse-relation view field on another object). - Engine-level test for the reserved-identifier collision. - `twenty-standard` guard test that its INDEX views/view fields stay on the derived scheme and stay system-owned. - Integration test: full engine provisioning of the INDEX view/view fields on object creation, same view id preserved across an object rename, and cascade delete on object deletion. ## Follow-up The full record-page stack (record-page view, its view fields, view field groups, page layout / tab / widget) is still built imperatively and moves into the engine in https://github.com/twentyhq/core-team-issues/issues/2721.
This commit is contained in:
Paul Rastoin
+2
-1
@@ -522,7 +522,8 @@ export class BackfillRecordPageLayoutsCommand extends ProvisionedWorkspaceComman
|
||||
const viewFields = computeFlatViewFieldsToCreate({
|
||||
objectFlatFieldMetadatas: objectFieldMetadatas,
|
||||
viewUniversalIdentifier: fieldsView.universalIdentifier,
|
||||
flatApplication: twentyStandardFlatApplication,
|
||||
applicationUniversalIdentifier:
|
||||
twentyStandardFlatApplication.universalIdentifier,
|
||||
labelIdentifierFieldMetadataUniversalIdentifier:
|
||||
customObject.labelIdentifierFieldMetadataUniversalIdentifier,
|
||||
excludeLabelIdentifier: true,
|
||||
|
||||
+42
-28
@@ -1,6 +1,7 @@
|
||||
import { Command } from 'nest-commander';
|
||||
import { getSearchFieldUniversalIdentifier } from 'twenty-shared/application';
|
||||
import { STANDARD_OBJECTS } from 'twenty-shared/metadata';
|
||||
import { ViewKey } from 'twenty-shared/types';
|
||||
import { isDefined } from 'twenty-shared/utils';
|
||||
|
||||
import { ProvisionedWorkspaceCommandRunner } from 'src/database/commands/command-runners/provisioned-workspace.command-runner';
|
||||
@@ -13,8 +14,8 @@ import { findFlatEntityByUniversalIdentifier } from 'src/engine/metadata-modules
|
||||
import { type FlatFieldMetadata } from 'src/engine/metadata-modules/flat-field-metadata/types/flat-field-metadata.type';
|
||||
import { type FlatObjectMetadata } from 'src/engine/metadata-modules/flat-object-metadata/types/flat-object-metadata.type';
|
||||
import { type FlatSearchFieldMetadata } from 'src/engine/metadata-modules/flat-search-field-metadata/types/flat-search-field-metadata.type';
|
||||
import { type FlatView } from 'src/engine/metadata-modules/flat-view/types/flat-view.type';
|
||||
import { type FlatViewField } from 'src/engine/metadata-modules/flat-view-field/types/flat-view-field.type';
|
||||
import { type FlatView } from 'src/engine/metadata-modules/flat-view/types/flat-view.type';
|
||||
import { WorkspaceCacheService } from 'src/engine/workspace-cache/services/workspace-cache.service';
|
||||
import { computeTwentyStandardApplicationAllFlatEntityMaps } from 'src/engine/workspace-manager/twenty-standard-application/utils/twenty-standard-application-all-flat-entity-maps.constant';
|
||||
import { WorkspaceMigrationValidateBuildAndRunService } from 'src/engine/workspace-manager/workspace-migration/services/workspace-migration-validate-build-and-run-service';
|
||||
@@ -159,15 +160,34 @@ export class AddMessageCampaignNameFieldCommand extends ProvisionedWorkspaceComm
|
||||
);
|
||||
}
|
||||
|
||||
viewFieldsToCreate.push({
|
||||
...standardNameViewField,
|
||||
position: this.computeNameViewFieldPosition({
|
||||
flatViewMaps,
|
||||
flatViewFieldMaps,
|
||||
standardNameViewField,
|
||||
isNameLabelIdentifier,
|
||||
}),
|
||||
});
|
||||
const campaignIndexFlatView =
|
||||
campaignObjectMetadata.viewUniversalIdentifiers
|
||||
.map(
|
||||
(viewUniversalIdentifier) =>
|
||||
flatViewMaps.byUniversalIdentifier[viewUniversalIdentifier],
|
||||
)
|
||||
.filter(isDefined)
|
||||
.find(
|
||||
(flatView) =>
|
||||
flatView.key === ViewKey.INDEX && !isDefined(flatView.deletedAt),
|
||||
);
|
||||
|
||||
if (isDefined(campaignIndexFlatView)) {
|
||||
viewFieldsToCreate.push({
|
||||
...standardNameViewField,
|
||||
viewUniversalIdentifier: campaignIndexFlatView.universalIdentifier,
|
||||
position: this.computeNameViewFieldPosition({
|
||||
campaignIndexFlatView,
|
||||
flatViewFieldMaps,
|
||||
standardNameViewField,
|
||||
isNameLabelIdentifier,
|
||||
}),
|
||||
});
|
||||
} else {
|
||||
this.logger.warn(
|
||||
`No INDEX view found for messageCampaign in workspace ${workspaceId}, skipping the name view column`,
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
const searchFieldMetadatasToCreate =
|
||||
@@ -255,32 +275,26 @@ export class AddMessageCampaignNameFieldCommand extends ProvisionedWorkspaceComm
|
||||
// it), above all of them otherwise (the validator forbids anything below the
|
||||
// label identifier).
|
||||
private computeNameViewFieldPosition({
|
||||
flatViewMaps,
|
||||
campaignIndexFlatView,
|
||||
flatViewFieldMaps,
|
||||
standardNameViewField,
|
||||
isNameLabelIdentifier,
|
||||
}: {
|
||||
flatViewMaps: FlatEntityMaps<FlatView>;
|
||||
campaignIndexFlatView: FlatView;
|
||||
flatViewFieldMaps: FlatEntityMaps<FlatViewField>;
|
||||
standardNameViewField: FlatViewField;
|
||||
isNameLabelIdentifier: boolean;
|
||||
}): number {
|
||||
const flatView = findFlatEntityByUniversalIdentifier<FlatView>({
|
||||
flatEntityMaps: flatViewMaps,
|
||||
universalIdentifier: standardNameViewField.viewUniversalIdentifier,
|
||||
});
|
||||
|
||||
if (!isDefined(flatView)) {
|
||||
return standardNameViewField.position;
|
||||
}
|
||||
|
||||
const otherViewFieldPositions = flatView.viewFieldUniversalIdentifiers
|
||||
.map(
|
||||
(viewFieldUniversalIdentifier) =>
|
||||
flatViewFieldMaps.byUniversalIdentifier[viewFieldUniversalIdentifier],
|
||||
)
|
||||
.filter(isDefined)
|
||||
.map(({ position }) => position);
|
||||
const otherViewFieldPositions =
|
||||
campaignIndexFlatView.viewFieldUniversalIdentifiers
|
||||
.map(
|
||||
(viewFieldUniversalIdentifier) =>
|
||||
flatViewFieldMaps.byUniversalIdentifier[
|
||||
viewFieldUniversalIdentifier
|
||||
],
|
||||
)
|
||||
.filter(isDefined)
|
||||
.map(({ position }) => position);
|
||||
|
||||
if (otherViewFieldPositions.length === 0) {
|
||||
return standardNameViewField.position;
|
||||
|
||||
+27
@@ -0,0 +1,27 @@
|
||||
import { Module } from '@nestjs/common';
|
||||
import { TypeOrmModule } from '@nestjs/typeorm';
|
||||
|
||||
import { WorkspaceIteratorModule } from 'src/database/commands/command-runners/workspace-iterator.module';
|
||||
import { ReconcileIndexViewUniversalIdentifierCommand } from 'src/database/commands/upgrade-version-command/2-26/2-26-workspace-command-1785255689000-reconcile-index-view-universal-identifier.command';
|
||||
import { DemoteAndBackfillApplicationIndexViewCommand } from 'src/database/commands/upgrade-version-command/2-26/2-26-workspace-command-1785255690000-demote-and-backfill-application-index-view.command';
|
||||
import { ApplicationModule } from 'src/engine/core-modules/application/application.module';
|
||||
import { ViewEntity } from 'src/engine/metadata-modules/view/entities/view.entity';
|
||||
import { WorkspaceCacheModule } from 'src/engine/workspace-cache/workspace-cache.module';
|
||||
import { WorkspaceMigrationRunnerModule } from 'src/engine/workspace-manager/workspace-migration/workspace-migration-runner/workspace-migration-runner.module';
|
||||
import { WorkspaceMigrationModule } from 'src/engine/workspace-manager/workspace-migration/workspace-migration.module';
|
||||
|
||||
@Module({
|
||||
imports: [
|
||||
TypeOrmModule.forFeature([ViewEntity]),
|
||||
ApplicationModule,
|
||||
WorkspaceCacheModule,
|
||||
WorkspaceMigrationModule,
|
||||
WorkspaceMigrationRunnerModule,
|
||||
WorkspaceIteratorModule,
|
||||
],
|
||||
providers: [
|
||||
ReconcileIndexViewUniversalIdentifierCommand,
|
||||
DemoteAndBackfillApplicationIndexViewCommand,
|
||||
],
|
||||
})
|
||||
export class V2_26_UpgradeVersionCommandModule {}
|
||||
+291
@@ -0,0 +1,291 @@
|
||||
import { InjectRepository } from '@nestjs/typeorm';
|
||||
|
||||
import { Command } from 'nest-commander';
|
||||
import {
|
||||
getSystemViewUniversalIdentifier,
|
||||
getViewFieldUniversalIdentifier,
|
||||
} from 'twenty-shared/application';
|
||||
import { ViewKey } from 'twenty-shared/types';
|
||||
import { isDefined } from 'twenty-shared/utils';
|
||||
import { Repository } from 'typeorm';
|
||||
|
||||
import { ProvisionedWorkspaceCommandRunner } from 'src/database/commands/command-runners/provisioned-workspace.command-runner';
|
||||
import { WorkspaceIteratorService } from 'src/database/commands/command-runners/workspace-iterator.service';
|
||||
import { type RunOnWorkspaceArgs } from 'src/database/commands/command-runners/workspace.command-runner';
|
||||
import { invalidateIndexViewReconcileCache } from 'src/database/commands/upgrade-version-command/2-26/utils/invalidate-index-view-reconcile-cache.util';
|
||||
import { ApplicationService } from 'src/engine/core-modules/application/application.service';
|
||||
import { RegisteredWorkspaceCommand } from 'src/engine/core-modules/upgrade/decorators/registered-workspace-command.decorator';
|
||||
import { type AllFlatEntityMaps } from 'src/engine/metadata-modules/flat-entity/types/all-flat-entity-maps.type';
|
||||
import { findManyFlatEntityByUniversalIdentifierInUniversalFlatEntityMaps } from 'src/engine/metadata-modules/flat-entity/utils/find-many-flat-entity-by-universal-identifier-in-universal-flat-entity-maps.util';
|
||||
import { ViewFieldEntity } from 'src/engine/metadata-modules/view-field/entities/view-field.entity';
|
||||
import { ViewEntity } from 'src/engine/metadata-modules/view/entities/view.entity';
|
||||
import { WorkspaceCacheService } from 'src/engine/workspace-cache/services/workspace-cache.service';
|
||||
import { WorkspaceMigrationRunnerService } from 'src/engine/workspace-manager/workspace-migration/workspace-migration-runner/services/workspace-migration-runner.service';
|
||||
|
||||
type ReownUpdate = {
|
||||
id: string;
|
||||
update: {
|
||||
universalIdentifier?: string;
|
||||
isSystemSideEffect?: boolean;
|
||||
};
|
||||
};
|
||||
|
||||
@RegisteredWorkspaceCommand('2.26.0', 1785255689000)
|
||||
@Command({
|
||||
name: 'upgrade:2-26:reconcile-index-view-universal-identifier',
|
||||
description:
|
||||
'Re-own the INDEX table views ("All {objectLabelPlural}", keyed on ViewKey.INDEX) of the twenty-standard and workspace-custom applications, and all their view fields, onto the engine convention: the view gets the name-free deterministic universal identifier (getSystemViewUniversalIdentifier, object identifier + INDEX key), each view field gets the derived getViewFieldUniversalIdentifier keyed on the application of the field it DISPLAYS — not the row attribution, which diverges when a user shows a hidden standard column and mints a workspace-custom view field on a standard field — so an app or user column on a standard INDEX view converges too, and both get isSystemSideEffect: true, as if provisioned by the metadata side-effect engine. INDEX views of other applications are handled by the demote-and-backfill command. Children reference the view by primary key, so the re-own is a lossless update.',
|
||||
})
|
||||
export class ReconcileIndexViewUniversalIdentifierCommand extends ProvisionedWorkspaceCommandRunner {
|
||||
constructor(
|
||||
protected readonly workspaceIteratorService: WorkspaceIteratorService,
|
||||
private readonly workspaceCacheService: WorkspaceCacheService,
|
||||
private readonly workspaceMigrationRunnerService: WorkspaceMigrationRunnerService,
|
||||
private readonly applicationService: ApplicationService,
|
||||
// eslint-disable-next-line twenty/prefer-workspace-scoped-repository
|
||||
@InjectRepository(ViewEntity)
|
||||
private readonly viewRepository: Repository<ViewEntity>,
|
||||
) {
|
||||
super(workspaceIteratorService);
|
||||
}
|
||||
|
||||
override async runOnWorkspace({
|
||||
workspaceId,
|
||||
options,
|
||||
}: RunOnWorkspaceArgs): Promise<void> {
|
||||
const isDryRun = options.dryRun ?? false;
|
||||
|
||||
const {
|
||||
flatViewMaps,
|
||||
flatViewFieldMaps,
|
||||
flatObjectMetadataMaps,
|
||||
flatFieldMetadataMaps,
|
||||
} = await this.workspaceCacheService.getOrRecompute(workspaceId, [
|
||||
'flatViewMaps',
|
||||
'flatViewFieldMaps',
|
||||
'flatObjectMetadataMaps',
|
||||
'flatFieldMetadataMaps',
|
||||
]);
|
||||
|
||||
const { twentyStandardFlatApplication, workspaceCustomFlatApplication } =
|
||||
await this.applicationService.findWorkspaceTwentyStandardAndCustomApplicationOrThrow(
|
||||
{ workspaceId },
|
||||
);
|
||||
const engineOwnedApplicationUniversalIdentifiers = new Set([
|
||||
twentyStandardFlatApplication.universalIdentifier,
|
||||
workspaceCustomFlatApplication.universalIdentifier,
|
||||
]);
|
||||
|
||||
const flatIndexViews = Object.values(flatViewMaps.byUniversalIdentifier)
|
||||
.filter(isDefined)
|
||||
.filter(
|
||||
(flatView) =>
|
||||
flatView.key === ViewKey.INDEX &&
|
||||
!isDefined(flatView.deletedAt) &&
|
||||
engineOwnedApplicationUniversalIdentifiers.has(
|
||||
flatView.applicationUniversalIdentifier,
|
||||
),
|
||||
);
|
||||
|
||||
const { viewUpdates, viewFieldUpdates } = this.computeReownUpdates({
|
||||
workspaceId,
|
||||
flatIndexViews,
|
||||
flatViewFieldMaps,
|
||||
flatObjectMetadataMaps,
|
||||
flatFieldMetadataMaps,
|
||||
});
|
||||
|
||||
if (viewUpdates.length === 0 && viewFieldUpdates.length === 0) {
|
||||
this.logger.log(
|
||||
`No INDEX view to reconcile for workspace ${workspaceId}`,
|
||||
);
|
||||
|
||||
return;
|
||||
}
|
||||
|
||||
this.logger.log(
|
||||
`${isDryRun ? '[DRY RUN] ' : ''}Reconciling ${viewUpdates.length} INDEX view(s) and ${viewFieldUpdates.length} view field(s) for workspace ${workspaceId}`,
|
||||
);
|
||||
|
||||
if (isDryRun) {
|
||||
return;
|
||||
}
|
||||
|
||||
await this.viewRepository.manager.transaction(async (entityManager) => {
|
||||
const transactionalViewRepository =
|
||||
entityManager.getRepository(ViewEntity);
|
||||
const transactionalViewFieldRepository =
|
||||
entityManager.getRepository(ViewFieldEntity);
|
||||
|
||||
for (const { id, update } of viewUpdates) {
|
||||
await transactionalViewRepository.update({ id, workspaceId }, update);
|
||||
}
|
||||
|
||||
for (const { id, update } of viewFieldUpdates) {
|
||||
await transactionalViewFieldRepository.update(
|
||||
{ id, workspaceId },
|
||||
update,
|
||||
);
|
||||
}
|
||||
});
|
||||
|
||||
await invalidateIndexViewReconcileCache({
|
||||
workspaceId,
|
||||
workspaceMigrationRunnerService: this.workspaceMigrationRunnerService,
|
||||
});
|
||||
|
||||
this.logger.log(
|
||||
`Reconciled ${viewUpdates.length} INDEX view(s) and ${viewFieldUpdates.length} view field(s) for workspace ${workspaceId}`,
|
||||
);
|
||||
}
|
||||
|
||||
private computeReownUpdates({
|
||||
workspaceId,
|
||||
flatIndexViews,
|
||||
flatViewFieldMaps,
|
||||
flatObjectMetadataMaps,
|
||||
flatFieldMetadataMaps,
|
||||
}: {
|
||||
workspaceId: string;
|
||||
flatIndexViews: NonNullable<
|
||||
AllFlatEntityMaps['flatViewMaps']['byUniversalIdentifier'][string]
|
||||
>[];
|
||||
flatViewFieldMaps: AllFlatEntityMaps['flatViewFieldMaps'];
|
||||
flatObjectMetadataMaps: AllFlatEntityMaps['flatObjectMetadataMaps'];
|
||||
flatFieldMetadataMaps: AllFlatEntityMaps['flatFieldMetadataMaps'];
|
||||
}): { viewUpdates: ReownUpdate[]; viewFieldUpdates: ReownUpdate[] } {
|
||||
const viewUpdates: ReownUpdate[] = [];
|
||||
const viewFieldUpdates: ReownUpdate[] = [];
|
||||
|
||||
for (const flatView of flatIndexViews) {
|
||||
const flatObjectMetadata =
|
||||
flatObjectMetadataMaps.byUniversalIdentifier[
|
||||
flatView.objectMetadataUniversalIdentifier
|
||||
];
|
||||
|
||||
if (!isDefined(flatObjectMetadata)) {
|
||||
this.logger.warn(
|
||||
`Missing object for INDEX view ${flatView.id} in workspace ${workspaceId}, skipping`,
|
||||
);
|
||||
continue;
|
||||
}
|
||||
|
||||
const derivedViewUniversalIdentifier = getSystemViewUniversalIdentifier({
|
||||
applicationUniversalIdentifier:
|
||||
flatObjectMetadata.applicationUniversalIdentifier,
|
||||
objectUniversalIdentifier: flatObjectMetadata.universalIdentifier,
|
||||
viewKey: ViewKey.INDEX,
|
||||
});
|
||||
|
||||
const viewUpdate = this.computeViewReownUpdate({
|
||||
flatView,
|
||||
derivedViewUniversalIdentifier,
|
||||
});
|
||||
|
||||
if (isDefined(viewUpdate)) {
|
||||
viewUpdates.push(viewUpdate);
|
||||
}
|
||||
|
||||
viewFieldUpdates.push(
|
||||
...findManyFlatEntityByUniversalIdentifierInUniversalFlatEntityMaps({
|
||||
flatEntityMaps: flatViewFieldMaps,
|
||||
universalIdentifiers: flatView.viewFieldUniversalIdentifiers,
|
||||
})
|
||||
.map((flatViewField) =>
|
||||
this.computeViewFieldReownUpdate({
|
||||
workspaceId,
|
||||
flatViewField,
|
||||
derivedViewUniversalIdentifier,
|
||||
flatFieldMetadataMaps,
|
||||
}),
|
||||
)
|
||||
.filter(isDefined),
|
||||
);
|
||||
}
|
||||
|
||||
return { viewUpdates, viewFieldUpdates };
|
||||
}
|
||||
|
||||
private computeViewReownUpdate({
|
||||
flatView,
|
||||
derivedViewUniversalIdentifier,
|
||||
}: {
|
||||
flatView: NonNullable<
|
||||
AllFlatEntityMaps['flatViewMaps']['byUniversalIdentifier'][string]
|
||||
>;
|
||||
derivedViewUniversalIdentifier: string;
|
||||
}): ReownUpdate | undefined {
|
||||
const update: ReownUpdate['update'] = {};
|
||||
|
||||
if (flatView.universalIdentifier !== derivedViewUniversalIdentifier) {
|
||||
update.universalIdentifier = derivedViewUniversalIdentifier;
|
||||
}
|
||||
if (!flatView.isSystemSideEffect) {
|
||||
update.isSystemSideEffect = true;
|
||||
}
|
||||
|
||||
if (Object.keys(update).length === 0) {
|
||||
return undefined;
|
||||
}
|
||||
|
||||
return { id: flatView.id, update };
|
||||
}
|
||||
|
||||
private computeViewFieldReownUpdate({
|
||||
workspaceId,
|
||||
flatViewField,
|
||||
derivedViewUniversalIdentifier,
|
||||
flatFieldMetadataMaps,
|
||||
}: {
|
||||
workspaceId: string;
|
||||
flatViewField: NonNullable<
|
||||
AllFlatEntityMaps['flatViewFieldMaps']['byUniversalIdentifier'][string]
|
||||
>;
|
||||
derivedViewUniversalIdentifier: string;
|
||||
flatFieldMetadataMaps: AllFlatEntityMaps['flatFieldMetadataMaps'];
|
||||
}): ReownUpdate | undefined {
|
||||
if (isDefined(flatViewField.deletedAt)) {
|
||||
return undefined;
|
||||
}
|
||||
|
||||
const flatFieldMetadata =
|
||||
flatFieldMetadataMaps.byUniversalIdentifier[
|
||||
flatViewField.fieldMetadataUniversalIdentifier
|
||||
];
|
||||
|
||||
if (!isDefined(flatFieldMetadata)) {
|
||||
this.logger.warn(
|
||||
`Missing field for INDEX view field ${flatViewField.id} in workspace ${workspaceId}, skipping`,
|
||||
);
|
||||
|
||||
return undefined;
|
||||
}
|
||||
|
||||
const derivedViewFieldUniversalIdentifier = getViewFieldUniversalIdentifier(
|
||||
{
|
||||
applicationUniversalIdentifier:
|
||||
flatFieldMetadata.applicationUniversalIdentifier,
|
||||
viewUniversalIdentifier: derivedViewUniversalIdentifier,
|
||||
fieldMetadataUniversalIdentifier:
|
||||
flatViewField.fieldMetadataUniversalIdentifier,
|
||||
},
|
||||
);
|
||||
|
||||
const update: ReownUpdate['update'] = {};
|
||||
|
||||
if (
|
||||
flatViewField.universalIdentifier !== derivedViewFieldUniversalIdentifier
|
||||
) {
|
||||
update.universalIdentifier = derivedViewFieldUniversalIdentifier;
|
||||
}
|
||||
if (!flatViewField.isSystemSideEffect) {
|
||||
update.isSystemSideEffect = true;
|
||||
}
|
||||
|
||||
if (Object.keys(update).length === 0) {
|
||||
return undefined;
|
||||
}
|
||||
|
||||
return { id: flatViewField.id, update };
|
||||
}
|
||||
}
|
||||
+411
@@ -0,0 +1,411 @@
|
||||
import { InjectRepository } from '@nestjs/typeorm';
|
||||
|
||||
import { Command } from 'nest-commander';
|
||||
import {
|
||||
getSystemViewUniversalIdentifier,
|
||||
getViewFieldUniversalIdentifier,
|
||||
} from 'twenty-shared/application';
|
||||
import { ViewKey } from 'twenty-shared/types';
|
||||
import { isDefined } from 'twenty-shared/utils';
|
||||
import { In, Repository } from 'typeorm';
|
||||
|
||||
import { ProvisionedWorkspaceCommandRunner } from 'src/database/commands/command-runners/provisioned-workspace.command-runner';
|
||||
import { WorkspaceIteratorService } from 'src/database/commands/command-runners/workspace-iterator.service';
|
||||
import { type RunOnWorkspaceArgs } from 'src/database/commands/command-runners/workspace.command-runner';
|
||||
import { invalidateIndexViewReconcileCache } from 'src/database/commands/upgrade-version-command/2-26/utils/invalidate-index-view-reconcile-cache.util';
|
||||
import { ApplicationService } from 'src/engine/core-modules/application/application.service';
|
||||
import { RegisteredWorkspaceCommand } from 'src/engine/core-modules/upgrade/decorators/registered-workspace-command.decorator';
|
||||
import { type AllFlatEntityMaps } from 'src/engine/metadata-modules/flat-entity/types/all-flat-entity-maps.type';
|
||||
import { DEFAULT_VIEW_FIELD_SIZE } from 'src/engine/metadata-modules/flat-view-field/constants/default-view-field-size.constant';
|
||||
import { computeFlatIndexViewToCreate } from 'src/engine/metadata-modules/object-metadata/utils/compute-flat-index-view-to-create.util';
|
||||
import { isFlatFieldMetadataDisplayableInDefaultView } from 'src/engine/metadata-modules/object-metadata/utils/is-flat-field-metadata-displayable-in-default-view.util';
|
||||
import { orderFlatFieldMetadatasForSystemIndexView } from 'src/engine/metadata-modules/object-metadata/utils/order-flat-field-metadatas-for-system-index-view.util';
|
||||
import { ViewEntity } from 'src/engine/metadata-modules/view/entities/view.entity';
|
||||
import { WorkspaceCacheService } from 'src/engine/workspace-cache/services/workspace-cache.service';
|
||||
import { WorkspaceMigrationValidateBuildAndRunService } from 'src/engine/workspace-manager/workspace-migration/services/workspace-migration-validate-build-and-run-service';
|
||||
import { type UniversalFlatViewField } from 'src/engine/workspace-manager/workspace-migration/universal-flat-entity/types/universal-flat-view-field.type';
|
||||
import { type UniversalFlatView } from 'src/engine/workspace-manager/workspace-migration/universal-flat-entity/types/universal-flat-view.type';
|
||||
import { WorkspaceMigrationRunnerService } from 'src/engine/workspace-manager/workspace-migration/workspace-migration-runner/services/workspace-migration-runner.service';
|
||||
|
||||
type BackfillOperationsByApplication = Map<
|
||||
string,
|
||||
{
|
||||
viewsToCreate: UniversalFlatView[];
|
||||
viewFieldsToCreate: UniversalFlatViewField[];
|
||||
}
|
||||
>;
|
||||
|
||||
@RegisteredWorkspaceCommand('2.26.0', 1785255690000)
|
||||
@Command({
|
||||
name: 'upgrade:2-26:demote-and-backfill-application-index-view',
|
||||
description:
|
||||
'Manifest-installed applications never had their INDEX view auto-provisioned: the ones that declared a view with ViewKey.INDEX authored it themselves under a manifest identifier. The engine is now the sole owner of the INDEX key, so every caller-authored (isSystemSideEffect: false) INDEX view owned by an application other than twenty-standard and workspace-custom is demoted to key: null — it becomes a plain additional view under its manifest identifier, and the flat view validator rejects caller-provided INDEX keys so a later app sync cannot promote it back. Every application object left without an engine-owned INDEX view then gets one backfilled through the workspace migration pipeline with its full view-field layout, converging upgraded installs with fresh installs. Views are committed before view fields across all applications because a view field belongs to the application owning its field, which can differ from the application owning the object (fields can be created on another application object). The backfill runs through the legacy pipeline path (no side-effect expansion: it replays a state the engine convention already defines) and is idempotent and retry-safe: engine-owned INDEX views are neither demoted nor re-backfilled, and view creation and view-field creation are gated independently, so a retry after a partial failure still backfills the missing view fields of an already-committed view.',
|
||||
})
|
||||
export class DemoteAndBackfillApplicationIndexViewCommand extends ProvisionedWorkspaceCommandRunner {
|
||||
constructor(
|
||||
protected readonly workspaceIteratorService: WorkspaceIteratorService,
|
||||
private readonly workspaceCacheService: WorkspaceCacheService,
|
||||
private readonly workspaceMigrationRunnerService: WorkspaceMigrationRunnerService,
|
||||
private readonly applicationService: ApplicationService,
|
||||
private readonly workspaceMigrationValidateBuildAndRunService: WorkspaceMigrationValidateBuildAndRunService,
|
||||
// eslint-disable-next-line twenty/prefer-workspace-scoped-repository
|
||||
@InjectRepository(ViewEntity)
|
||||
private readonly viewRepository: Repository<ViewEntity>,
|
||||
) {
|
||||
super(workspaceIteratorService);
|
||||
}
|
||||
|
||||
override async runOnWorkspace({
|
||||
workspaceId,
|
||||
options,
|
||||
}: RunOnWorkspaceArgs): Promise<void> {
|
||||
const isDryRun = options.dryRun ?? false;
|
||||
|
||||
const {
|
||||
flatViewMaps,
|
||||
flatViewFieldMaps,
|
||||
flatObjectMetadataMaps,
|
||||
flatFieldMetadataMaps,
|
||||
} = await this.workspaceCacheService.getOrRecompute(workspaceId, [
|
||||
'flatViewMaps',
|
||||
'flatViewFieldMaps',
|
||||
'flatObjectMetadataMaps',
|
||||
'flatFieldMetadataMaps',
|
||||
]);
|
||||
|
||||
const { twentyStandardFlatApplication, workspaceCustomFlatApplication } =
|
||||
await this.applicationService.findWorkspaceTwentyStandardAndCustomApplicationOrThrow(
|
||||
{ workspaceId },
|
||||
);
|
||||
const engineOwnedApplicationUniversalIdentifiers = new Set([
|
||||
twentyStandardFlatApplication.universalIdentifier,
|
||||
workspaceCustomFlatApplication.universalIdentifier,
|
||||
]);
|
||||
|
||||
const viewIdsToDemote: string[] = [];
|
||||
|
||||
const objectUniversalIdentifiersWithEngineIndexView = new Set<string>();
|
||||
|
||||
for (const flatView of Object.values(flatViewMaps.byUniversalIdentifier)) {
|
||||
if (
|
||||
!isDefined(flatView) ||
|
||||
flatView.key !== ViewKey.INDEX ||
|
||||
isDefined(flatView.deletedAt) ||
|
||||
engineOwnedApplicationUniversalIdentifiers.has(
|
||||
flatView.applicationUniversalIdentifier,
|
||||
)
|
||||
) {
|
||||
continue;
|
||||
}
|
||||
|
||||
if (flatView.isSystemSideEffect === true) {
|
||||
objectUniversalIdentifiersWithEngineIndexView.add(
|
||||
flatView.objectMetadataUniversalIdentifier,
|
||||
);
|
||||
continue;
|
||||
}
|
||||
|
||||
viewIdsToDemote.push(flatView.id);
|
||||
}
|
||||
|
||||
const backfillOperationsByApplication =
|
||||
this.computeBackfillOperationsByApplication({
|
||||
flatViewFieldMaps,
|
||||
flatObjectMetadataMaps,
|
||||
flatFieldMetadataMaps,
|
||||
engineOwnedApplicationUniversalIdentifiers,
|
||||
objectUniversalIdentifiersWithEngineIndexView,
|
||||
});
|
||||
|
||||
const { backfillViewCount, backfillViewFieldCount } = [
|
||||
...backfillOperationsByApplication.values(),
|
||||
].reduce(
|
||||
(counts, { viewsToCreate, viewFieldsToCreate }) => ({
|
||||
backfillViewCount: counts.backfillViewCount + viewsToCreate.length,
|
||||
backfillViewFieldCount:
|
||||
counts.backfillViewFieldCount + viewFieldsToCreate.length,
|
||||
}),
|
||||
{ backfillViewCount: 0, backfillViewFieldCount: 0 },
|
||||
);
|
||||
|
||||
if (
|
||||
viewIdsToDemote.length === 0 &&
|
||||
backfillViewCount === 0 &&
|
||||
backfillViewFieldCount === 0
|
||||
) {
|
||||
this.logger.log(
|
||||
`No application INDEX view to demote or backfill for workspace ${workspaceId}`,
|
||||
);
|
||||
|
||||
return;
|
||||
}
|
||||
|
||||
this.logger.log(
|
||||
`${isDryRun ? '[DRY RUN] ' : ''}Demoting ${viewIdsToDemote.length} application INDEX view(s) and backfilling ${backfillViewCount} engine INDEX view(s) with ${backfillViewFieldCount} view field(s) for workspace ${workspaceId}`,
|
||||
);
|
||||
|
||||
if (isDryRun) {
|
||||
return;
|
||||
}
|
||||
|
||||
if (viewIdsToDemote.length > 0) {
|
||||
// Single bulk statement, atomic on its own: every row gets the same
|
||||
// patch.
|
||||
await this.viewRepository.update(
|
||||
{ id: In(viewIdsToDemote), workspaceId },
|
||||
{ key: null },
|
||||
);
|
||||
|
||||
await invalidateIndexViewReconcileCache({
|
||||
workspaceId,
|
||||
workspaceMigrationRunnerService: this.workspaceMigrationRunnerService,
|
||||
});
|
||||
}
|
||||
|
||||
await this.runBackfillMigrations({
|
||||
workspaceId,
|
||||
backfillOperationsByApplication,
|
||||
});
|
||||
|
||||
this.logger.log(
|
||||
`Demoted ${viewIdsToDemote.length} application INDEX view(s) and backfilled ${backfillViewCount} engine INDEX view(s) with ${backfillViewFieldCount} view field(s) for workspace ${workspaceId}`,
|
||||
);
|
||||
}
|
||||
|
||||
private computeBackfillOperationsByApplication({
|
||||
flatViewFieldMaps,
|
||||
flatObjectMetadataMaps,
|
||||
flatFieldMetadataMaps,
|
||||
engineOwnedApplicationUniversalIdentifiers,
|
||||
objectUniversalIdentifiersWithEngineIndexView,
|
||||
}: {
|
||||
flatViewFieldMaps: AllFlatEntityMaps['flatViewFieldMaps'];
|
||||
flatObjectMetadataMaps: AllFlatEntityMaps['flatObjectMetadataMaps'];
|
||||
flatFieldMetadataMaps: AllFlatEntityMaps['flatFieldMetadataMaps'];
|
||||
engineOwnedApplicationUniversalIdentifiers: Set<string>;
|
||||
objectUniversalIdentifiersWithEngineIndexView: Set<string>;
|
||||
}): BackfillOperationsByApplication {
|
||||
const backfillOperationsByApplication: BackfillOperationsByApplication =
|
||||
new Map();
|
||||
|
||||
const getApplicationBucket = (applicationUniversalIdentifier: string) => {
|
||||
const existingBucket = backfillOperationsByApplication.get(
|
||||
applicationUniversalIdentifier,
|
||||
);
|
||||
|
||||
if (isDefined(existingBucket)) {
|
||||
return existingBucket;
|
||||
}
|
||||
|
||||
const newBucket = { viewsToCreate: [], viewFieldsToCreate: [] };
|
||||
|
||||
backfillOperationsByApplication.set(
|
||||
applicationUniversalIdentifier,
|
||||
newBucket,
|
||||
);
|
||||
|
||||
return newBucket;
|
||||
};
|
||||
|
||||
for (const flatObjectMetadata of Object.values(
|
||||
flatObjectMetadataMaps.byUniversalIdentifier,
|
||||
)) {
|
||||
if (
|
||||
!isDefined(flatObjectMetadata) ||
|
||||
engineOwnedApplicationUniversalIdentifiers.has(
|
||||
flatObjectMetadata.applicationUniversalIdentifier,
|
||||
)
|
||||
) {
|
||||
continue;
|
||||
}
|
||||
|
||||
const indexViewUniversalIdentifier = getSystemViewUniversalIdentifier({
|
||||
applicationUniversalIdentifier:
|
||||
flatObjectMetadata.applicationUniversalIdentifier,
|
||||
objectUniversalIdentifier: flatObjectMetadata.universalIdentifier,
|
||||
viewKey: ViewKey.INDEX,
|
||||
});
|
||||
|
||||
if (
|
||||
!objectUniversalIdentifiersWithEngineIndexView.has(
|
||||
flatObjectMetadata.universalIdentifier,
|
||||
)
|
||||
) {
|
||||
// computeFlatIndexViewToCreate derives the same universal identifier.
|
||||
getApplicationBucket(
|
||||
flatObjectMetadata.applicationUniversalIdentifier,
|
||||
).viewsToCreate.push(
|
||||
computeFlatIndexViewToCreate({
|
||||
objectMetadata: flatObjectMetadata,
|
||||
applicationUniversalIdentifier:
|
||||
flatObjectMetadata.applicationUniversalIdentifier,
|
||||
}),
|
||||
);
|
||||
}
|
||||
|
||||
const { labelIdentifierFieldMetadataUniversalIdentifier } =
|
||||
flatObjectMetadata;
|
||||
|
||||
const displayableFlatFieldMetadatas =
|
||||
flatObjectMetadata.fieldUniversalIdentifiers
|
||||
.map(
|
||||
(fieldUniversalIdentifier) =>
|
||||
flatFieldMetadataMaps.byUniversalIdentifier[
|
||||
fieldUniversalIdentifier
|
||||
],
|
||||
)
|
||||
.filter(isDefined)
|
||||
.filter((flatFieldMetadata) =>
|
||||
isFlatFieldMetadataDisplayableInDefaultView({
|
||||
flatFieldMetadata,
|
||||
labelIdentifierFieldMetadataUniversalIdentifier,
|
||||
}),
|
||||
);
|
||||
|
||||
// The label identifier view field must be strictly lowest and visible.
|
||||
const orderedDisplayableFlatFieldMetadatas =
|
||||
orderFlatFieldMetadatasForSystemIndexView({
|
||||
labelIdentifierFieldMetadataUniversalIdentifier,
|
||||
flatFieldMetadatas: displayableFlatFieldMetadatas,
|
||||
});
|
||||
|
||||
const createdAt = new Date().toISOString();
|
||||
|
||||
orderedDisplayableFlatFieldMetadatas.forEach(
|
||||
(flatFieldMetadata, position) => {
|
||||
const fieldApplicationUniversalIdentifier =
|
||||
flatFieldMetadata.applicationUniversalIdentifier;
|
||||
|
||||
const viewFieldUniversalIdentifier = getViewFieldUniversalIdentifier({
|
||||
applicationUniversalIdentifier: fieldApplicationUniversalIdentifier,
|
||||
viewUniversalIdentifier: indexViewUniversalIdentifier,
|
||||
fieldMetadataUniversalIdentifier:
|
||||
flatFieldMetadata.universalIdentifier,
|
||||
});
|
||||
|
||||
// Already backfilled by a previous (partially failed) run.
|
||||
if (
|
||||
isDefined(
|
||||
flatViewFieldMaps.byUniversalIdentifier[
|
||||
viewFieldUniversalIdentifier
|
||||
],
|
||||
)
|
||||
) {
|
||||
return;
|
||||
}
|
||||
|
||||
getApplicationBucket(
|
||||
fieldApplicationUniversalIdentifier,
|
||||
).viewFieldsToCreate.push({
|
||||
universalIdentifier: viewFieldUniversalIdentifier,
|
||||
applicationUniversalIdentifier: fieldApplicationUniversalIdentifier,
|
||||
fieldMetadataUniversalIdentifier:
|
||||
flatFieldMetadata.universalIdentifier,
|
||||
viewUniversalIdentifier: indexViewUniversalIdentifier,
|
||||
viewFieldGroupUniversalIdentifier: null,
|
||||
isVisible: true,
|
||||
size: DEFAULT_VIEW_FIELD_SIZE,
|
||||
position,
|
||||
aggregateOperation: null,
|
||||
isActive: true,
|
||||
isSystemSideEffect: true,
|
||||
universalOverrides: null,
|
||||
createdAt,
|
||||
updatedAt: createdAt,
|
||||
deletedAt: null,
|
||||
});
|
||||
},
|
||||
);
|
||||
}
|
||||
|
||||
return backfillOperationsByApplication;
|
||||
}
|
||||
|
||||
private async runBackfillMigrations({
|
||||
workspaceId,
|
||||
backfillOperationsByApplication,
|
||||
}: {
|
||||
workspaceId: string;
|
||||
backfillOperationsByApplication: BackfillOperationsByApplication;
|
||||
}): Promise<void> {
|
||||
for (const [
|
||||
applicationUniversalIdentifier,
|
||||
{ viewsToCreate },
|
||||
] of backfillOperationsByApplication.entries()) {
|
||||
if (viewsToCreate.length === 0) {
|
||||
continue;
|
||||
}
|
||||
|
||||
await this.runBackfillMigration({
|
||||
workspaceId,
|
||||
applicationUniversalIdentifier,
|
||||
allFlatEntityOperationByMetadataName: {
|
||||
view: {
|
||||
flatEntityToCreate: viewsToCreate,
|
||||
flatEntityToDelete: [],
|
||||
flatEntityToUpdate: [],
|
||||
},
|
||||
},
|
||||
});
|
||||
}
|
||||
|
||||
for (const [
|
||||
applicationUniversalIdentifier,
|
||||
{ viewFieldsToCreate },
|
||||
] of backfillOperationsByApplication.entries()) {
|
||||
if (viewFieldsToCreate.length === 0) {
|
||||
continue;
|
||||
}
|
||||
|
||||
await this.runBackfillMigration({
|
||||
workspaceId,
|
||||
applicationUniversalIdentifier,
|
||||
allFlatEntityOperationByMetadataName: {
|
||||
viewField: {
|
||||
flatEntityToCreate: viewFieldsToCreate,
|
||||
flatEntityToDelete: [],
|
||||
flatEntityToUpdate: [],
|
||||
},
|
||||
},
|
||||
});
|
||||
}
|
||||
}
|
||||
|
||||
private async runBackfillMigration({
|
||||
workspaceId,
|
||||
applicationUniversalIdentifier,
|
||||
allFlatEntityOperationByMetadataName,
|
||||
}: {
|
||||
workspaceId: string;
|
||||
applicationUniversalIdentifier: string;
|
||||
allFlatEntityOperationByMetadataName: Record<
|
||||
string,
|
||||
{
|
||||
flatEntityToCreate: unknown[];
|
||||
flatEntityToDelete: never[];
|
||||
flatEntityToUpdate: never[];
|
||||
}
|
||||
>;
|
||||
}): Promise<void> {
|
||||
const result =
|
||||
await this.workspaceMigrationValidateBuildAndRunService.validateBuildAndRunLegacyWorkspaceMigration(
|
||||
{
|
||||
isSystemBuild: true,
|
||||
workspaceId,
|
||||
applicationUniversalIdentifier,
|
||||
allFlatEntityOperationByMetadataName:
|
||||
allFlatEntityOperationByMetadataName as never,
|
||||
},
|
||||
);
|
||||
|
||||
if (result.status === 'fail') {
|
||||
this.logger.error(
|
||||
`Failed to backfill engine INDEX view(s) for application ${applicationUniversalIdentifier} in workspace ${workspaceId}:\n${JSON.stringify(result, null, 2)}`,
|
||||
);
|
||||
|
||||
throw new Error(
|
||||
`Failed to backfill engine INDEX view(s) for workspace ${workspaceId}`,
|
||||
);
|
||||
}
|
||||
}
|
||||
}
|
||||
+490
@@ -0,0 +1,490 @@
|
||||
import {
|
||||
getSystemViewUniversalIdentifier,
|
||||
getViewFieldUniversalIdentifier,
|
||||
} from 'twenty-shared/application';
|
||||
import { ViewKey } from 'twenty-shared/types';
|
||||
|
||||
import { type WorkspaceIteratorService } from 'src/database/commands/command-runners/workspace-iterator.service';
|
||||
import { ReconcileIndexViewUniversalIdentifierCommand } from 'src/database/commands/upgrade-version-command/2-26/2-26-workspace-command-1785255689000-reconcile-index-view-universal-identifier.command';
|
||||
import { type ApplicationService } from 'src/engine/core-modules/application/application.service';
|
||||
import { ViewFieldEntity } from 'src/engine/metadata-modules/view-field/entities/view-field.entity';
|
||||
import { ViewEntity } from 'src/engine/metadata-modules/view/entities/view.entity';
|
||||
import { type WorkspaceCacheService } from 'src/engine/workspace-cache/services/workspace-cache.service';
|
||||
import { type WorkspaceMigrationRunnerService } from 'src/engine/workspace-manager/workspace-migration/workspace-migration-runner/services/workspace-migration-runner.service';
|
||||
|
||||
const WORKSPACE_ID = '20202020-0000-4000-8000-000000000001';
|
||||
const STANDARD_APPLICATION_UNIVERSAL_IDENTIFIER =
|
||||
'20202020-0000-4000-8000-0000000000ad';
|
||||
const CUSTOM_APPLICATION_UNIVERSAL_IDENTIFIER =
|
||||
'20202020-0000-4000-8000-0000000000aa';
|
||||
const EXTERNAL_APPLICATION_UNIVERSAL_IDENTIFIER =
|
||||
'20202020-0000-4000-8000-0000000000ae';
|
||||
const STANDARD_OBJECT_UNIVERSAL_IDENTIFIER =
|
||||
'20202020-0000-4000-8000-0000000000bb';
|
||||
const CUSTOM_OBJECT_UNIVERSAL_IDENTIFIER =
|
||||
'20202020-0000-4000-8000-0000000000bc';
|
||||
const FIELD_UNIVERSAL_IDENTIFIER = '20202020-0000-4000-8000-0000000000cc';
|
||||
|
||||
const DERIVED_STANDARD_VIEW_UNIVERSAL_IDENTIFIER =
|
||||
getSystemViewUniversalIdentifier({
|
||||
applicationUniversalIdentifier: STANDARD_APPLICATION_UNIVERSAL_IDENTIFIER,
|
||||
objectUniversalIdentifier: STANDARD_OBJECT_UNIVERSAL_IDENTIFIER,
|
||||
viewKey: ViewKey.INDEX,
|
||||
});
|
||||
const DERIVED_STANDARD_VIEW_FIELD_UNIVERSAL_IDENTIFIER =
|
||||
getViewFieldUniversalIdentifier({
|
||||
applicationUniversalIdentifier: STANDARD_APPLICATION_UNIVERSAL_IDENTIFIER,
|
||||
viewUniversalIdentifier: DERIVED_STANDARD_VIEW_UNIVERSAL_IDENTIFIER,
|
||||
fieldMetadataUniversalIdentifier: FIELD_UNIVERSAL_IDENTIFIER,
|
||||
});
|
||||
|
||||
const STANDARD_OBJECT_METADATA = {
|
||||
universalIdentifier: STANDARD_OBJECT_UNIVERSAL_IDENTIFIER,
|
||||
applicationUniversalIdentifier: STANDARD_APPLICATION_UNIVERSAL_IDENTIFIER,
|
||||
};
|
||||
|
||||
// A custom object: its INDEX view was historically created with a random v4
|
||||
// universal identifier by ObjectMetadataService.createOneObject.
|
||||
const CUSTOM_OBJECT_METADATA = {
|
||||
universalIdentifier: CUSTOM_OBJECT_UNIVERSAL_IDENTIFIER,
|
||||
applicationUniversalIdentifier: CUSTOM_APPLICATION_UNIVERSAL_IDENTIFIER,
|
||||
};
|
||||
|
||||
type WorkspaceView = {
|
||||
id: string;
|
||||
universalIdentifier: string;
|
||||
key: ViewKey | null;
|
||||
applicationUniversalIdentifier?: string;
|
||||
deletedAt?: string | null;
|
||||
isSystemSideEffect?: boolean;
|
||||
objectMetadataUniversalIdentifier?: string;
|
||||
viewFieldUniversalIdentifiers?: string[];
|
||||
};
|
||||
|
||||
type WorkspaceViewField = {
|
||||
id: string;
|
||||
universalIdentifier: string;
|
||||
applicationUniversalIdentifier?: string;
|
||||
fieldMetadataUniversalIdentifier?: string;
|
||||
deletedAt?: string | null;
|
||||
isSystemSideEffect?: boolean;
|
||||
};
|
||||
|
||||
const buildFlatView = (view: WorkspaceView) => ({
|
||||
applicationUniversalIdentifier: STANDARD_APPLICATION_UNIVERSAL_IDENTIFIER,
|
||||
deletedAt: null,
|
||||
isSystemSideEffect: false,
|
||||
objectMetadataUniversalIdentifier: STANDARD_OBJECT_UNIVERSAL_IDENTIFIER,
|
||||
viewFieldUniversalIdentifiers: [],
|
||||
...view,
|
||||
});
|
||||
|
||||
const buildFlatViewField = (viewField: WorkspaceViewField) => ({
|
||||
applicationUniversalIdentifier: STANDARD_APPLICATION_UNIVERSAL_IDENTIFIER,
|
||||
fieldMetadataUniversalIdentifier: FIELD_UNIVERSAL_IDENTIFIER,
|
||||
deletedAt: null,
|
||||
isSystemSideEffect: false,
|
||||
...viewField,
|
||||
});
|
||||
|
||||
const buildFlatFieldMetadata = ({
|
||||
applicationUniversalIdentifier = STANDARD_APPLICATION_UNIVERSAL_IDENTIFIER,
|
||||
}: {
|
||||
applicationUniversalIdentifier?: string;
|
||||
} = {}) => ({
|
||||
universalIdentifier: FIELD_UNIVERSAL_IDENTIFIER,
|
||||
applicationUniversalIdentifier,
|
||||
});
|
||||
|
||||
const buildByUniversalIdentifierMap = <
|
||||
T extends { universalIdentifier: string },
|
||||
>(
|
||||
flatEntities: T[],
|
||||
) => ({
|
||||
byUniversalIdentifier: Object.fromEntries(
|
||||
flatEntities.map((flatEntity) => [
|
||||
flatEntity.universalIdentifier,
|
||||
flatEntity,
|
||||
]),
|
||||
),
|
||||
});
|
||||
|
||||
describe('ReconcileIndexViewUniversalIdentifierCommand', () => {
|
||||
let command: ReconcileIndexViewUniversalIdentifierCommand;
|
||||
let getOrRecomputeMock: jest.Mock;
|
||||
let invalidateCacheMock: jest.Mock;
|
||||
let viewUpdateMock: jest.Mock;
|
||||
let viewFieldUpdateMock: jest.Mock;
|
||||
|
||||
beforeEach(() => {
|
||||
jest.clearAllMocks();
|
||||
|
||||
getOrRecomputeMock = jest.fn();
|
||||
invalidateCacheMock = jest.fn().mockResolvedValue(undefined);
|
||||
viewUpdateMock = jest.fn().mockResolvedValue(undefined);
|
||||
viewFieldUpdateMock = jest.fn().mockResolvedValue(undefined);
|
||||
|
||||
const entityManagerMock = {
|
||||
getRepository: (entity: unknown) => {
|
||||
if (entity === ViewEntity) {
|
||||
return { update: viewUpdateMock };
|
||||
}
|
||||
if (entity === ViewFieldEntity) {
|
||||
return { update: viewFieldUpdateMock };
|
||||
}
|
||||
throw new Error('Unexpected repository');
|
||||
},
|
||||
};
|
||||
|
||||
const viewRepositoryMock = {
|
||||
manager: {
|
||||
transaction: jest.fn(
|
||||
async (callback: (entityManager: unknown) => Promise<void>) =>
|
||||
callback(entityManagerMock),
|
||||
),
|
||||
},
|
||||
};
|
||||
|
||||
command = new ReconcileIndexViewUniversalIdentifierCommand(
|
||||
{} as WorkspaceIteratorService,
|
||||
{
|
||||
getOrRecompute: getOrRecomputeMock,
|
||||
} as unknown as WorkspaceCacheService,
|
||||
{
|
||||
invalidateCache: invalidateCacheMock,
|
||||
} as unknown as WorkspaceMigrationRunnerService,
|
||||
{
|
||||
findWorkspaceTwentyStandardAndCustomApplicationOrThrow: jest
|
||||
.fn()
|
||||
.mockResolvedValue({
|
||||
twentyStandardFlatApplication: {
|
||||
universalIdentifier: STANDARD_APPLICATION_UNIVERSAL_IDENTIFIER,
|
||||
},
|
||||
workspaceCustomFlatApplication: {
|
||||
universalIdentifier: CUSTOM_APPLICATION_UNIVERSAL_IDENTIFIER,
|
||||
},
|
||||
}),
|
||||
} as unknown as ApplicationService,
|
||||
viewRepositoryMock as never,
|
||||
);
|
||||
});
|
||||
|
||||
const mockWorkspaceCache = ({
|
||||
views,
|
||||
viewFields = [],
|
||||
fieldMetadatas = [buildFlatFieldMetadata()],
|
||||
}: {
|
||||
views: ReturnType<typeof buildFlatView>[];
|
||||
viewFields?: ReturnType<typeof buildFlatViewField>[];
|
||||
fieldMetadatas?: ReturnType<typeof buildFlatFieldMetadata>[];
|
||||
}) => {
|
||||
getOrRecomputeMock.mockResolvedValue({
|
||||
flatViewMaps: buildByUniversalIdentifierMap(views),
|
||||
flatViewFieldMaps: buildByUniversalIdentifierMap(viewFields),
|
||||
flatObjectMetadataMaps: buildByUniversalIdentifierMap([
|
||||
STANDARD_OBJECT_METADATA,
|
||||
CUSTOM_OBJECT_METADATA,
|
||||
]),
|
||||
flatFieldMetadataMaps: buildByUniversalIdentifierMap(fieldMetadatas),
|
||||
});
|
||||
};
|
||||
|
||||
const runOnWorkspace = (dryRun = false) =>
|
||||
command.runOnWorkspace({
|
||||
workspaceId: WORKSPACE_ID,
|
||||
options: { dryRun },
|
||||
index: 0,
|
||||
total: 1,
|
||||
});
|
||||
|
||||
it('re-owns an underived standard INDEX view and its view fields', async () => {
|
||||
mockWorkspaceCache({
|
||||
views: [
|
||||
buildFlatView({
|
||||
id: 'view-id',
|
||||
universalIdentifier: 'legacy-view-uid',
|
||||
key: ViewKey.INDEX,
|
||||
viewFieldUniversalIdentifiers: ['legacy-view-field-uid'],
|
||||
}),
|
||||
],
|
||||
viewFields: [
|
||||
buildFlatViewField({
|
||||
id: 'view-field-id',
|
||||
universalIdentifier: 'legacy-view-field-uid',
|
||||
}),
|
||||
],
|
||||
});
|
||||
|
||||
await runOnWorkspace();
|
||||
|
||||
expect(viewUpdateMock).toHaveBeenCalledTimes(1);
|
||||
expect(viewUpdateMock).toHaveBeenCalledWith(
|
||||
{ id: 'view-id', workspaceId: WORKSPACE_ID },
|
||||
{
|
||||
universalIdentifier: DERIVED_STANDARD_VIEW_UNIVERSAL_IDENTIFIER,
|
||||
isSystemSideEffect: true,
|
||||
},
|
||||
);
|
||||
expect(viewFieldUpdateMock).toHaveBeenCalledTimes(1);
|
||||
expect(viewFieldUpdateMock).toHaveBeenCalledWith(
|
||||
{ id: 'view-field-id', workspaceId: WORKSPACE_ID },
|
||||
{
|
||||
universalIdentifier: DERIVED_STANDARD_VIEW_FIELD_UNIVERSAL_IDENTIFIER,
|
||||
isSystemSideEffect: true,
|
||||
},
|
||||
);
|
||||
expect(invalidateCacheMock).toHaveBeenCalledTimes(1);
|
||||
expect(invalidateCacheMock.mock.calls[0][0].allFlatEntityMapsKeys).toEqual(
|
||||
expect.arrayContaining([
|
||||
'flatViewMaps',
|
||||
'flatViewFieldMaps',
|
||||
'flatObjectMetadataMaps',
|
||||
'flatFieldMetadataMaps',
|
||||
'flatPageLayoutWidgetMaps',
|
||||
]),
|
||||
);
|
||||
});
|
||||
|
||||
it('re-owns a custom object INDEX view onto the workspace-custom derivation', async () => {
|
||||
const derivedCustomViewUniversalIdentifier =
|
||||
getSystemViewUniversalIdentifier({
|
||||
applicationUniversalIdentifier: CUSTOM_APPLICATION_UNIVERSAL_IDENTIFIER,
|
||||
objectUniversalIdentifier: CUSTOM_OBJECT_UNIVERSAL_IDENTIFIER,
|
||||
viewKey: ViewKey.INDEX,
|
||||
});
|
||||
|
||||
mockWorkspaceCache({
|
||||
views: [
|
||||
buildFlatView({
|
||||
id: 'custom-view-id',
|
||||
universalIdentifier: 'legacy-v4-view-uid',
|
||||
key: ViewKey.INDEX,
|
||||
applicationUniversalIdentifier:
|
||||
CUSTOM_APPLICATION_UNIVERSAL_IDENTIFIER,
|
||||
objectMetadataUniversalIdentifier: CUSTOM_OBJECT_UNIVERSAL_IDENTIFIER,
|
||||
isSystemSideEffect: true,
|
||||
}),
|
||||
],
|
||||
});
|
||||
|
||||
await runOnWorkspace();
|
||||
|
||||
expect(viewUpdateMock).toHaveBeenCalledTimes(1);
|
||||
expect(viewUpdateMock).toHaveBeenCalledWith(
|
||||
{ id: 'custom-view-id', workspaceId: WORKSPACE_ID },
|
||||
{ universalIdentifier: derivedCustomViewUniversalIdentifier },
|
||||
);
|
||||
});
|
||||
|
||||
it('ignores INDEX views owned by other applications', async () => {
|
||||
mockWorkspaceCache({
|
||||
views: [
|
||||
buildFlatView({
|
||||
id: 'external-view-id',
|
||||
universalIdentifier: 'external-view-uid',
|
||||
key: ViewKey.INDEX,
|
||||
applicationUniversalIdentifier:
|
||||
EXTERNAL_APPLICATION_UNIVERSAL_IDENTIFIER,
|
||||
}),
|
||||
],
|
||||
});
|
||||
|
||||
await runOnWorkspace();
|
||||
|
||||
expect(viewUpdateMock).not.toHaveBeenCalled();
|
||||
expect(invalidateCacheMock).not.toHaveBeenCalled();
|
||||
});
|
||||
|
||||
it('skips a soft-deleted INDEX view coexisting with an active one on the same object', async () => {
|
||||
mockWorkspaceCache({
|
||||
views: [
|
||||
buildFlatView({
|
||||
id: 'soft-deleted-view-id',
|
||||
universalIdentifier: 'soft-deleted-view-uid',
|
||||
key: ViewKey.INDEX,
|
||||
deletedAt: '2024-01-01T00:00:00.000Z',
|
||||
}),
|
||||
buildFlatView({
|
||||
id: 'active-view-id',
|
||||
universalIdentifier: 'active-view-uid',
|
||||
key: ViewKey.INDEX,
|
||||
}),
|
||||
],
|
||||
});
|
||||
|
||||
await runOnWorkspace();
|
||||
|
||||
expect(viewUpdateMock).toHaveBeenCalledTimes(1);
|
||||
expect(viewUpdateMock).toHaveBeenCalledWith(
|
||||
{ id: 'active-view-id', workspaceId: WORKSPACE_ID },
|
||||
{
|
||||
universalIdentifier: DERIVED_STANDARD_VIEW_UNIVERSAL_IDENTIFIER,
|
||||
isSystemSideEffect: true,
|
||||
},
|
||||
);
|
||||
});
|
||||
|
||||
it('re-owns only the active row of a soft-deleted + re-created view field pair', async () => {
|
||||
mockWorkspaceCache({
|
||||
views: [
|
||||
buildFlatView({
|
||||
id: 'view-id',
|
||||
universalIdentifier: 'legacy-view-uid',
|
||||
key: ViewKey.INDEX,
|
||||
viewFieldUniversalIdentifiers: [
|
||||
'soft-deleted-view-field-uid',
|
||||
'active-view-field-uid',
|
||||
],
|
||||
}),
|
||||
],
|
||||
viewFields: [
|
||||
buildFlatViewField({
|
||||
id: 'soft-deleted-view-field-id',
|
||||
universalIdentifier: 'soft-deleted-view-field-uid',
|
||||
deletedAt: '2024-01-01T00:00:00.000Z',
|
||||
}),
|
||||
buildFlatViewField({
|
||||
id: 'active-view-field-id',
|
||||
universalIdentifier: 'active-view-field-uid',
|
||||
}),
|
||||
],
|
||||
});
|
||||
|
||||
await runOnWorkspace();
|
||||
|
||||
expect(viewFieldUpdateMock).toHaveBeenCalledTimes(1);
|
||||
expect(viewFieldUpdateMock).toHaveBeenCalledWith(
|
||||
{ id: 'active-view-field-id', workspaceId: WORKSPACE_ID },
|
||||
{
|
||||
universalIdentifier: DERIVED_STANDARD_VIEW_FIELD_UNIVERSAL_IDENTIFIER,
|
||||
isSystemSideEffect: true,
|
||||
},
|
||||
);
|
||||
});
|
||||
|
||||
it('re-owns an external application view field living on a standard INDEX view', async () => {
|
||||
// e.g. an app that declared explicit columns on a standard INDEX view:
|
||||
// those must converge on the derived scheme so manifest deletion inference
|
||||
// never drops them once the app stops declaring them.
|
||||
const derivedExternalViewFieldUniversalIdentifier =
|
||||
getViewFieldUniversalIdentifier({
|
||||
applicationUniversalIdentifier:
|
||||
EXTERNAL_APPLICATION_UNIVERSAL_IDENTIFIER,
|
||||
viewUniversalIdentifier: DERIVED_STANDARD_VIEW_UNIVERSAL_IDENTIFIER,
|
||||
fieldMetadataUniversalIdentifier: FIELD_UNIVERSAL_IDENTIFIER,
|
||||
});
|
||||
|
||||
mockWorkspaceCache({
|
||||
views: [
|
||||
buildFlatView({
|
||||
id: 'view-id',
|
||||
universalIdentifier: 'legacy-view-uid',
|
||||
key: ViewKey.INDEX,
|
||||
viewFieldUniversalIdentifiers: ['external-app-view-field-uid'],
|
||||
}),
|
||||
],
|
||||
viewFields: [
|
||||
buildFlatViewField({
|
||||
id: 'external-app-view-field-id',
|
||||
universalIdentifier: 'external-app-view-field-uid',
|
||||
applicationUniversalIdentifier:
|
||||
EXTERNAL_APPLICATION_UNIVERSAL_IDENTIFIER,
|
||||
}),
|
||||
],
|
||||
fieldMetadatas: [
|
||||
buildFlatFieldMetadata({
|
||||
applicationUniversalIdentifier:
|
||||
EXTERNAL_APPLICATION_UNIVERSAL_IDENTIFIER,
|
||||
}),
|
||||
],
|
||||
});
|
||||
|
||||
await runOnWorkspace();
|
||||
|
||||
expect(viewFieldUpdateMock).toHaveBeenCalledWith(
|
||||
{ id: 'external-app-view-field-id', workspaceId: WORKSPACE_ID },
|
||||
{
|
||||
universalIdentifier: derivedExternalViewFieldUniversalIdentifier,
|
||||
isSystemSideEffect: true,
|
||||
},
|
||||
);
|
||||
});
|
||||
|
||||
it('derives a view field from the displayed field application, not the row attribution', async () => {
|
||||
mockWorkspaceCache({
|
||||
views: [
|
||||
buildFlatView({
|
||||
id: 'view-id',
|
||||
universalIdentifier: 'legacy-view-uid',
|
||||
key: ViewKey.INDEX,
|
||||
viewFieldUniversalIdentifiers: ['user-shown-view-field-uid'],
|
||||
}),
|
||||
],
|
||||
viewFields: [
|
||||
buildFlatViewField({
|
||||
id: 'user-shown-view-field-id',
|
||||
universalIdentifier: 'user-shown-view-field-uid',
|
||||
applicationUniversalIdentifier:
|
||||
CUSTOM_APPLICATION_UNIVERSAL_IDENTIFIER,
|
||||
}),
|
||||
],
|
||||
});
|
||||
|
||||
await runOnWorkspace();
|
||||
|
||||
expect(viewFieldUpdateMock).toHaveBeenCalledWith(
|
||||
{ id: 'user-shown-view-field-id', workspaceId: WORKSPACE_ID },
|
||||
{
|
||||
universalIdentifier: DERIVED_STANDARD_VIEW_FIELD_UNIVERSAL_IDENTIFIER,
|
||||
isSystemSideEffect: true,
|
||||
},
|
||||
);
|
||||
});
|
||||
|
||||
it('does nothing when the INDEX view and its view fields are already derived and system-owned', async () => {
|
||||
mockWorkspaceCache({
|
||||
views: [
|
||||
buildFlatView({
|
||||
id: 'view-id',
|
||||
universalIdentifier: DERIVED_STANDARD_VIEW_UNIVERSAL_IDENTIFIER,
|
||||
key: ViewKey.INDEX,
|
||||
isSystemSideEffect: true,
|
||||
viewFieldUniversalIdentifiers: [
|
||||
DERIVED_STANDARD_VIEW_FIELD_UNIVERSAL_IDENTIFIER,
|
||||
],
|
||||
}),
|
||||
],
|
||||
viewFields: [
|
||||
buildFlatViewField({
|
||||
id: 'view-field-id',
|
||||
universalIdentifier: DERIVED_STANDARD_VIEW_FIELD_UNIVERSAL_IDENTIFIER,
|
||||
isSystemSideEffect: true,
|
||||
}),
|
||||
],
|
||||
});
|
||||
|
||||
await runOnWorkspace();
|
||||
|
||||
expect(viewUpdateMock).not.toHaveBeenCalled();
|
||||
expect(viewFieldUpdateMock).not.toHaveBeenCalled();
|
||||
expect(invalidateCacheMock).not.toHaveBeenCalled();
|
||||
});
|
||||
|
||||
it('does not write in dry-run mode', async () => {
|
||||
mockWorkspaceCache({
|
||||
views: [
|
||||
buildFlatView({
|
||||
id: 'view-id',
|
||||
universalIdentifier: 'legacy-view-uid',
|
||||
key: ViewKey.INDEX,
|
||||
}),
|
||||
],
|
||||
});
|
||||
|
||||
await runOnWorkspace(true);
|
||||
|
||||
expect(viewUpdateMock).not.toHaveBeenCalled();
|
||||
expect(viewFieldUpdateMock).not.toHaveBeenCalled();
|
||||
expect(invalidateCacheMock).not.toHaveBeenCalled();
|
||||
});
|
||||
});
|
||||
+314
@@ -0,0 +1,314 @@
|
||||
import {
|
||||
getSystemViewUniversalIdentifier,
|
||||
getViewFieldUniversalIdentifier,
|
||||
} from 'twenty-shared/application';
|
||||
import { FieldMetadataType, ViewKey } from 'twenty-shared/types';
|
||||
import { In } from 'typeorm';
|
||||
|
||||
import { type WorkspaceIteratorService } from 'src/database/commands/command-runners/workspace-iterator.service';
|
||||
import { DemoteAndBackfillApplicationIndexViewCommand } from 'src/database/commands/upgrade-version-command/2-26/2-26-workspace-command-1785255690000-demote-and-backfill-application-index-view.command';
|
||||
import { type ApplicationService } from 'src/engine/core-modules/application/application.service';
|
||||
import { type WorkspaceCacheService } from 'src/engine/workspace-cache/services/workspace-cache.service';
|
||||
import { type WorkspaceMigrationValidateBuildAndRunService } from 'src/engine/workspace-manager/workspace-migration/services/workspace-migration-validate-build-and-run-service';
|
||||
import { type WorkspaceMigrationRunnerService } from 'src/engine/workspace-manager/workspace-migration/workspace-migration-runner/services/workspace-migration-runner.service';
|
||||
|
||||
const WORKSPACE_ID = '20202020-0000-4000-8000-000000000001';
|
||||
const STANDARD_APPLICATION_UNIVERSAL_IDENTIFIER =
|
||||
'20202020-0000-4000-8000-0000000000ad';
|
||||
const CUSTOM_APPLICATION_UNIVERSAL_IDENTIFIER =
|
||||
'20202020-0000-4000-8000-0000000000aa';
|
||||
const EXTERNAL_APPLICATION_UNIVERSAL_IDENTIFIER =
|
||||
'20202020-0000-4000-8000-0000000000ae';
|
||||
const OBJECT_UNIVERSAL_IDENTIFIER = '20202020-0000-4000-8000-0000000000bb';
|
||||
const FIELD_UNIVERSAL_IDENTIFIER = '20202020-0000-4000-8000-0000000000cc';
|
||||
|
||||
const DERIVED_VIEW_UNIVERSAL_IDENTIFIER = getSystemViewUniversalIdentifier({
|
||||
applicationUniversalIdentifier: EXTERNAL_APPLICATION_UNIVERSAL_IDENTIFIER,
|
||||
objectUniversalIdentifier: OBJECT_UNIVERSAL_IDENTIFIER,
|
||||
viewKey: ViewKey.INDEX,
|
||||
});
|
||||
|
||||
// An external application object with one displayable field.
|
||||
const OBJECT_METADATA = {
|
||||
universalIdentifier: OBJECT_UNIVERSAL_IDENTIFIER,
|
||||
applicationUniversalIdentifier: EXTERNAL_APPLICATION_UNIVERSAL_IDENTIFIER,
|
||||
labelIdentifierFieldMetadataUniversalIdentifier: FIELD_UNIVERSAL_IDENTIFIER,
|
||||
fieldUniversalIdentifiers: [FIELD_UNIVERSAL_IDENTIFIER],
|
||||
};
|
||||
|
||||
const FIELD_METADATA = {
|
||||
universalIdentifier: FIELD_UNIVERSAL_IDENTIFIER,
|
||||
applicationUniversalIdentifier: EXTERNAL_APPLICATION_UNIVERSAL_IDENTIFIER,
|
||||
name: 'name',
|
||||
type: FieldMetadataType.TEXT,
|
||||
};
|
||||
|
||||
const buildByUniversalIdentifierMap = <
|
||||
T extends { universalIdentifier: string },
|
||||
>(
|
||||
flatEntities: T[],
|
||||
) => ({
|
||||
byUniversalIdentifier: Object.fromEntries(
|
||||
flatEntities.map((flatEntity) => [
|
||||
flatEntity.universalIdentifier,
|
||||
flatEntity,
|
||||
]),
|
||||
),
|
||||
});
|
||||
|
||||
type WorkspaceView = {
|
||||
id: string;
|
||||
universalIdentifier: string;
|
||||
key: ViewKey | null;
|
||||
applicationUniversalIdentifier?: string;
|
||||
deletedAt?: string | null;
|
||||
isSystemSideEffect?: boolean;
|
||||
objectMetadataUniversalIdentifier?: string;
|
||||
};
|
||||
|
||||
const buildFlatView = (view: WorkspaceView) => ({
|
||||
applicationUniversalIdentifier: EXTERNAL_APPLICATION_UNIVERSAL_IDENTIFIER,
|
||||
deletedAt: null,
|
||||
isSystemSideEffect: false,
|
||||
objectMetadataUniversalIdentifier: OBJECT_UNIVERSAL_IDENTIFIER,
|
||||
...view,
|
||||
});
|
||||
|
||||
describe('DemoteAndBackfillApplicationIndexViewCommand', () => {
|
||||
let command: DemoteAndBackfillApplicationIndexViewCommand;
|
||||
let getOrRecomputeMock: jest.Mock;
|
||||
let invalidateCacheMock: jest.Mock;
|
||||
let viewUpdateMock: jest.Mock;
|
||||
let validateBuildAndRunWorkspaceMigrationMock: jest.Mock;
|
||||
|
||||
beforeEach(() => {
|
||||
jest.clearAllMocks();
|
||||
|
||||
getOrRecomputeMock = jest.fn();
|
||||
invalidateCacheMock = jest.fn().mockResolvedValue(undefined);
|
||||
viewUpdateMock = jest.fn().mockResolvedValue(undefined);
|
||||
validateBuildAndRunWorkspaceMigrationMock = jest
|
||||
.fn()
|
||||
.mockResolvedValue({ status: 'success' });
|
||||
|
||||
const viewRepositoryMock = {
|
||||
update: viewUpdateMock,
|
||||
};
|
||||
|
||||
command = new DemoteAndBackfillApplicationIndexViewCommand(
|
||||
{} as WorkspaceIteratorService,
|
||||
{
|
||||
getOrRecompute: getOrRecomputeMock,
|
||||
} as unknown as WorkspaceCacheService,
|
||||
{
|
||||
invalidateCache: invalidateCacheMock,
|
||||
} as unknown as WorkspaceMigrationRunnerService,
|
||||
{
|
||||
findWorkspaceTwentyStandardAndCustomApplicationOrThrow: jest
|
||||
.fn()
|
||||
.mockResolvedValue({
|
||||
twentyStandardFlatApplication: {
|
||||
universalIdentifier: STANDARD_APPLICATION_UNIVERSAL_IDENTIFIER,
|
||||
},
|
||||
workspaceCustomFlatApplication: {
|
||||
universalIdentifier: CUSTOM_APPLICATION_UNIVERSAL_IDENTIFIER,
|
||||
},
|
||||
}),
|
||||
} as unknown as ApplicationService,
|
||||
{
|
||||
validateBuildAndRunLegacyWorkspaceMigration:
|
||||
validateBuildAndRunWorkspaceMigrationMock,
|
||||
} as unknown as WorkspaceMigrationValidateBuildAndRunService,
|
||||
viewRepositoryMock as never,
|
||||
);
|
||||
});
|
||||
|
||||
const mockWorkspaceCache = ({
|
||||
views = [],
|
||||
viewFields = [],
|
||||
}: {
|
||||
views?: ReturnType<typeof buildFlatView>[];
|
||||
viewFields?: { universalIdentifier: string }[];
|
||||
}) => {
|
||||
getOrRecomputeMock.mockResolvedValue({
|
||||
flatViewMaps: buildByUniversalIdentifierMap(views),
|
||||
flatViewFieldMaps: buildByUniversalIdentifierMap(viewFields),
|
||||
flatObjectMetadataMaps: buildByUniversalIdentifierMap([OBJECT_METADATA]),
|
||||
flatFieldMetadataMaps: buildByUniversalIdentifierMap([FIELD_METADATA]),
|
||||
});
|
||||
};
|
||||
|
||||
const runOnWorkspace = (dryRun = false) =>
|
||||
command.runOnWorkspace({
|
||||
workspaceId: WORKSPACE_ID,
|
||||
options: { dryRun },
|
||||
index: 0,
|
||||
total: 1,
|
||||
});
|
||||
|
||||
it('demotes the application INDEX view and backfills the engine-owned one', async () => {
|
||||
mockWorkspaceCache({
|
||||
views: [
|
||||
buildFlatView({
|
||||
id: 'app-view-id',
|
||||
universalIdentifier: 'app-view-uid',
|
||||
key: ViewKey.INDEX,
|
||||
}),
|
||||
],
|
||||
});
|
||||
|
||||
await runOnWorkspace();
|
||||
|
||||
// The app view becomes a plain additional view under its manifest
|
||||
// identifier.
|
||||
expect(viewUpdateMock).toHaveBeenCalledTimes(1);
|
||||
expect(viewUpdateMock).toHaveBeenCalledWith(
|
||||
{ id: In(['app-view-id']), workspaceId: WORKSPACE_ID },
|
||||
{ key: null },
|
||||
);
|
||||
// Demotions are visible to the backfill pipeline validators.
|
||||
expect(invalidateCacheMock).toHaveBeenCalledTimes(1);
|
||||
|
||||
// Two-phase: every application's views are committed before any view
|
||||
// field, so cross-application view fields always find their parent view.
|
||||
expect(validateBuildAndRunWorkspaceMigrationMock).toHaveBeenCalledTimes(2);
|
||||
|
||||
const [viewPayload, viewFieldPayload] =
|
||||
validateBuildAndRunWorkspaceMigrationMock.mock.calls.map(
|
||||
([payload]) => payload,
|
||||
);
|
||||
|
||||
expect(viewPayload.applicationUniversalIdentifier).toBe(
|
||||
EXTERNAL_APPLICATION_UNIVERSAL_IDENTIFIER,
|
||||
);
|
||||
expect(
|
||||
viewPayload.allFlatEntityOperationByMetadataName.view.flatEntityToCreate,
|
||||
).toEqual([
|
||||
expect.objectContaining({
|
||||
universalIdentifier: DERIVED_VIEW_UNIVERSAL_IDENTIFIER,
|
||||
key: ViewKey.INDEX,
|
||||
isSystemSideEffect: true,
|
||||
}),
|
||||
]);
|
||||
expect(
|
||||
viewPayload.allFlatEntityOperationByMetadataName.viewField,
|
||||
).toBeUndefined();
|
||||
|
||||
expect(viewFieldPayload.applicationUniversalIdentifier).toBe(
|
||||
EXTERNAL_APPLICATION_UNIVERSAL_IDENTIFIER,
|
||||
);
|
||||
expect(
|
||||
viewFieldPayload.allFlatEntityOperationByMetadataName.view,
|
||||
).toBeUndefined();
|
||||
expect(
|
||||
viewFieldPayload.allFlatEntityOperationByMetadataName.viewField
|
||||
.flatEntityToCreate,
|
||||
).toEqual([
|
||||
expect.objectContaining({
|
||||
universalIdentifier: getViewFieldUniversalIdentifier({
|
||||
applicationUniversalIdentifier:
|
||||
EXTERNAL_APPLICATION_UNIVERSAL_IDENTIFIER,
|
||||
viewUniversalIdentifier: DERIVED_VIEW_UNIVERSAL_IDENTIFIER,
|
||||
fieldMetadataUniversalIdentifier: FIELD_UNIVERSAL_IDENTIFIER,
|
||||
}),
|
||||
fieldMetadataUniversalIdentifier: FIELD_UNIVERSAL_IDENTIFIER,
|
||||
isVisible: true,
|
||||
position: 0,
|
||||
isSystemSideEffect: true,
|
||||
}),
|
||||
]);
|
||||
});
|
||||
|
||||
it('backfills an application object that never had an INDEX view', async () => {
|
||||
mockWorkspaceCache({ views: [] });
|
||||
|
||||
await runOnWorkspace();
|
||||
|
||||
expect(viewUpdateMock).not.toHaveBeenCalled();
|
||||
expect(invalidateCacheMock).not.toHaveBeenCalled();
|
||||
expect(validateBuildAndRunWorkspaceMigrationMock).toHaveBeenCalledTimes(2);
|
||||
});
|
||||
|
||||
it('is idempotent: a fully backfilled object is neither demoted nor re-backfilled', async () => {
|
||||
mockWorkspaceCache({
|
||||
views: [
|
||||
buildFlatView({
|
||||
id: 'engine-view-id',
|
||||
universalIdentifier: DERIVED_VIEW_UNIVERSAL_IDENTIFIER,
|
||||
key: ViewKey.INDEX,
|
||||
isSystemSideEffect: true,
|
||||
}),
|
||||
],
|
||||
viewFields: [
|
||||
{
|
||||
universalIdentifier: getViewFieldUniversalIdentifier({
|
||||
applicationUniversalIdentifier:
|
||||
EXTERNAL_APPLICATION_UNIVERSAL_IDENTIFIER,
|
||||
viewUniversalIdentifier: DERIVED_VIEW_UNIVERSAL_IDENTIFIER,
|
||||
fieldMetadataUniversalIdentifier: FIELD_UNIVERSAL_IDENTIFIER,
|
||||
}),
|
||||
},
|
||||
],
|
||||
});
|
||||
|
||||
await runOnWorkspace();
|
||||
|
||||
expect(viewUpdateMock).not.toHaveBeenCalled();
|
||||
expect(validateBuildAndRunWorkspaceMigrationMock).not.toHaveBeenCalled();
|
||||
expect(invalidateCacheMock).not.toHaveBeenCalled();
|
||||
});
|
||||
|
||||
it('backfills the missing view fields of an already-committed engine INDEX view', async () => {
|
||||
mockWorkspaceCache({
|
||||
views: [
|
||||
buildFlatView({
|
||||
id: 'engine-view-id',
|
||||
universalIdentifier: DERIVED_VIEW_UNIVERSAL_IDENTIFIER,
|
||||
key: ViewKey.INDEX,
|
||||
isSystemSideEffect: true,
|
||||
}),
|
||||
],
|
||||
});
|
||||
|
||||
await runOnWorkspace();
|
||||
|
||||
expect(viewUpdateMock).not.toHaveBeenCalled();
|
||||
expect(validateBuildAndRunWorkspaceMigrationMock).toHaveBeenCalledTimes(1);
|
||||
|
||||
const [payload] = validateBuildAndRunWorkspaceMigrationMock.mock.calls[0];
|
||||
|
||||
expect(payload.allFlatEntityOperationByMetadataName.view).toBeUndefined();
|
||||
expect(
|
||||
payload.allFlatEntityOperationByMetadataName.viewField.flatEntityToCreate,
|
||||
).toEqual([
|
||||
expect.objectContaining({
|
||||
universalIdentifier: getViewFieldUniversalIdentifier({
|
||||
applicationUniversalIdentifier:
|
||||
EXTERNAL_APPLICATION_UNIVERSAL_IDENTIFIER,
|
||||
viewUniversalIdentifier: DERIVED_VIEW_UNIVERSAL_IDENTIFIER,
|
||||
fieldMetadataUniversalIdentifier: FIELD_UNIVERSAL_IDENTIFIER,
|
||||
}),
|
||||
viewUniversalIdentifier: DERIVED_VIEW_UNIVERSAL_IDENTIFIER,
|
||||
isSystemSideEffect: true,
|
||||
}),
|
||||
]);
|
||||
});
|
||||
|
||||
it('does not write in dry-run mode', async () => {
|
||||
mockWorkspaceCache({
|
||||
views: [
|
||||
buildFlatView({
|
||||
id: 'app-view-id',
|
||||
universalIdentifier: 'app-view-uid',
|
||||
key: ViewKey.INDEX,
|
||||
}),
|
||||
],
|
||||
});
|
||||
|
||||
await runOnWorkspace(true);
|
||||
|
||||
expect(viewUpdateMock).not.toHaveBeenCalled();
|
||||
expect(validateBuildAndRunWorkspaceMigrationMock).not.toHaveBeenCalled();
|
||||
expect(invalidateCacheMock).not.toHaveBeenCalled();
|
||||
});
|
||||
});
|
||||
+28
@@ -0,0 +1,28 @@
|
||||
import { getMetadataFlatEntityMapsKey } from 'src/engine/metadata-modules/flat-entity/utils/get-metadata-flat-entity-maps-key.util';
|
||||
import { getMetadataRelatedMetadataNames } from 'src/engine/metadata-modules/flat-entity/utils/get-metadata-related-metadata-names.util';
|
||||
import { type WorkspaceMigrationRunnerService } from 'src/engine/workspace-manager/workspace-migration/workspace-migration-runner/services/workspace-migration-runner.service';
|
||||
|
||||
export const invalidateIndexViewReconcileCache = async ({
|
||||
workspaceId,
|
||||
workspaceMigrationRunnerService,
|
||||
}: {
|
||||
workspaceId: string;
|
||||
workspaceMigrationRunnerService: WorkspaceMigrationRunnerService;
|
||||
}): Promise<void> => {
|
||||
const reconciledMetadataRelatedNames = [
|
||||
'view',
|
||||
...getMetadataRelatedMetadataNames('view'),
|
||||
'viewField',
|
||||
...getMetadataRelatedMetadataNames('viewField'),
|
||||
'pageLayoutWidget',
|
||||
] as const;
|
||||
|
||||
await workspaceMigrationRunnerService.invalidateCache({
|
||||
allFlatEntityMapsKeys: [
|
||||
...new Set(
|
||||
reconciledMetadataRelatedNames.map(getMetadataFlatEntityMapsKey),
|
||||
),
|
||||
],
|
||||
workspaceId,
|
||||
});
|
||||
};
|
||||
+2
@@ -25,6 +25,7 @@ import { V2_21_UpgradeVersionCommandModule } from 'src/database/commands/upgrade
|
||||
import { V2_22_UpgradeVersionCommandModule } from 'src/database/commands/upgrade-version-command/2-22/2-22-upgrade-version-command.module';
|
||||
import { V2_23_UpgradeVersionCommandModule } from 'src/database/commands/upgrade-version-command/2-23/2-23-upgrade-version-command.module';
|
||||
import { V2_25_UpgradeVersionCommandModule } from 'src/database/commands/upgrade-version-command/2-25/2-25-upgrade-version-command.module';
|
||||
import { V2_26_UpgradeVersionCommandModule } from 'src/database/commands/upgrade-version-command/2-26/2-26-upgrade-version-command.module';
|
||||
|
||||
@Module({
|
||||
imports: [
|
||||
@@ -53,6 +54,7 @@ import { V2_25_UpgradeVersionCommandModule } from 'src/database/commands/upgrade
|
||||
V2_22_UpgradeVersionCommandModule,
|
||||
V2_23_UpgradeVersionCommandModule,
|
||||
V2_25_UpgradeVersionCommandModule,
|
||||
V2_26_UpgradeVersionCommandModule,
|
||||
],
|
||||
})
|
||||
export class WorkspaceCommandProviderModule {}
|
||||
|
||||
Reference in New Issue
Block a user