Centralized side effects devxp basis (#22295)

# Introduction

This PR introduces a centralized, strictly-typed **metadata side-effect
engine** that unifies how system metadata side effects are derived and
applied across both metadata entry points — the **metadata GraphQL API**
and the **application sync / manifest** flow — and migrates the first
side effect end-to-end: **a unique scalar field owns its backing
single-field `UNIQUE` index** (full create / update / delete lifecycle).

## New conventions

- **Engine-owned companions**: metadata flagged `isSystemSideEffect:
true` is owned by the engine. Its deletion is never inferred from
absence in a manifest — it results from PG-level cascade or from a
delete side effect (a side effect always has a cause, its parent
metadata).
- **Reserved deterministic identifiers**: apps cannot declare metadata
reusing an engine-owned deterministic `universalIdentifier`. Doing so
fails validation with `RESERVED_SYSTEM_UNIVERSAL_IDENTIFIER` (until an
explicit override API exists).
- **Record-native operation matrix**: the operation matrix is keyed by
`universalIdentifier` (`AllFlatEntityOperationRecordByMetadataName`)
instead of arrays, making parent resolution and deduplication O(1).
Array-based API callers are transpiled to records at the
validate-build-and-run boundary.
- Twenty-sdk user-facing experience with system fields will only be
related to overrides.

# What this PR does

## 1. Side-effect engine (foundation)

- `MetadataSideEffectEngineService.expandWithSideEffects(...)` takes the
intention-carrying record matrix and returns it expanded with derived
side effects, or a structured failure.
- Handlers are registered via a typed **decorator + registry** pattern
(`MetadataSideEffectHandler({ operation, metadataName, name, description
})`), with runtime duplicate-name detection. Multiple handlers per
(operation, metadataName) are supported.
- Handler contract mirrors the validator pattern:
- receives the trigger flat entity, the live record matrix, and
**strictly-typed related flat entity maps**
(`MetadataFlatEntityAndRelatedFlatEntityMapsForSideEffect<P>`, derived
from declared companion metadata names — no loose
`Partial<AllFlatEntityMaps>` context)
- returns `MetadataSideEffectResult`: `success` (operations record) |
`noop` | `fail` (structured failure)
- **Non-recursion is structural**: triggers are read from the original
caller input, never from the expanded matrix, so a side effect can never
trigger another side effect.
- **Deduplication + collision detection**: side effects are deduped by
`universalIdentifier` per operation; a caller-declared entity colliding
with an engine-owned deterministic identifier is recorded as a
collision.
- **Unified failure channel**: handler failures and reserved-identifier
collisions are merged into the same `OrchestratorFailureReport` contract
as builder validation errors, and the run short-circuits (fail-closed,
nothing is applied).

## 2. First migrated side effect — unique field → backing unique index

Three handlers own the complete lifecycle of the deterministic
single-field `UNIQUE` index backing a unique scalar field:

- **create**: unique scalar field → generate the deterministic backing
index (`fieldUniqueBackingIndexOnCreate`)
- **update**: `isUnique` flips and renames of still-unique fields (the
index name — and therefore its deterministic identifier — derives from
the field name, so a rename drops the stale index and recreates the
deterministic one) (`fieldUniqueBackingIndexOnUpdate`)
- **delete**: cascade-delete the backing index
(`fieldUniqueBackingIndexOnDelete`)

Supporting rules:
- The primary key `id` field never spawns a backing index (uniqueness
comes from the PK constraint) — explicit `isPrimaryKeyFlatFieldMetadata`
guard.
- Parent object resolution is **optimistic-first**: an object created or
updated in the same batch wins over the workspace cache (so e.g.
renaming an object while flipping a field to unique builds the index
from the post-rename object), resolved in O(1) via the record matrix.
- A missing parent object is reported as a structured side-effect
failure, never silently skipped.

## 3. Path convergence — manifest and API share one flow

- The manifest sync now derives a from→to **record matrix** from the
cache and feeds `validateBuildAndRunWorkspaceMigrationFromRecord`, the
same flow the API uses — both paths converge on the engine.
- Manifest-side unique-index generation and API transpiler
system-unique-index handling were removed (declared/composite/relation
indexes stay untouched).
- New `WorkspaceMigrationFlatEntityMapsService` mutualizes
flat-entity-maps computation between the side-effect engine and the
builder: cache keys are derived from the caller metadata names (+
validation- and side-effect-related closures) instead of hardcoded
loads.
- App-scoping and pruning are folded into one shared primitive
(`getSubAllFlatEntityMapsByApplicationIdsOrThrow`): slicing dependency
maps to the involved applications always prunes dangling one-to-many
aggregators — callers can no longer forget it.
- **Behavior change**: an app extending another app's view with a view
field now syncs successfully (cross-app view-field extension), covered
by a dedicated integration test.

## 4. Backfill upgrade command (2.19)

`upgrade:2-19:backfill-system-unique-index-universal-identifier`
rewrites legacy system unique-index `universalIdentifier`s to their
deterministic value so the engine can own pre-existing indexes. The
backfill is **driven from `isUnique: true` fields** (mirroring the
engine ownership predicate — excludes PK / morph / relation fields) and
resolves each field's backing index in O(1).

# Bugs fixed along the way

- `database:reset` seeding failed with
`INDEX_FIELD_INVALID_DEFAULT_VALUE`: the engine derived a backing
`UNIQUE` index for the default `id` primary key. Fixed with the explicit
primary-key guard.
- `isUnique` updates on system-flagged standard fields (e.g.
auto-created `name`) did not trigger the backing-index side effect.
- Manifest sync crashed with "Could not find flat entity with universal
identifier ..." when app-scoped slices left dangling aggregator
references — fixed by centralizing pruning in the shared slice primitive
This commit is contained in:
Paul Rastoin
2026-07-03 18:13:20 +02:00
committed by GitHub
parent 566c3b6629
commit 43730d7748
59 changed files with 2840 additions and 691 deletions
@@ -3,16 +3,31 @@ import { TypeOrmModule } from '@nestjs/typeorm';
import { WorkspaceIteratorModule } from 'src/database/commands/command-runners/workspace-iterator.module';
import { BackfillWorkspaceCustomApplicationRegistrationCommand } from 'src/database/commands/upgrade-version-command/2-19/2-19-workspace-command-1782853718000-backfill-workspace-custom-application-registration.command';
import { ApplicationModule } from 'src/engine/core-modules/application/application.module';
import { BackfillSystemUniqueIndexUniversalIdentifierCommand } from 'src/database/commands/upgrade-version-command/2-19/2-19-workspace-command-1783093620000-backfill-system-unique-index-universal-identifier.command';
import { ApplicationEntity } from 'src/engine/core-modules/application/application.entity';
import { ApplicationModule } from 'src/engine/core-modules/application/application.module';
import { WorkspaceEntity } from 'src/engine/core-modules/workspace/workspace.entity';
import { IndexMetadataEntity } from 'src/engine/metadata-modules/index-metadata/index-metadata.entity';
import { WorkspaceMetadataVersionModule } from 'src/engine/metadata-modules/workspace-metadata-version/workspace-metadata-version.module';
import { WorkspaceCacheModule } from 'src/engine/workspace-cache/workspace-cache.module';
import { WorkspaceMigrationModule } from 'src/engine/workspace-manager/workspace-migration/workspace-migration.module';
@Module({
imports: [
ApplicationModule,
TypeOrmModule.forFeature([WorkspaceEntity, ApplicationEntity]),
TypeOrmModule.forFeature([
WorkspaceEntity,
ApplicationEntity,
IndexMetadataEntity,
]),
WorkspaceIteratorModule,
WorkspaceCacheModule,
WorkspaceMetadataVersionModule,
WorkspaceMigrationModule,
],
providers: [
BackfillWorkspaceCustomApplicationRegistrationCommand,
BackfillSystemUniqueIndexUniversalIdentifierCommand,
],
providers: [BackfillWorkspaceCustomApplicationRegistrationCommand],
})
export class V2_19_UpgradeVersionCommandModule {}
@@ -0,0 +1,175 @@
import { Command } from 'nest-commander';
import { InjectRepository } from '@nestjs/typeorm';
import { getIndexUniversalIdentifier } from 'twenty-shared/application';
import { isDefined } from 'twenty-shared/utils';
import { Repository } from 'typeorm';
import { ActiveOrSuspendedWorkspaceCommandRunner } from 'src/database/commands/command-runners/active-or-suspended-workspace.command-runner';
import { WorkspaceIteratorService } from 'src/database/commands/command-runners/workspace-iterator.service';
import { type RunOnWorkspaceArgs } from 'src/database/commands/command-runners/workspace.command-runner';
import { RegisteredWorkspaceCommand } from 'src/engine/core-modules/upgrade/decorators/registered-workspace-command.decorator';
import { findFlatEntityByIdInFlatEntityMaps } from 'src/engine/metadata-modules/flat-entity/utils/find-flat-entity-by-id-in-flat-entity-maps.util';
import { getMetadataFlatEntityMapsKey } from 'src/engine/metadata-modules/flat-entity/utils/get-metadata-flat-entity-maps-key.util';
import { getMetadataRelatedMetadataNames } from 'src/engine/metadata-modules/flat-entity/utils/get-metadata-related-metadata-names.util';
import { getMetadataSerializedRelationNames } from 'src/engine/metadata-modules/flat-entity/utils/get-metadata-serialized-relation-names.util';
import { isMorphOrRelationFlatFieldMetadata } from 'src/engine/metadata-modules/flat-field-metadata/utils/is-morph-or-relation-flat-field-metadata.util';
import { isPrimaryKeyFlatFieldMetadata } from 'src/engine/metadata-modules/flat-field-metadata/utils/is-primary-key-flat-field-metadata.util';
import { isSystemUniqueFlatIndexMetadata } from 'src/engine/metadata-modules/flat-index-metadata/utils/is-system-unique-flat-index-metadata.util';
import { IndexMetadataEntity } from 'src/engine/metadata-modules/index-metadata/index-metadata.entity';
import { WorkspaceMetadataVersionService } from 'src/engine/metadata-modules/workspace-metadata-version/services/workspace-metadata-version.service';
import { WorkspaceCacheService } from 'src/engine/workspace-cache/services/workspace-cache.service';
@RegisteredWorkspaceCommand('2.19.0', 1783093620000)
@Command({
name: 'upgrade:2-19:backfill-system-unique-index-universal-identifier',
description:
'Backfill the deterministic universal identifier of system unique indexes (the index backing a unique scalar field) so the metadata side-effect engine can own their lifecycle.',
})
export class BackfillSystemUniqueIndexUniversalIdentifierCommand extends ActiveOrSuspendedWorkspaceCommandRunner {
constructor(
protected readonly workspaceIteratorService: WorkspaceIteratorService,
private readonly workspaceCacheService: WorkspaceCacheService,
private readonly workspaceMetadataVersionService: WorkspaceMetadataVersionService,
// eslint-disable-next-line twenty/prefer-workspace-scoped-repository
@InjectRepository(IndexMetadataEntity)
private readonly indexMetadataRepository: Repository<IndexMetadataEntity>,
) {
super(workspaceIteratorService);
}
override async runOnWorkspace({
workspaceId,
options,
}: RunOnWorkspaceArgs): Promise<void> {
const isDryRun = options.dryRun ?? false;
const {
flatObjectMetadataMaps,
flatFieldMetadataMaps,
flatIndexMaps,
flatApplicationMaps,
} = await this.workspaceCacheService.getOrRecompute(workspaceId, [
'flatObjectMetadataMaps',
'flatFieldMetadataMaps',
'flatIndexMaps',
'flatApplicationMaps',
]);
const backingFlatIndexMetadataByFieldMetadataId = new Map(
Object.values(flatIndexMaps.byUniversalIdentifier)
.filter(isDefined)
.filter(
(flatIndexMetadata) =>
isSystemUniqueFlatIndexMetadata(flatIndexMetadata) &&
flatIndexMetadata.flatIndexFieldMetadatas.length === 1,
)
.map((flatIndexMetadata): [string, typeof flatIndexMetadata] => [
flatIndexMetadata.flatIndexFieldMetadatas[0].fieldMetadataId,
flatIndexMetadata,
]),
);
const indexesToBackfill = Object.values(
flatFieldMetadataMaps.byUniversalIdentifier,
)
.filter(isDefined)
.filter(
(flatFieldMetadata) =>
flatFieldMetadata.isUnique === true &&
!isMorphOrRelationFlatFieldMetadata(flatFieldMetadata) &&
!isPrimaryKeyFlatFieldMetadata(flatFieldMetadata),
)
.flatMap((flatFieldMetadata) => {
const flatObjectMetadata = findFlatEntityByIdInFlatEntityMaps({
flatEntityId: flatFieldMetadata.objectMetadataId,
flatEntityMaps: flatObjectMetadataMaps,
});
if (
!isDefined(flatObjectMetadata) ||
!isDefined(flatFieldMetadata.applicationId)
) {
return [];
}
const applicationUniversalIdentifier =
flatApplicationMaps.byId[flatFieldMetadata.applicationId]
?.universalIdentifier;
if (!isDefined(applicationUniversalIdentifier)) {
return [];
}
const backingFlatIndexMetadata =
backingFlatIndexMetadataByFieldMetadataId.get(flatFieldMetadata.id);
if (!isDefined(backingFlatIndexMetadata)) {
return [];
}
const deterministicUniversalIdentifier = getIndexUniversalIdentifier({
applicationUniversalIdentifier,
objectUniversalIdentifier: flatObjectMetadata.universalIdentifier,
name: backingFlatIndexMetadata.name,
});
if (
deterministicUniversalIdentifier ===
backingFlatIndexMetadata.universalIdentifier
) {
return [];
}
return [
{
id: backingFlatIndexMetadata.id,
name: backingFlatIndexMetadata.name,
deterministicUniversalIdentifier,
},
];
});
if (indexesToBackfill.length === 0) {
this.logger.log(
`No system unique index universal identifier to backfill for workspace ${workspaceId}`,
);
return;
}
this.logger.log(
`${isDryRun ? '[DRY RUN] ' : ''}Backfilling ${indexesToBackfill.length} system unique index universal identifier(s) for workspace ${workspaceId}: ${indexesToBackfill.map(({ name }) => name).join(', ')}`,
);
if (isDryRun) {
return;
}
for (const { id, deterministicUniversalIdentifier } of indexesToBackfill) {
await this.indexMetadataRepository.update(
{ id, workspaceId },
{ universalIdentifier: deterministicUniversalIdentifier },
);
}
const indexRelatedMetadataNames = [
'index',
...getMetadataRelatedMetadataNames('index'),
...getMetadataSerializedRelationNames('index'),
] as const;
const cacheKeysToFlush = [
...new Set(indexRelatedMetadataNames.map(getMetadataFlatEntityMapsKey)),
];
await this.workspaceCacheService.flush(workspaceId, cacheKeysToFlush);
await this.workspaceMetadataVersionService.incrementMetadataVersion(
workspaceId,
);
this.logger.log(
`Backfilled ${indexesToBackfill.length} system unique index universal identifier(s) for workspace ${workspaceId}`,
);
}
}
@@ -5,6 +5,7 @@ import { ALL_METADATA_NAME } from 'twenty-shared/metadata';
import { isDefined } from 'twenty-shared/utils';
import { ComputeApplicationManifestAllUniversalFlatEntityMapsService } from 'src/engine/core-modules/application/application-manifest/services/compute-application-manifest-all-universal-flat-entity-maps.service';
import { buildAllFlatEntityOperationRecordByMetadataNameFromFromTo } from 'src/engine/core-modules/application/application-manifest/utils/build-all-flat-entity-operation-record-by-metadata-name-from-from-to.util';
import { buildFromToAllUniversalFlatEntityMaps } from 'src/engine/core-modules/application/application-manifest/utils/build-from-to-all-universal-flat-entity-maps.util';
import { getApplicationSubAllFlatEntityMaps } from 'src/engine/core-modules/application/application-manifest/utils/get-application-sub-all-flat-entity-maps.util';
import {
@@ -173,11 +174,6 @@ export class ApplicationManifestMigrationService {
}> {
const now = new Date().toISOString();
const { twentyStandardFlatApplication } =
await this.applicationService.findWorkspaceTwentyStandardAndCustomApplicationOrThrow(
{ workspaceId },
);
const recomputeStart = performance.now();
const cacheResult = await this.workspaceCacheService.getOrRecompute(
workspaceId,
@@ -193,7 +189,8 @@ export class ApplicationManifestMigrationService {
ApplicationManifestMigrationService.name,
);
const { featureFlagsMap, ...existingAllFlatEntityMaps } = cacheResult;
const { featureFlagsMap: _featureFlagsMap, ...existingAllFlatEntityMaps } =
cacheResult;
const fromAllFlatEntityMaps = getApplicationSubAllFlatEntityMaps({
applicationIds: [ownerFlatApplication.id],
@@ -208,32 +205,27 @@ export class ApplicationManifestMigrationService {
workspaceId,
});
const dependencyAllFlatEntityMaps = getApplicationSubAllFlatEntityMaps({
applicationIds:
ownerFlatApplication.universalIdentifier ===
TWENTY_STANDARD_APPLICATION.universalIdentifier
? [twentyStandardFlatApplication.id]
: [ownerFlatApplication.id, twentyStandardFlatApplication.id],
fromAllFlatEntityMaps: existingAllFlatEntityMaps,
});
const allFlatEntityOperationRecordByMetadataName =
buildAllFlatEntityOperationRecordByMetadataNameFromFromTo({
fromAllFlatEntityMaps,
toAllUniversalFlatEntityMaps,
buildOptions: {
isSystemBuild: false,
inferDeletionFromMissingEntities: true,
applicationUniversalIdentifier:
ownerFlatApplication.universalIdentifier,
},
});
const validateBuildRunStart = performance.now();
const validateAndBuildResult =
await this.workspaceMigrationValidateBuildAndRunService.validateBuildAndRunWorkspaceMigrationFromTo(
await this.workspaceMigrationValidateBuildAndRunService.validateBuildAndRunWorkspaceMigrationFromRecord(
{
buildOptions: {
isSystemBuild: false,
inferDeletionFromMissingEntities: true,
applicationUniversalIdentifier:
ownerFlatApplication.universalIdentifier,
},
fromToAllFlatEntityMaps: buildFromToAllUniversalFlatEntityMaps({
fromAllFlatEntityMaps,
toAllUniversalFlatEntityMaps,
}),
allFlatEntityOperationRecordByMetadataName,
workspaceId,
dependencyAllFlatEntityMaps,
additionalCacheDataMaps: { featureFlagsMap },
isSystemBuild: false,
applicationUniversalIdentifier:
ownerFlatApplication.universalIdentifier,
dryRun,
},
);
@@ -40,7 +40,6 @@ import { fromAgentManifestToUniversalFlatAgent } from 'src/engine/core-modules/a
import { type EncryptedString } from 'src/engine/core-modules/secret-encryption/branded-strings/encrypted-string.type';
import { type PlaintextString } from 'src/engine/core-modules/secret-encryption/branded-strings/plaintext-string.type';
import { SecretEncryptionService } from 'src/engine/core-modules/secret-encryption/secret-encryption.service';
import { generateIndexForFlatFieldMetadata } from 'src/engine/metadata-modules/flat-field-metadata/utils/generate-index-for-flat-field-metadata.util';
import { createEmptyAllFlatEntityMaps } from 'src/engine/metadata-modules/flat-entity/constant/create-empty-all-flat-entity-maps.constant';
import { type AllFlatEntityMaps } from 'src/engine/metadata-modules/flat-entity/types/all-flat-entity-maps.type';
import { type UniversalFlatFieldMetadata } from 'src/engine/workspace-manager/workspace-migration/universal-flat-entity/types/universal-flat-field-metadata.type';
@@ -115,19 +114,6 @@ export class ComputeApplicationManifestAllUniversalFlatEntityMapsService {
universalFlatEntityMapsToMutate:
allUniversalFlatEntityMaps.flatFieldMetadataMaps,
});
if (flatFieldMetadata.isUnique) {
addUniversalFlatEntityToUniversalFlatEntityMapsThroughMutationOrThrow(
{
universalFlatEntity: generateIndexForFlatFieldMetadata({
flatFieldMetadata,
flatObjectMetadata,
}),
universalFlatEntityMapsToMutate:
allUniversalFlatEntityMaps.flatIndexMaps,
},
);
}
}
}
@@ -143,27 +129,6 @@ export class ComputeApplicationManifestAllUniversalFlatEntityMapsService {
universalFlatEntityMapsToMutate:
allUniversalFlatEntityMaps.flatFieldMetadataMaps,
});
if (flatFieldMetadata.isUnique) {
const flatObjectMetadata =
allUniversalFlatEntityMaps.flatObjectMetadataMaps
.byUniversalIdentifier[
flatFieldMetadata.objectMetadataUniversalIdentifier
];
if (isDefined(flatObjectMetadata)) {
addUniversalFlatEntityToUniversalFlatEntityMapsThroughMutationOrThrow(
{
universalFlatEntity: generateIndexForFlatFieldMetadata({
flatFieldMetadata,
flatObjectMetadata,
}),
universalFlatEntityMapsToMutate:
allUniversalFlatEntityMaps.flatIndexMaps,
},
);
}
}
}
const indexCountByObjectUniversalIdentifier = new Map<string, number>();
@@ -0,0 +1,149 @@
import {
ALL_METADATA_NAME,
type AllMetadataName,
} from 'twenty-shared/metadata';
import { isDefined } from 'twenty-shared/utils';
import { type AllFlatEntityMaps } from 'src/engine/metadata-modules/flat-entity/types/all-flat-entity-maps.type';
import {
type AllFlatEntityOperationRecordByMetadataName,
type FlatEntityOperationRecord,
} from 'src/engine/metadata-modules/flat-entity/types/all-flat-entity-operation-record-by-metadata-name.type';
import { type MetadataUniversalFlatEntity } from 'src/engine/metadata-modules/flat-entity/types/metadata-universal-flat-entity.type';
import { getMetadataFlatEntityMapsKey } from 'src/engine/metadata-modules/flat-entity/utils/get-metadata-flat-entity-maps-key.util';
import { isSystemUniqueFlatIndexMetadata } from 'src/engine/metadata-modules/flat-index-metadata/utils/is-system-unique-flat-index-metadata.util';
import { type MetadataUniversalFlatEntityMaps } from 'src/engine/workspace-manager/workspace-migration/universal-flat-entity/types/metadata-universal-flat-entity-maps.type';
import { compareTwoFlatEntity } from 'src/engine/workspace-manager/workspace-migration/universal-flat-entity/utils/compare-two-universal-flat-entity.util';
import { shouldInferDeletionFromMissingEntities } from 'src/engine/workspace-manager/workspace-migration/utils/should-infer-deletion-from-missing-entities.util';
import { type WorkspaceMigrationBuilderOptions } from 'src/engine/workspace-manager/workspace-migration/workspace-migration-builder/types/workspace-migration-builder-options.type';
const toRecordByUniversalIdentifier = <T extends AllMetadataName>(
flatEntities: MetadataUniversalFlatEntity<T>[],
): Record<string, MetadataUniversalFlatEntity<T>> =>
Object.fromEntries(
flatEntities.map((flatEntity) => [
flatEntity.universalIdentifier,
flatEntity,
]),
);
const buildFlatEntityOperationRecordForMetadata = <T extends AllMetadataName>({
metadataName,
fromFlatEntityMaps,
toFlatEntityMaps,
buildOptions,
}: {
metadataName: T;
fromFlatEntityMaps: MetadataUniversalFlatEntityMaps<T>;
toFlatEntityMaps: MetadataUniversalFlatEntityMaps<T>;
buildOptions: WorkspaceMigrationBuilderOptions;
}): FlatEntityOperationRecord<T> => {
const fromByUniversalIdentifier = fromFlatEntityMaps.byUniversalIdentifier;
const toByUniversalIdentifier = toFlatEntityMaps.byUniversalIdentifier;
const flatEntityToCreate = Object.values(toByUniversalIdentifier)
.filter(isDefined)
.filter(
(toFlatEntity) =>
!isDefined(fromByUniversalIdentifier[toFlatEntity.universalIdentifier]),
);
const flatEntityToDelete = shouldInferDeletionFromMissingEntities({
buildOptions,
metadataName,
})
? Object.values(fromByUniversalIdentifier)
.filter(isDefined)
.filter(
(fromFlatEntity) =>
!isDefined(
toByUniversalIdentifier[fromFlatEntity.universalIdentifier],
),
)
.filter((fromFlatEntity) => {
if (metadataName !== ALL_METADATA_NAME.index) {
return true;
}
return !isSystemUniqueFlatIndexMetadata(
fromFlatEntity as unknown as {
isSystemSideEffect: boolean;
isUnique: boolean;
},
);
})
: [];
const flatEntityToUpdate = Object.values(fromByUniversalIdentifier)
.filter(isDefined)
.map((fromFlatEntity) => {
const toFlatEntity =
toByUniversalIdentifier[fromFlatEntity.universalIdentifier];
if (!isDefined(toFlatEntity)) {
return undefined;
}
const update = compareTwoFlatEntity({
fromUniversalFlatEntity: fromFlatEntity,
toUniversalFlatEntity: toFlatEntity,
metadataName,
});
return isDefined(update) ? toFlatEntity : undefined;
})
.filter(isDefined);
return {
flatEntityToCreate: toRecordByUniversalIdentifier(flatEntityToCreate),
flatEntityToUpdate: toRecordByUniversalIdentifier(flatEntityToUpdate),
flatEntityToDelete: toRecordByUniversalIdentifier(flatEntityToDelete),
};
};
export const buildAllFlatEntityOperationRecordByMetadataNameFromFromTo = ({
fromAllFlatEntityMaps,
toAllUniversalFlatEntityMaps,
buildOptions,
}: {
fromAllFlatEntityMaps: AllFlatEntityMaps;
toAllUniversalFlatEntityMaps: AllFlatEntityMaps;
buildOptions: WorkspaceMigrationBuilderOptions;
}): AllFlatEntityOperationRecordByMetadataName => {
const allFlatEntityOperationRecordByMetadataName: AllFlatEntityOperationRecordByMetadataName =
{};
for (const metadataName of Object.values(ALL_METADATA_NAME)) {
const flatEntityMapsKey = getMetadataFlatEntityMapsKey(metadataName);
const flatEntityOperationRecord = buildFlatEntityOperationRecordForMetadata(
{
metadataName,
fromFlatEntityMaps: fromAllFlatEntityMaps[
flatEntityMapsKey
] as unknown as MetadataUniversalFlatEntityMaps<typeof metadataName>,
toFlatEntityMaps: toAllUniversalFlatEntityMaps[
flatEntityMapsKey
] as unknown as MetadataUniversalFlatEntityMaps<typeof metadataName>,
buildOptions,
},
);
if (
Object.keys(flatEntityOperationRecord.flatEntityToCreate).length === 0 &&
Object.keys(flatEntityOperationRecord.flatEntityToUpdate).length === 0 &&
Object.keys(flatEntityOperationRecord.flatEntityToDelete).length === 0
) {
continue;
}
(
allFlatEntityOperationRecordByMetadataName as Record<
string,
FlatEntityOperationRecord<AllMetadataName>
>
)[metadataName] = flatEntityOperationRecord;
}
return allFlatEntityOperationRecordByMetadataName;
};
@@ -1,11 +1,7 @@
import { ALL_METADATA_NAME } from 'twenty-shared/metadata';
import { createEmptyAllFlatEntityMaps } from 'src/engine/metadata-modules/flat-entity/constant/create-empty-all-flat-entity-maps.constant';
import { type AllFlatEntityMaps } from 'src/engine/metadata-modules/flat-entity/types/all-flat-entity-maps.type';
import { type MetadataFlatEntity } from 'src/engine/metadata-modules/flat-entity/types/metadata-flat-entity.type';
import { getMetadataFlatEntityMapsKey } from 'src/engine/metadata-modules/flat-entity/utils/get-metadata-flat-entity-maps-key.util';
import { getSubFlatEntityMapsByApplicationIdsOrThrow } from 'src/engine/metadata-modules/flat-entity/utils/get-sub-flat-entity-maps-by-application-ids-or-throw.util';
import { pruneDanglingForeignKeyAggregatorsInAllFlatEntityMapsThroughMutation } from 'src/engine/metadata-modules/flat-entity/utils/prune-dangling-foreign-key-aggregators-in-all-flat-entity-maps-through-mutation.util';
import { getSubAllFlatEntityMapsByApplicationIdsOrThrow } from 'src/engine/metadata-modules/flat-entity/utils/get-sub-all-flat-entity-maps-by-application-ids-or-throw.util';
export const getApplicationSubAllFlatEntityMaps = ({
applicationIds,
@@ -13,28 +9,9 @@ export const getApplicationSubAllFlatEntityMaps = ({
}: {
applicationIds: string[];
fromAllFlatEntityMaps: AllFlatEntityMaps;
}): AllFlatEntityMaps => {
const emptyAllFlatEntityMaps = createEmptyAllFlatEntityMaps();
for (const metadataName of Object.values(ALL_METADATA_NAME)) {
const flatEntityMapsKey = getMetadataFlatEntityMapsKey(metadataName);
const fromFlatEntityMaps = fromAllFlatEntityMaps[flatEntityMapsKey];
const applicationSubFlatEntityMaps =
getSubFlatEntityMapsByApplicationIdsOrThrow<
MetadataFlatEntity<typeof metadataName>
>({
applicationIds,
flatEntityMaps: fromFlatEntityMaps,
});
// @ts-expect-error Metadata flat entity maps cache key and metadataName colliding
emptyAllFlatEntityMaps[flatEntityMapsKey] = applicationSubFlatEntityMaps;
}
pruneDanglingForeignKeyAggregatorsInAllFlatEntityMapsThroughMutation({
allFlatEntityMapsToMutate: emptyAllFlatEntityMaps,
});
return emptyAllFlatEntityMaps;
};
}): AllFlatEntityMaps =>
getSubAllFlatEntityMapsByApplicationIdsOrThrow({
applicationIds,
metadataNames: Object.values(ALL_METADATA_NAME),
fromAllFlatEntityMaps,
}) as AllFlatEntityMaps;
@@ -0,0 +1,7 @@
import { type AllMetadataName } from 'twenty-shared/metadata';
export const ALL_METADATA_SIDE_EFFECT_COMPANION_METADATA_NAMES = {
fieldMetadata: ['index'],
} as const satisfies Partial<
Record<AllMetadataName, readonly AllMetadataName[]>
>;
@@ -0,0 +1,15 @@
import { type AllMetadataName } from 'twenty-shared/metadata';
import { type MetadataUniversalFlatEntity } from 'src/engine/metadata-modules/flat-entity/types/metadata-universal-flat-entity.type';
export type FlatEntityOperationRecord<T extends AllMetadataName> = {
flatEntityToCreate: Partial<
Record<string, MetadataUniversalFlatEntity<T> & { id?: string }>
>;
flatEntityToUpdate: Partial<Record<string, MetadataUniversalFlatEntity<T>>>;
flatEntityToDelete: Partial<Record<string, MetadataUniversalFlatEntity<T>>>;
};
export type AllFlatEntityOperationRecordByMetadataName = {
[P in AllMetadataName]?: FlatEntityOperationRecord<P>;
};
@@ -7,3 +7,7 @@ export type FlatEntityToCreateDeleteUpdate<T extends AllMetadataName> = {
flatEntityToCreate: (MetadataUniversalFlatEntity<T> & { id?: string })[];
flatEntityToDelete: MetadataUniversalFlatEntity<T>[];
};
export type AllFlatEntityOperationByMetadataName = {
[P in AllMetadataName]?: FlatEntityToCreateDeleteUpdate<P>;
};
@@ -0,0 +1,13 @@
import { type AllMetadataName } from 'twenty-shared/metadata';
import { type AllFlatEntityMaps } from 'src/engine/metadata-modules/flat-entity/types/all-flat-entity-maps.type';
import { type MetadataSideEffectRelatedMetadataNames } from 'src/engine/metadata-modules/flat-entity/types/metadata-side-effect-related-metadata-names.type';
import { type MetadataToFlatEntityMapsKey } from 'src/engine/metadata-modules/flat-entity/types/metadata-to-flat-entity-maps-key';
export type MetadataFlatEntityAndRelatedFlatEntityMapsForSideEffect<
T extends AllMetadataName,
> = Pick<
AllFlatEntityMaps,
| MetadataToFlatEntityMapsKey<T>
| MetadataToFlatEntityMapsKey<MetadataSideEffectRelatedMetadataNames<T>>
>;
@@ -0,0 +1,14 @@
import { type AllMetadataName } from 'twenty-shared/metadata';
import { type ALL_METADATA_SIDE_EFFECT_COMPANION_METADATA_NAMES } from 'src/engine/metadata-modules/flat-entity/constant/all-metadata-side-effect-companion-metadata-names.constant';
import { type MetadataManyToOneRelatedMetadataNames } from 'src/engine/metadata-modules/flat-entity/types/metadata-many-to-one-related-metadata-names.type';
export type MetadataSideEffectCompanionMetadataNames<
T extends AllMetadataName,
> = T extends keyof typeof ALL_METADATA_SIDE_EFFECT_COMPANION_METADATA_NAMES
? (typeof ALL_METADATA_SIDE_EFFECT_COMPANION_METADATA_NAMES)[T][number]
: never;
export type MetadataSideEffectRelatedMetadataNames<T extends AllMetadataName> =
| MetadataManyToOneRelatedMetadataNames<T>
| MetadataSideEffectCompanionMetadataNames<T>;
@@ -0,0 +1,16 @@
import { type AllMetadataName } from 'twenty-shared/metadata';
import { ALL_METADATA_SIDE_EFFECT_COMPANION_METADATA_NAMES } from 'src/engine/metadata-modules/flat-entity/constant/all-metadata-side-effect-companion-metadata-names.constant';
import { type MetadataSideEffectCompanionMetadataNames } from 'src/engine/metadata-modules/flat-entity/types/metadata-side-effect-related-metadata-names.type';
export const getMetadataSideEffectCompanionNames = <T extends AllMetadataName>(
metadataName: T,
): MetadataSideEffectCompanionMetadataNames<T>[] => {
const companionMetadataNamesByMetadataName =
ALL_METADATA_SIDE_EFFECT_COMPANION_METADATA_NAMES as Partial<
Record<AllMetadataName, readonly AllMetadataName[]>
>;
return (companionMetadataNamesByMetadataName[metadataName] ??
[]) as MetadataSideEffectCompanionMetadataNames<T>[];
};
@@ -0,0 +1,51 @@
import { type AllMetadataName } from 'twenty-shared/metadata';
import { isDefined } from 'twenty-shared/utils';
import {
FlatEntityMapsException,
FlatEntityMapsExceptionCode,
} from 'src/engine/metadata-modules/flat-entity/exceptions/flat-entity-maps.exception';
import { type AllFlatEntityMaps } from 'src/engine/metadata-modules/flat-entity/types/all-flat-entity-maps.type';
import { type MetadataFlatEntity } from 'src/engine/metadata-modules/flat-entity/types/metadata-flat-entity.type';
import { getMetadataFlatEntityMapsKey } from 'src/engine/metadata-modules/flat-entity/utils/get-metadata-flat-entity-maps-key.util';
import { getSubFlatEntityMapsByApplicationIdsOrThrow } from 'src/engine/metadata-modules/flat-entity/utils/get-sub-flat-entity-maps-by-application-ids-or-throw.util';
import { pruneDanglingForeignKeyAggregatorsInAllFlatEntityMapsThroughMutation } from 'src/engine/metadata-modules/flat-entity/utils/prune-dangling-foreign-key-aggregators-in-all-flat-entity-maps-through-mutation.util';
export const getSubAllFlatEntityMapsByApplicationIdsOrThrow = ({
applicationIds,
metadataNames,
fromAllFlatEntityMaps,
}: {
applicationIds: string[];
metadataNames: AllMetadataName[];
fromAllFlatEntityMaps: Partial<AllFlatEntityMaps>;
}): Partial<AllFlatEntityMaps> => {
const subAllFlatEntityMaps: Partial<AllFlatEntityMaps> = {};
for (const metadataName of metadataNames) {
const flatEntityMapsKey = getMetadataFlatEntityMapsKey(metadataName);
const fromFlatEntityMaps = fromAllFlatEntityMaps[flatEntityMapsKey];
if (!isDefined(fromFlatEntityMaps)) {
throw new FlatEntityMapsException(
`Missing flat entity maps for metadata "${metadataName}" while building application-scoped slice`,
FlatEntityMapsExceptionCode.INTERNAL_SERVER_ERROR,
);
}
// @ts-expect-error Metadata flat entity maps cache key and metadataName colliding
subAllFlatEntityMaps[flatEntityMapsKey] =
getSubFlatEntityMapsByApplicationIdsOrThrow<
MetadataFlatEntity<typeof metadataName>
>({
applicationIds,
flatEntityMaps: fromFlatEntityMaps,
});
}
pruneDanglingForeignKeyAggregatorsInAllFlatEntityMapsThroughMutation({
allFlatEntityMapsToMutate: subAllFlatEntityMaps,
});
return subAllFlatEntityMaps;
};
@@ -0,0 +1,8 @@
import { type AllMetadataName } from 'twenty-shared/metadata';
import { type MetadataUniversalFlatEntity } from 'src/engine/metadata-modules/flat-entity/types/metadata-universal-flat-entity.type';
export const isSystemSideEffectFlatEntity = <T extends AllMetadataName>(
flatEntity: MetadataUniversalFlatEntity<T>,
): boolean =>
'isSystemSideEffect' in flatEntity && flatEntity.isSystemSideEffect === true;
@@ -74,35 +74,49 @@ export const pruneDanglingForeignKeyAggregatorsInAllFlatEntityMapsThroughMutatio
({
allFlatEntityMapsToMutate,
}: {
allFlatEntityMapsToMutate: AllFlatEntityMaps;
allFlatEntityMapsToMutate: Partial<AllFlatEntityMaps>;
}): void => {
const looseAllFlatEntityMaps =
allFlatEntityMapsToMutate as unknown as LooseAllFlatEntityMaps;
allFlatEntityMapsToMutate as unknown as Partial<LooseAllFlatEntityMaps>;
for (const metadataName of Object.values(ALL_METADATA_NAME)) {
const parentFlatEntityMaps =
looseAllFlatEntityMaps[getMetadataFlatEntityMapsKey(metadataName)];
if (!isDefined(parentFlatEntityMaps)) {
continue;
}
const oneToManyRelations = Object.values(
ALL_ONE_TO_MANY_METADATA_RELATIONS[metadataName],
) as OneToManyRelation[];
const aggregatorsWithChildIdentifiers: OneToManyAggregatorWithChildIdentifiers[] =
oneToManyRelations.filter(isDefined).map((relation) => ({
aggregatorProperty: relation.universalFlatEntityForeignKeyAggregator,
childUniversalIdentifiers: new Set(
Object.keys(
looseAllFlatEntityMaps[
getMetadataFlatEntityMapsKey(relation.metadataName)
].byUniversalIdentifier,
),
),
}));
oneToManyRelations.filter(isDefined).flatMap((relation) => {
const childFlatEntityMaps =
looseAllFlatEntityMaps[
getMetadataFlatEntityMapsKey(relation.metadataName)
];
const childUniversalIdentifiers = isDefined(childFlatEntityMaps)
? new Set(Object.keys(childFlatEntityMaps.byUniversalIdentifier))
: new Set<string>();
return [
{
aggregatorProperty:
relation.universalFlatEntityForeignKeyAggregator,
childUniversalIdentifiers,
},
];
});
if (aggregatorsWithChildIdentifiers.length === 0) {
continue;
}
const flatEntityByUniversalIdentifier =
looseAllFlatEntityMaps[getMetadataFlatEntityMapsKey(metadataName)]
.byUniversalIdentifier;
parentFlatEntityMaps.byUniversalIdentifier;
for (const [universalIdentifier, parentFlatEntity] of Object.entries(
flatEntityByUniversalIdentifier,
@@ -0,0 +1,94 @@
import { isDefined } from 'twenty-shared/utils';
import {
FlatEntityMapsException,
FlatEntityMapsExceptionCode,
} from 'src/engine/metadata-modules/flat-entity/exceptions/flat-entity-maps.exception';
import { type AllFlatEntityOperationRecordByMetadataName } from 'src/engine/metadata-modules/flat-entity/types/all-flat-entity-operation-record-by-metadata-name.type';
import { type AllFlatEntityOperationByMetadataName } from 'src/engine/metadata-modules/flat-entity/types/flat-entity-to-create-delete-update.type';
type GenericUniversalFlatEntity = { universalIdentifier: string };
const toRecordByUniversalIdentifierOrThrow = ({
flatEntities,
metadataName,
operation,
}: {
flatEntities: GenericUniversalFlatEntity[];
metadataName: string;
operation: 'flatEntityToCreate' | 'flatEntityToUpdate' | 'flatEntityToDelete';
}): Record<string, GenericUniversalFlatEntity> => {
const recordByUniversalIdentifier: Record<
string,
GenericUniversalFlatEntity
> = {};
const seenUniversalIdentifiers = new Set<string>();
for (const flatEntity of flatEntities) {
if (seenUniversalIdentifiers.has(flatEntity.universalIdentifier)) {
throw new FlatEntityMapsException(
`Duplicate universalIdentifier "${flatEntity.universalIdentifier}" in ${operation} for metadata "${metadataName}"`,
FlatEntityMapsExceptionCode.ENTITY_ALREADY_EXISTS,
);
}
seenUniversalIdentifiers.add(flatEntity.universalIdentifier);
recordByUniversalIdentifier[flatEntity.universalIdentifier] = flatEntity;
}
return recordByUniversalIdentifier;
};
// Compat bridge: array-based API callers are transpiled into the canonical record
// matrix so the side-effect engine and downstream from/to computation stay record-native.
// To be removed once every caller produces records directly.
export const transpileFlatEntityOperationArrayToRecord = (
allFlatEntityOperationByMetadataName: AllFlatEntityOperationByMetadataName,
): AllFlatEntityOperationRecordByMetadataName => {
const genericMatrix =
allFlatEntityOperationByMetadataName as unknown as Record<
string,
| {
flatEntityToCreate: GenericUniversalFlatEntity[];
flatEntityToUpdate: GenericUniversalFlatEntity[];
flatEntityToDelete: GenericUniversalFlatEntity[];
}
| undefined
>;
const recordMatrix: Record<
string,
{
flatEntityToCreate: Record<string, GenericUniversalFlatEntity>;
flatEntityToUpdate: Record<string, GenericUniversalFlatEntity>;
flatEntityToDelete: Record<string, GenericUniversalFlatEntity>;
}
> = {};
for (const metadataName of Object.keys(genericMatrix)) {
const operations = genericMatrix[metadataName];
if (!isDefined(operations)) {
continue;
}
recordMatrix[metadataName] = {
flatEntityToCreate: toRecordByUniversalIdentifierOrThrow({
flatEntities: operations.flatEntityToCreate,
metadataName,
operation: 'flatEntityToCreate',
}),
flatEntityToUpdate: toRecordByUniversalIdentifierOrThrow({
flatEntities: operations.flatEntityToUpdate,
metadataName,
operation: 'flatEntityToUpdate',
}),
flatEntityToDelete: toRecordByUniversalIdentifierOrThrow({
flatEntities: operations.flatEntityToDelete,
metadataName,
operation: 'flatEntityToDelete',
}),
};
}
return recordMatrix as unknown as AllFlatEntityOperationRecordByMetadataName;
};
@@ -19,7 +19,6 @@ import { findFlatEntityByIdInFlatEntityMaps } from 'src/engine/metadata-modules/
import { type FieldInputTranspilationResult } from 'src/engine/metadata-modules/flat-field-metadata/types/field-input-transpilation-result.type';
import { fromMorphRelationCreateFieldInputToFlatFieldMetadatas } from 'src/engine/metadata-modules/flat-field-metadata/utils/from-morph-relation-create-field-input-to-flat-field-metadatas.util';
import { fromRelationCreateFieldInputToFlatFieldMetadatas } from 'src/engine/metadata-modules/flat-field-metadata/utils/from-relation-create-field-input-to-flat-field-metadatas.util';
import { generateIndexForFlatFieldMetadata } from 'src/engine/metadata-modules/flat-field-metadata/utils/generate-index-for-flat-field-metadata.util';
import { getDefaultFlatFieldMetadata } from 'src/engine/metadata-modules/flat-field-metadata/utils/get-default-flat-field-metadata-from-create-field-input.util';
import { type UniversalFlatFieldMetadata } from 'src/engine/workspace-manager/workspace-migration/universal-flat-entity/types/universal-flat-field-metadata.type';
import { type UniversalFlatIndexMetadata } from 'src/engine/workspace-manager/workspace-migration/universal-flat-entity/types/universal-flat-index-metadata.type';
@@ -186,17 +185,6 @@ export const fromCreateFieldInputToFlatFieldMetadatasToCreate = async ({
case FieldMetadataType.RICH_TEXT:
case FieldMetadataType.ACTOR:
case FieldMetadataType.ARRAY: {
const indexMetadatas: UniversalFlatIndexMetadata[] = [];
if (commonFlatFieldMetadata.isUnique) {
indexMetadatas.push(
generateIndexForFlatFieldMetadata({
flatFieldMetadata: commonFlatFieldMetadata,
flatObjectMetadata: parentFlatObjectMetadata,
}),
);
}
return {
status: 'success',
result: {
@@ -206,7 +194,7 @@ export const fromCreateFieldInputToFlatFieldMetadatasToCreate = async ({
type: createFieldInput.type,
},
],
indexMetadatas,
indexMetadatas: [],
},
};
}
@@ -14,7 +14,9 @@ import { findFlatEntityByUniversalIdentifierOrThrow } from 'src/engine/metadata-
import { findManyFlatEntityByUniversalIdentifierInUniversalFlatEntityMapsOrThrow } from 'src/engine/metadata-modules/flat-entity/utils/find-many-flat-entity-by-universal-identifier-in-universal-flat-entity-maps-or-throw.util';
import { computeFlatFieldMetadataRelatedFlatFieldMetadata } from 'src/engine/metadata-modules/flat-field-metadata/utils/compute-flat-field-metadata-related-flat-field-metadata.util';
import { computeSearchFieldMetadataDeletionForDeletedFields } from 'src/engine/metadata-modules/flat-field-metadata/utils/compute-search-field-metadata-deletion-for-deleted-fields.util';
import { isMorphOrRelationFlatFieldMetadata } from 'src/engine/metadata-modules/flat-field-metadata/utils/is-morph-or-relation-flat-field-metadata.util';
import { type FlatIndexMetadata } from 'src/engine/metadata-modules/flat-index-metadata/types/flat-index-metadata.type';
import { isSystemUniqueFlatIndexMetadata } from 'src/engine/metadata-modules/flat-index-metadata/utils/is-system-unique-flat-index-metadata.util';
import { generateFlatIndexMetadataWithNameOrThrow } from 'src/engine/metadata-modules/index-metadata/utils/generate-flat-index.util';
import { belongsToTwentyStandardApp } from 'src/engine/metadata-modules/utils/belongs-to-twenty-standard-app.util';
import { type UniversalFlatFieldMetadata } from 'src/engine/workspace-manager/workspace-migration/universal-flat-entity/types/universal-flat-field-metadata.type';
@@ -95,10 +97,38 @@ export const fromDeleteFieldInputToFlatFieldMetadatasToDelete = ({
...relatedFlatFieldMetadataToDelete,
];
const isEngineOwnedBackingUniqueIndex = (
flatIndexMetadata: FlatIndexMetadata,
): boolean => {
if (
!isSystemUniqueFlatIndexMetadata(flatIndexMetadata) ||
flatIndexMetadata.flatIndexFieldMetadatas.length !== 1
) {
return false;
}
const backingFlatFieldMetadata = findFlatEntityByIdInFlatEntityMaps({
flatEntityId:
flatIndexMetadata.flatIndexFieldMetadatas[0].fieldMetadataId,
flatEntityMaps: existingFlatFieldMetadataMaps,
});
return (
isDefined(backingFlatFieldMetadata) &&
backingFlatFieldMetadata.isUnique === true &&
!isMorphOrRelationFlatFieldMetadata(backingFlatFieldMetadata)
);
};
const flatIndexMap = new Map<string, FlatIndexMetadata>();
const allFlatIndexes = Object.values(
existingFlatIndexMaps.byUniversalIdentifier,
).filter(isDefined);
)
.filter(isDefined)
.filter(
(flatIndexMetadata) =>
!isEngineOwnedBackingUniqueIndex(flatIndexMetadata),
);
for (const flatFieldMetadata of flatFieldMetadatasToDelete) {
allFlatIndexes.forEach((flatIndex) => {
@@ -150,7 +150,6 @@ export const fromUpdateFieldInputToFlatFieldMetadata = ({
toFlatFieldMetadata,
flatViewMaps,
flatViewFieldMaps,
flatApplication,
});
if (sideEffectResult.status === 'fail') {
@@ -0,0 +1,44 @@
import { IndexType } from 'src/engine/metadata-modules/index-metadata/types/indexType.types';
import { generateDeterministicFlatIndexMetadataOrThrow } from 'src/engine/metadata-modules/index-metadata/utils/generate-deterministic-flat-index.util';
import { type UniversalFlatFieldMetadata } from 'src/engine/workspace-manager/workspace-migration/universal-flat-entity/types/universal-flat-field-metadata.type';
import { type UniversalFlatIndexMetadata } from 'src/engine/workspace-manager/workspace-migration/universal-flat-entity/types/universal-flat-index-metadata.type';
import { type UniversalFlatObjectMetadata } from 'src/engine/workspace-manager/workspace-migration/universal-flat-entity/types/universal-flat-object-metadata.type';
type GenerateDeterministicIndexForFlatFieldMetadataOrThrowArgs = {
flatFieldMetadata: UniversalFlatFieldMetadata;
flatObjectMetadata: UniversalFlatObjectMetadata;
};
export const generateDeterministicIndexForFlatFieldMetadataOrThrow = ({
flatFieldMetadata,
flatObjectMetadata,
}: GenerateDeterministicIndexForFlatFieldMetadataOrThrowArgs): UniversalFlatIndexMetadata => {
const createdAt = new Date().toISOString();
return generateDeterministicFlatIndexMetadataOrThrow({
flatObjectMetadata,
objectFlatFieldMetadatas: [flatFieldMetadata],
flatIndex: {
createdAt,
universalFlatIndexFieldMetadatas: [
{
createdAt,
fieldMetadataUniversalIdentifier:
flatFieldMetadata.universalIdentifier,
order: 0,
subFieldName: null,
updatedAt: createdAt,
},
],
indexType: IndexType.BTREE,
indexWhereClause: null,
isCustom: true,
isUnique: flatFieldMetadata.isUnique ?? false,
isSystemSideEffect: true,
objectMetadataUniversalIdentifier: flatObjectMetadata.universalIdentifier,
updatedAt: createdAt,
applicationUniversalIdentifier:
flatFieldMetadata.applicationUniversalIdentifier,
},
});
};
@@ -1,6 +1,5 @@
import { type FromTo } from 'twenty-shared/types';
import { type FlatApplication } from 'src/engine/core-modules/application/types/flat-application.type';
import { type AllFlatEntityMaps } from 'src/engine/metadata-modules/flat-entity/types/all-flat-entity-maps.type';
import { type FieldInputTranspilationResult } from 'src/engine/metadata-modules/flat-field-metadata/types/field-input-transpilation-result.type';
import { type FlatFieldMetadata } from 'src/engine/metadata-modules/flat-field-metadata/types/flat-field-metadata.type';
@@ -36,9 +35,7 @@ type HandleFlatFieldMetadataUpdateSideEffectArgs = FromTo<
| 'flatViewGroupMaps'
| 'flatViewMaps'
| 'flatViewFieldMaps'
> & {
flatApplication: FlatApplication;
};
>;
export const FLAT_FIELD_METADATA_UPDATE_EMPTY_SIDE_EFFECTS: FlatFieldMetadataUpdateSideEffects =
{
@@ -65,7 +62,6 @@ export const handleFlatFieldMetadataUpdateSideEffect = ({
flatViewGroupMaps,
flatViewMaps,
flatViewFieldMaps,
flatApplication,
}: HandleFlatFieldMetadataUpdateSideEffectArgs): FieldInputTranspilationResult<FlatFieldMetadataUpdateSideEffects> => {
const sideEffectResult = structuredClone(
FLAT_FIELD_METADATA_UPDATE_EMPTY_SIDE_EFFECTS,
@@ -125,7 +121,6 @@ export const handleFlatFieldMetadataUpdateSideEffect = ({
flatIndexMaps,
flatObjectMetadataMaps,
flatFieldMetadataMaps,
flatApplication,
});
if (indexChangesSideEffectResult.status === 'fail') {
@@ -1,19 +1,12 @@
import { msg } from '@lingui/core/macro';
import { type FromTo } from 'twenty-shared/types';
import { isDefined } from 'twenty-shared/utils';
import { type FlatApplication } from 'src/engine/core-modules/application/types/flat-application.type';
import { FieldMetadataExceptionCode } from 'src/engine/metadata-modules/field-metadata/field-metadata.exception';
import { type AllFlatEntityMaps } from 'src/engine/metadata-modules/flat-entity/types/all-flat-entity-maps.type';
import { findFlatEntityByIdInFlatEntityMapsOrThrow } from 'src/engine/metadata-modules/flat-entity/utils/find-flat-entity-by-id-in-flat-entity-maps-or-throw.util';
import { type FieldInputTranspilationResult } from 'src/engine/metadata-modules/flat-field-metadata/types/field-input-transpilation-result.type';
import { type FlatFieldMetadata } from 'src/engine/metadata-modules/flat-field-metadata/types/flat-field-metadata.type';
import { findFieldRelatedIndexes } from 'src/engine/metadata-modules/flat-field-metadata/utils/find-field-related-index.util';
import { generateIndexForFlatFieldMetadata } from 'src/engine/metadata-modules/flat-field-metadata/utils/generate-index-for-flat-field-metadata.util';
import { isMorphOrRelationFlatFieldMetadata } from 'src/engine/metadata-modules/flat-field-metadata/utils/is-morph-or-relation-flat-field-metadata.util';
import { recomputeIndexOnFlatFieldMetadataNameUpdate } from 'src/engine/metadata-modules/flat-field-metadata/utils/recompute-index-on-flat-field-metadata-name-update.util';
import { type FlatIndexMetadata } from 'src/engine/metadata-modules/flat-index-metadata/types/flat-index-metadata.type';
import { type FlatObjectMetadata } from 'src/engine/metadata-modules/flat-object-metadata/types/flat-object-metadata.type';
import { isSystemUniqueFlatIndexMetadata } from 'src/engine/metadata-modules/flat-index-metadata/utils/is-system-unique-flat-index-metadata.util';
import { type UniversalFlatIndexMetadata } from 'src/engine/workspace-manager/workspace-migration/universal-flat-entity/types/universal-flat-index-metadata.type';
export type FieldMetadataUpdateIndexSideEffect = {
@@ -29,9 +22,8 @@ type FromToFlatFieldMetadataAndFlatEntityMaps = FromTo<
Pick<
AllFlatEntityMaps,
'flatObjectMetadataMaps' | 'flatFieldMetadataMaps' | 'flatIndexMaps'
> & {
flatApplication: FlatApplication;
};
>;
const FIELD_METADATA_UPDATE_INDEX_SIDE_EFFECT: FieldMetadataUpdateIndexSideEffect =
{
flatIndexMetadatasToUpdate: [],
@@ -45,14 +37,8 @@ export const handleIndexChangesDuringFieldUpdate = ({
flatIndexMaps,
flatObjectMetadataMaps,
flatFieldMetadataMaps,
flatApplication,
}: FromToFlatFieldMetadataAndFlatEntityMaps): FieldInputTranspilationResult<FieldMetadataUpdateIndexSideEffect> => {
if (
!hasIndexRelevantChanges({
fromFlatFieldMetadata,
toFlatFieldMetadata,
})
) {
if (fromFlatFieldMetadata.name === toFlatFieldMetadata.name) {
return {
status: 'success',
result: FIELD_METADATA_UPDATE_INDEX_SIDE_EFFECT,
@@ -68,119 +54,17 @@ export const handleIndexChangesDuringFieldUpdate = ({
flatFieldMetadata: fromFlatFieldMetadata,
flatObjectMetadata,
flatIndexMaps,
});
}).filter(
(flatIndexMetadata) => !isSystemUniqueFlatIndexMetadata(flatIndexMetadata),
);
if (relatedIndexes.length === 0) {
return handleNoExistingIndexes({
toFlatFieldMetadata,
flatObjectMetadata,
});
}
return handleExistingIndexes({
toFlatFieldMetadata,
fromFlatFieldMetadata,
relatedIndexes,
flatObjectMetadata,
flatFieldMetadataMaps,
flatApplication,
});
};
const hasIndexRelevantChanges = ({
fromFlatFieldMetadata,
toFlatFieldMetadata,
}: FromTo<FlatFieldMetadata, 'flatFieldMetadata'>): boolean =>
fromFlatFieldMetadata.name !== toFlatFieldMetadata.name ||
fromFlatFieldMetadata.isUnique !== toFlatFieldMetadata.isUnique;
const handleNoExistingIndexes = ({
toFlatFieldMetadata,
flatObjectMetadata,
}: {
toFlatFieldMetadata: FlatFieldMetadata;
flatObjectMetadata: FlatObjectMetadata;
}): FieldInputTranspilationResult<FieldMetadataUpdateIndexSideEffect> => {
if (!toFlatFieldMetadata.isUnique) {
return {
status: 'success',
result: FIELD_METADATA_UPDATE_INDEX_SIDE_EFFECT,
};
}
const newIndex = generateIndexForFlatFieldMetadata({
flatFieldMetadata: toFlatFieldMetadata,
flatObjectMetadata,
});
return {
status: 'success',
result: {
...FIELD_METADATA_UPDATE_INDEX_SIDE_EFFECT,
flatIndexMetadatasToCreate: [newIndex],
},
};
};
const handleExistingIndexes = ({
fromFlatFieldMetadata,
toFlatFieldMetadata,
relatedIndexes,
flatObjectMetadata,
flatFieldMetadataMaps,
flatApplication,
}: {
relatedIndexes: FlatIndexMetadata[];
flatObjectMetadata: FlatObjectMetadata;
flatFieldMetadataMaps: AllFlatEntityMaps['flatFieldMetadataMaps'];
} & FromTo<FlatFieldMetadata, 'flatFieldMetadata'> & {
flatApplication: FlatApplication;
}): FieldInputTranspilationResult<FieldMetadataUpdateIndexSideEffect> => {
if (
toFlatFieldMetadata.isUnique === false &&
!isMorphOrRelationFlatFieldMetadata(fromFlatFieldMetadata)
) {
const expectedUniqueIndex = generateIndexForFlatFieldMetadata({
flatFieldMetadata: {
...fromFlatFieldMetadata,
isUnique: true,
},
flatObjectMetadata,
});
const uniqueIndexToDelete = relatedIndexes.find(
(index) => index.name === expectedUniqueIndex.name,
);
if (
isDefined(uniqueIndexToDelete) &&
((isDefined(uniqueIndexToDelete.applicationId) &&
uniqueIndexToDelete.applicationId !== flatApplication.id) ||
!uniqueIndexToDelete.isCustom)
) {
return {
status: 'fail',
errors: [
{
code: FieldMetadataExceptionCode.INVALID_FIELD_INPUT,
message:
'Cannot delete unique index that have not been created by the workspace custom application',
userFriendlyMessage: msg`Cannot delete unique index that have not been created by the workspace custom application`,
},
],
};
}
return {
status: 'success',
result: {
...FIELD_METADATA_UPDATE_INDEX_SIDE_EFFECT,
flatIndexMetadatasToDelete: uniqueIndexToDelete
? [uniqueIndexToDelete]
: [],
},
};
}
const updatedIndexes = recomputeIndexOnFlatFieldMetadataNameUpdate({
flatFieldMetadataMaps,
flatObjectMetadata,
@@ -0,0 +1,7 @@
import { type UniversalFlatFieldMetadata } from 'src/engine/workspace-manager/workspace-migration/universal-flat-entity/types/universal-flat-field-metadata.type';
// The primary key already enforces uniqueness via its constraint, so it must not
// spawn a redundant application-managed unique index.
export const isPrimaryKeyFlatFieldMetadata = (
flatFieldMetadata: Pick<UniversalFlatFieldMetadata, 'name'>,
): boolean => flatFieldMetadata.name === 'id';
@@ -0,0 +1,6 @@
export const isSystemUniqueFlatIndexMetadata = (flatIndexMetadata: {
isSystemSideEffect: boolean;
isUnique: boolean;
}): boolean =>
flatIndexMetadata.isSystemSideEffect === true &&
flatIndexMetadata.isUnique === true;
@@ -0,0 +1,96 @@
import { compositeTypeDefinitions, RelationType } from 'twenty-shared/types';
import { isDefined } from 'twenty-shared/utils';
import { computeCompositeColumnName } from 'src/engine/metadata-modules/field-metadata/utils/compute-column-name.util';
import { computeMorphOrRelationFieldJoinColumnName } from 'src/engine/metadata-modules/field-metadata/utils/compute-morph-or-relation-field-join-column-name.util';
import { isCompositeFieldMetadataType } from 'src/engine/metadata-modules/field-metadata/utils/is-composite-field-metadata-type.util';
import {
FlatEntityMapsException,
FlatEntityMapsExceptionCode,
} from 'src/engine/metadata-modules/flat-entity/exceptions/flat-entity-maps.exception';
import { isMorphOrRelationUniversalFlatFieldMetadata } from 'src/engine/metadata-modules/flat-field-metadata/utils/is-morph-or-relation-flat-field-metadata.util';
import { generateDeterministicIndexName } from 'src/engine/metadata-modules/index-metadata/utils/generate-deterministic-index-name';
import { type UniversalFlatFieldMetadata } from 'src/engine/workspace-manager/workspace-migration/universal-flat-entity/types/universal-flat-field-metadata.type';
import { type UniversalFlatIndexFieldMetadata } from 'src/engine/workspace-manager/workspace-migration/universal-flat-entity/types/universal-flat-index-metadata.type';
import { type UniversalFlatObjectMetadata } from 'src/engine/workspace-manager/workspace-migration/universal-flat-entity/types/universal-flat-object-metadata.type';
type ComputeFlatIndexNameArgs = {
flatObjectMetadata: UniversalFlatObjectMetadata;
objectFlatFieldMetadatas: UniversalFlatFieldMetadata[];
indexFields: Pick<
UniversalFlatIndexFieldMetadata,
'order' | 'fieldMetadataUniversalIdentifier' | 'subFieldName'
>[];
isUnique: boolean;
indexWhereClause: string | null;
};
export const computeFlatIndexNameOrThrow = ({
flatObjectMetadata,
objectFlatFieldMetadatas,
indexFields,
isUnique,
indexWhereClause,
}: ComputeFlatIndexNameArgs): string => {
const orderedIndexColumnNames = [...indexFields]
.sort((a, b) => a.order - b.order)
.map((flatIndexField) => {
const relatedFlatFieldMetadata = objectFlatFieldMetadatas.find(
(flatFieldMetadata) =>
flatFieldMetadata.universalIdentifier ===
flatIndexField.fieldMetadataUniversalIdentifier,
);
if (!isDefined(relatedFlatFieldMetadata)) {
throw new FlatEntityMapsException(
'Could not find flat index field related field in cache',
FlatEntityMapsExceptionCode.ENTITY_NOT_FOUND,
);
}
if (
isCompositeFieldMetadataType(relatedFlatFieldMetadata.type) &&
isDefined(flatIndexField.subFieldName)
) {
const property = compositeTypeDefinitions
.get(relatedFlatFieldMetadata.type)
?.properties.find(
(compositeProperty) =>
compositeProperty.name === flatIndexField.subFieldName,
);
if (!isDefined(property)) {
throw new FlatEntityMapsException(
`Composite sub-field "${flatIndexField.subFieldName}" not found on ${relatedFlatFieldMetadata.name}`,
FlatEntityMapsExceptionCode.ENTITY_NOT_FOUND,
);
}
return computeCompositeColumnName(
{
name: relatedFlatFieldMetadata.name,
type: relatedFlatFieldMetadata.type,
},
property,
);
}
const isManyToOneRelation =
isMorphOrRelationUniversalFlatFieldMetadata(relatedFlatFieldMetadata) &&
relatedFlatFieldMetadata.universalSettings?.relationType ===
RelationType.MANY_TO_ONE;
return isManyToOneRelation
? computeMorphOrRelationFieldJoinColumnName({
name: relatedFlatFieldMetadata.name,
})
: relatedFlatFieldMetadata.name;
});
return generateDeterministicIndexName({
flatObjectMetadata,
orderedIndexColumnNames,
isUnique,
indexWhereClause,
});
};
@@ -0,0 +1,56 @@
import { getIndexUniversalIdentifier } from 'twenty-shared/application';
import { computeFlatIndexNameOrThrow } from 'src/engine/metadata-modules/index-metadata/utils/compute-flat-index-name.util';
import { type UniversalFlatFieldMetadata } from 'src/engine/workspace-manager/workspace-migration/universal-flat-entity/types/universal-flat-field-metadata.type';
import {
type UniversalFlatIndexFieldMetadata,
type UniversalFlatIndexMetadata,
} from 'src/engine/workspace-manager/workspace-migration/universal-flat-entity/types/universal-flat-index-metadata.type';
import { type UniversalFlatObjectMetadata } from 'src/engine/workspace-manager/workspace-migration/universal-flat-entity/types/universal-flat-object-metadata.type';
type GenerateDeterministicFlatIndexArgs = {
flatObjectMetadata: UniversalFlatObjectMetadata;
objectFlatFieldMetadatas: UniversalFlatFieldMetadata[];
flatIndex: Omit<
UniversalFlatIndexMetadata,
'name' | 'universalIdentifier' | 'universalFlatIndexFieldMetadatas'
> & {
universalFlatIndexFieldMetadatas: Omit<
UniversalFlatIndexFieldMetadata,
'indexMetadataUniversalIdentifier'
>[];
};
};
export const generateDeterministicFlatIndexMetadataOrThrow = ({
flatObjectMetadata,
objectFlatFieldMetadatas,
flatIndex,
}: GenerateDeterministicFlatIndexArgs): UniversalFlatIndexMetadata => {
const name = computeFlatIndexNameOrThrow({
flatObjectMetadata,
objectFlatFieldMetadatas,
indexFields: flatIndex.universalFlatIndexFieldMetadatas,
isUnique: flatIndex.isUnique,
indexWhereClause: flatIndex.indexWhereClause,
});
const universalIdentifier = getIndexUniversalIdentifier({
applicationUniversalIdentifier: flatIndex.applicationUniversalIdentifier,
objectUniversalIdentifier: flatObjectMetadata.universalIdentifier,
name,
});
return {
...flatIndex,
name,
universalIdentifier,
universalFlatIndexFieldMetadatas:
flatIndex.universalFlatIndexFieldMetadatas.map(
(universalFlatIndexFieldMetadata) => ({
...universalFlatIndexFieldMetadata,
indexMetadataUniversalIdentifier: universalIdentifier,
}),
),
};
};
@@ -13,7 +13,7 @@ type GenerateDeterministicIndexNameArgs = {
// coexist; the unique-name constraint must not block that).
indexWhereClause?: string | null;
};
export const generateDeterministicIndexNameV2 = ({
export const generateDeterministicIndexName = ({
orderedIndexColumnNames,
flatObjectMetadata,
isUnique = false,
@@ -1,15 +1,4 @@
import { compositeTypeDefinitions, RelationType } from 'twenty-shared/types';
import { isDefined } from 'twenty-shared/utils';
import { computeCompositeColumnName } from 'src/engine/metadata-modules/field-metadata/utils/compute-column-name.util';
import { computeMorphOrRelationFieldJoinColumnName } from 'src/engine/metadata-modules/field-metadata/utils/compute-morph-or-relation-field-join-column-name.util';
import { isCompositeFieldMetadataType } from 'src/engine/metadata-modules/field-metadata/utils/is-composite-field-metadata-type.util';
import {
FlatEntityMapsException,
FlatEntityMapsExceptionCode,
} from 'src/engine/metadata-modules/flat-entity/exceptions/flat-entity-maps.exception';
import { isMorphOrRelationUniversalFlatFieldMetadata } from 'src/engine/metadata-modules/flat-field-metadata/utils/is-morph-or-relation-flat-field-metadata.util';
import { generateDeterministicIndexNameV2 } from 'src/engine/metadata-modules/index-metadata/utils/generate-deterministic-index-name-v2';
import { computeFlatIndexNameOrThrow } from 'src/engine/metadata-modules/index-metadata/utils/compute-flat-index-name.util';
import { type UniversalFlatFieldMetadata } from 'src/engine/workspace-manager/workspace-migration/universal-flat-entity/types/universal-flat-field-metadata.type';
import { type UniversalFlatIndexMetadata } from 'src/engine/workspace-manager/workspace-migration/universal-flat-entity/types/universal-flat-index-metadata.type';
import { type UniversalFlatObjectMetadata } from 'src/engine/workspace-manager/workspace-migration/universal-flat-entity/types/universal-flat-object-metadata.type';
@@ -25,69 +14,10 @@ export const generateFlatIndexMetadataWithNameOrThrow = ({
objectFlatFieldMetadatas,
flatIndex,
}: GenerateFlatIndexArgs): UniversalFlatIndexMetadata => {
const orderedIndexColumnNames = flatIndex.universalFlatIndexFieldMetadatas
.sort((a, b) => a.order - b.order)
.map((flatIndexField) => {
const relatedFlatFieldMetadata = objectFlatFieldMetadatas.find(
(flatFieldMetadata) =>
flatFieldMetadata.universalIdentifier ===
flatIndexField.fieldMetadataUniversalIdentifier,
);
if (!isDefined(relatedFlatFieldMetadata)) {
throw new FlatEntityMapsException(
'Could not find flat index field related field in cache',
FlatEntityMapsExceptionCode.ENTITY_NOT_FOUND,
);
}
// Composite parent with an explicit sub-field → single sub-column.
// Composite parent without sub-field falls through to the legacy
// scalar branch below, which produces a deterministic name based on
// the parent name (the runner handles the multi-column SQL expansion
// via isIncludedInUniqueConstraint).
if (
isCompositeFieldMetadataType(relatedFlatFieldMetadata.type) &&
isDefined(flatIndexField.subFieldName)
) {
const property = compositeTypeDefinitions
.get(relatedFlatFieldMetadata.type)
?.properties.find(
(compositeProperty) =>
compositeProperty.name === flatIndexField.subFieldName,
);
if (!isDefined(property)) {
throw new FlatEntityMapsException(
`Composite sub-field "${flatIndexField.subFieldName}" not found on ${relatedFlatFieldMetadata.name}`,
FlatEntityMapsExceptionCode.ENTITY_NOT_FOUND,
);
}
return computeCompositeColumnName(
{
name: relatedFlatFieldMetadata.name,
type: relatedFlatFieldMetadata.type,
},
property,
);
}
const isManyToOneRelation =
isMorphOrRelationUniversalFlatFieldMetadata(relatedFlatFieldMetadata) &&
relatedFlatFieldMetadata.universalSettings?.relationType ===
RelationType.MANY_TO_ONE;
return isManyToOneRelation
? computeMorphOrRelationFieldJoinColumnName({
name: relatedFlatFieldMetadata.name,
})
: relatedFlatFieldMetadata.name;
});
const name = generateDeterministicIndexNameV2({
const name = computeFlatIndexNameOrThrow({
flatObjectMetadata,
orderedIndexColumnNames,
objectFlatFieldMetadatas,
indexFields: flatIndex.universalFlatIndexFieldMetadatas,
isUnique: flatIndex.isUnique,
indexWhereClause: flatIndex.indexWhereClause,
});
@@ -0,0 +1,2 @@
export const METADATA_SIDE_EFFECT_HANDLER_METADATA_KEY =
'METADATA_SIDE_EFFECT_HANDLER_METADATA_KEY';
@@ -0,0 +1,4 @@
export enum MetadataSideEffectExceptionCode {
RESERVED_SYSTEM_UNIVERSAL_IDENTIFIER = 'RESERVED_SYSTEM_UNIVERSAL_IDENTIFIER',
SIDE_EFFECT_PARENT_METADATA_NOT_FOUND = 'SIDE_EFFECT_PARENT_METADATA_NOT_FOUND',
}
@@ -0,0 +1,74 @@
import { Injectable } from '@nestjs/common';
import { isDefined } from 'twenty-shared/utils';
import { generateDeterministicIndexForFlatFieldMetadataOrThrow } from 'src/engine/metadata-modules/flat-field-metadata/utils/generate-deterministic-index-for-flat-field-metadata-or-throw.util';
import { isMorphOrRelationUniversalFlatFieldMetadata } from 'src/engine/metadata-modules/flat-field-metadata/utils/is-morph-or-relation-flat-field-metadata.util';
import { isPrimaryKeyFlatFieldMetadata } from 'src/engine/metadata-modules/flat-field-metadata/utils/is-primary-key-flat-field-metadata.util';
import { buildFieldSideEffectParentNotFoundFailure } from 'src/engine/metadata-modules/metadata-side-effect/handlers/field-metadata/utils/build-field-side-effect-parent-not-found-failure.util';
import { resolveParentFlatObjectMetadataAfterStateForFieldSideEffect } from 'src/engine/metadata-modules/metadata-side-effect/handlers/field-metadata/utils/resolve-parent-flat-object-metadata-after-state-for-field-side-effect.util';
import {
type BuildSideEffectsArgs,
MetadataSideEffectHandler,
} from 'src/engine/metadata-modules/metadata-side-effect/interfaces/base-metadata-side-effect-handler.service';
import { type MetadataSideEffectResult } from 'src/engine/metadata-modules/metadata-side-effect/types/metadata-side-effect-result.type';
@Injectable()
export class FieldUniqueBackingIndexOnCreateSideEffectHandlerService extends MetadataSideEffectHandler(
{
operation: 'create',
metadataName: 'fieldMetadata',
name: 'fieldUniqueBackingIndexOnCreate',
description:
'When a unique scalar field is created, generate the single-field UNIQUE index that enforces its uniqueness constraint at the database level.',
},
) {
buildSideEffects({
flatEntity: flatFieldMetadata,
allFlatEntityOperationRecordByMetadataName,
relatedFlatEntityMaps,
}: BuildSideEffectsArgs<'fieldMetadata'>): MetadataSideEffectResult {
if (
flatFieldMetadata.isUnique !== true ||
isMorphOrRelationUniversalFlatFieldMetadata(flatFieldMetadata)
) {
return { status: 'noop' };
}
if (isPrimaryKeyFlatFieldMetadata(flatFieldMetadata)) {
return { status: 'noop' };
}
const parentFlatObjectMetadata =
resolveParentFlatObjectMetadataAfterStateForFieldSideEffect({
objectMetadataUniversalIdentifier:
flatFieldMetadata.objectMetadataUniversalIdentifier,
allFlatEntityOperationRecordByMetadataName,
relatedFlatEntityMaps,
});
if (!isDefined(parentFlatObjectMetadata)) {
return buildFieldSideEffectParentNotFoundFailure({
flatFieldMetadata,
operation: 'create',
});
}
const flatIndexMetadata =
generateDeterministicIndexForFlatFieldMetadataOrThrow({
flatFieldMetadata,
flatObjectMetadata: parentFlatObjectMetadata,
});
return {
status: 'success',
operations: {
index: {
flatEntityToCreate: {
[flatIndexMetadata.universalIdentifier]: flatIndexMetadata,
},
},
},
};
}
}
@@ -0,0 +1,74 @@
import { Injectable } from '@nestjs/common';
import { isDefined } from 'twenty-shared/utils';
import { generateDeterministicIndexForFlatFieldMetadataOrThrow } from 'src/engine/metadata-modules/flat-field-metadata/utils/generate-deterministic-index-for-flat-field-metadata-or-throw.util';
import { isMorphOrRelationUniversalFlatFieldMetadata } from 'src/engine/metadata-modules/flat-field-metadata/utils/is-morph-or-relation-flat-field-metadata.util';
import { buildFieldSideEffectParentNotFoundFailure } from 'src/engine/metadata-modules/metadata-side-effect/handlers/field-metadata/utils/build-field-side-effect-parent-not-found-failure.util';
import {
type BuildSideEffectsArgs,
MetadataSideEffectHandler,
} from 'src/engine/metadata-modules/metadata-side-effect/interfaces/base-metadata-side-effect-handler.service';
import { type MetadataSideEffectResult } from 'src/engine/metadata-modules/metadata-side-effect/types/metadata-side-effect-result.type';
@Injectable()
export class FieldUniqueBackingIndexOnDeleteSideEffectHandlerService extends MetadataSideEffectHandler(
{
operation: 'delete',
metadataName: 'fieldMetadata',
name: 'fieldUniqueBackingIndexOnDelete',
description:
'When a unique scalar field is deleted, cascade-delete the single-field UNIQUE index that backed its uniqueness constraint.',
},
) {
buildSideEffects({
flatEntity: flatFieldMetadata,
relatedFlatEntityMaps,
}: BuildSideEffectsArgs<'fieldMetadata'>): MetadataSideEffectResult {
if (
flatFieldMetadata.isUnique !== true ||
isMorphOrRelationUniversalFlatFieldMetadata(flatFieldMetadata)
) {
return { status: 'noop' };
}
const parentFlatObjectMetadata =
relatedFlatEntityMaps.flatObjectMetadataMaps.byUniversalIdentifier[
flatFieldMetadata.objectMetadataUniversalIdentifier
];
if (!isDefined(parentFlatObjectMetadata)) {
return buildFieldSideEffectParentNotFoundFailure({
flatFieldMetadata,
operation: 'delete',
});
}
const flatIndexMetadataToDelete =
generateDeterministicIndexForFlatFieldMetadataOrThrow({
flatFieldMetadata,
flatObjectMetadata: parentFlatObjectMetadata,
});
const indexExistsInWorkspace = isDefined(
relatedFlatEntityMaps.flatIndexMaps.byUniversalIdentifier[
flatIndexMetadataToDelete.universalIdentifier
],
);
if (!indexExistsInWorkspace) {
return { status: 'noop' };
}
return {
status: 'success',
operations: {
index: {
flatEntityToDelete: {
[flatIndexMetadataToDelete.universalIdentifier]:
flatIndexMetadataToDelete,
},
},
},
};
}
}
@@ -0,0 +1,149 @@
import { Injectable } from '@nestjs/common';
import { isDefined } from 'twenty-shared/utils';
import { generateDeterministicIndexForFlatFieldMetadataOrThrow } from 'src/engine/metadata-modules/flat-field-metadata/utils/generate-deterministic-index-for-flat-field-metadata-or-throw.util';
import { isMorphOrRelationUniversalFlatFieldMetadata } from 'src/engine/metadata-modules/flat-field-metadata/utils/is-morph-or-relation-flat-field-metadata.util';
import { isPrimaryKeyFlatFieldMetadata } from 'src/engine/metadata-modules/flat-field-metadata/utils/is-primary-key-flat-field-metadata.util';
import { buildFieldSideEffectParentNotFoundFailure } from 'src/engine/metadata-modules/metadata-side-effect/handlers/field-metadata/utils/build-field-side-effect-parent-not-found-failure.util';
import { resolveParentFlatObjectMetadataAfterStateForFieldSideEffect } from 'src/engine/metadata-modules/metadata-side-effect/handlers/field-metadata/utils/resolve-parent-flat-object-metadata-after-state-for-field-side-effect.util';
import {
type BuildSideEffectsArgs,
MetadataSideEffectHandler,
} from 'src/engine/metadata-modules/metadata-side-effect/interfaces/base-metadata-side-effect-handler.service';
import { type MetadataSideEffectResult } from 'src/engine/metadata-modules/metadata-side-effect/types/metadata-side-effect-result.type';
@Injectable()
export class FieldUniqueBackingIndexOnUpdateSideEffectHandlerService extends MetadataSideEffectHandler(
{
operation: 'update',
metadataName: 'fieldMetadata',
name: 'fieldUniqueBackingIndexOnUpdate',
description:
"Keep a unique scalar field's backing UNIQUE index in sync when its `isUnique` flag flips or the field is renamed (drop the stale index and recreate the deterministic one).",
},
) {
buildSideEffects({
flatEntity: flatFieldMetadata,
allFlatEntityOperationRecordByMetadataName,
relatedFlatEntityMaps,
}: BuildSideEffectsArgs<'fieldMetadata'>): MetadataSideEffectResult {
if (isMorphOrRelationUniversalFlatFieldMetadata(flatFieldMetadata)) {
return { status: 'noop' };
}
if (isPrimaryKeyFlatFieldMetadata(flatFieldMetadata)) {
return { status: 'noop' };
}
const existingFlatFieldMetadata =
relatedFlatEntityMaps.flatFieldMetadataMaps.byUniversalIdentifier[
flatFieldMetadata.universalIdentifier
];
if (!isDefined(existingFlatFieldMetadata)) {
return { status: 'noop' };
}
const wasRenamed =
existingFlatFieldMetadata.name !== flatFieldMetadata.name;
const uniquenessHasFlipped =
existingFlatFieldMetadata.isUnique !== flatFieldMetadata.isUnique;
const backingIndexMustFollowRename =
existingFlatFieldMetadata.isUnique === true &&
flatFieldMetadata.isUnique === true &&
wasRenamed;
if (!uniquenessHasFlipped && !backingIndexMustFollowRename) {
return { status: 'noop' };
}
const existingFlatObjectMetadata =
relatedFlatEntityMaps.flatObjectMetadataMaps.byUniversalIdentifier[
flatFieldMetadata.objectMetadataUniversalIdentifier
];
const optimisticFlatObjectMetadata =
resolveParentFlatObjectMetadataAfterStateForFieldSideEffect({
objectMetadataUniversalIdentifier:
flatFieldMetadata.objectMetadataUniversalIdentifier,
allFlatEntityOperationRecordByMetadataName,
relatedFlatEntityMaps,
});
const backingIndexMustBeDeleted =
existingFlatFieldMetadata.isUnique === true;
const backingIndexMustBeCreated = flatFieldMetadata.isUnique === true;
if (
(backingIndexMustBeDeleted && !isDefined(existingFlatObjectMetadata)) ||
(backingIndexMustBeCreated && !isDefined(optimisticFlatObjectMetadata))
) {
return buildFieldSideEffectParentNotFoundFailure({
flatFieldMetadata,
operation: 'update',
});
}
const previousFlatIndexMetadata =
backingIndexMustBeDeleted && isDefined(existingFlatObjectMetadata)
? generateDeterministicIndexForFlatFieldMetadataOrThrow({
flatFieldMetadata: {
...flatFieldMetadata,
name: existingFlatFieldMetadata.name,
isUnique: true,
},
flatObjectMetadata: existingFlatObjectMetadata,
})
: undefined;
const flatIndexMetadataToDelete =
isDefined(previousFlatIndexMetadata) &&
isDefined(
relatedFlatEntityMaps.flatIndexMaps.byUniversalIdentifier[
previousFlatIndexMetadata.universalIdentifier
],
)
? previousFlatIndexMetadata
: undefined;
const flatIndexMetadataToCreate =
backingIndexMustBeCreated && isDefined(optimisticFlatObjectMetadata)
? generateDeterministicIndexForFlatFieldMetadataOrThrow({
flatFieldMetadata: { ...flatFieldMetadata, isUnique: true },
flatObjectMetadata: optimisticFlatObjectMetadata,
})
: undefined;
if (
!isDefined(flatIndexMetadataToCreate) &&
!isDefined(flatIndexMetadataToDelete)
) {
return { status: 'noop' };
}
return {
status: 'success',
operations: {
index: {
...(isDefined(flatIndexMetadataToCreate)
? {
flatEntityToCreate: {
[flatIndexMetadataToCreate.universalIdentifier]:
flatIndexMetadataToCreate,
},
}
: {}),
...(isDefined(flatIndexMetadataToDelete)
? {
flatEntityToDelete: {
[flatIndexMetadataToDelete.universalIdentifier]:
flatIndexMetadataToDelete,
},
}
: {}),
},
},
};
}
}
@@ -0,0 +1,31 @@
import { msg, t } from '@lingui/core/macro';
import { type AllMetadataName } from 'twenty-shared/metadata';
import { type MetadataFlatEntity } from 'src/engine/metadata-modules/flat-entity/types/metadata-flat-entity.type';
import { type MetadataUniversalFlatEntity } from 'src/engine/metadata-modules/flat-entity/types/metadata-universal-flat-entity.type';
import { type WorkspaceMigrationActionType } from 'src/engine/metadata-modules/flat-entity/types/metadata-workspace-migration-action.type';
import { MetadataSideEffectExceptionCode } from 'src/engine/metadata-modules/metadata-side-effect/exceptions/metadata-side-effect-exception-code';
import { type MetadataSideEffectFailure } from 'src/engine/metadata-modules/metadata-side-effect/types/metadata-side-effect-result.type';
export const buildFieldSideEffectParentNotFoundFailure = ({
flatFieldMetadata,
operation,
}: {
flatFieldMetadata: MetadataUniversalFlatEntity<'fieldMetadata'>;
operation: WorkspaceMigrationActionType;
}): MetadataSideEffectFailure => ({
status: 'fail',
type: operation,
metadataName: 'fieldMetadata',
flatEntityMinimalInformation: {
universalIdentifier: flatFieldMetadata.universalIdentifier,
name: flatFieldMetadata.name,
} as Partial<MetadataFlatEntity<AllMetadataName>>,
errors: [
{
code: MetadataSideEffectExceptionCode.SIDE_EFFECT_PARENT_METADATA_NOT_FOUND,
message: t`Could not resolve parent object metadata "${flatFieldMetadata.objectMetadataUniversalIdentifier}" for field unique index side effect`,
userFriendlyMessage: msg`This field references an object that could not be found`,
},
],
});
@@ -0,0 +1,26 @@
import { type AllFlatEntityOperationRecordByMetadataName } from 'src/engine/metadata-modules/flat-entity/types/all-flat-entity-operation-record-by-metadata-name.type';
import { type MetadataFlatEntityAndRelatedFlatEntityMapsForSideEffect } from 'src/engine/metadata-modules/flat-entity/types/metadata-flat-entity-and-related-flat-entity-maps-for-side-effect.type';
import { type UniversalFlatObjectMetadata } from 'src/engine/workspace-manager/workspace-migration/universal-flat-entity/types/universal-flat-object-metadata.type';
export const resolveParentFlatObjectMetadataAfterStateForFieldSideEffect = ({
objectMetadataUniversalIdentifier,
allFlatEntityOperationRecordByMetadataName,
relatedFlatEntityMaps,
}: {
objectMetadataUniversalIdentifier: string;
allFlatEntityOperationRecordByMetadataName: AllFlatEntityOperationRecordByMetadataName;
relatedFlatEntityMaps: MetadataFlatEntityAndRelatedFlatEntityMapsForSideEffect<'fieldMetadata'>;
}): UniversalFlatObjectMetadata | undefined => {
const pendingFlatObjectMetadata =
allFlatEntityOperationRecordByMetadataName.objectMetadata
?.flatEntityToUpdate[objectMetadataUniversalIdentifier] ??
allFlatEntityOperationRecordByMetadataName.objectMetadata
?.flatEntityToCreate[objectMetadataUniversalIdentifier];
return (
pendingFlatObjectMetadata ??
relatedFlatEntityMaps.flatObjectMetadataMaps.byUniversalIdentifier[
objectMetadataUniversalIdentifier
]
);
};
@@ -0,0 +1,14 @@
import { Module } from '@nestjs/common';
import { FieldUniqueBackingIndexOnCreateSideEffectHandlerService } from 'src/engine/metadata-modules/metadata-side-effect/handlers/field-metadata/services/field-unique-backing-index-on-create-side-effect-handler.service';
import { FieldUniqueBackingIndexOnDeleteSideEffectHandlerService } from 'src/engine/metadata-modules/metadata-side-effect/handlers/field-metadata/services/field-unique-backing-index-on-delete-side-effect-handler.service';
import { FieldUniqueBackingIndexOnUpdateSideEffectHandlerService } from 'src/engine/metadata-modules/metadata-side-effect/handlers/field-metadata/services/field-unique-backing-index-on-update-side-effect-handler.service';
@Module({
providers: [
FieldUniqueBackingIndexOnCreateSideEffectHandlerService,
FieldUniqueBackingIndexOnUpdateSideEffectHandlerService,
FieldUniqueBackingIndexOnDeleteSideEffectHandlerService,
],
})
export class MetadataSideEffectHandlersModule {}
@@ -0,0 +1,61 @@
import { SetMetadata } from '@nestjs/common';
import { type AllMetadataName } from 'twenty-shared/metadata';
import { type AllFlatEntityOperationRecordByMetadataName } from 'src/engine/metadata-modules/flat-entity/types/all-flat-entity-operation-record-by-metadata-name.type';
import { type MetadataFlatEntityAndRelatedFlatEntityMapsForSideEffect } from 'src/engine/metadata-modules/flat-entity/types/metadata-flat-entity-and-related-flat-entity-maps-for-side-effect.type';
import { type MetadataUniversalFlatEntity } from 'src/engine/metadata-modules/flat-entity/types/metadata-universal-flat-entity.type';
import { METADATA_SIDE_EFFECT_HANDLER_METADATA_KEY } from 'src/engine/metadata-modules/metadata-side-effect/constants/metadata-side-effect-handler-metadata-key.constant';
import { type MetadataSideEffectContext } from 'src/engine/metadata-modules/metadata-side-effect/types/metadata-side-effect-context.type';
import { type MetadataSideEffectOperation } from 'src/engine/metadata-modules/metadata-side-effect/types/metadata-side-effect-operation.type';
import { type MetadataSideEffectResult } from 'src/engine/metadata-modules/metadata-side-effect/types/metadata-side-effect-result.type';
export type BuildSideEffectsArgs<P extends AllMetadataName> = {
flatEntity: MetadataUniversalFlatEntity<P>;
allFlatEntityOperationRecordByMetadataName: AllFlatEntityOperationRecordByMetadataName;
relatedFlatEntityMaps: MetadataFlatEntityAndRelatedFlatEntityMapsForSideEffect<P>;
context: MetadataSideEffectContext;
};
export abstract class BaseMetadataSideEffectHandlerService<
P extends AllMetadataName,
> {
public operation: MetadataSideEffectOperation;
public metadataName: P;
public sideEffectName: string;
public sideEffectDescription: string;
abstract buildSideEffects(
args: BuildSideEffectsArgs<P>,
): MetadataSideEffectResult;
}
type MetadataSideEffectHandlerDeclaration<P extends AllMetadataName> = {
operation: MetadataSideEffectOperation;
metadataName: P;
name: string;
description: string;
};
export function MetadataSideEffectHandler<P extends AllMetadataName>({
operation,
metadataName,
name,
description,
}: MetadataSideEffectHandlerDeclaration<P>): typeof BaseMetadataSideEffectHandlerService<P> {
abstract class SideEffectHandlerService extends BaseMetadataSideEffectHandlerService<P> {
operation = operation;
metadataName = metadataName;
sideEffectName = name;
sideEffectDescription = description;
}
SetMetadata(METADATA_SIDE_EFFECT_HANDLER_METADATA_KEY, {
operation,
metadataName,
name,
description,
})(SideEffectHandlerService);
return SideEffectHandlerService;
}
@@ -0,0 +1,16 @@
import { Module } from '@nestjs/common';
import { DiscoveryModule } from '@nestjs/core';
import { MetadataSideEffectHandlersModule } from 'src/engine/metadata-modules/metadata-side-effect/handlers/metadata-side-effect-handlers.module';
import { MetadataSideEffectHandlerRegistryService } from 'src/engine/metadata-modules/metadata-side-effect/registry/metadata-side-effect-handler-registry.service';
import { MetadataSideEffectEngineService } from 'src/engine/metadata-modules/metadata-side-effect/services/metadata-side-effect-engine.service';
@Module({
imports: [DiscoveryModule, MetadataSideEffectHandlersModule],
providers: [
MetadataSideEffectHandlerRegistryService,
MetadataSideEffectEngineService,
],
exports: [MetadataSideEffectEngineService],
})
export class MetadataSideEffectModule {}
@@ -0,0 +1,109 @@
import { Injectable, OnModuleInit } from '@nestjs/common';
import { DiscoveryService } from '@nestjs/core';
import { type AllMetadataName } from 'twenty-shared/metadata';
import { isDefined } from 'twenty-shared/utils';
import { METADATA_SIDE_EFFECT_HANDLER_METADATA_KEY } from 'src/engine/metadata-modules/metadata-side-effect/constants/metadata-side-effect-handler-metadata-key.constant';
import { MetadataSideEffectHandlersModule } from 'src/engine/metadata-modules/metadata-side-effect/handlers/metadata-side-effect-handlers.module';
import { type BaseMetadataSideEffectHandlerService } from 'src/engine/metadata-modules/metadata-side-effect/interfaces/base-metadata-side-effect-handler.service';
import {
buildMetadataSideEffectHandlerKey,
type MetadataSideEffectHandlerDescriptor,
type MetadataSideEffectHandlerKey,
type MetadataSideEffectOperation,
} from 'src/engine/metadata-modules/metadata-side-effect/types/metadata-side-effect-operation.type';
type RegisteredSideEffectHandler =
BaseMetadataSideEffectHandlerService<AllMetadataName>;
export type RegisteredMetadataSideEffectHandlerKey = {
operation: MetadataSideEffectOperation;
metadataName: AllMetadataName;
};
@Injectable()
export class MetadataSideEffectHandlerRegistryService implements OnModuleInit {
private readonly handlersByKey = new Map<
MetadataSideEffectHandlerKey,
RegisteredSideEffectHandler[]
>();
private readonly registeredHandlerKeys: RegisteredMetadataSideEffectHandlerKey[] =
[];
private readonly registeredSideEffectNames = new Set<string>();
constructor(private readonly discoveryService: DiscoveryService) {}
onModuleInit() {
this.discoverAndRegisterHandlers();
}
private discoverAndRegisterHandlers(): void {
const providers = this.discoveryService.getProviders({
include: [MetadataSideEffectHandlersModule],
});
providers.forEach((wrapper) => {
const { instance, metatype } = wrapper;
if (!instance || !metatype) return;
const descriptor: MetadataSideEffectHandlerDescriptor | undefined =
Reflect.getMetadata(
METADATA_SIDE_EFFECT_HANDLER_METADATA_KEY,
metatype,
);
if (
!isDefined(descriptor) ||
typeof instance.buildSideEffects !== 'function'
) {
return;
}
this.registerHandler(instance);
});
}
private registerHandler(instance: RegisteredSideEffectHandler): void {
if (this.registeredSideEffectNames.has(instance.sideEffectName)) {
throw new Error(
`Duplicate metadata side-effect name "${instance.sideEffectName}". Side-effect names must be unique.`,
);
}
this.registeredSideEffectNames.add(instance.sideEffectName);
const handlerKey = buildMetadataSideEffectHandlerKey(
instance.operation,
instance.metadataName,
);
const existingHandlers = this.handlersByKey.get(handlerKey);
if (isDefined(existingHandlers)) {
existingHandlers.push(instance);
return;
}
this.handlersByKey.set(handlerKey, [instance]);
this.registeredHandlerKeys.push({
operation: instance.operation,
metadataName: instance.metadataName,
});
}
getHandlers(
operation: MetadataSideEffectOperation,
metadataName: AllMetadataName,
): RegisteredSideEffectHandler[] {
return (
this.handlersByKey.get(
buildMetadataSideEffectHandlerKey(operation, metadataName),
) ?? []
);
}
getRegisteredHandlerKeys(): RegisteredMetadataSideEffectHandlerKey[] {
return this.registeredHandlerKeys;
}
}
@@ -0,0 +1,329 @@
import { Injectable } from '@nestjs/common';
import { type AllMetadataName } from 'twenty-shared/metadata';
import { isDefined } from 'twenty-shared/utils';
import { type AllFlatEntityMaps } from 'src/engine/metadata-modules/flat-entity/types/all-flat-entity-maps.type';
import { type AllFlatEntityOperationRecordByMetadataName } from 'src/engine/metadata-modules/flat-entity/types/all-flat-entity-operation-record-by-metadata-name.type';
import { type MetadataFlatEntityAndRelatedFlatEntityMapsForSideEffect } from 'src/engine/metadata-modules/flat-entity/types/metadata-flat-entity-and-related-flat-entity-maps-for-side-effect.type';
import { type MetadataUniversalFlatEntity } from 'src/engine/metadata-modules/flat-entity/types/metadata-universal-flat-entity.type';
import { getMetadataManyToOneRelatedNames } from 'src/engine/metadata-modules/flat-entity/utils/get-metadata-many-to-one-related-names.util';
import { getMetadataSideEffectCompanionNames } from 'src/engine/metadata-modules/flat-entity/utils/get-metadata-side-effect-companion-names.util';
import { isSystemSideEffectFlatEntity } from 'src/engine/metadata-modules/flat-entity/utils/is-system-side-effect-flat-entity.util';
import { MetadataSideEffectHandlerRegistryService } from 'src/engine/metadata-modules/metadata-side-effect/registry/metadata-side-effect-handler-registry.service';
import { type MetadataSideEffectContext } from 'src/engine/metadata-modules/metadata-side-effect/types/metadata-side-effect-context.type';
import { type MetadataSideEffectExpansionResult } from 'src/engine/metadata-modules/metadata-side-effect/types/metadata-side-effect-expansion-result.type';
import {
METADATA_SIDE_EFFECT_OPERATIONS,
type MetadataSideEffectOperation,
} from 'src/engine/metadata-modules/metadata-side-effect/types/metadata-side-effect-operation.type';
import { type MetadataSideEffectFailure } from 'src/engine/metadata-modules/metadata-side-effect/types/metadata-side-effect-result.type';
import { type SystemSideEffectUniversalIdentifierCollision } from 'src/engine/metadata-modules/metadata-side-effect/types/system-side-effect-universal-identifier-collision.type';
import { mapSystemSideEffectCollisionToFailure } from 'src/engine/metadata-modules/metadata-side-effect/utils/map-system-side-effect-collision-to-failure.util';
import { EMPTY_ORCHESTRATOR_FAILURE_REPORT } from 'src/engine/workspace-manager/workspace-migration/constant/empty-orchestrator-failure-report.constant';
import { pushToOrchestratorFailureReport } from 'src/engine/workspace-manager/workspace-migration/utils/merge-orchestrator-failure-reports.util';
type GenericUniversalFlatEntity = { universalIdentifier: string };
type GenericFlatEntityOperationRecord = {
flatEntityToCreate: Record<string, GenericUniversalFlatEntity>;
flatEntityToUpdate: Record<string, GenericUniversalFlatEntity>;
flatEntityToDelete: Record<string, GenericUniversalFlatEntity>;
};
type GenericAllFlatEntityOperationRecordByMetadataName = Record<
string,
GenericFlatEntityOperationRecord | undefined
>;
type GenericPartialFlatEntityOperationRecord = {
flatEntityToCreate?: Record<string, GenericUniversalFlatEntity>;
flatEntityToUpdate?: Record<string, GenericUniversalFlatEntity>;
flatEntityToDelete?: Record<string, GenericUniversalFlatEntity>;
};
type GenericMetadataSideEffectOperationsByMetadataName = Record<
string,
GenericPartialFlatEntityOperationRecord | undefined
>;
const OPERATION_TO_FLAT_ENTITY_RECORD_KEY = {
create: 'flatEntityToCreate',
update: 'flatEntityToUpdate',
delete: 'flatEntityToDelete',
} as const satisfies Record<
MetadataSideEffectOperation,
keyof GenericFlatEntityOperationRecord
>;
@Injectable()
export class MetadataSideEffectEngineService {
constructor(
private readonly metadataSideEffectHandlerRegistryService: MetadataSideEffectHandlerRegistryService,
) {}
getSideEffectRelatedMetadataNames(
triggerMetadataNames: AllMetadataName[],
): AllMetadataName[] {
const relatedMetadataNames = new Set<AllMetadataName>();
for (const {
metadataName,
} of this.metadataSideEffectHandlerRegistryService.getRegisteredHandlerKeys()) {
if (!triggerMetadataNames.includes(metadataName)) {
continue;
}
for (const relatedMetadataName of [
metadataName,
...getMetadataManyToOneRelatedNames(metadataName),
...getMetadataSideEffectCompanionNames(metadataName),
]) {
relatedMetadataNames.add(relatedMetadataName);
}
}
return [...relatedMetadataNames];
}
expandWithSideEffects({
allFlatEntityOperationRecordByMetadataName,
sideEffectRelatedFlatEntityMaps,
context,
}: {
allFlatEntityOperationRecordByMetadataName: AllFlatEntityOperationRecordByMetadataName;
sideEffectRelatedFlatEntityMaps: Partial<AllFlatEntityMaps>;
context: MetadataSideEffectContext;
}): MetadataSideEffectExpansionResult {
const expandedMatrix = this.cloneMatrix(
allFlatEntityOperationRecordByMetadataName,
);
const systemSideEffectUniversalIdentifierCollisions: SystemSideEffectUniversalIdentifierCollision[] =
[];
const sideEffectFailures: MetadataSideEffectFailure[] = [];
const triggerMatrix =
allFlatEntityOperationRecordByMetadataName as unknown as GenericAllFlatEntityOperationRecordByMetadataName;
for (const {
operation,
metadataName,
} of this.metadataSideEffectHandlerRegistryService.getRegisteredHandlerKeys()) {
const handlers =
this.metadataSideEffectHandlerRegistryService.getHandlers(
operation,
metadataName,
);
if (handlers.length === 0) {
continue;
}
const triggerFlatEntities = Object.values(
triggerMatrix[metadataName]?.[
OPERATION_TO_FLAT_ENTITY_RECORD_KEY[operation]
] ?? {},
);
for (const triggerFlatEntity of triggerFlatEntities) {
for (const handler of handlers) {
const sideEffectResult = handler.buildSideEffects({
flatEntity:
triggerFlatEntity as unknown as MetadataUniversalFlatEntity<AllMetadataName>,
allFlatEntityOperationRecordByMetadataName:
expandedMatrix as unknown as AllFlatEntityOperationRecordByMetadataName,
relatedFlatEntityMaps:
sideEffectRelatedFlatEntityMaps as unknown as MetadataFlatEntityAndRelatedFlatEntityMapsForSideEffect<AllMetadataName>,
context,
});
if (sideEffectResult.status === 'fail') {
sideEffectFailures.push(sideEffectResult);
continue;
}
if (sideEffectResult.status === 'noop') {
continue;
}
this.mergeSideEffectsIntoMatrix({
expandedMatrix,
sideEffectOperations:
sideEffectResult.operations as unknown as GenericMetadataSideEffectOperationsByMetadataName,
systemSideEffectUniversalIdentifierCollisions,
});
}
}
}
const allSideEffectFailures: MetadataSideEffectFailure[] = [
...sideEffectFailures,
...systemSideEffectUniversalIdentifierCollisions.map(
mapSystemSideEffectCollisionToFailure,
),
];
if (allSideEffectFailures.length > 0) {
const report = EMPTY_ORCHESTRATOR_FAILURE_REPORT();
for (const sideEffectFailure of allSideEffectFailures) {
pushToOrchestratorFailureReport({
report,
metadataName: sideEffectFailure.metadataName,
items: [sideEffectFailure],
});
}
return {
status: 'fail',
report,
};
}
return {
status: 'success',
allFlatEntityOperationRecordByMetadataName:
expandedMatrix as unknown as AllFlatEntityOperationRecordByMetadataName,
};
}
private mergeSideEffectsIntoMatrix({
expandedMatrix,
sideEffectOperations,
systemSideEffectUniversalIdentifierCollisions,
}: {
expandedMatrix: GenericAllFlatEntityOperationRecordByMetadataName;
sideEffectOperations: GenericMetadataSideEffectOperationsByMetadataName;
systemSideEffectUniversalIdentifierCollisions: SystemSideEffectUniversalIdentifierCollision[];
}): void {
for (const metadataName of Object.keys(sideEffectOperations)) {
const operationBuckets = sideEffectOperations[metadataName];
if (!isDefined(operationBuckets)) {
continue;
}
for (const operation of METADATA_SIDE_EFFECT_OPERATIONS) {
const sideEffectFlatEntities = Object.values(
operationBuckets[OPERATION_TO_FLAT_ENTITY_RECORD_KEY[operation]] ??
{},
);
for (const sideEffectFlatEntity of sideEffectFlatEntities) {
this.addToOperationIfAbsent({
expandedMatrix,
operation,
metadataName,
flatEntity: sideEffectFlatEntity,
systemSideEffectUniversalIdentifierCollisions,
});
}
}
}
}
private cloneMatrix(
allFlatEntityOperationRecordByMetadataName: AllFlatEntityOperationRecordByMetadataName,
): GenericAllFlatEntityOperationRecordByMetadataName {
const genericMatrix =
allFlatEntityOperationRecordByMetadataName as unknown as GenericAllFlatEntityOperationRecordByMetadataName;
const clonedMatrix: GenericAllFlatEntityOperationRecordByMetadataName = {};
for (const metadataName of Object.keys(genericMatrix)) {
const operations = genericMatrix[metadataName];
if (!isDefined(operations)) {
continue;
}
clonedMatrix[metadataName] = {
flatEntityToCreate: { ...operations.flatEntityToCreate },
flatEntityToUpdate: { ...operations.flatEntityToUpdate },
flatEntityToDelete: { ...operations.flatEntityToDelete },
};
}
return clonedMatrix;
}
private addToOperationIfAbsent({
expandedMatrix,
operation,
metadataName,
flatEntity,
systemSideEffectUniversalIdentifierCollisions,
}: {
expandedMatrix: GenericAllFlatEntityOperationRecordByMetadataName;
operation: MetadataSideEffectOperation;
metadataName: string;
flatEntity: GenericUniversalFlatEntity;
systemSideEffectUniversalIdentifierCollisions: SystemSideEffectUniversalIdentifierCollision[];
}): void {
const operations = (expandedMatrix[metadataName] ??= {
flatEntityToCreate: {},
flatEntityToUpdate: {},
flatEntityToDelete: {},
});
const flatEntityRecordKey = OPERATION_TO_FLAT_ENTITY_RECORD_KEY[operation];
const flatEntityRecord = operations[flatEntityRecordKey];
const existingFlatEntity = flatEntityRecord[flatEntity.universalIdentifier];
if (isDefined(existingFlatEntity)) {
this.recordUniversalIdentifierCollisionIfNeeded({
existingFlatEntity,
operation,
metadataName,
flatEntity,
systemSideEffectUniversalIdentifierCollisions,
});
return;
}
flatEntityRecord[flatEntity.universalIdentifier] = flatEntity;
}
private recordUniversalIdentifierCollisionIfNeeded({
existingFlatEntity,
operation,
metadataName,
flatEntity,
systemSideEffectUniversalIdentifierCollisions,
}: {
existingFlatEntity: GenericUniversalFlatEntity;
operation: MetadataSideEffectOperation;
metadataName: string;
flatEntity: GenericUniversalFlatEntity;
systemSideEffectUniversalIdentifierCollisions: SystemSideEffectUniversalIdentifierCollision[];
}): void {
const isIncomingSystemSideEffect = isSystemSideEffectFlatEntity(
flatEntity as unknown as MetadataUniversalFlatEntity<AllMetadataName>,
);
if (!isIncomingSystemSideEffect) {
return;
}
if (
isSystemSideEffectFlatEntity(
existingFlatEntity as unknown as MetadataUniversalFlatEntity<AllMetadataName>,
)
) {
return;
}
systemSideEffectUniversalIdentifierCollisions.push({
metadataName: metadataName as AllMetadataName,
operation,
universalIdentifier: flatEntity.universalIdentifier,
name: this.extractFlatEntityName(flatEntity),
});
}
private extractFlatEntityName(
flatEntity: GenericUniversalFlatEntity,
): string | undefined {
const { name } = flatEntity as { name?: unknown };
return typeof name === 'string' ? name : undefined;
}
}
@@ -0,0 +1,5 @@
import { type WorkspaceMigrationBuilderOptions } from 'src/engine/workspace-manager/workspace-migration/workspace-migration-builder/types/workspace-migration-builder-options.type';
export type MetadataSideEffectContext = {
buildOptions: WorkspaceMigrationBuilderOptions;
};
@@ -0,0 +1,15 @@
import { type AllFlatEntityOperationRecordByMetadataName } from 'src/engine/metadata-modules/flat-entity/types/all-flat-entity-operation-record-by-metadata-name.type';
import { type OrchestratorFailureReport } from 'src/engine/workspace-manager/workspace-migration/types/workspace-migration-orchestrator.type';
// Same contract as the builder (WorkspaceMigrationOrchestratorFailedResult):
// collisions and handler failures are merged into a single OrchestratorFailureReport
// so callers handle side-effect and builder failures through one uniform channel.
export type MetadataSideEffectExpansionResult =
| {
status: 'success';
allFlatEntityOperationRecordByMetadataName: AllFlatEntityOperationRecordByMetadataName;
}
| {
status: 'fail';
report: OrchestratorFailureReport;
};
@@ -0,0 +1,24 @@
import { type AllMetadataName } from 'twenty-shared/metadata';
export type MetadataSideEffectOperation = 'create' | 'update' | 'delete';
export const METADATA_SIDE_EFFECT_OPERATIONS = [
'create',
'update',
'delete',
] as const satisfies readonly MetadataSideEffectOperation[];
export type MetadataSideEffectHandlerKey =
`${MetadataSideEffectOperation}:${AllMetadataName}`;
export const buildMetadataSideEffectHandlerKey = (
operation: MetadataSideEffectOperation,
metadataName: AllMetadataName,
): MetadataSideEffectHandlerKey => `${operation}:${metadataName}`;
export type MetadataSideEffectHandlerDescriptor = {
operation: MetadataSideEffectOperation;
metadataName: AllMetadataName;
name: string;
description: string;
};
@@ -0,0 +1,17 @@
import { type AllMetadataName } from 'twenty-shared/metadata';
import { type MetadataUniversalFlatEntity } from 'src/engine/metadata-modules/flat-entity/types/metadata-universal-flat-entity.type';
// Record-native, partial companion operations a handler emits. Each bucket is keyed by
// universalIdentifier so the engine merges/deduplicates against the matrix in O(1),
// matching the shape it consumes (AllFlatEntityOperationRecordByMetadataName).
export type MetadataSideEffectOperationsByMetadataName = {
[P in AllMetadataName]?: {
flatEntityToCreate?: Record<
string,
MetadataUniversalFlatEntity<P> & { id?: string }
>;
flatEntityToUpdate?: Record<string, MetadataUniversalFlatEntity<P>>;
flatEntityToDelete?: Record<string, MetadataUniversalFlatEntity<P>>;
};
};
@@ -0,0 +1,27 @@
import { type AllMetadataName } from 'twenty-shared/metadata';
import { type WorkspaceMigrationActionType } from 'src/engine/metadata-modules/flat-entity/types/metadata-workspace-migration-action.type';
import { type MetadataSideEffectOperationsByMetadataName } from 'src/engine/metadata-modules/metadata-side-effect/types/metadata-side-effect-operations-by-metadata-name.type';
import { type FailedFlatEntityValidation } from 'src/engine/workspace-manager/workspace-migration/workspace-migration-builder/builders/types/failed-flat-entity-validation.type';
// A single failure emitted by a side-effect handler, using the exact same shape
// as the builder's per-entity validation failure (a `status: 'fail'` tag spread
// over a FailedFlatEntityValidation), so both sources merge into one report.
export type MetadataSideEffectFailure = {
status: 'fail';
} & FailedFlatEntityValidation<AllMetadataName, WorkspaceMigrationActionType>;
// Discriminated union of the three handler outcomes: `noop` when there is
// nothing to do, `success` when it produced companion operations to merge, or
// `fail` (fail-slow) with a validation failure the engine aggregates instead of
// throwing. `success` therefore always carries operations, keeping the empty
// case explicit as `noop` rather than a success with `operations: {}`.
export type MetadataSideEffectResult =
| {
status: 'noop';
}
| {
status: 'success';
operations: MetadataSideEffectOperationsByMetadataName;
}
| MetadataSideEffectFailure;
@@ -0,0 +1,10 @@
import { type AllMetadataName } from 'twenty-shared/metadata';
import { type MetadataSideEffectOperation } from 'src/engine/metadata-modules/metadata-side-effect/types/metadata-side-effect-operation.type';
export type SystemSideEffectUniversalIdentifierCollision = {
metadataName: AllMetadataName;
operation: MetadataSideEffectOperation;
universalIdentifier: string;
name?: string;
};
@@ -0,0 +1,29 @@
import { msg, t } from '@lingui/core/macro';
import { type AllMetadataName } from 'twenty-shared/metadata';
import { type MetadataFlatEntity } from 'src/engine/metadata-modules/flat-entity/types/metadata-flat-entity.type';
import { MetadataSideEffectExceptionCode } from 'src/engine/metadata-modules/metadata-side-effect/exceptions/metadata-side-effect-exception-code';
import { type MetadataSideEffectFailure } from 'src/engine/metadata-modules/metadata-side-effect/types/metadata-side-effect-result.type';
import { type SystemSideEffectUniversalIdentifierCollision } from 'src/engine/metadata-modules/metadata-side-effect/types/system-side-effect-universal-identifier-collision.type';
// A reserved-identifier collision is just another side-effect failure: mapping it
// to the shared MetadataSideEffectFailure shape lets the engine merge collisions
// and handler failures into a single report.
export const mapSystemSideEffectCollisionToFailure = (
collision: SystemSideEffectUniversalIdentifierCollision,
): MetadataSideEffectFailure => ({
status: 'fail',
type: collision.operation,
metadataName: collision.metadataName,
flatEntityMinimalInformation: {
universalIdentifier: collision.universalIdentifier,
...(collision.name !== undefined ? { name: collision.name } : {}),
} as Partial<MetadataFlatEntity<AllMetadataName>>,
errors: [
{
code: MetadataSideEffectExceptionCode.RESERVED_SYSTEM_UNIVERSAL_IDENTIFIER,
message: t`Universal identifier is reserved for system-managed metadata`,
userFriendlyMessage: msg`This identifier is reserved by the system`,
},
],
});
@@ -36,6 +36,7 @@ export const buildCompanyStandardFlatIndexMetadatas = ({
indexName: 'domainNameUniqueIndex',
relatedFieldNames: ['domainName'],
isUnique: true,
hasDeterministicUniversalIdentifier: true,
},
standardObjectMetadataRelatedEntityIds,
dependencyFlatEntityMaps,
@@ -36,6 +36,7 @@ export const buildPersonStandardFlatIndexMetadatas = ({
indexName: 'emailsUniqueIndex',
relatedFieldNames: ['emails'],
isUnique: true,
hasDeterministicUniversalIdentifier: true,
},
standardObjectMetadataRelatedEntityIds,
dependencyFlatEntityMaps,
@@ -24,6 +24,7 @@ export const buildWorkspaceMemberStandardFlatIndexMetadatas = ({
indexName: 'userEmailUniqueIndex',
relatedFieldNames: ['userEmail'],
isUnique: true,
hasDeterministicUniversalIdentifier: true,
},
standardObjectMetadataRelatedEntityIds,
dependencyFlatEntityMaps,
@@ -1,3 +1,4 @@
import { getIndexUniversalIdentifier } from 'twenty-shared/application';
import { STANDARD_OBJECTS } from 'twenty-shared/metadata';
import { isDefined } from 'twenty-shared/utils';
import { v4 } from 'uuid';
@@ -9,7 +10,7 @@ import {
type FlatIndexMetadata,
} from 'src/engine/metadata-modules/flat-index-metadata/types/flat-index-metadata.type';
import { IndexType } from 'src/engine/metadata-modules/index-metadata/types/indexType.types';
import { generateFlatIndexMetadataWithNameOrThrow } from 'src/engine/metadata-modules/index-metadata/utils/generate-flat-index.util';
import { computeFlatIndexNameOrThrow } from 'src/engine/metadata-modules/index-metadata/utils/compute-flat-index-name.util';
import { TWENTY_STANDARD_APPLICATION } from 'src/engine/workspace-manager/twenty-standard-application/constants/twenty-standard-applications';
import { type AllStandardObjectFieldName } from 'src/engine/workspace-manager/twenty-standard-application/types/all-standard-object-field-name.type';
import { type AllStandardObjectIndexName } from 'src/engine/workspace-manager/twenty-standard-application/types/all-standard-object-index-name.type';
@@ -20,6 +21,7 @@ import { type UniversalFlatIndexFieldMetadata } from 'src/engine/workspace-manag
export type CreateStandardIndexOptions<O extends AllStandardObjectName> = {
indexName: AllStandardObjectIndexName<O>;
relatedFieldNames: AllStandardObjectFieldName<O>[];
hasDeterministicUniversalIdentifier?: boolean;
} & Partial<
Pick<FlatIndexMetadata, 'indexType' | 'indexWhereClause' | 'isUnique'>
>;
@@ -42,6 +44,7 @@ export const createStandardIndexFlatMetadata = <
indexType = IndexType.BTREE,
indexWhereClause = null,
isUnique = false,
hasDeterministicUniversalIdentifier = false,
},
standardObjectMetadataRelatedEntityIds,
dependencyFlatEntityMaps: { flatFieldMetadataMaps, flatObjectMetadataMaps },
@@ -88,41 +91,55 @@ export const createStandardIndexFlatMetadata = <
const indexId = v4();
const unviersalFlatIndex = generateFlatIndexMetadataWithNameOrThrow({
flatIndex: {
createdAt: now,
applicationUniversalIdentifier:
TWENTY_STANDARD_APPLICATION.universalIdentifier,
indexType,
indexWhereClause,
isCustom: false,
isUnique,
isSystemSideEffect: true,
objectMetadataUniversalIdentifier: flatObjectMetadata.universalIdentifier,
universalIdentifier: indexDefinition.universalIdentifier,
updatedAt: now,
universalFlatIndexFieldMetadatas:
flatFieldMetadatas.map<UniversalFlatIndexFieldMetadata>(
(
{ universalIdentifier: fieldMetadataUniversalIdentifier },
index,
) => ({
createdAt: now,
order: index,
subFieldName: null,
updatedAt: now,
fieldMetadataUniversalIdentifier,
indexMetadataUniversalIdentifier:
indexDefinition.universalIdentifier,
}),
),
},
const computedIndexName = computeFlatIndexNameOrThrow({
flatObjectMetadata,
objectFlatFieldMetadatas: flatFieldMetadatas,
indexFields: flatFieldMetadatas.map((flatFieldMetadata, index) => ({
order: index,
fieldMetadataUniversalIdentifier: flatFieldMetadata.universalIdentifier,
subFieldName: null,
})),
isUnique,
indexWhereClause,
});
const universalIdentifier = hasDeterministicUniversalIdentifier
? getIndexUniversalIdentifier({
applicationUniversalIdentifier:
TWENTY_STANDARD_APPLICATION.universalIdentifier,
objectUniversalIdentifier: flatObjectMetadata.universalIdentifier,
name: computedIndexName,
})
: indexDefinition.universalIdentifier;
const universalFlatIndex = {
createdAt: now,
applicationUniversalIdentifier:
TWENTY_STANDARD_APPLICATION.universalIdentifier,
indexType,
indexWhereClause,
isCustom: false,
isUnique,
isSystemSideEffect: true,
name: computedIndexName,
objectMetadataUniversalIdentifier: flatObjectMetadata.universalIdentifier,
universalIdentifier,
updatedAt: now,
universalFlatIndexFieldMetadatas:
flatFieldMetadatas.map<UniversalFlatIndexFieldMetadata>(
({ universalIdentifier: fieldMetadataUniversalIdentifier }, index) => ({
createdAt: now,
order: index,
subFieldName: null,
updatedAt: now,
fieldMetadataUniversalIdentifier,
indexMetadataUniversalIdentifier: universalIdentifier,
}),
),
};
return {
...unviersalFlatIndex,
...universalFlatIndex,
applicationId: twentyStandardApplicationId,
id: v4(),
flatIndexFieldMetadatas: relatedFieldIds.map<FlatIndexFieldMetadata>(
@@ -0,0 +1,341 @@
import { Injectable } from '@nestjs/common';
import { AllMetadataName } from 'twenty-shared/metadata';
import { isDefined } from 'twenty-shared/utils';
import { FlatApplicationCacheMaps } from 'src/engine/core-modules/application/types/flat-application-cache-maps.type';
import { ALL_MANY_TO_ONE_METADATA_RELATIONS } from 'src/engine/metadata-modules/flat-entity/constant/all-many-to-one-metadata-relations.constant';
import {
FlatEntityMapsException,
FlatEntityMapsExceptionCode,
} from 'src/engine/metadata-modules/flat-entity/exceptions/flat-entity-maps.exception';
import { AllFlatEntityMaps } from 'src/engine/metadata-modules/flat-entity/types/all-flat-entity-maps.type';
import { AllFlatEntityOperationRecordByMetadataName } from 'src/engine/metadata-modules/flat-entity/types/all-flat-entity-operation-record-by-metadata-name.type';
import { MetadataUniversalFlatEntity } from 'src/engine/metadata-modules/flat-entity/types/metadata-universal-flat-entity.type';
import { getMetadataFlatEntityMapsKey } from 'src/engine/metadata-modules/flat-entity/utils/get-metadata-flat-entity-maps-key.util';
import { getMetadataRelatedMetadataNamesForValidation } from 'src/engine/metadata-modules/flat-entity/utils/get-metadata-related-metadata-names-for-validation.util';
import { getSubAllFlatEntityMapsByApplicationIdsOrThrow } from 'src/engine/metadata-modules/flat-entity/utils/get-sub-all-flat-entity-maps-by-application-ids-or-throw.util';
import { MetadataSideEffectEngineService } from 'src/engine/metadata-modules/metadata-side-effect/services/metadata-side-effect-engine.service';
import { WorkspaceCacheService } from 'src/engine/workspace-cache/services/workspace-cache.service';
import { TWENTY_STANDARD_APPLICATION } from 'src/engine/workspace-manager/twenty-standard-application/constants/twenty-standard-applications';
import { WORKSPACE_MIGRATION_ADDITIONAL_CACHE_DATA_MAPS_KEY } from 'src/engine/workspace-manager/workspace-migration/constant/workspace-migration-additional-cache-data-maps-key.constant';
import { IdByUniversalIdentifierByMetadataName } from 'src/engine/workspace-manager/workspace-migration/services/utils/enrich-create-workspace-migration-action-with-ids.util';
import { WorkspaceMigrationBuilderAdditionalCacheDataMaps } from 'src/engine/workspace-manager/workspace-migration/types/workspace-migration-builder-additional-cache-data-maps.type';
import { FromToAllUniversalFlatEntityMaps } from 'src/engine/workspace-manager/workspace-migration/types/workspace-migration-orchestrator.type';
import { computeUniversalFlatEntityMapsFromToThroughMutation } from 'src/engine/workspace-manager/workspace-migration/utils/compute-universal-flat-entity-maps-from-to-through-mutation.util';
import { InferDeletionFromMissingEntities } from 'src/engine/workspace-manager/workspace-migration/workspace-migration-builder/types/infer-deletion-from-missing-entities.type';
export type WorkspaceMigrationRelatedFlatEntityMaps =
Partial<AllFlatEntityMaps> & WorkspaceMigrationBuilderAdditionalCacheDataMaps;
type FlatEntityMapsBundle = {
flatApplicationMaps: FlatApplicationCacheMaps;
allRelatedFlatEntityMaps: WorkspaceMigrationRelatedFlatEntityMaps;
allMetadataNameCacheToCompute: AllMetadataName[];
};
type ComputeFromToAllFlatEntityMapsAndBuildOptionsArgs = {
allFlatEntityOperationRecordByMetadataName: AllFlatEntityOperationRecordByMetadataName;
applicationUniversalIdentifier: string;
} & FlatEntityMapsBundle;
@Injectable()
export class WorkspaceMigrationFlatEntityMapsService {
constructor(
private readonly workspaceCacheService: WorkspaceCacheService,
private readonly metadataSideEffectEngineService: MetadataSideEffectEngineService,
) {}
async getOrRecomputeAllRelatedFlatEntityMaps({
workspaceId,
callerMetadataNames,
}: {
workspaceId: string;
callerMetadataNames: AllMetadataName[];
}): Promise<FlatEntityMapsBundle> {
const allMetadataNameCacheToCompute = [
...new Set(
[
...callerMetadataNames,
...this.metadataSideEffectEngineService.getSideEffectRelatedMetadataNames(
callerMetadataNames,
),
].flatMap((metadataName) => [
metadataName,
...getMetadataRelatedMetadataNamesForValidation(metadataName),
]),
),
];
const { flatApplicationMaps, ...allRelatedFlatEntityMaps } =
await this.workspaceCacheService.getOrRecompute(workspaceId, [
...allMetadataNameCacheToCompute.map(getMetadataFlatEntityMapsKey),
...WORKSPACE_MIGRATION_ADDITIONAL_CACHE_DATA_MAPS_KEY,
'flatApplicationMaps',
]);
return {
flatApplicationMaps,
allRelatedFlatEntityMaps,
allMetadataNameCacheToCompute,
};
}
computeFromToAllFlatEntityMapsAndBuildOptions({
allFlatEntityOperationRecordByMetadataName,
applicationUniversalIdentifier,
flatApplicationMaps,
allRelatedFlatEntityMaps,
allMetadataNameCacheToCompute,
}: ComputeFromToAllFlatEntityMapsAndBuildOptionsArgs): {
fromToAllFlatEntityMaps: FromToAllUniversalFlatEntityMaps;
inferDeletionFromMissingEntities: InferDeletionFromMissingEntities;
dependencyAllFlatEntityMaps: Partial<AllFlatEntityMaps>;
additionalCacheDataMaps: WorkspaceMigrationBuilderAdditionalCacheDataMaps;
idByUniversalIdentifierByMetadataName: IdByUniversalIdentifierByMetadataName;
} {
const { dependencyAllFlatEntityMaps, additionalCacheDataMaps } =
this.computeDependencyAndAdditionalCacheDataMaps({
allFlatEntityOperationRecordByMetadataName,
applicationUniversalIdentifier,
flatApplicationMaps,
allRelatedFlatEntityMaps,
allMetadataNameCacheToCompute,
});
const fromToAllFlatEntityMaps: FromToAllUniversalFlatEntityMaps = {};
const idByUniversalIdentifierByMetadataName: IdByUniversalIdentifierByMetadataName =
{};
const inferDeletionFromMissingEntities: InferDeletionFromMissingEntities =
{};
const allMetadataNameToCompare = Object.keys(
allFlatEntityOperationRecordByMetadataName,
) as AllMetadataName[];
for (const metadataName of allMetadataNameToCompare) {
const flatEntityOperations =
allFlatEntityOperationRecordByMetadataName[metadataName];
if (!isDefined(flatEntityOperations)) {
throw new FlatEntityMapsException(
`Could not load flat entity maps to compare for ${metadataName}, should never occur`,
FlatEntityMapsExceptionCode.INTERNAL_SERVER_ERROR,
);
}
// The record matrix is the canonical form; the from/to mutation helper still
// consumes arrays, so we flatten each bucket at this boundary only.
const flatEntityToCreate = Object.values(
flatEntityOperations.flatEntityToCreate,
);
const flatEntityToUpdate = Object.values(
flatEntityOperations.flatEntityToUpdate,
);
const flatEntityToDelete = Object.values(
flatEntityOperations.flatEntityToDelete,
);
const idByUniversalIdentifier = Object.fromEntries(
flatEntityToCreate
.filter(
(
flatEntity,
): flatEntity is MetadataUniversalFlatEntity<
typeof metadataName
> & { id: string } => isDefined(flatEntity.id),
)
.map((flatEntity) => [flatEntity.universalIdentifier, flatEntity.id]),
);
if (Object.keys(idByUniversalIdentifier).length > 0) {
idByUniversalIdentifierByMetadataName[metadataName] =
idByUniversalIdentifier;
}
const flatEntityMapsKey = getMetadataFlatEntityMapsKey(metadataName);
const flatEntityMaps = allRelatedFlatEntityMaps[flatEntityMapsKey];
if (!isDefined(flatEntityMaps)) {
throw new FlatEntityMapsException(
`Flat entity maps for ${metadataName} were not pre-fetched; the up-front cache key union must cover every metadata name the side-effect expansion can add`,
FlatEntityMapsExceptionCode.INTERNAL_SERVER_ERROR,
);
}
// @ts-expect-error Metadata flat entity maps cache key and metadataName colliding
fromToAllFlatEntityMaps[flatEntityMapsKey] =
computeUniversalFlatEntityMapsFromToThroughMutation({
flatEntityMaps: structuredClone(flatEntityMaps),
flatEntityToCreate,
flatEntityToDelete,
flatEntityToUpdate,
});
if (flatEntityToDelete.length > 0) {
inferDeletionFromMissingEntities[metadataName] = true;
}
}
return {
fromToAllFlatEntityMaps,
inferDeletionFromMissingEntities,
dependencyAllFlatEntityMaps,
additionalCacheDataMaps,
idByUniversalIdentifierByMetadataName,
};
}
private computeDependencyAndAdditionalCacheDataMaps({
allFlatEntityOperationRecordByMetadataName,
applicationUniversalIdentifier,
flatApplicationMaps,
allRelatedFlatEntityMaps,
allMetadataNameCacheToCompute,
}: ComputeFromToAllFlatEntityMapsAndBuildOptionsArgs) {
const applicationIds = this.computeAllInvolvedApplicationIds({
allFlatEntityOperationRecordByMetadataName,
flatApplicationMaps,
applicationUniversalIdentifier,
allRelatedFlatEntityMaps,
});
const dependencyAllFlatEntityMaps =
getSubAllFlatEntityMapsByApplicationIdsOrThrow({
applicationIds,
metadataNames: allMetadataNameCacheToCompute,
fromAllFlatEntityMaps: allRelatedFlatEntityMaps,
});
const additionalCacheDataMaps =
WORKSPACE_MIGRATION_ADDITIONAL_CACHE_DATA_MAPS_KEY.reduce<WorkspaceMigrationBuilderAdditionalCacheDataMaps>(
(acc, additionalCacheDataMapsKey) => {
return {
...acc,
[additionalCacheDataMapsKey]:
allRelatedFlatEntityMaps[additionalCacheDataMapsKey],
};
},
{} as WorkspaceMigrationBuilderAdditionalCacheDataMaps,
);
return {
dependencyAllFlatEntityMaps,
additionalCacheDataMaps,
};
}
private computeAllInvolvedApplicationIds({
allFlatEntityOperationRecordByMetadataName,
flatApplicationMaps,
applicationUniversalIdentifier,
allRelatedFlatEntityMaps,
}: {
allFlatEntityOperationRecordByMetadataName: AllFlatEntityOperationRecordByMetadataName;
flatApplicationMaps: FlatApplicationCacheMaps;
applicationUniversalIdentifier: string;
allRelatedFlatEntityMaps: Partial<AllFlatEntityMaps>;
}): string[] {
const applicationIds = new Set<string>();
const applicationId =
flatApplicationMaps.idByUniversalIdentifier[
applicationUniversalIdentifier
];
const twentyStandardApplicationId =
flatApplicationMaps.idByUniversalIdentifier[
TWENTY_STANDARD_APPLICATION.universalIdentifier
];
if (!isDefined(twentyStandardApplicationId) || !isDefined(applicationId)) {
throw new FlatEntityMapsException(
'Application to build and its dependent application not found',
FlatEntityMapsExceptionCode.ENTITY_NOT_FOUND,
);
}
applicationIds.add(applicationId);
const isBuildingTwentyStandardApplication =
applicationUniversalIdentifier ===
TWENTY_STANDARD_APPLICATION.universalIdentifier;
if (!isBuildingTwentyStandardApplication) {
applicationIds.add(twentyStandardApplicationId);
}
for (const metadataName of Object.keys(
allFlatEntityOperationRecordByMetadataName,
) as AllMetadataName[]) {
const flatEntityOperations =
allFlatEntityOperationRecordByMetadataName[metadataName];
if (!isDefined(flatEntityOperations)) {
continue;
}
const { flatEntityToCreate, flatEntityToUpdate, flatEntityToDelete } =
flatEntityOperations;
const relations = ALL_MANY_TO_ONE_METADATA_RELATIONS[metadataName];
for (const flatEntity of [
...Object.values(flatEntityToCreate),
...Object.values(flatEntityToUpdate),
...Object.values(flatEntityToDelete),
]) {
const entityApplicationId =
flatApplicationMaps.idByUniversalIdentifier[
flatEntity.applicationUniversalIdentifier
];
if (isDefined(entityApplicationId)) {
applicationIds.add(entityApplicationId);
}
for (const relation of Object.values(relations) as ({
foreignKey: string;
metadataName: AllMetadataName;
isNullable: boolean;
universalForeignKey: string;
} | null)[]) {
if (!isDefined(relation)) {
continue;
}
const { universalForeignKey, metadataName: targetMetadataName } =
relation;
const referencedUniversalIdentifier =
flatEntity[universalForeignKey as keyof typeof flatEntity];
if (!isDefined(referencedUniversalIdentifier)) {
continue;
}
const targetFlatEntityMaps =
allRelatedFlatEntityMaps[
getMetadataFlatEntityMapsKey(
targetMetadataName as AllMetadataName,
)
];
if (!isDefined(targetFlatEntityMaps)) {
continue;
}
const referencedEntity =
targetFlatEntityMaps.byUniversalIdentifier[
referencedUniversalIdentifier
];
if (isDefined(referencedEntity)) {
applicationIds.add(referencedEntity.applicationId);
}
}
}
}
return [...applicationIds];
}
}
@@ -4,53 +4,43 @@ import {
AllMetadataName,
WorkspaceMigrationV2ExceptionCode,
} from 'twenty-shared/metadata';
import { isDefined } from 'twenty-shared/utils';
import { FlatApplicationCacheMaps } from 'src/engine/core-modules/application/types/flat-application-cache-maps.type';
import { LoggerService } from 'src/engine/core-modules/logger/logger.service';
import { TwentyConfigService } from 'src/engine/core-modules/twenty-config/twenty-config.service';
import { ALL_MANY_TO_ONE_METADATA_RELATIONS } from 'src/engine/metadata-modules/flat-entity/constant/all-many-to-one-metadata-relations.constant';
import { createEmptyFlatEntityMaps } from 'src/engine/metadata-modules/flat-entity/constant/create-empty-flat-entity-maps.constant';
import {
FlatEntityMapsException,
FlatEntityMapsExceptionCode,
} from 'src/engine/metadata-modules/flat-entity/exceptions/flat-entity-maps.exception';
import { AllFlatEntityMaps } from 'src/engine/metadata-modules/flat-entity/types/all-flat-entity-maps.type';
import { FlatEntityToCreateDeleteUpdate } from 'src/engine/metadata-modules/flat-entity/types/flat-entity-to-create-delete-update.type';
import { MetadataFlatEntity } from 'src/engine/metadata-modules/flat-entity/types/metadata-flat-entity.type';
import { MetadataUniversalFlatEntity } from 'src/engine/metadata-modules/flat-entity/types/metadata-universal-flat-entity.type';
import { getMetadataFlatEntityMapsKey } from 'src/engine/metadata-modules/flat-entity/utils/get-metadata-flat-entity-maps-key.util';
import { AllFlatEntityOperationRecordByMetadataName } from 'src/engine/metadata-modules/flat-entity/types/all-flat-entity-operation-record-by-metadata-name.type';
import { AllFlatEntityOperationByMetadataName } from 'src/engine/metadata-modules/flat-entity/types/flat-entity-to-create-delete-update.type';
import { getFlatEntityMapsExceptionContext } from 'src/engine/metadata-modules/flat-entity/utils/get-flat-entity-maps-exception-context.util';
import { getMetadataRelatedMetadataNamesForValidation } from 'src/engine/metadata-modules/flat-entity/utils/get-metadata-related-metadata-names-for-validation.util';
import { getSubFlatEntityMapsByApplicationIdsOrThrow } from 'src/engine/metadata-modules/flat-entity/utils/get-sub-flat-entity-maps-by-application-ids-or-throw.util';
import { transpileFlatEntityOperationArrayToRecord } from 'src/engine/metadata-modules/flat-entity/utils/transpile-flat-entity-operation-array-to-record.util';
import { MetadataSideEffectEngineService } from 'src/engine/metadata-modules/metadata-side-effect/services/metadata-side-effect-engine.service';
import { MetadataEventEmitter } from 'src/engine/subscriptions/metadata-event/metadata-event-emitter';
import { WorkspaceCacheService } from 'src/engine/workspace-cache/services/workspace-cache.service';
import { TWENTY_STANDARD_APPLICATION } from 'src/engine/workspace-manager/twenty-standard-application/constants/twenty-standard-applications';
import { WorkspaceMigrationV2Exception } from 'src/engine/workspace-manager/workspace-migration.exception';
import { WORKSPACE_MIGRATION_ADDITIONAL_CACHE_DATA_MAPS_KEY } from 'src/engine/workspace-manager/workspace-migration/constant/workspace-migration-additional-cache-data-maps-key.constant';
import {
enrichCreateWorkspaceMigrationActionsWithIds,
IdByUniversalIdentifierByMetadataName,
} from 'src/engine/workspace-manager/workspace-migration/services/utils/enrich-create-workspace-migration-action-with-ids.util';
import { WorkspaceMigrationBuildOrchestratorService } from 'src/engine/workspace-manager/workspace-migration/services/workspace-migration-build-orchestrator.service';
import { WorkspaceMigrationBuilderAdditionalCacheDataMaps } from 'src/engine/workspace-manager/workspace-migration/types/workspace-migration-builder-additional-cache-data-maps.type';
import { WorkspaceMigrationFlatEntityMapsService } from 'src/engine/workspace-manager/workspace-migration/services/workspace-migration-flat-entity-maps.service';
import {
FromToAllUniversalFlatEntityMaps,
WorkspaceMigrationOrchestratorBuildArgs,
WorkspaceMigrationOrchestratorFailedResult,
WorkspaceMigrationOrchestratorSuccessfulResult,
} from 'src/engine/workspace-manager/workspace-migration/types/workspace-migration-orchestrator.type';
import { computeUniversalFlatEntityMapsFromToThroughMutation } from 'src/engine/workspace-manager/workspace-migration/utils/compute-universal-flat-entity-maps-from-to-through-mutation.util';
import { InferDeletionFromMissingEntities } from 'src/engine/workspace-manager/workspace-migration/workspace-migration-builder/types/infer-deletion-from-missing-entities.type';
import { WorkspaceMigrationRunnerService } from 'src/engine/workspace-manager/workspace-migration/workspace-migration-runner/services/workspace-migration-runner.service';
type ValidateBuildAndRunWorkspaceMigrationFromMatriceArgs = {
workspaceId: string;
allFlatEntityOperationByMetadataName: {
[P in AllMetadataName]?: FlatEntityToCreateDeleteUpdate<P>;
};
allFlatEntityOperationByMetadataName: AllFlatEntityOperationByMetadataName;
isSystemBuild?: boolean;
applicationUniversalIdentifier: string;
dryRun?: boolean;
};
type ValidateBuildAndRunWorkspaceMigrationFromRecordArgs = {
workspaceId: string;
allFlatEntityOperationRecordByMetadataName: AllFlatEntityOperationRecordByMetadataName;
isSystemBuild?: boolean;
applicationUniversalIdentifier: string;
dryRun?: boolean;
};
@Injectable()
@@ -60,8 +50,9 @@ export class WorkspaceMigrationValidateBuildAndRunService {
constructor(
private readonly workspaceMigrationRunnerService: WorkspaceMigrationRunnerService,
private readonly workspaceMigrationBuildOrchestratorService: WorkspaceMigrationBuildOrchestratorService,
private readonly workspaceCacheService: WorkspaceCacheService,
private readonly workspaceMigrationFlatEntityMapsService: WorkspaceMigrationFlatEntityMapsService,
private readonly metadataEventEmitter: MetadataEventEmitter,
private readonly metadataSideEffectEngineService: MetadataSideEffectEngineService,
private readonly logger: LoggerService,
twentyConfigService: TwentyConfigService,
) {
@@ -70,291 +61,6 @@ export class WorkspaceMigrationValidateBuildAndRunService {
this.isDebugEnabled = logLevels.includes('debug');
}
private computeAllInvolvedApplicationIds({
allFlatEntityOperationByMetadataName,
flatApplicationMaps,
applicationUniversalIdentifier,
allRelatedFlatEntityMaps,
}: Pick<
ValidateBuildAndRunWorkspaceMigrationFromMatriceArgs,
'allFlatEntityOperationByMetadataName' | 'applicationUniversalIdentifier'
> & {
flatApplicationMaps: FlatApplicationCacheMaps;
allRelatedFlatEntityMaps: Partial<AllFlatEntityMaps>;
}): string[] {
const applicationIds = new Set<string>();
const applicationId =
flatApplicationMaps.idByUniversalIdentifier[
applicationUniversalIdentifier
];
const twentyStandardApplicationId =
flatApplicationMaps.idByUniversalIdentifier[
TWENTY_STANDARD_APPLICATION.universalIdentifier
];
if (!isDefined(twentyStandardApplicationId) || !isDefined(applicationId)) {
throw new FlatEntityMapsException(
'Application to build and its dependent application not found',
FlatEntityMapsExceptionCode.ENTITY_NOT_FOUND,
);
}
applicationIds.add(applicationId);
const isBuildingTwentyStandardApplication =
applicationUniversalIdentifier ===
TWENTY_STANDARD_APPLICATION.universalIdentifier;
if (!isBuildingTwentyStandardApplication) {
applicationIds.add(twentyStandardApplicationId);
}
for (const metadataName of Object.keys(
allFlatEntityOperationByMetadataName,
) as AllMetadataName[]) {
const flatEntityOperations =
allFlatEntityOperationByMetadataName[metadataName];
if (!isDefined(flatEntityOperations)) {
continue;
}
const { flatEntityToCreate, flatEntityToUpdate, flatEntityToDelete } =
flatEntityOperations;
const relations = ALL_MANY_TO_ONE_METADATA_RELATIONS[metadataName];
for (const flatEntity of [
...flatEntityToCreate,
...flatEntityToUpdate,
...flatEntityToDelete,
]) {
const entityApplicationId =
flatApplicationMaps.idByUniversalIdentifier[
flatEntity.applicationUniversalIdentifier
];
if (isDefined(entityApplicationId)) {
applicationIds.add(entityApplicationId);
}
for (const relation of Object.values(relations) as ({
foreignKey: string;
metadataName: AllMetadataName;
isNullable: boolean;
universalForeignKey: string;
} | null)[]) {
if (!isDefined(relation)) {
continue;
}
const { universalForeignKey, metadataName: targetMetadataName } =
relation;
const referencedUniversalIdentifier =
flatEntity[universalForeignKey as keyof typeof flatEntity];
if (!isDefined(referencedUniversalIdentifier)) {
continue;
}
const targetFlatEntityMaps =
allRelatedFlatEntityMaps[
getMetadataFlatEntityMapsKey(
targetMetadataName as AllMetadataName,
)
];
if (!isDefined(targetFlatEntityMaps)) {
continue;
}
const referencedEntity =
targetFlatEntityMaps.byUniversalIdentifier[
referencedUniversalIdentifier
];
if (isDefined(referencedEntity)) {
applicationIds.add(referencedEntity.applicationId);
}
}
}
}
return [...applicationIds];
}
private async computeAllRelatedFlatEntityMaps({
allFlatEntityOperationByMetadataName,
workspaceId,
applicationUniversalIdentifier,
}: ValidateBuildAndRunWorkspaceMigrationFromMatriceArgs) {
const allMetadataNameToCompare = Object.keys(
allFlatEntityOperationByMetadataName,
) as AllMetadataName[];
const allMetadataNameCacheToCompute = [
...new Set([
...allMetadataNameToCompare,
...allMetadataNameToCompare.flatMap(
getMetadataRelatedMetadataNamesForValidation,
),
]),
];
const allFlatEntityMapsCacheKeysToCompute =
allMetadataNameCacheToCompute.map(getMetadataFlatEntityMapsKey);
const { flatApplicationMaps, ...allRelatedFlatEntityMaps } =
await this.workspaceCacheService.getOrRecompute(workspaceId, [
...allFlatEntityMapsCacheKeysToCompute,
...WORKSPACE_MIGRATION_ADDITIONAL_CACHE_DATA_MAPS_KEY,
'flatApplicationMaps',
]);
const initialAccumulator = allMetadataNameCacheToCompute.reduce<
Partial<AllFlatEntityMaps>
>(
(allFlatEntityMaps, metadataName) => ({
...allFlatEntityMaps,
[getMetadataFlatEntityMapsKey(metadataName)]:
createEmptyFlatEntityMaps(),
}),
{},
);
const applicationIds = this.computeAllInvolvedApplicationIds({
allFlatEntityOperationByMetadataName,
flatApplicationMaps,
applicationUniversalIdentifier,
allRelatedFlatEntityMaps,
});
const dependencyAllFlatEntityMaps = allMetadataNameCacheToCompute.reduce(
(allFlatEntityMaps, metadataName) => {
const metadataFlatEntityMapsKey =
getMetadataFlatEntityMapsKey(metadataName);
return {
...allFlatEntityMaps,
[metadataFlatEntityMapsKey]:
getSubFlatEntityMapsByApplicationIdsOrThrow<
MetadataFlatEntity<typeof metadataName>
>({
applicationIds,
flatEntityMaps:
allRelatedFlatEntityMaps[metadataFlatEntityMapsKey],
}),
};
},
initialAccumulator,
);
const additionalCacheDataMaps =
WORKSPACE_MIGRATION_ADDITIONAL_CACHE_DATA_MAPS_KEY.reduce<WorkspaceMigrationBuilderAdditionalCacheDataMaps>(
(acc, additionalCacheDataMapsKey) => {
return {
...acc,
[additionalCacheDataMapsKey]:
allRelatedFlatEntityMaps[additionalCacheDataMapsKey],
};
},
{} as WorkspaceMigrationBuilderAdditionalCacheDataMaps,
);
return {
allRelatedFlatEntityMaps,
dependencyAllFlatEntityMaps,
additionalCacheDataMaps,
};
}
private async computeFromToAllFlatEntityMapsAndBuildOptions({
allFlatEntityOperationByMetadataName,
workspaceId,
applicationUniversalIdentifier,
}: ValidateBuildAndRunWorkspaceMigrationFromMatriceArgs): Promise<{
fromToAllFlatEntityMaps: FromToAllUniversalFlatEntityMaps;
inferDeletionFromMissingEntities: InferDeletionFromMissingEntities;
dependencyAllFlatEntityMaps: Partial<AllFlatEntityMaps>;
additionalCacheDataMaps: WorkspaceMigrationBuilderAdditionalCacheDataMaps;
idByUniversalIdentifierByMetadataName: IdByUniversalIdentifierByMetadataName;
}> {
const {
allRelatedFlatEntityMaps,
dependencyAllFlatEntityMaps,
additionalCacheDataMaps,
} = await this.computeAllRelatedFlatEntityMaps({
allFlatEntityOperationByMetadataName,
workspaceId,
applicationUniversalIdentifier,
});
const fromToAllFlatEntityMaps: FromToAllUniversalFlatEntityMaps = {};
const idByUniversalIdentifierByMetadataName: IdByUniversalIdentifierByMetadataName =
{};
const inferDeletionFromMissingEntities: InferDeletionFromMissingEntities =
{};
const allMetadataNameToCompare = Object.keys(
allFlatEntityOperationByMetadataName,
) as AllMetadataName[];
for (const metadataName of allMetadataNameToCompare) {
const flatEntityOperations =
allFlatEntityOperationByMetadataName[metadataName];
if (!isDefined(flatEntityOperations)) {
throw new FlatEntityMapsException(
`Could not load flat entity maps to compare for ${metadataName}, should never occur`,
FlatEntityMapsExceptionCode.INTERNAL_SERVER_ERROR,
);
}
const { flatEntityToCreate, flatEntityToDelete, flatEntityToUpdate } =
flatEntityOperations;
const idByUniversalIdentifier = Object.fromEntries(
flatEntityToCreate
.filter(
(
flatEntity,
): flatEntity is MetadataUniversalFlatEntity<
typeof metadataName
> & { id: string } => isDefined(flatEntity.id),
)
.map((flatEntity) => [flatEntity.universalIdentifier, flatEntity.id]),
);
if (Object.keys(idByUniversalIdentifier).length > 0) {
idByUniversalIdentifierByMetadataName[metadataName] =
idByUniversalIdentifier;
}
const flatEntityMapsKey = getMetadataFlatEntityMapsKey(metadataName);
const flatEntityMaps = allRelatedFlatEntityMaps[flatEntityMapsKey];
// @ts-expect-error Metadata flat entity maps cache key and metadataName colliding
fromToAllFlatEntityMaps[flatEntityMapsKey] =
computeUniversalFlatEntityMapsFromToThroughMutation({
flatEntityMaps: structuredClone(flatEntityMaps),
flatEntityToCreate,
flatEntityToDelete,
flatEntityToUpdate,
});
if (flatEntityToDelete.length > 0) {
inferDeletionFromMissingEntities[metadataName] = true;
}
}
return {
fromToAllFlatEntityMaps,
inferDeletionFromMissingEntities,
dependencyAllFlatEntityMaps,
additionalCacheDataMaps,
idByUniversalIdentifierByMetadataName,
};
}
public async validateBuildAndRunWorkspaceMigrationFromTo(
args: WorkspaceMigrationOrchestratorBuildArgs & {
idByUniversalIdentifierByMetadataName?: IdByUniversalIdentifierByMetadataName;
@@ -456,25 +162,87 @@ export class WorkspaceMigrationValidateBuildAndRunService {
}
public async validateBuildAndRunWorkspaceMigration({
allFlatEntityOperationByMetadataName: allFlatEntities,
allFlatEntityOperationByMetadataName,
workspaceId,
isSystemBuild = false,
applicationUniversalIdentifier,
dryRun,
}: ValidateBuildAndRunWorkspaceMigrationFromMatriceArgs): Promise<
| WorkspaceMigrationOrchestratorFailedResult
| WorkspaceMigrationOrchestratorSuccessfulResult
| (WorkspaceMigrationOrchestratorSuccessfulResult & {
hasSchemaMetadataChanged: boolean;
})
> {
return await this.validateBuildAndRunWorkspaceMigrationFromRecord({
allFlatEntityOperationRecordByMetadataName:
transpileFlatEntityOperationArrayToRecord(
allFlatEntityOperationByMetadataName,
),
workspaceId,
isSystemBuild,
applicationUniversalIdentifier,
dryRun,
});
}
public async validateBuildAndRunWorkspaceMigrationFromRecord({
allFlatEntityOperationRecordByMetadataName,
workspaceId,
isSystemBuild = false,
applicationUniversalIdentifier,
dryRun,
}: ValidateBuildAndRunWorkspaceMigrationFromRecordArgs): Promise<
| WorkspaceMigrationOrchestratorFailedResult
| (WorkspaceMigrationOrchestratorSuccessfulResult & {
hasSchemaMetadataChanged: boolean;
})
> {
const callerMetadataNames = Object.keys(
allFlatEntityOperationRecordByMetadataName,
) as AllMetadataName[];
const {
flatApplicationMaps,
allRelatedFlatEntityMaps,
allMetadataNameCacheToCompute,
} =
await this.workspaceMigrationFlatEntityMapsService.getOrRecomputeAllRelatedFlatEntityMaps(
{
workspaceId,
callerMetadataNames,
},
);
const sideEffectExpansionResult =
this.metadataSideEffectEngineService.expandWithSideEffects({
allFlatEntityOperationRecordByMetadataName,
sideEffectRelatedFlatEntityMaps: allRelatedFlatEntityMaps,
context: {
buildOptions: { isSystemBuild, applicationUniversalIdentifier },
},
});
if (sideEffectExpansionResult.status === 'fail') {
return sideEffectExpansionResult;
}
const {
fromToAllFlatEntityMaps,
inferDeletionFromMissingEntities,
dependencyAllFlatEntityMaps,
additionalCacheDataMaps,
idByUniversalIdentifierByMetadataName,
} = await this.computeFromToAllFlatEntityMapsAndBuildOptions({
allFlatEntityOperationByMetadataName: allFlatEntities,
workspaceId,
applicationUniversalIdentifier,
});
} =
this.workspaceMigrationFlatEntityMapsService.computeFromToAllFlatEntityMapsAndBuildOptions(
{
allFlatEntityOperationRecordByMetadataName:
sideEffectExpansionResult.allFlatEntityOperationRecordByMetadataName,
applicationUniversalIdentifier,
flatApplicationMaps,
allRelatedFlatEntityMaps,
allMetadataNameCacheToCompute,
},
);
return await this.validateBuildAndRunWorkspaceMigrationFromTo({
buildOptions: {
@@ -487,6 +255,7 @@ export class WorkspaceMigrationValidateBuildAndRunService {
dependencyAllFlatEntityMaps,
additionalCacheDataMaps,
idByUniversalIdentifierByMetadataName,
dryRun,
});
}
}
@@ -154,8 +154,8 @@ export class UpdateFieldActionHandlerService extends WorkspaceMigrationRunnerAct
// isUnique is derived from IndexMetadata at cache build time and has
// no underlying column on fieldMetadata. It travels in the update
// payload only so per-type validators (e.g. FILES rejection) can run
// — the actual state change is handled by the side-effect index
// create/delete in handleIndexChangesDuringFieldUpdate.
// — the actual state change is handled by the metadata side-effect
// engine, which owns the backing unique index lifecycle.
const { isUnique: _droppedIsUnique, ...persistedUpdate } = update;
if (Object.keys(persistedUpdate).length === 0) {
@@ -1,7 +1,9 @@
import { Module } from '@nestjs/common';
import { MetadataSideEffectModule } from 'src/engine/metadata-modules/metadata-side-effect/metadata-side-effect.module';
import { WorkspaceCacheModule } from 'src/engine/workspace-cache/workspace-cache.module';
import { WorkspaceMigrationBuildOrchestratorService } from 'src/engine/workspace-manager/workspace-migration/services/workspace-migration-build-orchestrator.service';
import { WorkspaceMigrationFlatEntityMapsService } from 'src/engine/workspace-manager/workspace-migration/services/workspace-migration-flat-entity-maps.service';
import { WorkspaceMigrationValidateBuildAndRunService } from 'src/engine/workspace-manager/workspace-migration/services/workspace-migration-validate-build-and-run-service';
import { WorkspaceMigrationBuilderModule } from 'src/engine/workspace-manager/workspace-migration/workspace-migration-builder/workspace-migration-builder.module';
import { WorkspaceMigrationRunnerModule } from 'src/engine/workspace-manager/workspace-migration/workspace-migration-runner/workspace-migration-runner.module';
@@ -11,10 +13,12 @@ import { WorkspaceMigrationRunnerModule } from 'src/engine/workspace-manager/wor
WorkspaceMigrationBuilderModule,
WorkspaceMigrationRunnerModule,
WorkspaceCacheModule,
MetadataSideEffectModule,
],
providers: [
WorkspaceMigrationValidateBuildAndRunService,
WorkspaceMigrationBuildOrchestratorService,
WorkspaceMigrationFlatEntityMapsService,
],
exports: [
WorkspaceMigrationValidateBuildAndRunService,
@@ -0,0 +1,37 @@
// Jest Snapshot v1, https://goo.gl/fbAQLP
exports[`Sync application should fail on unique index side-effect collision should fail when a declared index reuses the reserved deterministic identifier of the engine-owned unique backing index 1`] = `
{
"extensions": {
"code": "METADATA_VALIDATION_FAILED",
"errors": {
"index": [
{
"errors": [
{
"code": "RESERVED_SYSTEM_UNIVERSAL_IDENTIFIER",
"message": "Universal identifier is reserved for system-managed metadata",
"userFriendlyMessage": "This identifier is reserved by the system",
},
],
"flatEntityMinimalInformation": {
"name": "IDX_UNIQUE_790b15e5a08e02973ecff530bce",
"universalIdentifier": Any<String>,
},
"metadataName": "index",
"status": "fail",
"type": "create",
},
],
},
"message": "Validation failed for 1 index",
"summary": {
"index": 1,
"totalErrors": 1,
},
"userFriendlyMessage": "This identifier is reserved by the system",
},
"message": "Validation errors occurred while syncing application manifest metadata",
"name": "GraphQLError",
}
`;
@@ -0,0 +1,119 @@
import { expectOneNotInternalServerErrorSnapshot } from 'test/integration/graphql/utils/expect-one-not-internal-server-error-snapshot.util';
import { buildBaseManifest } from 'test/integration/metadata/suites/application/utils/build-base-manifest.util';
import { buildDefaultObjectManifest } from 'test/integration/metadata/suites/application/utils/build-default-object-manifest.util';
import { cleanupApplicationAndAppRegistration } from 'test/integration/metadata/suites/application/utils/cleanup-application-and-app-registration.util';
import { setupApplicationForSync } from 'test/integration/metadata/suites/application/utils/setup-application-for-sync.util';
import { syncApplication } from 'test/integration/metadata/suites/application/utils/sync-application.util';
import {
getIndexUniversalIdentifier,
type Manifest,
} from 'twenty-shared/application';
import { FieldMetadataType } from 'twenty-shared/types';
import { v4 as uuidv4 } from 'uuid';
import { generateDeterministicIndexName } from 'src/engine/metadata-modules/index-metadata/utils/generate-deterministic-index-name';
import { type UniversalFlatObjectMetadata } from 'src/engine/workspace-manager/workspace-migration/universal-flat-entity/types/universal-flat-object-metadata.type';
const TEST_APP_ID = uuidv4();
const TEST_ROLE_ID = uuidv4();
const OBJECT_NAME_SINGULAR = 'collidingIndexObject';
const UNIQUE_FIELD_NAME = 'externalId';
const computeSystemUniqueIndexUniversalIdentifier = (
objectUniversalIdentifier: string,
): string => {
const name = generateDeterministicIndexName({
flatObjectMetadata: {
nameSingular: OBJECT_NAME_SINGULAR,
applicationUniversalIdentifier: TEST_APP_ID,
} as UniversalFlatObjectMetadata,
orderedIndexColumnNames: [UNIQUE_FIELD_NAME],
isUnique: true,
indexWhereClause: null,
});
return getIndexUniversalIdentifier({
applicationUniversalIdentifier: TEST_APP_ID,
objectUniversalIdentifier,
name,
});
};
// The object is otherwise fully valid (all system fields present) so the reserved-identifier
// violation is the only validation error surfaced.
const buildCollidingManifest = (): Manifest => {
const objectUniversalIdentifier = uuidv4();
const uniqueFieldUniversalIdentifier = uuidv4();
const object = buildDefaultObjectManifest({
nameSingular: OBJECT_NAME_SINGULAR,
namePlural: 'collidingIndexObjects',
labelSingular: 'Colliding Index Object',
labelPlural: 'Colliding Index Objects',
description: 'Object used to test unique index side-effect collision',
universalIdentifier: objectUniversalIdentifier,
additionalFields: [
{
universalIdentifier: uniqueFieldUniversalIdentifier,
type: FieldMetadataType.TEXT,
name: UNIQUE_FIELD_NAME,
label: 'External ID',
description: 'Unique external identifier',
icon: 'IconId',
isUnique: true,
isNullable: true,
},
],
});
return buildBaseManifest({
appId: TEST_APP_ID,
roleId: TEST_ROLE_ID,
overrides: {
objects: [object],
fields: [],
indexes: [
{
universalIdentifier: computeSystemUniqueIndexUniversalIdentifier(
objectUniversalIdentifier,
),
objectUniversalIdentifier,
isUnique: true,
fields: [
{
universalIdentifier: uuidv4(),
fieldUniversalIdentifier: uniqueFieldUniversalIdentifier,
},
],
},
],
},
});
};
describe('Sync application should fail on unique index side-effect collision', () => {
beforeAll(async () => {
await setupApplicationForSync({
applicationUniversalIdentifier: TEST_APP_ID,
name: 'Test Unique Index Collision App',
description: 'App for testing unique index side-effect collision',
sourcePath: 'test-unique-index-collision',
});
}, 60000);
afterAll(async () => {
await cleanupApplicationAndAppRegistration({
applicationUniversalIdentifier: TEST_APP_ID,
});
});
it('should fail when a declared index reuses the reserved deterministic identifier of the engine-owned unique backing index', async () => {
const { errors } = await syncApplication({
manifest: buildCollidingManifest(),
expectToFail: true,
});
expectOneNotInternalServerErrorSnapshot({ errors });
}, 60000);
});
@@ -2,9 +2,13 @@ import { buildBaseManifest } from 'test/integration/metadata/suites/application/
import { cleanupApplicationAndAppRegistration } from 'test/integration/metadata/suites/application/utils/cleanup-application-and-app-registration.util';
import { setupApplicationForSync } from 'test/integration/metadata/suites/application/utils/setup-application-for-sync.util';
import { syncApplication } from 'test/integration/metadata/suites/application/utils/sync-application.util';
import { findManyObjectMetadataWithIndexes } from 'test/integration/metadata/suites/object-metadata/utils/find-many-object-metadata-with-indexes.util';
import { findViewFields } from 'test/integration/metadata/suites/view-field/utils/find-view-fields.util';
import { findViews } from 'test/integration/metadata/suites/view/utils/find-views.util';
import type { FieldManifest } from 'twenty-shared/application';
import { STANDARD_OBJECTS } from 'twenty-shared/metadata';
import { FieldMetadataType, ViewType } from 'twenty-shared/types';
import { isDefined } from 'twenty-shared/utils';
import { v4 as uuidv4 } from 'uuid';
const APP_A_ID = uuidv4();
@@ -19,6 +23,8 @@ const APP_B_VIEW_FIELD_ID = uuidv4();
const PERSON_OBJECT_UNIVERSAL_IDENTIFIER =
STANDARD_OBJECTS.person.universalIdentifier;
const APP_A_VIEW_NAME = 'App A View';
const appBPersonField: FieldManifest = {
universalIdentifier: APP_B_FIELD_ID,
type: FieldMetadataType.TEXT,
@@ -29,7 +35,7 @@ const appBPersonField: FieldManifest = {
objectUniversalIdentifier: PERSON_OBJECT_UNIVERSAL_IDENTIFIER,
};
describe('Sync application should fail when creating a view field on a view owned by another app', () => {
describe('Sync application should succeed when extending another app view with a view field', () => {
beforeAll(async () => {
await setupApplicationForSync({
applicationUniversalIdentifier: APP_A_ID,
@@ -41,7 +47,7 @@ describe('Sync application should fail when creating a view field on a view owne
await setupApplicationForSync({
applicationUniversalIdentifier: APP_B_ID,
name: 'App B',
description: 'App attempting to add a view field on App A view',
description: 'App extending App A view with a view field',
sourcePath: 'test-cross-app-view-field-app-b',
});
@@ -50,10 +56,19 @@ describe('Sync application should fail when creating a view field on a view owne
appId: APP_A_ID,
roleId: APP_A_ROLE_ID,
overrides: {
// Role labels are unique workspace-wide (PG constraint), so each app
// must ship a distinct label
roles: [
{
universalIdentifier: APP_A_ROLE_ID,
label: 'App A Role',
description: 'Role owned by App A',
},
],
views: [
{
universalIdentifier: APP_A_VIEW_ID,
name: 'App A View',
name: APP_A_VIEW_NAME,
objectUniversalIdentifier: PERSON_OBJECT_UNIVERSAL_IDENTIFIER,
type: ViewType.TABLE,
icon: 'IconList',
@@ -74,12 +89,19 @@ describe('Sync application should fail when creating a view field on a view owne
});
});
it('rejects a standalone view field from App B targeting an App A view', async () => {
it('accepts a standalone view field from App B targeting an App A view', async () => {
const { errors } = await syncApplication({
manifest: buildBaseManifest({
appId: APP_B_ID,
roleId: APP_B_ROLE_ID,
overrides: {
roles: [
{
universalIdentifier: APP_B_ROLE_ID,
label: 'App B Role',
description: 'Role owned by App B',
},
],
fields: [appBPersonField],
viewFields: [
{
@@ -93,17 +115,49 @@ describe('Sync application should fail when creating a view field on a view owne
],
},
}),
expectToFail: true,
expectToFail: false,
});
expect(errors).toBeDefined();
expect(errors.length).toBeGreaterThan(0);
expect(isDefined(errors)).toBe(false);
const [error] = errors;
const objects = await findManyObjectMetadataWithIndexes({
expectToFail: false,
});
expect(error.extensions.code).toBe('METADATA_VALIDATION_FAILED');
expect(error.extensions.summary.totalErrors).toBe(1);
expect(error.extensions.summary.viewField).toBe(1);
expect(error.extensions.message).toMatch(/viewField/);
const personObject = objects.find(
(objectMetadata) =>
objectMetadata.universalIdentifier ===
PERSON_OBJECT_UNIVERSAL_IDENTIFIER,
);
expect(personObject).toBeDefined();
const appBField = personObject?.fieldsList.find(
(field) => field.name === appBPersonField.name,
);
expect(appBField).toBeDefined();
const { data: viewsData } = await findViews({
objectMetadataId: personObject?.id,
expectToFail: false,
});
const appAView = viewsData?.getViews.find(
(view) => view.name === APP_A_VIEW_NAME,
);
expect(appAView).toBeDefined();
const { data: viewFieldsData } = await findViewFields({
viewId: appAView?.id ?? '',
expectToFail: false,
});
const appBViewField = (viewFieldsData?.getViewFields ?? []).find(
(viewField) => viewField.fieldMetadataId === appBField?.id,
);
expect(appBViewField).toBeDefined();
}, 60000);
});
@@ -0,0 +1,245 @@
import { buildBaseManifest } from 'test/integration/metadata/suites/application/utils/build-base-manifest.util';
import { buildDefaultObjectManifest } from 'test/integration/metadata/suites/application/utils/build-default-object-manifest.util';
import { cleanupApplicationAndAppRegistration } from 'test/integration/metadata/suites/application/utils/cleanup-application-and-app-registration.util';
import { setupApplicationForSync } from 'test/integration/metadata/suites/application/utils/setup-application-for-sync.util';
import { syncApplication } from 'test/integration/metadata/suites/application/utils/sync-application.util';
import { findManyObjectMetadataWithIndexes } from 'test/integration/metadata/suites/object-metadata/utils/find-many-object-metadata-with-indexes.util';
import {
getFieldUniversalIdentifier,
type FieldManifest,
} from 'twenty-shared/application';
import { FieldMetadataType } from 'twenty-shared/types';
import { v4 as uuidv4 } from 'uuid';
import { generateDeterministicIndexName } from 'src/engine/metadata-modules/index-metadata/utils/generate-deterministic-index-name';
import { type UniversalFlatObjectMetadata } from 'src/engine/workspace-manager/workspace-migration/universal-flat-entity/types/universal-flat-object-metadata.type';
const TEST_APP_ID = uuidv4();
const TEST_ROLE_ID = uuidv4();
const TEST_OBJECT = buildDefaultObjectManifest({
nameSingular: 'uniqueRenameObject',
namePlural: 'uniqueRenameObjects',
labelSingular: 'Unique Rename Object',
labelPlural: 'Unique Rename Objects',
description: 'Object used to test unique field renames',
icon: 'IconTag',
});
const buildUniqueFieldManifest = ({
universalIdentifier,
name,
}: {
universalIdentifier: string;
name: string;
}): FieldManifest => ({
universalIdentifier,
type: FieldMetadataType.TEXT,
name,
label: 'Unique Reference',
description: 'Unique reference field',
icon: 'IconId',
isUnique: true,
isNullable: true,
objectUniversalIdentifier: TEST_OBJECT.universalIdentifier,
});
const buildManifest = (fields: FieldManifest[]) =>
buildBaseManifest({
appId: TEST_APP_ID,
roleId: TEST_ROLE_ID,
overrides: { objects: [TEST_OBJECT], fields },
});
// Mirrors the engine's derivation so the test asserts the exact backing index
// the side-effect engine is expected to own for a given field name.
const computeExpectedUniqueIndexName = (fieldName: string): string =>
generateDeterministicIndexName({
flatObjectMetadata: {
nameSingular: TEST_OBJECT.nameSingular,
applicationUniversalIdentifier: TEST_APP_ID,
} as UniversalFlatObjectMetadata,
orderedIndexColumnNames: [fieldName],
isUnique: true,
indexWhereClause: null,
});
const findTestObject = async () => {
const objects = await findManyObjectMetadataWithIndexes({
expectToFail: false,
});
const object = objects.find(
(objectMetadata) =>
objectMetadata.universalIdentifier === TEST_OBJECT.universalIdentifier,
);
expect(object).toBeDefined();
return object!;
};
describe('Sync application unique field rename', () => {
beforeEach(async () => {
await setupApplicationForSync({
applicationUniversalIdentifier: TEST_APP_ID,
name: 'Unique Field Rename App',
description: 'App for testing unique field rename behaviors',
sourcePath: 'test-unique-field-rename',
});
}, 60000);
afterEach(async () => {
await cleanupApplicationAndAppRegistration({
applicationUniversalIdentifier: TEST_APP_ID,
});
});
it('should drop and recreate the field and its backing unique index when the rename changes the field universal identifier', async () => {
const computeNameDerivedFieldUniversalIdentifier = (name: string) =>
getFieldUniversalIdentifier({
applicationUniversalIdentifier: TEST_APP_ID,
objectUniversalIdentifier: TEST_OBJECT.universalIdentifier,
name,
});
await syncApplication({
manifest: buildManifest([
buildUniqueFieldManifest({
universalIdentifier:
computeNameDerivedFieldUniversalIdentifier('tata'),
name: 'tata',
}),
]),
expectToFail: false,
});
const objectAfterFirstSync = await findTestObject();
const tataField = objectAfterFirstSync.fieldsList.find(
(field) => field.name === 'tata',
);
expect(tataField).toBeDefined();
expect(
objectAfterFirstSync.indexMetadataList.find(
(index) => index.name === computeExpectedUniqueIndexName('tata'),
),
).toBeDefined();
await syncApplication({
manifest: buildManifest([
buildUniqueFieldManifest({
universalIdentifier:
computeNameDerivedFieldUniversalIdentifier('toto'),
name: 'toto',
}),
]),
expectToFail: false,
});
const objectAfterSecondSync = await findTestObject();
const totoField = objectAfterSecondSync.fieldsList.find(
(field) => field.name === 'toto',
);
expect(totoField).toBeDefined();
expect(totoField?.universalIdentifier).toBe(
computeNameDerivedFieldUniversalIdentifier('toto'),
);
// A different metadata id proves the field was dropped and recreated,
// not updated in place.
expect(totoField?.id).not.toBe(tataField?.id);
expect(
objectAfterSecondSync.fieldsList.find((field) => field.name === 'tata'),
).toBeUndefined();
const totoUniqueIndex = objectAfterSecondSync.indexMetadataList.find(
(index) => index.name === computeExpectedUniqueIndexName('toto'),
);
expect(totoUniqueIndex).toBeDefined();
expect(totoUniqueIndex?.isUnique).toBe(true);
expect(
totoUniqueIndex?.indexFieldMetadataList.some(
(indexField) => indexField.fieldMetadataId === totoField?.id,
),
).toBe(true);
expect(
objectAfterSecondSync.indexMetadataList.find(
(index) => index.name === computeExpectedUniqueIndexName('tata'),
),
).toBeUndefined();
}, 60000);
it('should update the field in place and swap its backing unique index when the rename keeps a static universal identifier', async () => {
const staticFieldUniversalIdentifier = uuidv4();
await syncApplication({
manifest: buildManifest([
buildUniqueFieldManifest({
universalIdentifier: staticFieldUniversalIdentifier,
name: 'tata',
}),
]),
expectToFail: false,
});
const objectAfterFirstSync = await findTestObject();
const tataField = objectAfterFirstSync.fieldsList.find(
(field) => field.name === 'tata',
);
expect(tataField).toBeDefined();
expect(tataField?.universalIdentifier).toBe(staticFieldUniversalIdentifier);
expect(
objectAfterFirstSync.indexMetadataList.find(
(index) => index.name === computeExpectedUniqueIndexName('tata'),
),
).toBeDefined();
await syncApplication({
manifest: buildManifest([
buildUniqueFieldManifest({
universalIdentifier: staticFieldUniversalIdentifier,
name: 'toto',
}),
]),
expectToFail: false,
});
const objectAfterSecondSync = await findTestObject();
const renamedField = objectAfterSecondSync.fieldsList.find(
(field) => field.universalIdentifier === staticFieldUniversalIdentifier,
);
expect(renamedField).toBeDefined();
expect(renamedField?.name).toBe('toto');
// The same metadata id proves the field was updated in place, not dropped
// and recreated.
expect(renamedField?.id).toBe(tataField?.id);
expect(
objectAfterSecondSync.fieldsList.find((field) => field.name === 'tata'),
).toBeUndefined();
const totoUniqueIndex = objectAfterSecondSync.indexMetadataList.find(
(index) => index.name === computeExpectedUniqueIndexName('toto'),
);
expect(totoUniqueIndex).toBeDefined();
expect(totoUniqueIndex?.isUnique).toBe(true);
expect(
totoUniqueIndex?.indexFieldMetadataList.some(
(indexField) => indexField.fieldMetadataId === renamedField?.id,
),
).toBe(true);
expect(
objectAfterSecondSync.indexMetadataList.find(
(index) => index.name === computeExpectedUniqueIndexName('tata'),
),
).toBeUndefined();
}, 60000);
});