Centralized side effects devxp basis (#22295)

# Introduction

This PR introduces a centralized, strictly-typed **metadata side-effect
engine** that unifies how system metadata side effects are derived and
applied across both metadata entry points — the **metadata GraphQL API**
and the **application sync / manifest** flow — and migrates the first
side effect end-to-end: **a unique scalar field owns its backing
single-field `UNIQUE` index** (full create / update / delete lifecycle).

## New conventions

- **Engine-owned companions**: metadata flagged `isSystemSideEffect:
true` is owned by the engine. Its deletion is never inferred from
absence in a manifest — it results from PG-level cascade or from a
delete side effect (a side effect always has a cause, its parent
metadata).
- **Reserved deterministic identifiers**: apps cannot declare metadata
reusing an engine-owned deterministic `universalIdentifier`. Doing so
fails validation with `RESERVED_SYSTEM_UNIVERSAL_IDENTIFIER` (until an
explicit override API exists).
- **Record-native operation matrix**: the operation matrix is keyed by
`universalIdentifier` (`AllFlatEntityOperationRecordByMetadataName`)
instead of arrays, making parent resolution and deduplication O(1).
Array-based API callers are transpiled to records at the
validate-build-and-run boundary.
- Twenty-sdk user-facing experience with system fields will only be
related to overrides.

# What this PR does

## 1. Side-effect engine (foundation)

- `MetadataSideEffectEngineService.expandWithSideEffects(...)` takes the
intention-carrying record matrix and returns it expanded with derived
side effects, or a structured failure.
- Handlers are registered via a typed **decorator + registry** pattern
(`MetadataSideEffectHandler({ operation, metadataName, name, description
})`), with runtime duplicate-name detection. Multiple handlers per
(operation, metadataName) are supported.
- Handler contract mirrors the validator pattern:
- receives the trigger flat entity, the live record matrix, and
**strictly-typed related flat entity maps**
(`MetadataFlatEntityAndRelatedFlatEntityMapsForSideEffect<P>`, derived
from declared companion metadata names — no loose
`Partial<AllFlatEntityMaps>` context)
- returns `MetadataSideEffectResult`: `success` (operations record) |
`noop` | `fail` (structured failure)
- **Non-recursion is structural**: triggers are read from the original
caller input, never from the expanded matrix, so a side effect can never
trigger another side effect.
- **Deduplication + collision detection**: side effects are deduped by
`universalIdentifier` per operation; a caller-declared entity colliding
with an engine-owned deterministic identifier is recorded as a
collision.
- **Unified failure channel**: handler failures and reserved-identifier
collisions are merged into the same `OrchestratorFailureReport` contract
as builder validation errors, and the run short-circuits (fail-closed,
nothing is applied).

## 2. First migrated side effect — unique field → backing unique index

Three handlers own the complete lifecycle of the deterministic
single-field `UNIQUE` index backing a unique scalar field:

- **create**: unique scalar field → generate the deterministic backing
index (`fieldUniqueBackingIndexOnCreate`)
- **update**: `isUnique` flips and renames of still-unique fields (the
index name — and therefore its deterministic identifier — derives from
the field name, so a rename drops the stale index and recreates the
deterministic one) (`fieldUniqueBackingIndexOnUpdate`)
- **delete**: cascade-delete the backing index
(`fieldUniqueBackingIndexOnDelete`)

Supporting rules:
- The primary key `id` field never spawns a backing index (uniqueness
comes from the PK constraint) — explicit `isPrimaryKeyFlatFieldMetadata`
guard.
- Parent object resolution is **optimistic-first**: an object created or
updated in the same batch wins over the workspace cache (so e.g.
renaming an object while flipping a field to unique builds the index
from the post-rename object), resolved in O(1) via the record matrix.
- A missing parent object is reported as a structured side-effect
failure, never silently skipped.

## 3. Path convergence — manifest and API share one flow

- The manifest sync now derives a from→to **record matrix** from the
cache and feeds `validateBuildAndRunWorkspaceMigrationFromRecord`, the
same flow the API uses — both paths converge on the engine.
- Manifest-side unique-index generation and API transpiler
system-unique-index handling were removed (declared/composite/relation
indexes stay untouched).
- New `WorkspaceMigrationFlatEntityMapsService` mutualizes
flat-entity-maps computation between the side-effect engine and the
builder: cache keys are derived from the caller metadata names (+
validation- and side-effect-related closures) instead of hardcoded
loads.
- App-scoping and pruning are folded into one shared primitive
(`getSubAllFlatEntityMapsByApplicationIdsOrThrow`): slicing dependency
maps to the involved applications always prunes dangling one-to-many
aggregators — callers can no longer forget it.
- **Behavior change**: an app extending another app's view with a view
field now syncs successfully (cross-app view-field extension), covered
by a dedicated integration test.

## 4. Backfill upgrade command (2.19)

`upgrade:2-19:backfill-system-unique-index-universal-identifier`
rewrites legacy system unique-index `universalIdentifier`s to their
deterministic value so the engine can own pre-existing indexes. The
backfill is **driven from `isUnique: true` fields** (mirroring the
engine ownership predicate — excludes PK / morph / relation fields) and
resolves each field's backing index in O(1).

# Bugs fixed along the way

- `database:reset` seeding failed with
`INDEX_FIELD_INVALID_DEFAULT_VALUE`: the engine derived a backing
`UNIQUE` index for the default `id` primary key. Fixed with the explicit
primary-key guard.
- `isUnique` updates on system-flagged standard fields (e.g.
auto-created `name`) did not trigger the backing-index side effect.
- Manifest sync crashed with "Could not find flat entity with universal
identifier ..." when app-scoped slices left dangling aggregator
references — fixed by centralizing pruning in the shared slice primitive
This commit is contained in:
Paul Rastoin
2026-07-03 18:13:20 +02:00
committed by GitHub
parent 566c3b6629
commit 43730d7748
59 changed files with 2840 additions and 691 deletions
@@ -0,0 +1,7 @@
import { type AllMetadataName } from 'twenty-shared/metadata';
export const ALL_METADATA_SIDE_EFFECT_COMPANION_METADATA_NAMES = {
fieldMetadata: ['index'],
} as const satisfies Partial<
Record<AllMetadataName, readonly AllMetadataName[]>
>;
@@ -0,0 +1,15 @@
import { type AllMetadataName } from 'twenty-shared/metadata';
import { type MetadataUniversalFlatEntity } from 'src/engine/metadata-modules/flat-entity/types/metadata-universal-flat-entity.type';
export type FlatEntityOperationRecord<T extends AllMetadataName> = {
flatEntityToCreate: Partial<
Record<string, MetadataUniversalFlatEntity<T> & { id?: string }>
>;
flatEntityToUpdate: Partial<Record<string, MetadataUniversalFlatEntity<T>>>;
flatEntityToDelete: Partial<Record<string, MetadataUniversalFlatEntity<T>>>;
};
export type AllFlatEntityOperationRecordByMetadataName = {
[P in AllMetadataName]?: FlatEntityOperationRecord<P>;
};
@@ -7,3 +7,7 @@ export type FlatEntityToCreateDeleteUpdate<T extends AllMetadataName> = {
flatEntityToCreate: (MetadataUniversalFlatEntity<T> & { id?: string })[];
flatEntityToDelete: MetadataUniversalFlatEntity<T>[];
};
export type AllFlatEntityOperationByMetadataName = {
[P in AllMetadataName]?: FlatEntityToCreateDeleteUpdate<P>;
};
@@ -0,0 +1,13 @@
import { type AllMetadataName } from 'twenty-shared/metadata';
import { type AllFlatEntityMaps } from 'src/engine/metadata-modules/flat-entity/types/all-flat-entity-maps.type';
import { type MetadataSideEffectRelatedMetadataNames } from 'src/engine/metadata-modules/flat-entity/types/metadata-side-effect-related-metadata-names.type';
import { type MetadataToFlatEntityMapsKey } from 'src/engine/metadata-modules/flat-entity/types/metadata-to-flat-entity-maps-key';
export type MetadataFlatEntityAndRelatedFlatEntityMapsForSideEffect<
T extends AllMetadataName,
> = Pick<
AllFlatEntityMaps,
| MetadataToFlatEntityMapsKey<T>
| MetadataToFlatEntityMapsKey<MetadataSideEffectRelatedMetadataNames<T>>
>;
@@ -0,0 +1,14 @@
import { type AllMetadataName } from 'twenty-shared/metadata';
import { type ALL_METADATA_SIDE_EFFECT_COMPANION_METADATA_NAMES } from 'src/engine/metadata-modules/flat-entity/constant/all-metadata-side-effect-companion-metadata-names.constant';
import { type MetadataManyToOneRelatedMetadataNames } from 'src/engine/metadata-modules/flat-entity/types/metadata-many-to-one-related-metadata-names.type';
export type MetadataSideEffectCompanionMetadataNames<
T extends AllMetadataName,
> = T extends keyof typeof ALL_METADATA_SIDE_EFFECT_COMPANION_METADATA_NAMES
? (typeof ALL_METADATA_SIDE_EFFECT_COMPANION_METADATA_NAMES)[T][number]
: never;
export type MetadataSideEffectRelatedMetadataNames<T extends AllMetadataName> =
| MetadataManyToOneRelatedMetadataNames<T>
| MetadataSideEffectCompanionMetadataNames<T>;
@@ -0,0 +1,16 @@
import { type AllMetadataName } from 'twenty-shared/metadata';
import { ALL_METADATA_SIDE_EFFECT_COMPANION_METADATA_NAMES } from 'src/engine/metadata-modules/flat-entity/constant/all-metadata-side-effect-companion-metadata-names.constant';
import { type MetadataSideEffectCompanionMetadataNames } from 'src/engine/metadata-modules/flat-entity/types/metadata-side-effect-related-metadata-names.type';
export const getMetadataSideEffectCompanionNames = <T extends AllMetadataName>(
metadataName: T,
): MetadataSideEffectCompanionMetadataNames<T>[] => {
const companionMetadataNamesByMetadataName =
ALL_METADATA_SIDE_EFFECT_COMPANION_METADATA_NAMES as Partial<
Record<AllMetadataName, readonly AllMetadataName[]>
>;
return (companionMetadataNamesByMetadataName[metadataName] ??
[]) as MetadataSideEffectCompanionMetadataNames<T>[];
};
@@ -0,0 +1,51 @@
import { type AllMetadataName } from 'twenty-shared/metadata';
import { isDefined } from 'twenty-shared/utils';
import {
FlatEntityMapsException,
FlatEntityMapsExceptionCode,
} from 'src/engine/metadata-modules/flat-entity/exceptions/flat-entity-maps.exception';
import { type AllFlatEntityMaps } from 'src/engine/metadata-modules/flat-entity/types/all-flat-entity-maps.type';
import { type MetadataFlatEntity } from 'src/engine/metadata-modules/flat-entity/types/metadata-flat-entity.type';
import { getMetadataFlatEntityMapsKey } from 'src/engine/metadata-modules/flat-entity/utils/get-metadata-flat-entity-maps-key.util';
import { getSubFlatEntityMapsByApplicationIdsOrThrow } from 'src/engine/metadata-modules/flat-entity/utils/get-sub-flat-entity-maps-by-application-ids-or-throw.util';
import { pruneDanglingForeignKeyAggregatorsInAllFlatEntityMapsThroughMutation } from 'src/engine/metadata-modules/flat-entity/utils/prune-dangling-foreign-key-aggregators-in-all-flat-entity-maps-through-mutation.util';
export const getSubAllFlatEntityMapsByApplicationIdsOrThrow = ({
applicationIds,
metadataNames,
fromAllFlatEntityMaps,
}: {
applicationIds: string[];
metadataNames: AllMetadataName[];
fromAllFlatEntityMaps: Partial<AllFlatEntityMaps>;
}): Partial<AllFlatEntityMaps> => {
const subAllFlatEntityMaps: Partial<AllFlatEntityMaps> = {};
for (const metadataName of metadataNames) {
const flatEntityMapsKey = getMetadataFlatEntityMapsKey(metadataName);
const fromFlatEntityMaps = fromAllFlatEntityMaps[flatEntityMapsKey];
if (!isDefined(fromFlatEntityMaps)) {
throw new FlatEntityMapsException(
`Missing flat entity maps for metadata "${metadataName}" while building application-scoped slice`,
FlatEntityMapsExceptionCode.INTERNAL_SERVER_ERROR,
);
}
// @ts-expect-error Metadata flat entity maps cache key and metadataName colliding
subAllFlatEntityMaps[flatEntityMapsKey] =
getSubFlatEntityMapsByApplicationIdsOrThrow<
MetadataFlatEntity<typeof metadataName>
>({
applicationIds,
flatEntityMaps: fromFlatEntityMaps,
});
}
pruneDanglingForeignKeyAggregatorsInAllFlatEntityMapsThroughMutation({
allFlatEntityMapsToMutate: subAllFlatEntityMaps,
});
return subAllFlatEntityMaps;
};
@@ -0,0 +1,8 @@
import { type AllMetadataName } from 'twenty-shared/metadata';
import { type MetadataUniversalFlatEntity } from 'src/engine/metadata-modules/flat-entity/types/metadata-universal-flat-entity.type';
export const isSystemSideEffectFlatEntity = <T extends AllMetadataName>(
flatEntity: MetadataUniversalFlatEntity<T>,
): boolean =>
'isSystemSideEffect' in flatEntity && flatEntity.isSystemSideEffect === true;
@@ -74,35 +74,49 @@ export const pruneDanglingForeignKeyAggregatorsInAllFlatEntityMapsThroughMutatio
({
allFlatEntityMapsToMutate,
}: {
allFlatEntityMapsToMutate: AllFlatEntityMaps;
allFlatEntityMapsToMutate: Partial<AllFlatEntityMaps>;
}): void => {
const looseAllFlatEntityMaps =
allFlatEntityMapsToMutate as unknown as LooseAllFlatEntityMaps;
allFlatEntityMapsToMutate as unknown as Partial<LooseAllFlatEntityMaps>;
for (const metadataName of Object.values(ALL_METADATA_NAME)) {
const parentFlatEntityMaps =
looseAllFlatEntityMaps[getMetadataFlatEntityMapsKey(metadataName)];
if (!isDefined(parentFlatEntityMaps)) {
continue;
}
const oneToManyRelations = Object.values(
ALL_ONE_TO_MANY_METADATA_RELATIONS[metadataName],
) as OneToManyRelation[];
const aggregatorsWithChildIdentifiers: OneToManyAggregatorWithChildIdentifiers[] =
oneToManyRelations.filter(isDefined).map((relation) => ({
aggregatorProperty: relation.universalFlatEntityForeignKeyAggregator,
childUniversalIdentifiers: new Set(
Object.keys(
looseAllFlatEntityMaps[
getMetadataFlatEntityMapsKey(relation.metadataName)
].byUniversalIdentifier,
),
),
}));
oneToManyRelations.filter(isDefined).flatMap((relation) => {
const childFlatEntityMaps =
looseAllFlatEntityMaps[
getMetadataFlatEntityMapsKey(relation.metadataName)
];
const childUniversalIdentifiers = isDefined(childFlatEntityMaps)
? new Set(Object.keys(childFlatEntityMaps.byUniversalIdentifier))
: new Set<string>();
return [
{
aggregatorProperty:
relation.universalFlatEntityForeignKeyAggregator,
childUniversalIdentifiers,
},
];
});
if (aggregatorsWithChildIdentifiers.length === 0) {
continue;
}
const flatEntityByUniversalIdentifier =
looseAllFlatEntityMaps[getMetadataFlatEntityMapsKey(metadataName)]
.byUniversalIdentifier;
parentFlatEntityMaps.byUniversalIdentifier;
for (const [universalIdentifier, parentFlatEntity] of Object.entries(
flatEntityByUniversalIdentifier,
@@ -0,0 +1,94 @@
import { isDefined } from 'twenty-shared/utils';
import {
FlatEntityMapsException,
FlatEntityMapsExceptionCode,
} from 'src/engine/metadata-modules/flat-entity/exceptions/flat-entity-maps.exception';
import { type AllFlatEntityOperationRecordByMetadataName } from 'src/engine/metadata-modules/flat-entity/types/all-flat-entity-operation-record-by-metadata-name.type';
import { type AllFlatEntityOperationByMetadataName } from 'src/engine/metadata-modules/flat-entity/types/flat-entity-to-create-delete-update.type';
type GenericUniversalFlatEntity = { universalIdentifier: string };
const toRecordByUniversalIdentifierOrThrow = ({
flatEntities,
metadataName,
operation,
}: {
flatEntities: GenericUniversalFlatEntity[];
metadataName: string;
operation: 'flatEntityToCreate' | 'flatEntityToUpdate' | 'flatEntityToDelete';
}): Record<string, GenericUniversalFlatEntity> => {
const recordByUniversalIdentifier: Record<
string,
GenericUniversalFlatEntity
> = {};
const seenUniversalIdentifiers = new Set<string>();
for (const flatEntity of flatEntities) {
if (seenUniversalIdentifiers.has(flatEntity.universalIdentifier)) {
throw new FlatEntityMapsException(
`Duplicate universalIdentifier "${flatEntity.universalIdentifier}" in ${operation} for metadata "${metadataName}"`,
FlatEntityMapsExceptionCode.ENTITY_ALREADY_EXISTS,
);
}
seenUniversalIdentifiers.add(flatEntity.universalIdentifier);
recordByUniversalIdentifier[flatEntity.universalIdentifier] = flatEntity;
}
return recordByUniversalIdentifier;
};
// Compat bridge: array-based API callers are transpiled into the canonical record
// matrix so the side-effect engine and downstream from/to computation stay record-native.
// To be removed once every caller produces records directly.
export const transpileFlatEntityOperationArrayToRecord = (
allFlatEntityOperationByMetadataName: AllFlatEntityOperationByMetadataName,
): AllFlatEntityOperationRecordByMetadataName => {
const genericMatrix =
allFlatEntityOperationByMetadataName as unknown as Record<
string,
| {
flatEntityToCreate: GenericUniversalFlatEntity[];
flatEntityToUpdate: GenericUniversalFlatEntity[];
flatEntityToDelete: GenericUniversalFlatEntity[];
}
| undefined
>;
const recordMatrix: Record<
string,
{
flatEntityToCreate: Record<string, GenericUniversalFlatEntity>;
flatEntityToUpdate: Record<string, GenericUniversalFlatEntity>;
flatEntityToDelete: Record<string, GenericUniversalFlatEntity>;
}
> = {};
for (const metadataName of Object.keys(genericMatrix)) {
const operations = genericMatrix[metadataName];
if (!isDefined(operations)) {
continue;
}
recordMatrix[metadataName] = {
flatEntityToCreate: toRecordByUniversalIdentifierOrThrow({
flatEntities: operations.flatEntityToCreate,
metadataName,
operation: 'flatEntityToCreate',
}),
flatEntityToUpdate: toRecordByUniversalIdentifierOrThrow({
flatEntities: operations.flatEntityToUpdate,
metadataName,
operation: 'flatEntityToUpdate',
}),
flatEntityToDelete: toRecordByUniversalIdentifierOrThrow({
flatEntities: operations.flatEntityToDelete,
metadataName,
operation: 'flatEntityToDelete',
}),
};
}
return recordMatrix as unknown as AllFlatEntityOperationRecordByMetadataName;
};
@@ -19,7 +19,6 @@ import { findFlatEntityByIdInFlatEntityMaps } from 'src/engine/metadata-modules/
import { type FieldInputTranspilationResult } from 'src/engine/metadata-modules/flat-field-metadata/types/field-input-transpilation-result.type';
import { fromMorphRelationCreateFieldInputToFlatFieldMetadatas } from 'src/engine/metadata-modules/flat-field-metadata/utils/from-morph-relation-create-field-input-to-flat-field-metadatas.util';
import { fromRelationCreateFieldInputToFlatFieldMetadatas } from 'src/engine/metadata-modules/flat-field-metadata/utils/from-relation-create-field-input-to-flat-field-metadatas.util';
import { generateIndexForFlatFieldMetadata } from 'src/engine/metadata-modules/flat-field-metadata/utils/generate-index-for-flat-field-metadata.util';
import { getDefaultFlatFieldMetadata } from 'src/engine/metadata-modules/flat-field-metadata/utils/get-default-flat-field-metadata-from-create-field-input.util';
import { type UniversalFlatFieldMetadata } from 'src/engine/workspace-manager/workspace-migration/universal-flat-entity/types/universal-flat-field-metadata.type';
import { type UniversalFlatIndexMetadata } from 'src/engine/workspace-manager/workspace-migration/universal-flat-entity/types/universal-flat-index-metadata.type';
@@ -186,17 +185,6 @@ export const fromCreateFieldInputToFlatFieldMetadatasToCreate = async ({
case FieldMetadataType.RICH_TEXT:
case FieldMetadataType.ACTOR:
case FieldMetadataType.ARRAY: {
const indexMetadatas: UniversalFlatIndexMetadata[] = [];
if (commonFlatFieldMetadata.isUnique) {
indexMetadatas.push(
generateIndexForFlatFieldMetadata({
flatFieldMetadata: commonFlatFieldMetadata,
flatObjectMetadata: parentFlatObjectMetadata,
}),
);
}
return {
status: 'success',
result: {
@@ -206,7 +194,7 @@ export const fromCreateFieldInputToFlatFieldMetadatasToCreate = async ({
type: createFieldInput.type,
},
],
indexMetadatas,
indexMetadatas: [],
},
};
}
@@ -14,7 +14,9 @@ import { findFlatEntityByUniversalIdentifierOrThrow } from 'src/engine/metadata-
import { findManyFlatEntityByUniversalIdentifierInUniversalFlatEntityMapsOrThrow } from 'src/engine/metadata-modules/flat-entity/utils/find-many-flat-entity-by-universal-identifier-in-universal-flat-entity-maps-or-throw.util';
import { computeFlatFieldMetadataRelatedFlatFieldMetadata } from 'src/engine/metadata-modules/flat-field-metadata/utils/compute-flat-field-metadata-related-flat-field-metadata.util';
import { computeSearchFieldMetadataDeletionForDeletedFields } from 'src/engine/metadata-modules/flat-field-metadata/utils/compute-search-field-metadata-deletion-for-deleted-fields.util';
import { isMorphOrRelationFlatFieldMetadata } from 'src/engine/metadata-modules/flat-field-metadata/utils/is-morph-or-relation-flat-field-metadata.util';
import { type FlatIndexMetadata } from 'src/engine/metadata-modules/flat-index-metadata/types/flat-index-metadata.type';
import { isSystemUniqueFlatIndexMetadata } from 'src/engine/metadata-modules/flat-index-metadata/utils/is-system-unique-flat-index-metadata.util';
import { generateFlatIndexMetadataWithNameOrThrow } from 'src/engine/metadata-modules/index-metadata/utils/generate-flat-index.util';
import { belongsToTwentyStandardApp } from 'src/engine/metadata-modules/utils/belongs-to-twenty-standard-app.util';
import { type UniversalFlatFieldMetadata } from 'src/engine/workspace-manager/workspace-migration/universal-flat-entity/types/universal-flat-field-metadata.type';
@@ -95,10 +97,38 @@ export const fromDeleteFieldInputToFlatFieldMetadatasToDelete = ({
...relatedFlatFieldMetadataToDelete,
];
const isEngineOwnedBackingUniqueIndex = (
flatIndexMetadata: FlatIndexMetadata,
): boolean => {
if (
!isSystemUniqueFlatIndexMetadata(flatIndexMetadata) ||
flatIndexMetadata.flatIndexFieldMetadatas.length !== 1
) {
return false;
}
const backingFlatFieldMetadata = findFlatEntityByIdInFlatEntityMaps({
flatEntityId:
flatIndexMetadata.flatIndexFieldMetadatas[0].fieldMetadataId,
flatEntityMaps: existingFlatFieldMetadataMaps,
});
return (
isDefined(backingFlatFieldMetadata) &&
backingFlatFieldMetadata.isUnique === true &&
!isMorphOrRelationFlatFieldMetadata(backingFlatFieldMetadata)
);
};
const flatIndexMap = new Map<string, FlatIndexMetadata>();
const allFlatIndexes = Object.values(
existingFlatIndexMaps.byUniversalIdentifier,
).filter(isDefined);
)
.filter(isDefined)
.filter(
(flatIndexMetadata) =>
!isEngineOwnedBackingUniqueIndex(flatIndexMetadata),
);
for (const flatFieldMetadata of flatFieldMetadatasToDelete) {
allFlatIndexes.forEach((flatIndex) => {
@@ -150,7 +150,6 @@ export const fromUpdateFieldInputToFlatFieldMetadata = ({
toFlatFieldMetadata,
flatViewMaps,
flatViewFieldMaps,
flatApplication,
});
if (sideEffectResult.status === 'fail') {
@@ -0,0 +1,44 @@
import { IndexType } from 'src/engine/metadata-modules/index-metadata/types/indexType.types';
import { generateDeterministicFlatIndexMetadataOrThrow } from 'src/engine/metadata-modules/index-metadata/utils/generate-deterministic-flat-index.util';
import { type UniversalFlatFieldMetadata } from 'src/engine/workspace-manager/workspace-migration/universal-flat-entity/types/universal-flat-field-metadata.type';
import { type UniversalFlatIndexMetadata } from 'src/engine/workspace-manager/workspace-migration/universal-flat-entity/types/universal-flat-index-metadata.type';
import { type UniversalFlatObjectMetadata } from 'src/engine/workspace-manager/workspace-migration/universal-flat-entity/types/universal-flat-object-metadata.type';
type GenerateDeterministicIndexForFlatFieldMetadataOrThrowArgs = {
flatFieldMetadata: UniversalFlatFieldMetadata;
flatObjectMetadata: UniversalFlatObjectMetadata;
};
export const generateDeterministicIndexForFlatFieldMetadataOrThrow = ({
flatFieldMetadata,
flatObjectMetadata,
}: GenerateDeterministicIndexForFlatFieldMetadataOrThrowArgs): UniversalFlatIndexMetadata => {
const createdAt = new Date().toISOString();
return generateDeterministicFlatIndexMetadataOrThrow({
flatObjectMetadata,
objectFlatFieldMetadatas: [flatFieldMetadata],
flatIndex: {
createdAt,
universalFlatIndexFieldMetadatas: [
{
createdAt,
fieldMetadataUniversalIdentifier:
flatFieldMetadata.universalIdentifier,
order: 0,
subFieldName: null,
updatedAt: createdAt,
},
],
indexType: IndexType.BTREE,
indexWhereClause: null,
isCustom: true,
isUnique: flatFieldMetadata.isUnique ?? false,
isSystemSideEffect: true,
objectMetadataUniversalIdentifier: flatObjectMetadata.universalIdentifier,
updatedAt: createdAt,
applicationUniversalIdentifier:
flatFieldMetadata.applicationUniversalIdentifier,
},
});
};
@@ -1,6 +1,5 @@
import { type FromTo } from 'twenty-shared/types';
import { type FlatApplication } from 'src/engine/core-modules/application/types/flat-application.type';
import { type AllFlatEntityMaps } from 'src/engine/metadata-modules/flat-entity/types/all-flat-entity-maps.type';
import { type FieldInputTranspilationResult } from 'src/engine/metadata-modules/flat-field-metadata/types/field-input-transpilation-result.type';
import { type FlatFieldMetadata } from 'src/engine/metadata-modules/flat-field-metadata/types/flat-field-metadata.type';
@@ -36,9 +35,7 @@ type HandleFlatFieldMetadataUpdateSideEffectArgs = FromTo<
| 'flatViewGroupMaps'
| 'flatViewMaps'
| 'flatViewFieldMaps'
> & {
flatApplication: FlatApplication;
};
>;
export const FLAT_FIELD_METADATA_UPDATE_EMPTY_SIDE_EFFECTS: FlatFieldMetadataUpdateSideEffects =
{
@@ -65,7 +62,6 @@ export const handleFlatFieldMetadataUpdateSideEffect = ({
flatViewGroupMaps,
flatViewMaps,
flatViewFieldMaps,
flatApplication,
}: HandleFlatFieldMetadataUpdateSideEffectArgs): FieldInputTranspilationResult<FlatFieldMetadataUpdateSideEffects> => {
const sideEffectResult = structuredClone(
FLAT_FIELD_METADATA_UPDATE_EMPTY_SIDE_EFFECTS,
@@ -125,7 +121,6 @@ export const handleFlatFieldMetadataUpdateSideEffect = ({
flatIndexMaps,
flatObjectMetadataMaps,
flatFieldMetadataMaps,
flatApplication,
});
if (indexChangesSideEffectResult.status === 'fail') {
@@ -1,19 +1,12 @@
import { msg } from '@lingui/core/macro';
import { type FromTo } from 'twenty-shared/types';
import { isDefined } from 'twenty-shared/utils';
import { type FlatApplication } from 'src/engine/core-modules/application/types/flat-application.type';
import { FieldMetadataExceptionCode } from 'src/engine/metadata-modules/field-metadata/field-metadata.exception';
import { type AllFlatEntityMaps } from 'src/engine/metadata-modules/flat-entity/types/all-flat-entity-maps.type';
import { findFlatEntityByIdInFlatEntityMapsOrThrow } from 'src/engine/metadata-modules/flat-entity/utils/find-flat-entity-by-id-in-flat-entity-maps-or-throw.util';
import { type FieldInputTranspilationResult } from 'src/engine/metadata-modules/flat-field-metadata/types/field-input-transpilation-result.type';
import { type FlatFieldMetadata } from 'src/engine/metadata-modules/flat-field-metadata/types/flat-field-metadata.type';
import { findFieldRelatedIndexes } from 'src/engine/metadata-modules/flat-field-metadata/utils/find-field-related-index.util';
import { generateIndexForFlatFieldMetadata } from 'src/engine/metadata-modules/flat-field-metadata/utils/generate-index-for-flat-field-metadata.util';
import { isMorphOrRelationFlatFieldMetadata } from 'src/engine/metadata-modules/flat-field-metadata/utils/is-morph-or-relation-flat-field-metadata.util';
import { recomputeIndexOnFlatFieldMetadataNameUpdate } from 'src/engine/metadata-modules/flat-field-metadata/utils/recompute-index-on-flat-field-metadata-name-update.util';
import { type FlatIndexMetadata } from 'src/engine/metadata-modules/flat-index-metadata/types/flat-index-metadata.type';
import { type FlatObjectMetadata } from 'src/engine/metadata-modules/flat-object-metadata/types/flat-object-metadata.type';
import { isSystemUniqueFlatIndexMetadata } from 'src/engine/metadata-modules/flat-index-metadata/utils/is-system-unique-flat-index-metadata.util';
import { type UniversalFlatIndexMetadata } from 'src/engine/workspace-manager/workspace-migration/universal-flat-entity/types/universal-flat-index-metadata.type';
export type FieldMetadataUpdateIndexSideEffect = {
@@ -29,9 +22,8 @@ type FromToFlatFieldMetadataAndFlatEntityMaps = FromTo<
Pick<
AllFlatEntityMaps,
'flatObjectMetadataMaps' | 'flatFieldMetadataMaps' | 'flatIndexMaps'
> & {
flatApplication: FlatApplication;
};
>;
const FIELD_METADATA_UPDATE_INDEX_SIDE_EFFECT: FieldMetadataUpdateIndexSideEffect =
{
flatIndexMetadatasToUpdate: [],
@@ -45,14 +37,8 @@ export const handleIndexChangesDuringFieldUpdate = ({
flatIndexMaps,
flatObjectMetadataMaps,
flatFieldMetadataMaps,
flatApplication,
}: FromToFlatFieldMetadataAndFlatEntityMaps): FieldInputTranspilationResult<FieldMetadataUpdateIndexSideEffect> => {
if (
!hasIndexRelevantChanges({
fromFlatFieldMetadata,
toFlatFieldMetadata,
})
) {
if (fromFlatFieldMetadata.name === toFlatFieldMetadata.name) {
return {
status: 'success',
result: FIELD_METADATA_UPDATE_INDEX_SIDE_EFFECT,
@@ -68,119 +54,17 @@ export const handleIndexChangesDuringFieldUpdate = ({
flatFieldMetadata: fromFlatFieldMetadata,
flatObjectMetadata,
flatIndexMaps,
});
}).filter(
(flatIndexMetadata) => !isSystemUniqueFlatIndexMetadata(flatIndexMetadata),
);
if (relatedIndexes.length === 0) {
return handleNoExistingIndexes({
toFlatFieldMetadata,
flatObjectMetadata,
});
}
return handleExistingIndexes({
toFlatFieldMetadata,
fromFlatFieldMetadata,
relatedIndexes,
flatObjectMetadata,
flatFieldMetadataMaps,
flatApplication,
});
};
const hasIndexRelevantChanges = ({
fromFlatFieldMetadata,
toFlatFieldMetadata,
}: FromTo<FlatFieldMetadata, 'flatFieldMetadata'>): boolean =>
fromFlatFieldMetadata.name !== toFlatFieldMetadata.name ||
fromFlatFieldMetadata.isUnique !== toFlatFieldMetadata.isUnique;
const handleNoExistingIndexes = ({
toFlatFieldMetadata,
flatObjectMetadata,
}: {
toFlatFieldMetadata: FlatFieldMetadata;
flatObjectMetadata: FlatObjectMetadata;
}): FieldInputTranspilationResult<FieldMetadataUpdateIndexSideEffect> => {
if (!toFlatFieldMetadata.isUnique) {
return {
status: 'success',
result: FIELD_METADATA_UPDATE_INDEX_SIDE_EFFECT,
};
}
const newIndex = generateIndexForFlatFieldMetadata({
flatFieldMetadata: toFlatFieldMetadata,
flatObjectMetadata,
});
return {
status: 'success',
result: {
...FIELD_METADATA_UPDATE_INDEX_SIDE_EFFECT,
flatIndexMetadatasToCreate: [newIndex],
},
};
};
const handleExistingIndexes = ({
fromFlatFieldMetadata,
toFlatFieldMetadata,
relatedIndexes,
flatObjectMetadata,
flatFieldMetadataMaps,
flatApplication,
}: {
relatedIndexes: FlatIndexMetadata[];
flatObjectMetadata: FlatObjectMetadata;
flatFieldMetadataMaps: AllFlatEntityMaps['flatFieldMetadataMaps'];
} & FromTo<FlatFieldMetadata, 'flatFieldMetadata'> & {
flatApplication: FlatApplication;
}): FieldInputTranspilationResult<FieldMetadataUpdateIndexSideEffect> => {
if (
toFlatFieldMetadata.isUnique === false &&
!isMorphOrRelationFlatFieldMetadata(fromFlatFieldMetadata)
) {
const expectedUniqueIndex = generateIndexForFlatFieldMetadata({
flatFieldMetadata: {
...fromFlatFieldMetadata,
isUnique: true,
},
flatObjectMetadata,
});
const uniqueIndexToDelete = relatedIndexes.find(
(index) => index.name === expectedUniqueIndex.name,
);
if (
isDefined(uniqueIndexToDelete) &&
((isDefined(uniqueIndexToDelete.applicationId) &&
uniqueIndexToDelete.applicationId !== flatApplication.id) ||
!uniqueIndexToDelete.isCustom)
) {
return {
status: 'fail',
errors: [
{
code: FieldMetadataExceptionCode.INVALID_FIELD_INPUT,
message:
'Cannot delete unique index that have not been created by the workspace custom application',
userFriendlyMessage: msg`Cannot delete unique index that have not been created by the workspace custom application`,
},
],
};
}
return {
status: 'success',
result: {
...FIELD_METADATA_UPDATE_INDEX_SIDE_EFFECT,
flatIndexMetadatasToDelete: uniqueIndexToDelete
? [uniqueIndexToDelete]
: [],
},
};
}
const updatedIndexes = recomputeIndexOnFlatFieldMetadataNameUpdate({
flatFieldMetadataMaps,
flatObjectMetadata,
@@ -0,0 +1,7 @@
import { type UniversalFlatFieldMetadata } from 'src/engine/workspace-manager/workspace-migration/universal-flat-entity/types/universal-flat-field-metadata.type';
// The primary key already enforces uniqueness via its constraint, so it must not
// spawn a redundant application-managed unique index.
export const isPrimaryKeyFlatFieldMetadata = (
flatFieldMetadata: Pick<UniversalFlatFieldMetadata, 'name'>,
): boolean => flatFieldMetadata.name === 'id';
@@ -0,0 +1,6 @@
export const isSystemUniqueFlatIndexMetadata = (flatIndexMetadata: {
isSystemSideEffect: boolean;
isUnique: boolean;
}): boolean =>
flatIndexMetadata.isSystemSideEffect === true &&
flatIndexMetadata.isUnique === true;
@@ -0,0 +1,96 @@
import { compositeTypeDefinitions, RelationType } from 'twenty-shared/types';
import { isDefined } from 'twenty-shared/utils';
import { computeCompositeColumnName } from 'src/engine/metadata-modules/field-metadata/utils/compute-column-name.util';
import { computeMorphOrRelationFieldJoinColumnName } from 'src/engine/metadata-modules/field-metadata/utils/compute-morph-or-relation-field-join-column-name.util';
import { isCompositeFieldMetadataType } from 'src/engine/metadata-modules/field-metadata/utils/is-composite-field-metadata-type.util';
import {
FlatEntityMapsException,
FlatEntityMapsExceptionCode,
} from 'src/engine/metadata-modules/flat-entity/exceptions/flat-entity-maps.exception';
import { isMorphOrRelationUniversalFlatFieldMetadata } from 'src/engine/metadata-modules/flat-field-metadata/utils/is-morph-or-relation-flat-field-metadata.util';
import { generateDeterministicIndexName } from 'src/engine/metadata-modules/index-metadata/utils/generate-deterministic-index-name';
import { type UniversalFlatFieldMetadata } from 'src/engine/workspace-manager/workspace-migration/universal-flat-entity/types/universal-flat-field-metadata.type';
import { type UniversalFlatIndexFieldMetadata } from 'src/engine/workspace-manager/workspace-migration/universal-flat-entity/types/universal-flat-index-metadata.type';
import { type UniversalFlatObjectMetadata } from 'src/engine/workspace-manager/workspace-migration/universal-flat-entity/types/universal-flat-object-metadata.type';
type ComputeFlatIndexNameArgs = {
flatObjectMetadata: UniversalFlatObjectMetadata;
objectFlatFieldMetadatas: UniversalFlatFieldMetadata[];
indexFields: Pick<
UniversalFlatIndexFieldMetadata,
'order' | 'fieldMetadataUniversalIdentifier' | 'subFieldName'
>[];
isUnique: boolean;
indexWhereClause: string | null;
};
export const computeFlatIndexNameOrThrow = ({
flatObjectMetadata,
objectFlatFieldMetadatas,
indexFields,
isUnique,
indexWhereClause,
}: ComputeFlatIndexNameArgs): string => {
const orderedIndexColumnNames = [...indexFields]
.sort((a, b) => a.order - b.order)
.map((flatIndexField) => {
const relatedFlatFieldMetadata = objectFlatFieldMetadatas.find(
(flatFieldMetadata) =>
flatFieldMetadata.universalIdentifier ===
flatIndexField.fieldMetadataUniversalIdentifier,
);
if (!isDefined(relatedFlatFieldMetadata)) {
throw new FlatEntityMapsException(
'Could not find flat index field related field in cache',
FlatEntityMapsExceptionCode.ENTITY_NOT_FOUND,
);
}
if (
isCompositeFieldMetadataType(relatedFlatFieldMetadata.type) &&
isDefined(flatIndexField.subFieldName)
) {
const property = compositeTypeDefinitions
.get(relatedFlatFieldMetadata.type)
?.properties.find(
(compositeProperty) =>
compositeProperty.name === flatIndexField.subFieldName,
);
if (!isDefined(property)) {
throw new FlatEntityMapsException(
`Composite sub-field "${flatIndexField.subFieldName}" not found on ${relatedFlatFieldMetadata.name}`,
FlatEntityMapsExceptionCode.ENTITY_NOT_FOUND,
);
}
return computeCompositeColumnName(
{
name: relatedFlatFieldMetadata.name,
type: relatedFlatFieldMetadata.type,
},
property,
);
}
const isManyToOneRelation =
isMorphOrRelationUniversalFlatFieldMetadata(relatedFlatFieldMetadata) &&
relatedFlatFieldMetadata.universalSettings?.relationType ===
RelationType.MANY_TO_ONE;
return isManyToOneRelation
? computeMorphOrRelationFieldJoinColumnName({
name: relatedFlatFieldMetadata.name,
})
: relatedFlatFieldMetadata.name;
});
return generateDeterministicIndexName({
flatObjectMetadata,
orderedIndexColumnNames,
isUnique,
indexWhereClause,
});
};
@@ -0,0 +1,56 @@
import { getIndexUniversalIdentifier } from 'twenty-shared/application';
import { computeFlatIndexNameOrThrow } from 'src/engine/metadata-modules/index-metadata/utils/compute-flat-index-name.util';
import { type UniversalFlatFieldMetadata } from 'src/engine/workspace-manager/workspace-migration/universal-flat-entity/types/universal-flat-field-metadata.type';
import {
type UniversalFlatIndexFieldMetadata,
type UniversalFlatIndexMetadata,
} from 'src/engine/workspace-manager/workspace-migration/universal-flat-entity/types/universal-flat-index-metadata.type';
import { type UniversalFlatObjectMetadata } from 'src/engine/workspace-manager/workspace-migration/universal-flat-entity/types/universal-flat-object-metadata.type';
type GenerateDeterministicFlatIndexArgs = {
flatObjectMetadata: UniversalFlatObjectMetadata;
objectFlatFieldMetadatas: UniversalFlatFieldMetadata[];
flatIndex: Omit<
UniversalFlatIndexMetadata,
'name' | 'universalIdentifier' | 'universalFlatIndexFieldMetadatas'
> & {
universalFlatIndexFieldMetadatas: Omit<
UniversalFlatIndexFieldMetadata,
'indexMetadataUniversalIdentifier'
>[];
};
};
export const generateDeterministicFlatIndexMetadataOrThrow = ({
flatObjectMetadata,
objectFlatFieldMetadatas,
flatIndex,
}: GenerateDeterministicFlatIndexArgs): UniversalFlatIndexMetadata => {
const name = computeFlatIndexNameOrThrow({
flatObjectMetadata,
objectFlatFieldMetadatas,
indexFields: flatIndex.universalFlatIndexFieldMetadatas,
isUnique: flatIndex.isUnique,
indexWhereClause: flatIndex.indexWhereClause,
});
const universalIdentifier = getIndexUniversalIdentifier({
applicationUniversalIdentifier: flatIndex.applicationUniversalIdentifier,
objectUniversalIdentifier: flatObjectMetadata.universalIdentifier,
name,
});
return {
...flatIndex,
name,
universalIdentifier,
universalFlatIndexFieldMetadatas:
flatIndex.universalFlatIndexFieldMetadatas.map(
(universalFlatIndexFieldMetadata) => ({
...universalFlatIndexFieldMetadata,
indexMetadataUniversalIdentifier: universalIdentifier,
}),
),
};
};
@@ -13,7 +13,7 @@ type GenerateDeterministicIndexNameArgs = {
// coexist; the unique-name constraint must not block that).
indexWhereClause?: string | null;
};
export const generateDeterministicIndexNameV2 = ({
export const generateDeterministicIndexName = ({
orderedIndexColumnNames,
flatObjectMetadata,
isUnique = false,
@@ -1,15 +1,4 @@
import { compositeTypeDefinitions, RelationType } from 'twenty-shared/types';
import { isDefined } from 'twenty-shared/utils';
import { computeCompositeColumnName } from 'src/engine/metadata-modules/field-metadata/utils/compute-column-name.util';
import { computeMorphOrRelationFieldJoinColumnName } from 'src/engine/metadata-modules/field-metadata/utils/compute-morph-or-relation-field-join-column-name.util';
import { isCompositeFieldMetadataType } from 'src/engine/metadata-modules/field-metadata/utils/is-composite-field-metadata-type.util';
import {
FlatEntityMapsException,
FlatEntityMapsExceptionCode,
} from 'src/engine/metadata-modules/flat-entity/exceptions/flat-entity-maps.exception';
import { isMorphOrRelationUniversalFlatFieldMetadata } from 'src/engine/metadata-modules/flat-field-metadata/utils/is-morph-or-relation-flat-field-metadata.util';
import { generateDeterministicIndexNameV2 } from 'src/engine/metadata-modules/index-metadata/utils/generate-deterministic-index-name-v2';
import { computeFlatIndexNameOrThrow } from 'src/engine/metadata-modules/index-metadata/utils/compute-flat-index-name.util';
import { type UniversalFlatFieldMetadata } from 'src/engine/workspace-manager/workspace-migration/universal-flat-entity/types/universal-flat-field-metadata.type';
import { type UniversalFlatIndexMetadata } from 'src/engine/workspace-manager/workspace-migration/universal-flat-entity/types/universal-flat-index-metadata.type';
import { type UniversalFlatObjectMetadata } from 'src/engine/workspace-manager/workspace-migration/universal-flat-entity/types/universal-flat-object-metadata.type';
@@ -25,69 +14,10 @@ export const generateFlatIndexMetadataWithNameOrThrow = ({
objectFlatFieldMetadatas,
flatIndex,
}: GenerateFlatIndexArgs): UniversalFlatIndexMetadata => {
const orderedIndexColumnNames = flatIndex.universalFlatIndexFieldMetadatas
.sort((a, b) => a.order - b.order)
.map((flatIndexField) => {
const relatedFlatFieldMetadata = objectFlatFieldMetadatas.find(
(flatFieldMetadata) =>
flatFieldMetadata.universalIdentifier ===
flatIndexField.fieldMetadataUniversalIdentifier,
);
if (!isDefined(relatedFlatFieldMetadata)) {
throw new FlatEntityMapsException(
'Could not find flat index field related field in cache',
FlatEntityMapsExceptionCode.ENTITY_NOT_FOUND,
);
}
// Composite parent with an explicit sub-field → single sub-column.
// Composite parent without sub-field falls through to the legacy
// scalar branch below, which produces a deterministic name based on
// the parent name (the runner handles the multi-column SQL expansion
// via isIncludedInUniqueConstraint).
if (
isCompositeFieldMetadataType(relatedFlatFieldMetadata.type) &&
isDefined(flatIndexField.subFieldName)
) {
const property = compositeTypeDefinitions
.get(relatedFlatFieldMetadata.type)
?.properties.find(
(compositeProperty) =>
compositeProperty.name === flatIndexField.subFieldName,
);
if (!isDefined(property)) {
throw new FlatEntityMapsException(
`Composite sub-field "${flatIndexField.subFieldName}" not found on ${relatedFlatFieldMetadata.name}`,
FlatEntityMapsExceptionCode.ENTITY_NOT_FOUND,
);
}
return computeCompositeColumnName(
{
name: relatedFlatFieldMetadata.name,
type: relatedFlatFieldMetadata.type,
},
property,
);
}
const isManyToOneRelation =
isMorphOrRelationUniversalFlatFieldMetadata(relatedFlatFieldMetadata) &&
relatedFlatFieldMetadata.universalSettings?.relationType ===
RelationType.MANY_TO_ONE;
return isManyToOneRelation
? computeMorphOrRelationFieldJoinColumnName({
name: relatedFlatFieldMetadata.name,
})
: relatedFlatFieldMetadata.name;
});
const name = generateDeterministicIndexNameV2({
const name = computeFlatIndexNameOrThrow({
flatObjectMetadata,
orderedIndexColumnNames,
objectFlatFieldMetadatas,
indexFields: flatIndex.universalFlatIndexFieldMetadatas,
isUnique: flatIndex.isUnique,
indexWhereClause: flatIndex.indexWhereClause,
});
@@ -0,0 +1,2 @@
export const METADATA_SIDE_EFFECT_HANDLER_METADATA_KEY =
'METADATA_SIDE_EFFECT_HANDLER_METADATA_KEY';
@@ -0,0 +1,4 @@
export enum MetadataSideEffectExceptionCode {
RESERVED_SYSTEM_UNIVERSAL_IDENTIFIER = 'RESERVED_SYSTEM_UNIVERSAL_IDENTIFIER',
SIDE_EFFECT_PARENT_METADATA_NOT_FOUND = 'SIDE_EFFECT_PARENT_METADATA_NOT_FOUND',
}
@@ -0,0 +1,74 @@
import { Injectable } from '@nestjs/common';
import { isDefined } from 'twenty-shared/utils';
import { generateDeterministicIndexForFlatFieldMetadataOrThrow } from 'src/engine/metadata-modules/flat-field-metadata/utils/generate-deterministic-index-for-flat-field-metadata-or-throw.util';
import { isMorphOrRelationUniversalFlatFieldMetadata } from 'src/engine/metadata-modules/flat-field-metadata/utils/is-morph-or-relation-flat-field-metadata.util';
import { isPrimaryKeyFlatFieldMetadata } from 'src/engine/metadata-modules/flat-field-metadata/utils/is-primary-key-flat-field-metadata.util';
import { buildFieldSideEffectParentNotFoundFailure } from 'src/engine/metadata-modules/metadata-side-effect/handlers/field-metadata/utils/build-field-side-effect-parent-not-found-failure.util';
import { resolveParentFlatObjectMetadataAfterStateForFieldSideEffect } from 'src/engine/metadata-modules/metadata-side-effect/handlers/field-metadata/utils/resolve-parent-flat-object-metadata-after-state-for-field-side-effect.util';
import {
type BuildSideEffectsArgs,
MetadataSideEffectHandler,
} from 'src/engine/metadata-modules/metadata-side-effect/interfaces/base-metadata-side-effect-handler.service';
import { type MetadataSideEffectResult } from 'src/engine/metadata-modules/metadata-side-effect/types/metadata-side-effect-result.type';
@Injectable()
export class FieldUniqueBackingIndexOnCreateSideEffectHandlerService extends MetadataSideEffectHandler(
{
operation: 'create',
metadataName: 'fieldMetadata',
name: 'fieldUniqueBackingIndexOnCreate',
description:
'When a unique scalar field is created, generate the single-field UNIQUE index that enforces its uniqueness constraint at the database level.',
},
) {
buildSideEffects({
flatEntity: flatFieldMetadata,
allFlatEntityOperationRecordByMetadataName,
relatedFlatEntityMaps,
}: BuildSideEffectsArgs<'fieldMetadata'>): MetadataSideEffectResult {
if (
flatFieldMetadata.isUnique !== true ||
isMorphOrRelationUniversalFlatFieldMetadata(flatFieldMetadata)
) {
return { status: 'noop' };
}
if (isPrimaryKeyFlatFieldMetadata(flatFieldMetadata)) {
return { status: 'noop' };
}
const parentFlatObjectMetadata =
resolveParentFlatObjectMetadataAfterStateForFieldSideEffect({
objectMetadataUniversalIdentifier:
flatFieldMetadata.objectMetadataUniversalIdentifier,
allFlatEntityOperationRecordByMetadataName,
relatedFlatEntityMaps,
});
if (!isDefined(parentFlatObjectMetadata)) {
return buildFieldSideEffectParentNotFoundFailure({
flatFieldMetadata,
operation: 'create',
});
}
const flatIndexMetadata =
generateDeterministicIndexForFlatFieldMetadataOrThrow({
flatFieldMetadata,
flatObjectMetadata: parentFlatObjectMetadata,
});
return {
status: 'success',
operations: {
index: {
flatEntityToCreate: {
[flatIndexMetadata.universalIdentifier]: flatIndexMetadata,
},
},
},
};
}
}
@@ -0,0 +1,74 @@
import { Injectable } from '@nestjs/common';
import { isDefined } from 'twenty-shared/utils';
import { generateDeterministicIndexForFlatFieldMetadataOrThrow } from 'src/engine/metadata-modules/flat-field-metadata/utils/generate-deterministic-index-for-flat-field-metadata-or-throw.util';
import { isMorphOrRelationUniversalFlatFieldMetadata } from 'src/engine/metadata-modules/flat-field-metadata/utils/is-morph-or-relation-flat-field-metadata.util';
import { buildFieldSideEffectParentNotFoundFailure } from 'src/engine/metadata-modules/metadata-side-effect/handlers/field-metadata/utils/build-field-side-effect-parent-not-found-failure.util';
import {
type BuildSideEffectsArgs,
MetadataSideEffectHandler,
} from 'src/engine/metadata-modules/metadata-side-effect/interfaces/base-metadata-side-effect-handler.service';
import { type MetadataSideEffectResult } from 'src/engine/metadata-modules/metadata-side-effect/types/metadata-side-effect-result.type';
@Injectable()
export class FieldUniqueBackingIndexOnDeleteSideEffectHandlerService extends MetadataSideEffectHandler(
{
operation: 'delete',
metadataName: 'fieldMetadata',
name: 'fieldUniqueBackingIndexOnDelete',
description:
'When a unique scalar field is deleted, cascade-delete the single-field UNIQUE index that backed its uniqueness constraint.',
},
) {
buildSideEffects({
flatEntity: flatFieldMetadata,
relatedFlatEntityMaps,
}: BuildSideEffectsArgs<'fieldMetadata'>): MetadataSideEffectResult {
if (
flatFieldMetadata.isUnique !== true ||
isMorphOrRelationUniversalFlatFieldMetadata(flatFieldMetadata)
) {
return { status: 'noop' };
}
const parentFlatObjectMetadata =
relatedFlatEntityMaps.flatObjectMetadataMaps.byUniversalIdentifier[
flatFieldMetadata.objectMetadataUniversalIdentifier
];
if (!isDefined(parentFlatObjectMetadata)) {
return buildFieldSideEffectParentNotFoundFailure({
flatFieldMetadata,
operation: 'delete',
});
}
const flatIndexMetadataToDelete =
generateDeterministicIndexForFlatFieldMetadataOrThrow({
flatFieldMetadata,
flatObjectMetadata: parentFlatObjectMetadata,
});
const indexExistsInWorkspace = isDefined(
relatedFlatEntityMaps.flatIndexMaps.byUniversalIdentifier[
flatIndexMetadataToDelete.universalIdentifier
],
);
if (!indexExistsInWorkspace) {
return { status: 'noop' };
}
return {
status: 'success',
operations: {
index: {
flatEntityToDelete: {
[flatIndexMetadataToDelete.universalIdentifier]:
flatIndexMetadataToDelete,
},
},
},
};
}
}
@@ -0,0 +1,149 @@
import { Injectable } from '@nestjs/common';
import { isDefined } from 'twenty-shared/utils';
import { generateDeterministicIndexForFlatFieldMetadataOrThrow } from 'src/engine/metadata-modules/flat-field-metadata/utils/generate-deterministic-index-for-flat-field-metadata-or-throw.util';
import { isMorphOrRelationUniversalFlatFieldMetadata } from 'src/engine/metadata-modules/flat-field-metadata/utils/is-morph-or-relation-flat-field-metadata.util';
import { isPrimaryKeyFlatFieldMetadata } from 'src/engine/metadata-modules/flat-field-metadata/utils/is-primary-key-flat-field-metadata.util';
import { buildFieldSideEffectParentNotFoundFailure } from 'src/engine/metadata-modules/metadata-side-effect/handlers/field-metadata/utils/build-field-side-effect-parent-not-found-failure.util';
import { resolveParentFlatObjectMetadataAfterStateForFieldSideEffect } from 'src/engine/metadata-modules/metadata-side-effect/handlers/field-metadata/utils/resolve-parent-flat-object-metadata-after-state-for-field-side-effect.util';
import {
type BuildSideEffectsArgs,
MetadataSideEffectHandler,
} from 'src/engine/metadata-modules/metadata-side-effect/interfaces/base-metadata-side-effect-handler.service';
import { type MetadataSideEffectResult } from 'src/engine/metadata-modules/metadata-side-effect/types/metadata-side-effect-result.type';
@Injectable()
export class FieldUniqueBackingIndexOnUpdateSideEffectHandlerService extends MetadataSideEffectHandler(
{
operation: 'update',
metadataName: 'fieldMetadata',
name: 'fieldUniqueBackingIndexOnUpdate',
description:
"Keep a unique scalar field's backing UNIQUE index in sync when its `isUnique` flag flips or the field is renamed (drop the stale index and recreate the deterministic one).",
},
) {
buildSideEffects({
flatEntity: flatFieldMetadata,
allFlatEntityOperationRecordByMetadataName,
relatedFlatEntityMaps,
}: BuildSideEffectsArgs<'fieldMetadata'>): MetadataSideEffectResult {
if (isMorphOrRelationUniversalFlatFieldMetadata(flatFieldMetadata)) {
return { status: 'noop' };
}
if (isPrimaryKeyFlatFieldMetadata(flatFieldMetadata)) {
return { status: 'noop' };
}
const existingFlatFieldMetadata =
relatedFlatEntityMaps.flatFieldMetadataMaps.byUniversalIdentifier[
flatFieldMetadata.universalIdentifier
];
if (!isDefined(existingFlatFieldMetadata)) {
return { status: 'noop' };
}
const wasRenamed =
existingFlatFieldMetadata.name !== flatFieldMetadata.name;
const uniquenessHasFlipped =
existingFlatFieldMetadata.isUnique !== flatFieldMetadata.isUnique;
const backingIndexMustFollowRename =
existingFlatFieldMetadata.isUnique === true &&
flatFieldMetadata.isUnique === true &&
wasRenamed;
if (!uniquenessHasFlipped && !backingIndexMustFollowRename) {
return { status: 'noop' };
}
const existingFlatObjectMetadata =
relatedFlatEntityMaps.flatObjectMetadataMaps.byUniversalIdentifier[
flatFieldMetadata.objectMetadataUniversalIdentifier
];
const optimisticFlatObjectMetadata =
resolveParentFlatObjectMetadataAfterStateForFieldSideEffect({
objectMetadataUniversalIdentifier:
flatFieldMetadata.objectMetadataUniversalIdentifier,
allFlatEntityOperationRecordByMetadataName,
relatedFlatEntityMaps,
});
const backingIndexMustBeDeleted =
existingFlatFieldMetadata.isUnique === true;
const backingIndexMustBeCreated = flatFieldMetadata.isUnique === true;
if (
(backingIndexMustBeDeleted && !isDefined(existingFlatObjectMetadata)) ||
(backingIndexMustBeCreated && !isDefined(optimisticFlatObjectMetadata))
) {
return buildFieldSideEffectParentNotFoundFailure({
flatFieldMetadata,
operation: 'update',
});
}
const previousFlatIndexMetadata =
backingIndexMustBeDeleted && isDefined(existingFlatObjectMetadata)
? generateDeterministicIndexForFlatFieldMetadataOrThrow({
flatFieldMetadata: {
...flatFieldMetadata,
name: existingFlatFieldMetadata.name,
isUnique: true,
},
flatObjectMetadata: existingFlatObjectMetadata,
})
: undefined;
const flatIndexMetadataToDelete =
isDefined(previousFlatIndexMetadata) &&
isDefined(
relatedFlatEntityMaps.flatIndexMaps.byUniversalIdentifier[
previousFlatIndexMetadata.universalIdentifier
],
)
? previousFlatIndexMetadata
: undefined;
const flatIndexMetadataToCreate =
backingIndexMustBeCreated && isDefined(optimisticFlatObjectMetadata)
? generateDeterministicIndexForFlatFieldMetadataOrThrow({
flatFieldMetadata: { ...flatFieldMetadata, isUnique: true },
flatObjectMetadata: optimisticFlatObjectMetadata,
})
: undefined;
if (
!isDefined(flatIndexMetadataToCreate) &&
!isDefined(flatIndexMetadataToDelete)
) {
return { status: 'noop' };
}
return {
status: 'success',
operations: {
index: {
...(isDefined(flatIndexMetadataToCreate)
? {
flatEntityToCreate: {
[flatIndexMetadataToCreate.universalIdentifier]:
flatIndexMetadataToCreate,
},
}
: {}),
...(isDefined(flatIndexMetadataToDelete)
? {
flatEntityToDelete: {
[flatIndexMetadataToDelete.universalIdentifier]:
flatIndexMetadataToDelete,
},
}
: {}),
},
},
};
}
}
@@ -0,0 +1,31 @@
import { msg, t } from '@lingui/core/macro';
import { type AllMetadataName } from 'twenty-shared/metadata';
import { type MetadataFlatEntity } from 'src/engine/metadata-modules/flat-entity/types/metadata-flat-entity.type';
import { type MetadataUniversalFlatEntity } from 'src/engine/metadata-modules/flat-entity/types/metadata-universal-flat-entity.type';
import { type WorkspaceMigrationActionType } from 'src/engine/metadata-modules/flat-entity/types/metadata-workspace-migration-action.type';
import { MetadataSideEffectExceptionCode } from 'src/engine/metadata-modules/metadata-side-effect/exceptions/metadata-side-effect-exception-code';
import { type MetadataSideEffectFailure } from 'src/engine/metadata-modules/metadata-side-effect/types/metadata-side-effect-result.type';
export const buildFieldSideEffectParentNotFoundFailure = ({
flatFieldMetadata,
operation,
}: {
flatFieldMetadata: MetadataUniversalFlatEntity<'fieldMetadata'>;
operation: WorkspaceMigrationActionType;
}): MetadataSideEffectFailure => ({
status: 'fail',
type: operation,
metadataName: 'fieldMetadata',
flatEntityMinimalInformation: {
universalIdentifier: flatFieldMetadata.universalIdentifier,
name: flatFieldMetadata.name,
} as Partial<MetadataFlatEntity<AllMetadataName>>,
errors: [
{
code: MetadataSideEffectExceptionCode.SIDE_EFFECT_PARENT_METADATA_NOT_FOUND,
message: t`Could not resolve parent object metadata "${flatFieldMetadata.objectMetadataUniversalIdentifier}" for field unique index side effect`,
userFriendlyMessage: msg`This field references an object that could not be found`,
},
],
});
@@ -0,0 +1,26 @@
import { type AllFlatEntityOperationRecordByMetadataName } from 'src/engine/metadata-modules/flat-entity/types/all-flat-entity-operation-record-by-metadata-name.type';
import { type MetadataFlatEntityAndRelatedFlatEntityMapsForSideEffect } from 'src/engine/metadata-modules/flat-entity/types/metadata-flat-entity-and-related-flat-entity-maps-for-side-effect.type';
import { type UniversalFlatObjectMetadata } from 'src/engine/workspace-manager/workspace-migration/universal-flat-entity/types/universal-flat-object-metadata.type';
export const resolveParentFlatObjectMetadataAfterStateForFieldSideEffect = ({
objectMetadataUniversalIdentifier,
allFlatEntityOperationRecordByMetadataName,
relatedFlatEntityMaps,
}: {
objectMetadataUniversalIdentifier: string;
allFlatEntityOperationRecordByMetadataName: AllFlatEntityOperationRecordByMetadataName;
relatedFlatEntityMaps: MetadataFlatEntityAndRelatedFlatEntityMapsForSideEffect<'fieldMetadata'>;
}): UniversalFlatObjectMetadata | undefined => {
const pendingFlatObjectMetadata =
allFlatEntityOperationRecordByMetadataName.objectMetadata
?.flatEntityToUpdate[objectMetadataUniversalIdentifier] ??
allFlatEntityOperationRecordByMetadataName.objectMetadata
?.flatEntityToCreate[objectMetadataUniversalIdentifier];
return (
pendingFlatObjectMetadata ??
relatedFlatEntityMaps.flatObjectMetadataMaps.byUniversalIdentifier[
objectMetadataUniversalIdentifier
]
);
};
@@ -0,0 +1,14 @@
import { Module } from '@nestjs/common';
import { FieldUniqueBackingIndexOnCreateSideEffectHandlerService } from 'src/engine/metadata-modules/metadata-side-effect/handlers/field-metadata/services/field-unique-backing-index-on-create-side-effect-handler.service';
import { FieldUniqueBackingIndexOnDeleteSideEffectHandlerService } from 'src/engine/metadata-modules/metadata-side-effect/handlers/field-metadata/services/field-unique-backing-index-on-delete-side-effect-handler.service';
import { FieldUniqueBackingIndexOnUpdateSideEffectHandlerService } from 'src/engine/metadata-modules/metadata-side-effect/handlers/field-metadata/services/field-unique-backing-index-on-update-side-effect-handler.service';
@Module({
providers: [
FieldUniqueBackingIndexOnCreateSideEffectHandlerService,
FieldUniqueBackingIndexOnUpdateSideEffectHandlerService,
FieldUniqueBackingIndexOnDeleteSideEffectHandlerService,
],
})
export class MetadataSideEffectHandlersModule {}
@@ -0,0 +1,61 @@
import { SetMetadata } from '@nestjs/common';
import { type AllMetadataName } from 'twenty-shared/metadata';
import { type AllFlatEntityOperationRecordByMetadataName } from 'src/engine/metadata-modules/flat-entity/types/all-flat-entity-operation-record-by-metadata-name.type';
import { type MetadataFlatEntityAndRelatedFlatEntityMapsForSideEffect } from 'src/engine/metadata-modules/flat-entity/types/metadata-flat-entity-and-related-flat-entity-maps-for-side-effect.type';
import { type MetadataUniversalFlatEntity } from 'src/engine/metadata-modules/flat-entity/types/metadata-universal-flat-entity.type';
import { METADATA_SIDE_EFFECT_HANDLER_METADATA_KEY } from 'src/engine/metadata-modules/metadata-side-effect/constants/metadata-side-effect-handler-metadata-key.constant';
import { type MetadataSideEffectContext } from 'src/engine/metadata-modules/metadata-side-effect/types/metadata-side-effect-context.type';
import { type MetadataSideEffectOperation } from 'src/engine/metadata-modules/metadata-side-effect/types/metadata-side-effect-operation.type';
import { type MetadataSideEffectResult } from 'src/engine/metadata-modules/metadata-side-effect/types/metadata-side-effect-result.type';
export type BuildSideEffectsArgs<P extends AllMetadataName> = {
flatEntity: MetadataUniversalFlatEntity<P>;
allFlatEntityOperationRecordByMetadataName: AllFlatEntityOperationRecordByMetadataName;
relatedFlatEntityMaps: MetadataFlatEntityAndRelatedFlatEntityMapsForSideEffect<P>;
context: MetadataSideEffectContext;
};
export abstract class BaseMetadataSideEffectHandlerService<
P extends AllMetadataName,
> {
public operation: MetadataSideEffectOperation;
public metadataName: P;
public sideEffectName: string;
public sideEffectDescription: string;
abstract buildSideEffects(
args: BuildSideEffectsArgs<P>,
): MetadataSideEffectResult;
}
type MetadataSideEffectHandlerDeclaration<P extends AllMetadataName> = {
operation: MetadataSideEffectOperation;
metadataName: P;
name: string;
description: string;
};
export function MetadataSideEffectHandler<P extends AllMetadataName>({
operation,
metadataName,
name,
description,
}: MetadataSideEffectHandlerDeclaration<P>): typeof BaseMetadataSideEffectHandlerService<P> {
abstract class SideEffectHandlerService extends BaseMetadataSideEffectHandlerService<P> {
operation = operation;
metadataName = metadataName;
sideEffectName = name;
sideEffectDescription = description;
}
SetMetadata(METADATA_SIDE_EFFECT_HANDLER_METADATA_KEY, {
operation,
metadataName,
name,
description,
})(SideEffectHandlerService);
return SideEffectHandlerService;
}
@@ -0,0 +1,16 @@
import { Module } from '@nestjs/common';
import { DiscoveryModule } from '@nestjs/core';
import { MetadataSideEffectHandlersModule } from 'src/engine/metadata-modules/metadata-side-effect/handlers/metadata-side-effect-handlers.module';
import { MetadataSideEffectHandlerRegistryService } from 'src/engine/metadata-modules/metadata-side-effect/registry/metadata-side-effect-handler-registry.service';
import { MetadataSideEffectEngineService } from 'src/engine/metadata-modules/metadata-side-effect/services/metadata-side-effect-engine.service';
@Module({
imports: [DiscoveryModule, MetadataSideEffectHandlersModule],
providers: [
MetadataSideEffectHandlerRegistryService,
MetadataSideEffectEngineService,
],
exports: [MetadataSideEffectEngineService],
})
export class MetadataSideEffectModule {}
@@ -0,0 +1,109 @@
import { Injectable, OnModuleInit } from '@nestjs/common';
import { DiscoveryService } from '@nestjs/core';
import { type AllMetadataName } from 'twenty-shared/metadata';
import { isDefined } from 'twenty-shared/utils';
import { METADATA_SIDE_EFFECT_HANDLER_METADATA_KEY } from 'src/engine/metadata-modules/metadata-side-effect/constants/metadata-side-effect-handler-metadata-key.constant';
import { MetadataSideEffectHandlersModule } from 'src/engine/metadata-modules/metadata-side-effect/handlers/metadata-side-effect-handlers.module';
import { type BaseMetadataSideEffectHandlerService } from 'src/engine/metadata-modules/metadata-side-effect/interfaces/base-metadata-side-effect-handler.service';
import {
buildMetadataSideEffectHandlerKey,
type MetadataSideEffectHandlerDescriptor,
type MetadataSideEffectHandlerKey,
type MetadataSideEffectOperation,
} from 'src/engine/metadata-modules/metadata-side-effect/types/metadata-side-effect-operation.type';
type RegisteredSideEffectHandler =
BaseMetadataSideEffectHandlerService<AllMetadataName>;
export type RegisteredMetadataSideEffectHandlerKey = {
operation: MetadataSideEffectOperation;
metadataName: AllMetadataName;
};
@Injectable()
export class MetadataSideEffectHandlerRegistryService implements OnModuleInit {
private readonly handlersByKey = new Map<
MetadataSideEffectHandlerKey,
RegisteredSideEffectHandler[]
>();
private readonly registeredHandlerKeys: RegisteredMetadataSideEffectHandlerKey[] =
[];
private readonly registeredSideEffectNames = new Set<string>();
constructor(private readonly discoveryService: DiscoveryService) {}
onModuleInit() {
this.discoverAndRegisterHandlers();
}
private discoverAndRegisterHandlers(): void {
const providers = this.discoveryService.getProviders({
include: [MetadataSideEffectHandlersModule],
});
providers.forEach((wrapper) => {
const { instance, metatype } = wrapper;
if (!instance || !metatype) return;
const descriptor: MetadataSideEffectHandlerDescriptor | undefined =
Reflect.getMetadata(
METADATA_SIDE_EFFECT_HANDLER_METADATA_KEY,
metatype,
);
if (
!isDefined(descriptor) ||
typeof instance.buildSideEffects !== 'function'
) {
return;
}
this.registerHandler(instance);
});
}
private registerHandler(instance: RegisteredSideEffectHandler): void {
if (this.registeredSideEffectNames.has(instance.sideEffectName)) {
throw new Error(
`Duplicate metadata side-effect name "${instance.sideEffectName}". Side-effect names must be unique.`,
);
}
this.registeredSideEffectNames.add(instance.sideEffectName);
const handlerKey = buildMetadataSideEffectHandlerKey(
instance.operation,
instance.metadataName,
);
const existingHandlers = this.handlersByKey.get(handlerKey);
if (isDefined(existingHandlers)) {
existingHandlers.push(instance);
return;
}
this.handlersByKey.set(handlerKey, [instance]);
this.registeredHandlerKeys.push({
operation: instance.operation,
metadataName: instance.metadataName,
});
}
getHandlers(
operation: MetadataSideEffectOperation,
metadataName: AllMetadataName,
): RegisteredSideEffectHandler[] {
return (
this.handlersByKey.get(
buildMetadataSideEffectHandlerKey(operation, metadataName),
) ?? []
);
}
getRegisteredHandlerKeys(): RegisteredMetadataSideEffectHandlerKey[] {
return this.registeredHandlerKeys;
}
}
@@ -0,0 +1,329 @@
import { Injectable } from '@nestjs/common';
import { type AllMetadataName } from 'twenty-shared/metadata';
import { isDefined } from 'twenty-shared/utils';
import { type AllFlatEntityMaps } from 'src/engine/metadata-modules/flat-entity/types/all-flat-entity-maps.type';
import { type AllFlatEntityOperationRecordByMetadataName } from 'src/engine/metadata-modules/flat-entity/types/all-flat-entity-operation-record-by-metadata-name.type';
import { type MetadataFlatEntityAndRelatedFlatEntityMapsForSideEffect } from 'src/engine/metadata-modules/flat-entity/types/metadata-flat-entity-and-related-flat-entity-maps-for-side-effect.type';
import { type MetadataUniversalFlatEntity } from 'src/engine/metadata-modules/flat-entity/types/metadata-universal-flat-entity.type';
import { getMetadataManyToOneRelatedNames } from 'src/engine/metadata-modules/flat-entity/utils/get-metadata-many-to-one-related-names.util';
import { getMetadataSideEffectCompanionNames } from 'src/engine/metadata-modules/flat-entity/utils/get-metadata-side-effect-companion-names.util';
import { isSystemSideEffectFlatEntity } from 'src/engine/metadata-modules/flat-entity/utils/is-system-side-effect-flat-entity.util';
import { MetadataSideEffectHandlerRegistryService } from 'src/engine/metadata-modules/metadata-side-effect/registry/metadata-side-effect-handler-registry.service';
import { type MetadataSideEffectContext } from 'src/engine/metadata-modules/metadata-side-effect/types/metadata-side-effect-context.type';
import { type MetadataSideEffectExpansionResult } from 'src/engine/metadata-modules/metadata-side-effect/types/metadata-side-effect-expansion-result.type';
import {
METADATA_SIDE_EFFECT_OPERATIONS,
type MetadataSideEffectOperation,
} from 'src/engine/metadata-modules/metadata-side-effect/types/metadata-side-effect-operation.type';
import { type MetadataSideEffectFailure } from 'src/engine/metadata-modules/metadata-side-effect/types/metadata-side-effect-result.type';
import { type SystemSideEffectUniversalIdentifierCollision } from 'src/engine/metadata-modules/metadata-side-effect/types/system-side-effect-universal-identifier-collision.type';
import { mapSystemSideEffectCollisionToFailure } from 'src/engine/metadata-modules/metadata-side-effect/utils/map-system-side-effect-collision-to-failure.util';
import { EMPTY_ORCHESTRATOR_FAILURE_REPORT } from 'src/engine/workspace-manager/workspace-migration/constant/empty-orchestrator-failure-report.constant';
import { pushToOrchestratorFailureReport } from 'src/engine/workspace-manager/workspace-migration/utils/merge-orchestrator-failure-reports.util';
type GenericUniversalFlatEntity = { universalIdentifier: string };
type GenericFlatEntityOperationRecord = {
flatEntityToCreate: Record<string, GenericUniversalFlatEntity>;
flatEntityToUpdate: Record<string, GenericUniversalFlatEntity>;
flatEntityToDelete: Record<string, GenericUniversalFlatEntity>;
};
type GenericAllFlatEntityOperationRecordByMetadataName = Record<
string,
GenericFlatEntityOperationRecord | undefined
>;
type GenericPartialFlatEntityOperationRecord = {
flatEntityToCreate?: Record<string, GenericUniversalFlatEntity>;
flatEntityToUpdate?: Record<string, GenericUniversalFlatEntity>;
flatEntityToDelete?: Record<string, GenericUniversalFlatEntity>;
};
type GenericMetadataSideEffectOperationsByMetadataName = Record<
string,
GenericPartialFlatEntityOperationRecord | undefined
>;
const OPERATION_TO_FLAT_ENTITY_RECORD_KEY = {
create: 'flatEntityToCreate',
update: 'flatEntityToUpdate',
delete: 'flatEntityToDelete',
} as const satisfies Record<
MetadataSideEffectOperation,
keyof GenericFlatEntityOperationRecord
>;
@Injectable()
export class MetadataSideEffectEngineService {
constructor(
private readonly metadataSideEffectHandlerRegistryService: MetadataSideEffectHandlerRegistryService,
) {}
getSideEffectRelatedMetadataNames(
triggerMetadataNames: AllMetadataName[],
): AllMetadataName[] {
const relatedMetadataNames = new Set<AllMetadataName>();
for (const {
metadataName,
} of this.metadataSideEffectHandlerRegistryService.getRegisteredHandlerKeys()) {
if (!triggerMetadataNames.includes(metadataName)) {
continue;
}
for (const relatedMetadataName of [
metadataName,
...getMetadataManyToOneRelatedNames(metadataName),
...getMetadataSideEffectCompanionNames(metadataName),
]) {
relatedMetadataNames.add(relatedMetadataName);
}
}
return [...relatedMetadataNames];
}
expandWithSideEffects({
allFlatEntityOperationRecordByMetadataName,
sideEffectRelatedFlatEntityMaps,
context,
}: {
allFlatEntityOperationRecordByMetadataName: AllFlatEntityOperationRecordByMetadataName;
sideEffectRelatedFlatEntityMaps: Partial<AllFlatEntityMaps>;
context: MetadataSideEffectContext;
}): MetadataSideEffectExpansionResult {
const expandedMatrix = this.cloneMatrix(
allFlatEntityOperationRecordByMetadataName,
);
const systemSideEffectUniversalIdentifierCollisions: SystemSideEffectUniversalIdentifierCollision[] =
[];
const sideEffectFailures: MetadataSideEffectFailure[] = [];
const triggerMatrix =
allFlatEntityOperationRecordByMetadataName as unknown as GenericAllFlatEntityOperationRecordByMetadataName;
for (const {
operation,
metadataName,
} of this.metadataSideEffectHandlerRegistryService.getRegisteredHandlerKeys()) {
const handlers =
this.metadataSideEffectHandlerRegistryService.getHandlers(
operation,
metadataName,
);
if (handlers.length === 0) {
continue;
}
const triggerFlatEntities = Object.values(
triggerMatrix[metadataName]?.[
OPERATION_TO_FLAT_ENTITY_RECORD_KEY[operation]
] ?? {},
);
for (const triggerFlatEntity of triggerFlatEntities) {
for (const handler of handlers) {
const sideEffectResult = handler.buildSideEffects({
flatEntity:
triggerFlatEntity as unknown as MetadataUniversalFlatEntity<AllMetadataName>,
allFlatEntityOperationRecordByMetadataName:
expandedMatrix as unknown as AllFlatEntityOperationRecordByMetadataName,
relatedFlatEntityMaps:
sideEffectRelatedFlatEntityMaps as unknown as MetadataFlatEntityAndRelatedFlatEntityMapsForSideEffect<AllMetadataName>,
context,
});
if (sideEffectResult.status === 'fail') {
sideEffectFailures.push(sideEffectResult);
continue;
}
if (sideEffectResult.status === 'noop') {
continue;
}
this.mergeSideEffectsIntoMatrix({
expandedMatrix,
sideEffectOperations:
sideEffectResult.operations as unknown as GenericMetadataSideEffectOperationsByMetadataName,
systemSideEffectUniversalIdentifierCollisions,
});
}
}
}
const allSideEffectFailures: MetadataSideEffectFailure[] = [
...sideEffectFailures,
...systemSideEffectUniversalIdentifierCollisions.map(
mapSystemSideEffectCollisionToFailure,
),
];
if (allSideEffectFailures.length > 0) {
const report = EMPTY_ORCHESTRATOR_FAILURE_REPORT();
for (const sideEffectFailure of allSideEffectFailures) {
pushToOrchestratorFailureReport({
report,
metadataName: sideEffectFailure.metadataName,
items: [sideEffectFailure],
});
}
return {
status: 'fail',
report,
};
}
return {
status: 'success',
allFlatEntityOperationRecordByMetadataName:
expandedMatrix as unknown as AllFlatEntityOperationRecordByMetadataName,
};
}
private mergeSideEffectsIntoMatrix({
expandedMatrix,
sideEffectOperations,
systemSideEffectUniversalIdentifierCollisions,
}: {
expandedMatrix: GenericAllFlatEntityOperationRecordByMetadataName;
sideEffectOperations: GenericMetadataSideEffectOperationsByMetadataName;
systemSideEffectUniversalIdentifierCollisions: SystemSideEffectUniversalIdentifierCollision[];
}): void {
for (const metadataName of Object.keys(sideEffectOperations)) {
const operationBuckets = sideEffectOperations[metadataName];
if (!isDefined(operationBuckets)) {
continue;
}
for (const operation of METADATA_SIDE_EFFECT_OPERATIONS) {
const sideEffectFlatEntities = Object.values(
operationBuckets[OPERATION_TO_FLAT_ENTITY_RECORD_KEY[operation]] ??
{},
);
for (const sideEffectFlatEntity of sideEffectFlatEntities) {
this.addToOperationIfAbsent({
expandedMatrix,
operation,
metadataName,
flatEntity: sideEffectFlatEntity,
systemSideEffectUniversalIdentifierCollisions,
});
}
}
}
}
private cloneMatrix(
allFlatEntityOperationRecordByMetadataName: AllFlatEntityOperationRecordByMetadataName,
): GenericAllFlatEntityOperationRecordByMetadataName {
const genericMatrix =
allFlatEntityOperationRecordByMetadataName as unknown as GenericAllFlatEntityOperationRecordByMetadataName;
const clonedMatrix: GenericAllFlatEntityOperationRecordByMetadataName = {};
for (const metadataName of Object.keys(genericMatrix)) {
const operations = genericMatrix[metadataName];
if (!isDefined(operations)) {
continue;
}
clonedMatrix[metadataName] = {
flatEntityToCreate: { ...operations.flatEntityToCreate },
flatEntityToUpdate: { ...operations.flatEntityToUpdate },
flatEntityToDelete: { ...operations.flatEntityToDelete },
};
}
return clonedMatrix;
}
private addToOperationIfAbsent({
expandedMatrix,
operation,
metadataName,
flatEntity,
systemSideEffectUniversalIdentifierCollisions,
}: {
expandedMatrix: GenericAllFlatEntityOperationRecordByMetadataName;
operation: MetadataSideEffectOperation;
metadataName: string;
flatEntity: GenericUniversalFlatEntity;
systemSideEffectUniversalIdentifierCollisions: SystemSideEffectUniversalIdentifierCollision[];
}): void {
const operations = (expandedMatrix[metadataName] ??= {
flatEntityToCreate: {},
flatEntityToUpdate: {},
flatEntityToDelete: {},
});
const flatEntityRecordKey = OPERATION_TO_FLAT_ENTITY_RECORD_KEY[operation];
const flatEntityRecord = operations[flatEntityRecordKey];
const existingFlatEntity = flatEntityRecord[flatEntity.universalIdentifier];
if (isDefined(existingFlatEntity)) {
this.recordUniversalIdentifierCollisionIfNeeded({
existingFlatEntity,
operation,
metadataName,
flatEntity,
systemSideEffectUniversalIdentifierCollisions,
});
return;
}
flatEntityRecord[flatEntity.universalIdentifier] = flatEntity;
}
private recordUniversalIdentifierCollisionIfNeeded({
existingFlatEntity,
operation,
metadataName,
flatEntity,
systemSideEffectUniversalIdentifierCollisions,
}: {
existingFlatEntity: GenericUniversalFlatEntity;
operation: MetadataSideEffectOperation;
metadataName: string;
flatEntity: GenericUniversalFlatEntity;
systemSideEffectUniversalIdentifierCollisions: SystemSideEffectUniversalIdentifierCollision[];
}): void {
const isIncomingSystemSideEffect = isSystemSideEffectFlatEntity(
flatEntity as unknown as MetadataUniversalFlatEntity<AllMetadataName>,
);
if (!isIncomingSystemSideEffect) {
return;
}
if (
isSystemSideEffectFlatEntity(
existingFlatEntity as unknown as MetadataUniversalFlatEntity<AllMetadataName>,
)
) {
return;
}
systemSideEffectUniversalIdentifierCollisions.push({
metadataName: metadataName as AllMetadataName,
operation,
universalIdentifier: flatEntity.universalIdentifier,
name: this.extractFlatEntityName(flatEntity),
});
}
private extractFlatEntityName(
flatEntity: GenericUniversalFlatEntity,
): string | undefined {
const { name } = flatEntity as { name?: unknown };
return typeof name === 'string' ? name : undefined;
}
}
@@ -0,0 +1,5 @@
import { type WorkspaceMigrationBuilderOptions } from 'src/engine/workspace-manager/workspace-migration/workspace-migration-builder/types/workspace-migration-builder-options.type';
export type MetadataSideEffectContext = {
buildOptions: WorkspaceMigrationBuilderOptions;
};
@@ -0,0 +1,15 @@
import { type AllFlatEntityOperationRecordByMetadataName } from 'src/engine/metadata-modules/flat-entity/types/all-flat-entity-operation-record-by-metadata-name.type';
import { type OrchestratorFailureReport } from 'src/engine/workspace-manager/workspace-migration/types/workspace-migration-orchestrator.type';
// Same contract as the builder (WorkspaceMigrationOrchestratorFailedResult):
// collisions and handler failures are merged into a single OrchestratorFailureReport
// so callers handle side-effect and builder failures through one uniform channel.
export type MetadataSideEffectExpansionResult =
| {
status: 'success';
allFlatEntityOperationRecordByMetadataName: AllFlatEntityOperationRecordByMetadataName;
}
| {
status: 'fail';
report: OrchestratorFailureReport;
};
@@ -0,0 +1,24 @@
import { type AllMetadataName } from 'twenty-shared/metadata';
export type MetadataSideEffectOperation = 'create' | 'update' | 'delete';
export const METADATA_SIDE_EFFECT_OPERATIONS = [
'create',
'update',
'delete',
] as const satisfies readonly MetadataSideEffectOperation[];
export type MetadataSideEffectHandlerKey =
`${MetadataSideEffectOperation}:${AllMetadataName}`;
export const buildMetadataSideEffectHandlerKey = (
operation: MetadataSideEffectOperation,
metadataName: AllMetadataName,
): MetadataSideEffectHandlerKey => `${operation}:${metadataName}`;
export type MetadataSideEffectHandlerDescriptor = {
operation: MetadataSideEffectOperation;
metadataName: AllMetadataName;
name: string;
description: string;
};
@@ -0,0 +1,17 @@
import { type AllMetadataName } from 'twenty-shared/metadata';
import { type MetadataUniversalFlatEntity } from 'src/engine/metadata-modules/flat-entity/types/metadata-universal-flat-entity.type';
// Record-native, partial companion operations a handler emits. Each bucket is keyed by
// universalIdentifier so the engine merges/deduplicates against the matrix in O(1),
// matching the shape it consumes (AllFlatEntityOperationRecordByMetadataName).
export type MetadataSideEffectOperationsByMetadataName = {
[P in AllMetadataName]?: {
flatEntityToCreate?: Record<
string,
MetadataUniversalFlatEntity<P> & { id?: string }
>;
flatEntityToUpdate?: Record<string, MetadataUniversalFlatEntity<P>>;
flatEntityToDelete?: Record<string, MetadataUniversalFlatEntity<P>>;
};
};
@@ -0,0 +1,27 @@
import { type AllMetadataName } from 'twenty-shared/metadata';
import { type WorkspaceMigrationActionType } from 'src/engine/metadata-modules/flat-entity/types/metadata-workspace-migration-action.type';
import { type MetadataSideEffectOperationsByMetadataName } from 'src/engine/metadata-modules/metadata-side-effect/types/metadata-side-effect-operations-by-metadata-name.type';
import { type FailedFlatEntityValidation } from 'src/engine/workspace-manager/workspace-migration/workspace-migration-builder/builders/types/failed-flat-entity-validation.type';
// A single failure emitted by a side-effect handler, using the exact same shape
// as the builder's per-entity validation failure (a `status: 'fail'` tag spread
// over a FailedFlatEntityValidation), so both sources merge into one report.
export type MetadataSideEffectFailure = {
status: 'fail';
} & FailedFlatEntityValidation<AllMetadataName, WorkspaceMigrationActionType>;
// Discriminated union of the three handler outcomes: `noop` when there is
// nothing to do, `success` when it produced companion operations to merge, or
// `fail` (fail-slow) with a validation failure the engine aggregates instead of
// throwing. `success` therefore always carries operations, keeping the empty
// case explicit as `noop` rather than a success with `operations: {}`.
export type MetadataSideEffectResult =
| {
status: 'noop';
}
| {
status: 'success';
operations: MetadataSideEffectOperationsByMetadataName;
}
| MetadataSideEffectFailure;
@@ -0,0 +1,10 @@
import { type AllMetadataName } from 'twenty-shared/metadata';
import { type MetadataSideEffectOperation } from 'src/engine/metadata-modules/metadata-side-effect/types/metadata-side-effect-operation.type';
export type SystemSideEffectUniversalIdentifierCollision = {
metadataName: AllMetadataName;
operation: MetadataSideEffectOperation;
universalIdentifier: string;
name?: string;
};
@@ -0,0 +1,29 @@
import { msg, t } from '@lingui/core/macro';
import { type AllMetadataName } from 'twenty-shared/metadata';
import { type MetadataFlatEntity } from 'src/engine/metadata-modules/flat-entity/types/metadata-flat-entity.type';
import { MetadataSideEffectExceptionCode } from 'src/engine/metadata-modules/metadata-side-effect/exceptions/metadata-side-effect-exception-code';
import { type MetadataSideEffectFailure } from 'src/engine/metadata-modules/metadata-side-effect/types/metadata-side-effect-result.type';
import { type SystemSideEffectUniversalIdentifierCollision } from 'src/engine/metadata-modules/metadata-side-effect/types/system-side-effect-universal-identifier-collision.type';
// A reserved-identifier collision is just another side-effect failure: mapping it
// to the shared MetadataSideEffectFailure shape lets the engine merge collisions
// and handler failures into a single report.
export const mapSystemSideEffectCollisionToFailure = (
collision: SystemSideEffectUniversalIdentifierCollision,
): MetadataSideEffectFailure => ({
status: 'fail',
type: collision.operation,
metadataName: collision.metadataName,
flatEntityMinimalInformation: {
universalIdentifier: collision.universalIdentifier,
...(collision.name !== undefined ? { name: collision.name } : {}),
} as Partial<MetadataFlatEntity<AllMetadataName>>,
errors: [
{
code: MetadataSideEffectExceptionCode.RESERVED_SYSTEM_UNIVERSAL_IDENTIFIER,
message: t`Universal identifier is reserved for system-managed metadata`,
userFriendlyMessage: msg`This identifier is reserved by the system`,
},
],
});