Centralized side effects devxp basis (#22295)

# Introduction

This PR introduces a centralized, strictly-typed **metadata side-effect
engine** that unifies how system metadata side effects are derived and
applied across both metadata entry points — the **metadata GraphQL API**
and the **application sync / manifest** flow — and migrates the first
side effect end-to-end: **a unique scalar field owns its backing
single-field `UNIQUE` index** (full create / update / delete lifecycle).

## New conventions

- **Engine-owned companions**: metadata flagged `isSystemSideEffect:
true` is owned by the engine. Its deletion is never inferred from
absence in a manifest — it results from PG-level cascade or from a
delete side effect (a side effect always has a cause, its parent
metadata).
- **Reserved deterministic identifiers**: apps cannot declare metadata
reusing an engine-owned deterministic `universalIdentifier`. Doing so
fails validation with `RESERVED_SYSTEM_UNIVERSAL_IDENTIFIER` (until an
explicit override API exists).
- **Record-native operation matrix**: the operation matrix is keyed by
`universalIdentifier` (`AllFlatEntityOperationRecordByMetadataName`)
instead of arrays, making parent resolution and deduplication O(1).
Array-based API callers are transpiled to records at the
validate-build-and-run boundary.
- Twenty-sdk user-facing experience with system fields will only be
related to overrides.

# What this PR does

## 1. Side-effect engine (foundation)

- `MetadataSideEffectEngineService.expandWithSideEffects(...)` takes the
intention-carrying record matrix and returns it expanded with derived
side effects, or a structured failure.
- Handlers are registered via a typed **decorator + registry** pattern
(`MetadataSideEffectHandler({ operation, metadataName, name, description
})`), with runtime duplicate-name detection. Multiple handlers per
(operation, metadataName) are supported.
- Handler contract mirrors the validator pattern:
- receives the trigger flat entity, the live record matrix, and
**strictly-typed related flat entity maps**
(`MetadataFlatEntityAndRelatedFlatEntityMapsForSideEffect<P>`, derived
from declared companion metadata names — no loose
`Partial<AllFlatEntityMaps>` context)
- returns `MetadataSideEffectResult`: `success` (operations record) |
`noop` | `fail` (structured failure)
- **Non-recursion is structural**: triggers are read from the original
caller input, never from the expanded matrix, so a side effect can never
trigger another side effect.
- **Deduplication + collision detection**: side effects are deduped by
`universalIdentifier` per operation; a caller-declared entity colliding
with an engine-owned deterministic identifier is recorded as a
collision.
- **Unified failure channel**: handler failures and reserved-identifier
collisions are merged into the same `OrchestratorFailureReport` contract
as builder validation errors, and the run short-circuits (fail-closed,
nothing is applied).

## 2. First migrated side effect — unique field → backing unique index

Three handlers own the complete lifecycle of the deterministic
single-field `UNIQUE` index backing a unique scalar field:

- **create**: unique scalar field → generate the deterministic backing
index (`fieldUniqueBackingIndexOnCreate`)
- **update**: `isUnique` flips and renames of still-unique fields (the
index name — and therefore its deterministic identifier — derives from
the field name, so a rename drops the stale index and recreates the
deterministic one) (`fieldUniqueBackingIndexOnUpdate`)
- **delete**: cascade-delete the backing index
(`fieldUniqueBackingIndexOnDelete`)

Supporting rules:
- The primary key `id` field never spawns a backing index (uniqueness
comes from the PK constraint) — explicit `isPrimaryKeyFlatFieldMetadata`
guard.
- Parent object resolution is **optimistic-first**: an object created or
updated in the same batch wins over the workspace cache (so e.g.
renaming an object while flipping a field to unique builds the index
from the post-rename object), resolved in O(1) via the record matrix.
- A missing parent object is reported as a structured side-effect
failure, never silently skipped.

## 3. Path convergence — manifest and API share one flow

- The manifest sync now derives a from→to **record matrix** from the
cache and feeds `validateBuildAndRunWorkspaceMigrationFromRecord`, the
same flow the API uses — both paths converge on the engine.
- Manifest-side unique-index generation and API transpiler
system-unique-index handling were removed (declared/composite/relation
indexes stay untouched).
- New `WorkspaceMigrationFlatEntityMapsService` mutualizes
flat-entity-maps computation between the side-effect engine and the
builder: cache keys are derived from the caller metadata names (+
validation- and side-effect-related closures) instead of hardcoded
loads.
- App-scoping and pruning are folded into one shared primitive
(`getSubAllFlatEntityMapsByApplicationIdsOrThrow`): slicing dependency
maps to the involved applications always prunes dangling one-to-many
aggregators — callers can no longer forget it.
- **Behavior change**: an app extending another app's view with a view
field now syncs successfully (cross-app view-field extension), covered
by a dedicated integration test.

## 4. Backfill upgrade command (2.19)

`upgrade:2-19:backfill-system-unique-index-universal-identifier`
rewrites legacy system unique-index `universalIdentifier`s to their
deterministic value so the engine can own pre-existing indexes. The
backfill is **driven from `isUnique: true` fields** (mirroring the
engine ownership predicate — excludes PK / morph / relation fields) and
resolves each field's backing index in O(1).

# Bugs fixed along the way

- `database:reset` seeding failed with
`INDEX_FIELD_INVALID_DEFAULT_VALUE`: the engine derived a backing
`UNIQUE` index for the default `id` primary key. Fixed with the explicit
primary-key guard.
- `isUnique` updates on system-flagged standard fields (e.g.
auto-created `name`) did not trigger the backing-index side effect.
- Manifest sync crashed with "Could not find flat entity with universal
identifier ..." when app-scoped slices left dangling aggregator
references — fixed by centralizing pruning in the shared slice primitive

packages/twenty-server/src/engine/workspace-manager/workspace-migration/services/workspace-migration-flat-entity-maps.service.ts

@@ -0,0 +1,341 @@
+import { Injectable } from '@nestjs/common';
+
+import { AllMetadataName } from 'twenty-shared/metadata';
+import { isDefined } from 'twenty-shared/utils';
+
+import { FlatApplicationCacheMaps } from 'src/engine/core-modules/application/types/flat-application-cache-maps.type';
+import { ALL_MANY_TO_ONE_METADATA_RELATIONS } from 'src/engine/metadata-modules/flat-entity/constant/all-many-to-one-metadata-relations.constant';
+import {
+  FlatEntityMapsException,
+  FlatEntityMapsExceptionCode,
+} from 'src/engine/metadata-modules/flat-entity/exceptions/flat-entity-maps.exception';
+import { AllFlatEntityMaps } from 'src/engine/metadata-modules/flat-entity/types/all-flat-entity-maps.type';
+import { AllFlatEntityOperationRecordByMetadataName } from 'src/engine/metadata-modules/flat-entity/types/all-flat-entity-operation-record-by-metadata-name.type';
+import { MetadataUniversalFlatEntity } from 'src/engine/metadata-modules/flat-entity/types/metadata-universal-flat-entity.type';
+import { getMetadataFlatEntityMapsKey } from 'src/engine/metadata-modules/flat-entity/utils/get-metadata-flat-entity-maps-key.util';
+import { getMetadataRelatedMetadataNamesForValidation } from 'src/engine/metadata-modules/flat-entity/utils/get-metadata-related-metadata-names-for-validation.util';
+import { getSubAllFlatEntityMapsByApplicationIdsOrThrow } from 'src/engine/metadata-modules/flat-entity/utils/get-sub-all-flat-entity-maps-by-application-ids-or-throw.util';
+import { MetadataSideEffectEngineService } from 'src/engine/metadata-modules/metadata-side-effect/services/metadata-side-effect-engine.service';
+import { WorkspaceCacheService } from 'src/engine/workspace-cache/services/workspace-cache.service';
+import { TWENTY_STANDARD_APPLICATION } from 'src/engine/workspace-manager/twenty-standard-application/constants/twenty-standard-applications';
+import { WORKSPACE_MIGRATION_ADDITIONAL_CACHE_DATA_MAPS_KEY } from 'src/engine/workspace-manager/workspace-migration/constant/workspace-migration-additional-cache-data-maps-key.constant';
+import { IdByUniversalIdentifierByMetadataName } from 'src/engine/workspace-manager/workspace-migration/services/utils/enrich-create-workspace-migration-action-with-ids.util';
+import { WorkspaceMigrationBuilderAdditionalCacheDataMaps } from 'src/engine/workspace-manager/workspace-migration/types/workspace-migration-builder-additional-cache-data-maps.type';
+import { FromToAllUniversalFlatEntityMaps } from 'src/engine/workspace-manager/workspace-migration/types/workspace-migration-orchestrator.type';
+import { computeUniversalFlatEntityMapsFromToThroughMutation } from 'src/engine/workspace-manager/workspace-migration/utils/compute-universal-flat-entity-maps-from-to-through-mutation.util';
+import { InferDeletionFromMissingEntities } from 'src/engine/workspace-manager/workspace-migration/workspace-migration-builder/types/infer-deletion-from-missing-entities.type';
+
+export type WorkspaceMigrationRelatedFlatEntityMaps =
+  Partial<AllFlatEntityMaps> & WorkspaceMigrationBuilderAdditionalCacheDataMaps;
+
+type FlatEntityMapsBundle = {
+  flatApplicationMaps: FlatApplicationCacheMaps;
+  allRelatedFlatEntityMaps: WorkspaceMigrationRelatedFlatEntityMaps;
+  allMetadataNameCacheToCompute: AllMetadataName[];
+};
+
+type ComputeFromToAllFlatEntityMapsAndBuildOptionsArgs = {
+  allFlatEntityOperationRecordByMetadataName: AllFlatEntityOperationRecordByMetadataName;
+  applicationUniversalIdentifier: string;
+} & FlatEntityMapsBundle;
+
+@Injectable()
+export class WorkspaceMigrationFlatEntityMapsService {
+  constructor(
+    private readonly workspaceCacheService: WorkspaceCacheService,
+    private readonly metadataSideEffectEngineService: MetadataSideEffectEngineService,
+  ) {}
+
+  async getOrRecomputeAllRelatedFlatEntityMaps({
+    workspaceId,
+    callerMetadataNames,
+  }: {
+    workspaceId: string;
+    callerMetadataNames: AllMetadataName[];
+  }): Promise<FlatEntityMapsBundle> {
+    const allMetadataNameCacheToCompute = [
+      ...new Set(
+        [
+          ...callerMetadataNames,
+          ...this.metadataSideEffectEngineService.getSideEffectRelatedMetadataNames(
+            callerMetadataNames,
+          ),
+        ].flatMap((metadataName) => [
+          metadataName,
+          ...getMetadataRelatedMetadataNamesForValidation(metadataName),
+        ]),
+      ),
+    ];
+
+    const { flatApplicationMaps, ...allRelatedFlatEntityMaps } =
+      await this.workspaceCacheService.getOrRecompute(workspaceId, [
+        ...allMetadataNameCacheToCompute.map(getMetadataFlatEntityMapsKey),
+        ...WORKSPACE_MIGRATION_ADDITIONAL_CACHE_DATA_MAPS_KEY,
+        'flatApplicationMaps',
+      ]);
+
+    return {
+      flatApplicationMaps,
+      allRelatedFlatEntityMaps,
+      allMetadataNameCacheToCompute,
+    };
+  }
+
+  computeFromToAllFlatEntityMapsAndBuildOptions({
+    allFlatEntityOperationRecordByMetadataName,
+    applicationUniversalIdentifier,
+    flatApplicationMaps,
+    allRelatedFlatEntityMaps,
+    allMetadataNameCacheToCompute,
+  }: ComputeFromToAllFlatEntityMapsAndBuildOptionsArgs): {
+    fromToAllFlatEntityMaps: FromToAllUniversalFlatEntityMaps;
+    inferDeletionFromMissingEntities: InferDeletionFromMissingEntities;
+    dependencyAllFlatEntityMaps: Partial<AllFlatEntityMaps>;
+    additionalCacheDataMaps: WorkspaceMigrationBuilderAdditionalCacheDataMaps;
+    idByUniversalIdentifierByMetadataName: IdByUniversalIdentifierByMetadataName;
+  } {
+    const { dependencyAllFlatEntityMaps, additionalCacheDataMaps } =
+      this.computeDependencyAndAdditionalCacheDataMaps({
+        allFlatEntityOperationRecordByMetadataName,
+        applicationUniversalIdentifier,
+        flatApplicationMaps,
+        allRelatedFlatEntityMaps,
+        allMetadataNameCacheToCompute,
+      });
+
+    const fromToAllFlatEntityMaps: FromToAllUniversalFlatEntityMaps = {};
+    const idByUniversalIdentifierByMetadataName: IdByUniversalIdentifierByMetadataName =
+      {};
+    const inferDeletionFromMissingEntities: InferDeletionFromMissingEntities =
+      {};
+    const allMetadataNameToCompare = Object.keys(
+      allFlatEntityOperationRecordByMetadataName,
+    ) as AllMetadataName[];
+
+    for (const metadataName of allMetadataNameToCompare) {
+      const flatEntityOperations =
+        allFlatEntityOperationRecordByMetadataName[metadataName];
+
+      if (!isDefined(flatEntityOperations)) {
+        throw new FlatEntityMapsException(
+          `Could not load flat entity maps to compare for ${metadataName}, should never occur`,
+          FlatEntityMapsExceptionCode.INTERNAL_SERVER_ERROR,
+        );
+      }
+
+      // The record matrix is the canonical form; the from/to mutation helper still
+      // consumes arrays, so we flatten each bucket at this boundary only.
+      const flatEntityToCreate = Object.values(
+        flatEntityOperations.flatEntityToCreate,
+      );
+      const flatEntityToUpdate = Object.values(
+        flatEntityOperations.flatEntityToUpdate,
+      );
+      const flatEntityToDelete = Object.values(
+        flatEntityOperations.flatEntityToDelete,
+      );
+
+      const idByUniversalIdentifier = Object.fromEntries(
+        flatEntityToCreate
+          .filter(
+            (
+              flatEntity,
+            ): flatEntity is MetadataUniversalFlatEntity<
+              typeof metadataName
+            > & { id: string } => isDefined(flatEntity.id),
+          )
+          .map((flatEntity) => [flatEntity.universalIdentifier, flatEntity.id]),
+      );
+
+      if (Object.keys(idByUniversalIdentifier).length > 0) {
+        idByUniversalIdentifierByMetadataName[metadataName] =
+          idByUniversalIdentifier;
+      }
+
+      const flatEntityMapsKey = getMetadataFlatEntityMapsKey(metadataName);
+      const flatEntityMaps = allRelatedFlatEntityMaps[flatEntityMapsKey];
+
+      if (!isDefined(flatEntityMaps)) {
+        throw new FlatEntityMapsException(
+          `Flat entity maps for ${metadataName} were not pre-fetched; the up-front cache key union must cover every metadata name the side-effect expansion can add`,
+          FlatEntityMapsExceptionCode.INTERNAL_SERVER_ERROR,
+        );
+      }
+
+      // @ts-expect-error Metadata flat entity maps cache key and metadataName colliding
+      fromToAllFlatEntityMaps[flatEntityMapsKey] =
+        computeUniversalFlatEntityMapsFromToThroughMutation({
+          flatEntityMaps: structuredClone(flatEntityMaps),
+          flatEntityToCreate,
+          flatEntityToDelete,
+          flatEntityToUpdate,
+        });
+
+      if (flatEntityToDelete.length > 0) {
+        inferDeletionFromMissingEntities[metadataName] = true;
+      }
+    }
+
+    return {
+      fromToAllFlatEntityMaps,
+      inferDeletionFromMissingEntities,
+      dependencyAllFlatEntityMaps,
+      additionalCacheDataMaps,
+      idByUniversalIdentifierByMetadataName,
+    };
+  }
+
+  private computeDependencyAndAdditionalCacheDataMaps({
+    allFlatEntityOperationRecordByMetadataName,
+    applicationUniversalIdentifier,
+    flatApplicationMaps,
+    allRelatedFlatEntityMaps,
+    allMetadataNameCacheToCompute,
+  }: ComputeFromToAllFlatEntityMapsAndBuildOptionsArgs) {
+    const applicationIds = this.computeAllInvolvedApplicationIds({
+      allFlatEntityOperationRecordByMetadataName,
+      flatApplicationMaps,
+      applicationUniversalIdentifier,
+      allRelatedFlatEntityMaps,
+    });
+
+    const dependencyAllFlatEntityMaps =
+      getSubAllFlatEntityMapsByApplicationIdsOrThrow({
+        applicationIds,
+        metadataNames: allMetadataNameCacheToCompute,
+        fromAllFlatEntityMaps: allRelatedFlatEntityMaps,
+      });
+
+    const additionalCacheDataMaps =
+      WORKSPACE_MIGRATION_ADDITIONAL_CACHE_DATA_MAPS_KEY.reduce<WorkspaceMigrationBuilderAdditionalCacheDataMaps>(
+        (acc, additionalCacheDataMapsKey) => {
+          return {
+            ...acc,
+            [additionalCacheDataMapsKey]:
+              allRelatedFlatEntityMaps[additionalCacheDataMapsKey],
+          };
+        },
+        {} as WorkspaceMigrationBuilderAdditionalCacheDataMaps,
+      );
+
+    return {
+      dependencyAllFlatEntityMaps,
+      additionalCacheDataMaps,
+    };
+  }
+
+  private computeAllInvolvedApplicationIds({
+    allFlatEntityOperationRecordByMetadataName,
+    flatApplicationMaps,
+    applicationUniversalIdentifier,
+    allRelatedFlatEntityMaps,
+  }: {
+    allFlatEntityOperationRecordByMetadataName: AllFlatEntityOperationRecordByMetadataName;
+    flatApplicationMaps: FlatApplicationCacheMaps;
+    applicationUniversalIdentifier: string;
+    allRelatedFlatEntityMaps: Partial<AllFlatEntityMaps>;
+  }): string[] {
+    const applicationIds = new Set<string>();
+
+    const applicationId =
+      flatApplicationMaps.idByUniversalIdentifier[
+        applicationUniversalIdentifier
+      ];
+
+    const twentyStandardApplicationId =
+      flatApplicationMaps.idByUniversalIdentifier[
+        TWENTY_STANDARD_APPLICATION.universalIdentifier
+      ];
+
+    if (!isDefined(twentyStandardApplicationId) || !isDefined(applicationId)) {
+      throw new FlatEntityMapsException(
+        'Application to build and its dependent application not found',
+        FlatEntityMapsExceptionCode.ENTITY_NOT_FOUND,
+      );
+    }
+
+    applicationIds.add(applicationId);
+
+    const isBuildingTwentyStandardApplication =
+      applicationUniversalIdentifier ===
+      TWENTY_STANDARD_APPLICATION.universalIdentifier;
+
+    if (!isBuildingTwentyStandardApplication) {
+      applicationIds.add(twentyStandardApplicationId);
+    }
+
+    for (const metadataName of Object.keys(
+      allFlatEntityOperationRecordByMetadataName,
+    ) as AllMetadataName[]) {
+      const flatEntityOperations =
+        allFlatEntityOperationRecordByMetadataName[metadataName];
+
+      if (!isDefined(flatEntityOperations)) {
+        continue;
+      }
+
+      const { flatEntityToCreate, flatEntityToUpdate, flatEntityToDelete } =
+        flatEntityOperations;
+
+      const relations = ALL_MANY_TO_ONE_METADATA_RELATIONS[metadataName];
+
+      for (const flatEntity of [
+        ...Object.values(flatEntityToCreate),
+        ...Object.values(flatEntityToUpdate),
+        ...Object.values(flatEntityToDelete),
+      ]) {
+        const entityApplicationId =
+          flatApplicationMaps.idByUniversalIdentifier[
+            flatEntity.applicationUniversalIdentifier
+          ];
+
+        if (isDefined(entityApplicationId)) {
+          applicationIds.add(entityApplicationId);
+        }
+
+        for (const relation of Object.values(relations) as ({
+          foreignKey: string;
+          metadataName: AllMetadataName;
+          isNullable: boolean;
+          universalForeignKey: string;
+        } | null)[]) {
+          if (!isDefined(relation)) {
+            continue;
+          }
+
+          const { universalForeignKey, metadataName: targetMetadataName } =
+            relation;
+
+          const referencedUniversalIdentifier =
+            flatEntity[universalForeignKey as keyof typeof flatEntity];
+
+          if (!isDefined(referencedUniversalIdentifier)) {
+            continue;
+          }
+
+          const targetFlatEntityMaps =
+            allRelatedFlatEntityMaps[
+              getMetadataFlatEntityMapsKey(
+                targetMetadataName as AllMetadataName,
+              )
+            ];
+
+          if (!isDefined(targetFlatEntityMaps)) {
+            continue;
+          }
+
+          const referencedEntity =
+            targetFlatEntityMaps.byUniversalIdentifier[
+              referencedUniversalIdentifier
+            ];
+
+          if (isDefined(referencedEntity)) {
+            applicationIds.add(referencedEntity.applicationId);
+          }
+        }
+      }
+    }
+
+    return [...applicationIds];
+  }
+}

packages/twenty-server/src/engine/workspace-manager/workspace-migration/services/workspace-migration-validate-build-and-run-service.ts

@@ -4,53 +4,43 @@ import {
   AllMetadataName,
   WorkspaceMigrationV2ExceptionCode,
 } from 'twenty-shared/metadata';
-import { isDefined } from 'twenty-shared/utils';
 
-import { FlatApplicationCacheMaps } from 'src/engine/core-modules/application/types/flat-application-cache-maps.type';
 import { LoggerService } from 'src/engine/core-modules/logger/logger.service';
 import { TwentyConfigService } from 'src/engine/core-modules/twenty-config/twenty-config.service';
-import { ALL_MANY_TO_ONE_METADATA_RELATIONS } from 'src/engine/metadata-modules/flat-entity/constant/all-many-to-one-metadata-relations.constant';
-import { createEmptyFlatEntityMaps } from 'src/engine/metadata-modules/flat-entity/constant/create-empty-flat-entity-maps.constant';
-import {
-  FlatEntityMapsException,
-  FlatEntityMapsExceptionCode,
-} from 'src/engine/metadata-modules/flat-entity/exceptions/flat-entity-maps.exception';
-import { AllFlatEntityMaps } from 'src/engine/metadata-modules/flat-entity/types/all-flat-entity-maps.type';
-import { FlatEntityToCreateDeleteUpdate } from 'src/engine/metadata-modules/flat-entity/types/flat-entity-to-create-delete-update.type';
-import { MetadataFlatEntity } from 'src/engine/metadata-modules/flat-entity/types/metadata-flat-entity.type';
-import { MetadataUniversalFlatEntity } from 'src/engine/metadata-modules/flat-entity/types/metadata-universal-flat-entity.type';
-import { getMetadataFlatEntityMapsKey } from 'src/engine/metadata-modules/flat-entity/utils/get-metadata-flat-entity-maps-key.util';
+import { AllFlatEntityOperationRecordByMetadataName } from 'src/engine/metadata-modules/flat-entity/types/all-flat-entity-operation-record-by-metadata-name.type';
+import { AllFlatEntityOperationByMetadataName } from 'src/engine/metadata-modules/flat-entity/types/flat-entity-to-create-delete-update.type';
 import { getFlatEntityMapsExceptionContext } from 'src/engine/metadata-modules/flat-entity/utils/get-flat-entity-maps-exception-context.util';
-import { getMetadataRelatedMetadataNamesForValidation } from 'src/engine/metadata-modules/flat-entity/utils/get-metadata-related-metadata-names-for-validation.util';
-import { getSubFlatEntityMapsByApplicationIdsOrThrow } from 'src/engine/metadata-modules/flat-entity/utils/get-sub-flat-entity-maps-by-application-ids-or-throw.util';
+import { transpileFlatEntityOperationArrayToRecord } from 'src/engine/metadata-modules/flat-entity/utils/transpile-flat-entity-operation-array-to-record.util';
+import { MetadataSideEffectEngineService } from 'src/engine/metadata-modules/metadata-side-effect/services/metadata-side-effect-engine.service';
 import { MetadataEventEmitter } from 'src/engine/subscriptions/metadata-event/metadata-event-emitter';
-import { WorkspaceCacheService } from 'src/engine/workspace-cache/services/workspace-cache.service';
-import { TWENTY_STANDARD_APPLICATION } from 'src/engine/workspace-manager/twenty-standard-application/constants/twenty-standard-applications';
 import { WorkspaceMigrationV2Exception } from 'src/engine/workspace-manager/workspace-migration.exception';
-import { WORKSPACE_MIGRATION_ADDITIONAL_CACHE_DATA_MAPS_KEY } from 'src/engine/workspace-manager/workspace-migration/constant/workspace-migration-additional-cache-data-maps-key.constant';
 import {
   enrichCreateWorkspaceMigrationActionsWithIds,
   IdByUniversalIdentifierByMetadataName,
 } from 'src/engine/workspace-manager/workspace-migration/services/utils/enrich-create-workspace-migration-action-with-ids.util';
 import { WorkspaceMigrationBuildOrchestratorService } from 'src/engine/workspace-manager/workspace-migration/services/workspace-migration-build-orchestrator.service';
-import { WorkspaceMigrationBuilderAdditionalCacheDataMaps } from 'src/engine/workspace-manager/workspace-migration/types/workspace-migration-builder-additional-cache-data-maps.type';
+import { WorkspaceMigrationFlatEntityMapsService } from 'src/engine/workspace-manager/workspace-migration/services/workspace-migration-flat-entity-maps.service';
 import {
-  FromToAllUniversalFlatEntityMaps,
   WorkspaceMigrationOrchestratorBuildArgs,
   WorkspaceMigrationOrchestratorFailedResult,
   WorkspaceMigrationOrchestratorSuccessfulResult,
 } from 'src/engine/workspace-manager/workspace-migration/types/workspace-migration-orchestrator.type';
-import { computeUniversalFlatEntityMapsFromToThroughMutation } from 'src/engine/workspace-manager/workspace-migration/utils/compute-universal-flat-entity-maps-from-to-through-mutation.util';
-import { InferDeletionFromMissingEntities } from 'src/engine/workspace-manager/workspace-migration/workspace-migration-builder/types/infer-deletion-from-missing-entities.type';
 import { WorkspaceMigrationRunnerService } from 'src/engine/workspace-manager/workspace-migration/workspace-migration-runner/services/workspace-migration-runner.service';
 
 type ValidateBuildAndRunWorkspaceMigrationFromMatriceArgs = {
   workspaceId: string;
-  allFlatEntityOperationByMetadataName: {
-    [P in AllMetadataName]?: FlatEntityToCreateDeleteUpdate<P>;
-  };
+  allFlatEntityOperationByMetadataName: AllFlatEntityOperationByMetadataName;
   isSystemBuild?: boolean;
   applicationUniversalIdentifier: string;
+  dryRun?: boolean;
+};
+
+type ValidateBuildAndRunWorkspaceMigrationFromRecordArgs = {
+  workspaceId: string;
+  allFlatEntityOperationRecordByMetadataName: AllFlatEntityOperationRecordByMetadataName;
+  isSystemBuild?: boolean;
+  applicationUniversalIdentifier: string;
+  dryRun?: boolean;
 };
 
 @Injectable()
@@ -60,8 +50,9 @@ export class WorkspaceMigrationValidateBuildAndRunService {
   constructor(
     private readonly workspaceMigrationRunnerService: WorkspaceMigrationRunnerService,
     private readonly workspaceMigrationBuildOrchestratorService: WorkspaceMigrationBuildOrchestratorService,
-    private readonly workspaceCacheService: WorkspaceCacheService,
+    private readonly workspaceMigrationFlatEntityMapsService: WorkspaceMigrationFlatEntityMapsService,
     private readonly metadataEventEmitter: MetadataEventEmitter,
+    private readonly metadataSideEffectEngineService: MetadataSideEffectEngineService,
     private readonly logger: LoggerService,
     twentyConfigService: TwentyConfigService,
   ) {
@@ -70,291 +61,6 @@ export class WorkspaceMigrationValidateBuildAndRunService {
     this.isDebugEnabled = logLevels.includes('debug');
   }
 
-  private computeAllInvolvedApplicationIds({
-    allFlatEntityOperationByMetadataName,
-    flatApplicationMaps,
-    applicationUniversalIdentifier,
-    allRelatedFlatEntityMaps,
-  }: Pick<
-    ValidateBuildAndRunWorkspaceMigrationFromMatriceArgs,
-    'allFlatEntityOperationByMetadataName' | 'applicationUniversalIdentifier'
-  > & {
-    flatApplicationMaps: FlatApplicationCacheMaps;
-    allRelatedFlatEntityMaps: Partial<AllFlatEntityMaps>;
-  }): string[] {
-    const applicationIds = new Set<string>();
-
-    const applicationId =
-      flatApplicationMaps.idByUniversalIdentifier[
-        applicationUniversalIdentifier
-      ];
-
-    const twentyStandardApplicationId =
-      flatApplicationMaps.idByUniversalIdentifier[
-        TWENTY_STANDARD_APPLICATION.universalIdentifier
-      ];
-
-    if (!isDefined(twentyStandardApplicationId) || !isDefined(applicationId)) {
-      throw new FlatEntityMapsException(
-        'Application to build and its dependent application not found',
-        FlatEntityMapsExceptionCode.ENTITY_NOT_FOUND,
-      );
-    }
-
-    applicationIds.add(applicationId);
-
-    const isBuildingTwentyStandardApplication =
-      applicationUniversalIdentifier ===
-      TWENTY_STANDARD_APPLICATION.universalIdentifier;
-
-    if (!isBuildingTwentyStandardApplication) {
-      applicationIds.add(twentyStandardApplicationId);
-    }
-
-    for (const metadataName of Object.keys(
-      allFlatEntityOperationByMetadataName,
-    ) as AllMetadataName[]) {
-      const flatEntityOperations =
-        allFlatEntityOperationByMetadataName[metadataName];
-
-      if (!isDefined(flatEntityOperations)) {
-        continue;
-      }
-
-      const { flatEntityToCreate, flatEntityToUpdate, flatEntityToDelete } =
-        flatEntityOperations;
-
-      const relations = ALL_MANY_TO_ONE_METADATA_RELATIONS[metadataName];
-
-      for (const flatEntity of [
-        ...flatEntityToCreate,
-        ...flatEntityToUpdate,
-        ...flatEntityToDelete,
-      ]) {
-        const entityApplicationId =
-          flatApplicationMaps.idByUniversalIdentifier[
-            flatEntity.applicationUniversalIdentifier
-          ];
-
-        if (isDefined(entityApplicationId)) {
-          applicationIds.add(entityApplicationId);
-        }
-
-        for (const relation of Object.values(relations) as ({
-          foreignKey: string;
-          metadataName: AllMetadataName;
-          isNullable: boolean;
-          universalForeignKey: string;
-        } | null)[]) {
-          if (!isDefined(relation)) {
-            continue;
-          }
-
-          const { universalForeignKey, metadataName: targetMetadataName } =
-            relation;
-
-          const referencedUniversalIdentifier =
-            flatEntity[universalForeignKey as keyof typeof flatEntity];
-
-          if (!isDefined(referencedUniversalIdentifier)) {
-            continue;
-          }
-
-          const targetFlatEntityMaps =
-            allRelatedFlatEntityMaps[
-              getMetadataFlatEntityMapsKey(
-                targetMetadataName as AllMetadataName,
-              )
-            ];
-
-          if (!isDefined(targetFlatEntityMaps)) {
-            continue;
-          }
-
-          const referencedEntity =
-            targetFlatEntityMaps.byUniversalIdentifier[
-              referencedUniversalIdentifier
-            ];
-
-          if (isDefined(referencedEntity)) {
-            applicationIds.add(referencedEntity.applicationId);
-          }
-        }
-      }
-    }
-
-    return [...applicationIds];
-  }
-
-  private async computeAllRelatedFlatEntityMaps({
-    allFlatEntityOperationByMetadataName,
-    workspaceId,
-    applicationUniversalIdentifier,
-  }: ValidateBuildAndRunWorkspaceMigrationFromMatriceArgs) {
-    const allMetadataNameToCompare = Object.keys(
-      allFlatEntityOperationByMetadataName,
-    ) as AllMetadataName[];
-    const allMetadataNameCacheToCompute = [
-      ...new Set([
-        ...allMetadataNameToCompare,
-        ...allMetadataNameToCompare.flatMap(
-          getMetadataRelatedMetadataNamesForValidation,
-        ),
-      ]),
-    ];
-    const allFlatEntityMapsCacheKeysToCompute =
-      allMetadataNameCacheToCompute.map(getMetadataFlatEntityMapsKey);
-
-    const { flatApplicationMaps, ...allRelatedFlatEntityMaps } =
-      await this.workspaceCacheService.getOrRecompute(workspaceId, [
-        ...allFlatEntityMapsCacheKeysToCompute,
-        ...WORKSPACE_MIGRATION_ADDITIONAL_CACHE_DATA_MAPS_KEY,
-        'flatApplicationMaps',
-      ]);
-
-    const initialAccumulator = allMetadataNameCacheToCompute.reduce<
-      Partial<AllFlatEntityMaps>
-    >(
-      (allFlatEntityMaps, metadataName) => ({
-        ...allFlatEntityMaps,
-        [getMetadataFlatEntityMapsKey(metadataName)]:
-          createEmptyFlatEntityMaps(),
-      }),
-      {},
-    );
-
-    const applicationIds = this.computeAllInvolvedApplicationIds({
-      allFlatEntityOperationByMetadataName,
-      flatApplicationMaps,
-      applicationUniversalIdentifier,
-      allRelatedFlatEntityMaps,
-    });
-
-    const dependencyAllFlatEntityMaps = allMetadataNameCacheToCompute.reduce(
-      (allFlatEntityMaps, metadataName) => {
-        const metadataFlatEntityMapsKey =
-          getMetadataFlatEntityMapsKey(metadataName);
-
-        return {
-          ...allFlatEntityMaps,
-          [metadataFlatEntityMapsKey]:
-            getSubFlatEntityMapsByApplicationIdsOrThrow<
-              MetadataFlatEntity<typeof metadataName>
-            >({
-              applicationIds,
-              flatEntityMaps:
-                allRelatedFlatEntityMaps[metadataFlatEntityMapsKey],
-            }),
-        };
-      },
-      initialAccumulator,
-    );
-
-    const additionalCacheDataMaps =
-      WORKSPACE_MIGRATION_ADDITIONAL_CACHE_DATA_MAPS_KEY.reduce<WorkspaceMigrationBuilderAdditionalCacheDataMaps>(
-        (acc, additionalCacheDataMapsKey) => {
-          return {
-            ...acc,
-            [additionalCacheDataMapsKey]:
-              allRelatedFlatEntityMaps[additionalCacheDataMapsKey],
-          };
-        },
-        {} as WorkspaceMigrationBuilderAdditionalCacheDataMaps,
-      );
-
-    return {
-      allRelatedFlatEntityMaps,
-      dependencyAllFlatEntityMaps,
-      additionalCacheDataMaps,
-    };
-  }
-
-  private async computeFromToAllFlatEntityMapsAndBuildOptions({
-    allFlatEntityOperationByMetadataName,
-    workspaceId,
-    applicationUniversalIdentifier,
-  }: ValidateBuildAndRunWorkspaceMigrationFromMatriceArgs): Promise<{
-    fromToAllFlatEntityMaps: FromToAllUniversalFlatEntityMaps;
-    inferDeletionFromMissingEntities: InferDeletionFromMissingEntities;
-    dependencyAllFlatEntityMaps: Partial<AllFlatEntityMaps>;
-    additionalCacheDataMaps: WorkspaceMigrationBuilderAdditionalCacheDataMaps;
-    idByUniversalIdentifierByMetadataName: IdByUniversalIdentifierByMetadataName;
-  }> {
-    const {
-      allRelatedFlatEntityMaps,
-      dependencyAllFlatEntityMaps,
-      additionalCacheDataMaps,
-    } = await this.computeAllRelatedFlatEntityMaps({
-      allFlatEntityOperationByMetadataName,
-      workspaceId,
-      applicationUniversalIdentifier,
-    });
-
-    const fromToAllFlatEntityMaps: FromToAllUniversalFlatEntityMaps = {};
-    const idByUniversalIdentifierByMetadataName: IdByUniversalIdentifierByMetadataName =
-      {};
-    const inferDeletionFromMissingEntities: InferDeletionFromMissingEntities =
-      {};
-    const allMetadataNameToCompare = Object.keys(
-      allFlatEntityOperationByMetadataName,
-    ) as AllMetadataName[];
-
-    for (const metadataName of allMetadataNameToCompare) {
-      const flatEntityOperations =
-        allFlatEntityOperationByMetadataName[metadataName];
-
-      if (!isDefined(flatEntityOperations)) {
-        throw new FlatEntityMapsException(
-          `Could not load flat entity maps to compare for ${metadataName}, should never occur`,
-          FlatEntityMapsExceptionCode.INTERNAL_SERVER_ERROR,
-        );
-      }
-      const { flatEntityToCreate, flatEntityToDelete, flatEntityToUpdate } =
-        flatEntityOperations;
-
-      const idByUniversalIdentifier = Object.fromEntries(
-        flatEntityToCreate
-          .filter(
-            (
-              flatEntity,
-            ): flatEntity is MetadataUniversalFlatEntity<
-              typeof metadataName
-            > & { id: string } => isDefined(flatEntity.id),
-          )
-          .map((flatEntity) => [flatEntity.universalIdentifier, flatEntity.id]),
-      );
-
-      if (Object.keys(idByUniversalIdentifier).length > 0) {
-        idByUniversalIdentifierByMetadataName[metadataName] =
-          idByUniversalIdentifier;
-      }
-
-      const flatEntityMapsKey = getMetadataFlatEntityMapsKey(metadataName);
-      const flatEntityMaps = allRelatedFlatEntityMaps[flatEntityMapsKey];
-
-      // @ts-expect-error Metadata flat entity maps cache key and metadataName colliding
-      fromToAllFlatEntityMaps[flatEntityMapsKey] =
-        computeUniversalFlatEntityMapsFromToThroughMutation({
-          flatEntityMaps: structuredClone(flatEntityMaps),
-          flatEntityToCreate,
-          flatEntityToDelete,
-          flatEntityToUpdate,
-        });
-
-      if (flatEntityToDelete.length > 0) {
-        inferDeletionFromMissingEntities[metadataName] = true;
-      }
-    }
-
-    return {
-      fromToAllFlatEntityMaps,
-      inferDeletionFromMissingEntities,
-      dependencyAllFlatEntityMaps,
-      additionalCacheDataMaps,
-      idByUniversalIdentifierByMetadataName,
-    };
-  }
-
   public async validateBuildAndRunWorkspaceMigrationFromTo(
     args: WorkspaceMigrationOrchestratorBuildArgs & {
       idByUniversalIdentifierByMetadataName?: IdByUniversalIdentifierByMetadataName;
@@ -456,25 +162,87 @@ export class WorkspaceMigrationValidateBuildAndRunService {
   }
 
   public async validateBuildAndRunWorkspaceMigration({
-    allFlatEntityOperationByMetadataName: allFlatEntities,
+    allFlatEntityOperationByMetadataName,
     workspaceId,
     isSystemBuild = false,
     applicationUniversalIdentifier,
+    dryRun,
   }: ValidateBuildAndRunWorkspaceMigrationFromMatriceArgs): Promise<
     | WorkspaceMigrationOrchestratorFailedResult
-    | WorkspaceMigrationOrchestratorSuccessfulResult
+    | (WorkspaceMigrationOrchestratorSuccessfulResult & {
+        hasSchemaMetadataChanged: boolean;
+      })
   > {
+    return await this.validateBuildAndRunWorkspaceMigrationFromRecord({
+      allFlatEntityOperationRecordByMetadataName:
+        transpileFlatEntityOperationArrayToRecord(
+          allFlatEntityOperationByMetadataName,
+        ),
+      workspaceId,
+      isSystemBuild,
+      applicationUniversalIdentifier,
+      dryRun,
+    });
+  }
+
+  public async validateBuildAndRunWorkspaceMigrationFromRecord({
+    allFlatEntityOperationRecordByMetadataName,
+    workspaceId,
+    isSystemBuild = false,
+    applicationUniversalIdentifier,
+    dryRun,
+  }: ValidateBuildAndRunWorkspaceMigrationFromRecordArgs): Promise<
+    | WorkspaceMigrationOrchestratorFailedResult
+    | (WorkspaceMigrationOrchestratorSuccessfulResult & {
+        hasSchemaMetadataChanged: boolean;
+      })
+  > {
+    const callerMetadataNames = Object.keys(
+      allFlatEntityOperationRecordByMetadataName,
+    ) as AllMetadataName[];
+
+    const {
+      flatApplicationMaps,
+      allRelatedFlatEntityMaps,
+      allMetadataNameCacheToCompute,
+    } =
+      await this.workspaceMigrationFlatEntityMapsService.getOrRecomputeAllRelatedFlatEntityMaps(
+        {
+          workspaceId,
+          callerMetadataNames,
+        },
+      );
+
+    const sideEffectExpansionResult =
+      this.metadataSideEffectEngineService.expandWithSideEffects({
+        allFlatEntityOperationRecordByMetadataName,
+        sideEffectRelatedFlatEntityMaps: allRelatedFlatEntityMaps,
+        context: {
+          buildOptions: { isSystemBuild, applicationUniversalIdentifier },
+        },
+      });
+
+    if (sideEffectExpansionResult.status === 'fail') {
+      return sideEffectExpansionResult;
+    }
+
     const {
       fromToAllFlatEntityMaps,
       inferDeletionFromMissingEntities,
       dependencyAllFlatEntityMaps,
       additionalCacheDataMaps,
       idByUniversalIdentifierByMetadataName,
-    } = await this.computeFromToAllFlatEntityMapsAndBuildOptions({
-      allFlatEntityOperationByMetadataName: allFlatEntities,
-      workspaceId,
-      applicationUniversalIdentifier,
-    });
+    } =
+      this.workspaceMigrationFlatEntityMapsService.computeFromToAllFlatEntityMapsAndBuildOptions(
+        {
+          allFlatEntityOperationRecordByMetadataName:
+            sideEffectExpansionResult.allFlatEntityOperationRecordByMetadataName,
+          applicationUniversalIdentifier,
+          flatApplicationMaps,
+          allRelatedFlatEntityMaps,
+          allMetadataNameCacheToCompute,
+        },
+      );
 
     return await this.validateBuildAndRunWorkspaceMigrationFromTo({
       buildOptions: {
@@ -487,6 +255,7 @@ export class WorkspaceMigrationValidateBuildAndRunService {
       dependencyAllFlatEntityMaps,
       additionalCacheDataMaps,
       idByUniversalIdentifierByMetadataName,
+      dryRun,
     });
   }
 }

packages/twenty-server/src/engine/workspace-manager/workspace-migration/workspace-migration-runner/action-handlers/field/services/update-field-action-handler.service.ts

@@ -154,8 +154,8 @@ export class UpdateFieldActionHandlerService extends WorkspaceMigrationRunnerAct
     // isUnique is derived from IndexMetadata at cache build time and has
     // no underlying column on fieldMetadata. It travels in the update
     // payload only so per-type validators (e.g. FILES rejection) can run
-    // — the actual state change is handled by the side-effect index
-    // create/delete in handleIndexChangesDuringFieldUpdate.
+    // — the actual state change is handled by the metadata side-effect
+    // engine, which owns the backing unique index lifecycle.
     const { isUnique: _droppedIsUnique, ...persistedUpdate } = update;
 
     if (Object.keys(persistedUpdate).length === 0) {

packages/twenty-server/src/engine/workspace-manager/workspace-migration/workspace-migration.module.ts

@@ -1,7 +1,9 @@
 import { Module } from '@nestjs/common';
 
+import { MetadataSideEffectModule } from 'src/engine/metadata-modules/metadata-side-effect/metadata-side-effect.module';
 import { WorkspaceCacheModule } from 'src/engine/workspace-cache/workspace-cache.module';
 import { WorkspaceMigrationBuildOrchestratorService } from 'src/engine/workspace-manager/workspace-migration/services/workspace-migration-build-orchestrator.service';
+import { WorkspaceMigrationFlatEntityMapsService } from 'src/engine/workspace-manager/workspace-migration/services/workspace-migration-flat-entity-maps.service';
 import { WorkspaceMigrationValidateBuildAndRunService } from 'src/engine/workspace-manager/workspace-migration/services/workspace-migration-validate-build-and-run-service';
 import { WorkspaceMigrationBuilderModule } from 'src/engine/workspace-manager/workspace-migration/workspace-migration-builder/workspace-migration-builder.module';
 import { WorkspaceMigrationRunnerModule } from 'src/engine/workspace-manager/workspace-migration/workspace-migration-runner/workspace-migration-runner.module';
@@ -11,10 +13,12 @@ import { WorkspaceMigrationRunnerModule } from 'src/engine/workspace-manager/wor
     WorkspaceMigrationBuilderModule,
     WorkspaceMigrationRunnerModule,
     WorkspaceCacheModule,
+    MetadataSideEffectModule,
   ],
   providers: [
     WorkspaceMigrationValidateBuildAndRunService,
     WorkspaceMigrationBuildOrchestratorService,
+    WorkspaceMigrationFlatEntityMapsService,
   ],
   exports: [
     WorkspaceMigrationValidateBuildAndRunService,