fix(server): backfill isSystemSideEffect on system fields provisioned before 2.15 (#22850)

## Context The `isSystemSideEffect` column was introduced in **2.15** via a fast instance command that added it with `DEFAULT false`. That stamped `false` onto every pre-existing `fieldMetadata` row — including the 8 engine-owned system fields (`id`, `createdAt`, `updatedAt`, `deletedAt`, `createdBy`, `updatedBy`, `position`, `searchVector`) of every object provisioned before 2.15, regardless of the creation path (API metadata **and** manifest sync). The per-workspace backfill that should have re-flagged those existing rows was explicitly deferred as out of scope in #21673 ("PR 2") and never shipped for `fieldMetadata`. Because `isSystemSideEffect` is configured with `toCompare: false`, no later sync ever repaired the stale value either. Since **2.20** the SDK no longer declares system fields in manifests. On an up-to-date instance, `twenty plan` against an unchanged app therefore diffs those stale-`false` system fields as **missing from the manifest**, and they fall through the `isSystemSideEffectFlatEntity` exclusion in `buildAllFlatEntityOperationRecordByMetadataNameFromFromTo`. Deletion inference then emits them as deletes, which the validator rejects: ``` Sync failed with 144 errors fieldMetadata: 144 errors 1..144. FIELD_MUTATION_NOT_ALLOWED: System fields cannot be deleted ``` (144 = 8 system fields × 18 custom objects, as reported on a production 2.20 instance.) ## What this PR does Adds a **2.21 workspace command** (`upgrade:2-21:backfill-system-field-is-system-side-effect`) that iterates active/suspended workspaces and flags the 8 system fields as `isSystemSideEffect: true`. - **Resolution by deterministic universal identifier**: for each object × reserved system field name it recomputes `getFieldUniversalIdentifier(applicationUID, objectUID, name)` and looks the row up in the flat maps. This is safe (and preferable to matching by `name`) because the 2.19 backfill already took over system field UIDs for every application, so an author-declared field reusing a reserved name keeps its own identifier and is never touched. An extra `isSystem` guard warn-and-skips any mismatch. - **All applications** are covered (installed apps, workspace custom app, twenty-standard): the stale flag is a function of *when* a row was provisioned, not *how*. Installed/custom apps are the acute `twenty plan` delete trap; twenty-standard has no trap today but flagging is a zero-diff no-op (`toCompare: false`) and a prerequisite for the end-state ownership invariant. - **`name` is intentionally excluded**: the 2.20 slow instance command deliberately flipped it to `false` (caller-provided default, not engine-owned); re-flagging it would undo that migration. - Supports `--dry-run`, updates only the collected rows, and invalidates the `flatFieldMetadataMaps` workspace cache after the write (a raw repository update does not invalidate it). ## Related - Resolves the pre-2.15 regression tail of twentyhq/core-team-issues#2635 - Follow-up to twentyhq/core-team-issues#2642 (system field side-effect engine migration)  <a href="https://cubic.dev/pr/twentyhq/twenty/pull/22850?utm_source=github" target="_blank" rel="noopener noreferrer" data-no-image-dialog="true"><picture><source media="(prefers-color-scheme: dark)" srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source media="(prefers-color-scheme: light)" srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img alt="Review in cubic" src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a>  --------- Co-authored-by: Weiko <corentin@twenty.com>
2026-07-13 10:55:54 +02:00
parent b94e38c2d9
commit 652adc3c03
3 changed files with 175 additions and 0 deletions
@@ -0,0 +1,20 @@
+import { Module } from '@nestjs/common';
+import { TypeOrmModule } from '@nestjs/typeorm';
+
+import { WorkspaceIteratorModule } from 'src/database/commands/command-runners/workspace-iterator.module';
+import { BackfillSystemFieldIsSystemSideEffectCommand } from 'src/database/commands/upgrade-version-command/2-21/2-21-workspace-command-1783925862946-backfill-system-field-is-system-side-effect.command';
+import { ApplicationEntity } from 'src/engine/core-modules/application/application.entity';
+import { FieldMetadataEntity } from 'src/engine/metadata-modules/field-metadata/field-metadata.entity';
+import { WorkspaceCacheModule } from 'src/engine/workspace-cache/workspace-cache.module';
+import { WorkspaceMigrationRunnerModule } from 'src/engine/workspace-manager/workspace-migration/workspace-migration-runner/workspace-migration-runner.module';
+
+@Module({
+  imports: [
+    TypeOrmModule.forFeature([ApplicationEntity, FieldMetadataEntity]),
+    WorkspaceCacheModule,
+    WorkspaceIteratorModule,
+    WorkspaceMigrationRunnerModule,
+  ],
+  providers: [BackfillSystemFieldIsSystemSideEffectCommand],
+})
+export class V2_21_UpgradeVersionCommandModule {}
@@ -0,0 +1,153 @@
+import { InjectRepository } from '@nestjs/typeorm';
+
+import { Command } from 'nest-commander';
+import { getFieldUniversalIdentifier } from 'twenty-shared/application';
+import { isDefined } from 'twenty-shared/utils';
+import { In, Repository } from 'typeorm';
+
+import { ActiveOrSuspendedWorkspaceCommandRunner } from 'src/database/commands/command-runners/active-or-suspended-workspace.command-runner';
+import { WorkspaceIteratorService } from 'src/database/commands/command-runners/workspace-iterator.service';
+import { type RunOnWorkspaceArgs } from 'src/database/commands/command-runners/workspace.command-runner';
+import { ApplicationEntity } from 'src/engine/core-modules/application/application.entity';
+import { RegisteredWorkspaceCommand } from 'src/engine/core-modules/upgrade/decorators/registered-workspace-command.decorator';
+import { FieldMetadataEntity } from 'src/engine/metadata-modules/field-metadata/field-metadata.entity';
+import { findFlatEntityByIdInFlatEntityMaps } from 'src/engine/metadata-modules/flat-entity/utils/find-flat-entity-by-id-in-flat-entity-maps.util';
+import { getMetadataFlatEntityMapsKey } from 'src/engine/metadata-modules/flat-entity/utils/get-metadata-flat-entity-maps-key.util';
+import { PARTIAL_SYSTEM_FLAT_FIELD_METADATAS } from 'src/engine/metadata-modules/object-metadata/constants/partial-system-flat-field-metadatas.constant';
+import { WorkspaceCacheService } from 'src/engine/workspace-cache/services/workspace-cache.service';
+import { WorkspaceMigrationRunnerService } from 'src/engine/workspace-manager/workspace-migration/workspace-migration-runner/services/workspace-migration-runner.service';
+
+const SYSTEM_FIELD_NAMES = new Set<string>([
+  PARTIAL_SYSTEM_FLAT_FIELD_METADATAS.id.name,
+  PARTIAL_SYSTEM_FLAT_FIELD_METADATAS.createdAt.name,
+  PARTIAL_SYSTEM_FLAT_FIELD_METADATAS.updatedAt.name,
+  PARTIAL_SYSTEM_FLAT_FIELD_METADATAS.deletedAt.name,
+  PARTIAL_SYSTEM_FLAT_FIELD_METADATAS.createdBy.name,
+  PARTIAL_SYSTEM_FLAT_FIELD_METADATAS.updatedBy.name,
+  PARTIAL_SYSTEM_FLAT_FIELD_METADATAS.position.name,
+  PARTIAL_SYSTEM_FLAT_FIELD_METADATAS.searchVector.name,
+]);
+
+@RegisteredWorkspaceCommand('2.21.0', 1783925862946)
+@Command({
+  name: 'upgrade:2-21:backfill-system-field-is-system-side-effect',
+  description:
+    'Flag existing system fields (id, createdAt, updatedAt, deletedAt, createdBy, updatedBy, position, searchVector) as isSystemSideEffect: true so manifest sync deletion inference excludes them.',
+})
+export class BackfillSystemFieldIsSystemSideEffectCommand extends ActiveOrSuspendedWorkspaceCommandRunner {
+  constructor(
+    protected readonly workspaceIteratorService: WorkspaceIteratorService,
+    private readonly workspaceCacheService: WorkspaceCacheService,
+    private readonly workspaceMigrationRunnerService: WorkspaceMigrationRunnerService,
+    @InjectRepository(ApplicationEntity)
+    private readonly applicationRepository: Repository<ApplicationEntity>,
+    @InjectRepository(FieldMetadataEntity)
+    private readonly fieldMetadataRepository: Repository<FieldMetadataEntity>,
+  ) {
+    super(workspaceIteratorService);
+  }
+
+  override async runOnWorkspace({
+    workspaceId,
+    options,
+  }: RunOnWorkspaceArgs): Promise<void> {
+    const isDryRun = options.dryRun ?? false;
+
+    const applications = await this.applicationRepository.find({
+      select: ['id', 'universalIdentifier'],
+      where: { workspaceId },
+      withDeleted: true,
+    });
+    const applicationUniversalIdentifierById = new Map(
+      applications.map((application) => [
+        application.id,
+        application.universalIdentifier,
+      ]),
+    );
+
+    const { flatFieldMetadataMaps, flatObjectMetadataMaps } =
+      await this.workspaceCacheService.getOrRecompute(workspaceId, [
+        'flatFieldMetadataMaps',
+        'flatObjectMetadataMaps',
+      ]);
+
+    const fieldMetadataIdsToFlag: string[] = [];
+
+    for (const flatFieldMetadata of Object.values(
+      flatFieldMetadataMaps.byUniversalIdentifier,
+    )) {
+      if (
+        !isDefined(flatFieldMetadata) ||
+        !SYSTEM_FIELD_NAMES.has(flatFieldMetadata.name) ||
+        flatFieldMetadata.isSystemSideEffect
+      ) {
+        continue;
+      }
+
+      const flatObjectMetadata = findFlatEntityByIdInFlatEntityMaps({
+        flatEntityMaps: flatObjectMetadataMaps,
+        flatEntityId: flatFieldMetadata.objectMetadataId,
+      });
+      const applicationUniversalIdentifier =
+        applicationUniversalIdentifierById.get(flatFieldMetadata.applicationId);
+
+      if (
+        !isDefined(flatObjectMetadata) ||
+        !isDefined(applicationUniversalIdentifier)
+      ) {
+        this.logger.warn(
+          `Missing object or application for field ${flatFieldMetadata.name} (${flatFieldMetadata.id}) in workspace ${workspaceId}, skipping`,
+        );
+        continue;
+      }
+
+      const systemFieldUniversalIdentifier = getFieldUniversalIdentifier({
+        applicationUniversalIdentifier,
+        objectUniversalIdentifier: flatObjectMetadata.universalIdentifier,
+        name: flatFieldMetadata.name,
+      });
+
+      if (
+        systemFieldUniversalIdentifier !== flatFieldMetadata.universalIdentifier
+      ) {
+        this.logger.warn(
+          `System field ${flatFieldMetadata.name} (${flatFieldMetadata.id}) does not carry its deterministic universal identifier in workspace ${workspaceId}, skipping`,
+        );
+
+        continue;
+      }
+
+      fieldMetadataIdsToFlag.push(flatFieldMetadata.id);
+    }
+
+    if (fieldMetadataIdsToFlag.length === 0) {
+      this.logger.log(
+        `No system field isSystemSideEffect flag to backfill for workspace ${workspaceId}`,
+      );
+
+      return;
+    }
+
+    this.logger.log(
+      `${isDryRun ? '[DRY RUN] ' : ''}Flagging ${fieldMetadataIdsToFlag.length} system field(s) as isSystemSideEffect for workspace ${workspaceId}`,
+    );
+
+    if (isDryRun) {
+      return;
+    }
+
+    await this.fieldMetadataRepository.update(
+      { id: In(fieldMetadataIdsToFlag), workspaceId },
+      { isSystemSideEffect: true },
+    );
+
+    await this.workspaceMigrationRunnerService.invalidateCache({
+      allFlatEntityMapsKeys: [getMetadataFlatEntityMapsKey('fieldMetadata')],
+      workspaceId,
+    });
+
+    this.logger.log(
+      `Flagged ${fieldMetadataIdsToFlag.length} system field(s) as isSystemSideEffect for workspace ${workspaceId}`,
+    );
+  }
+}
@@ -21,6 +21,7 @@ import { V2_17_UpgradeVersionCommandModule } from 'src/database/commands/upgrade
 import { V2_18_UpgradeVersionCommandModule } from 'src/database/commands/upgrade-version-command/2-18/2-18-upgrade-version-command.module';
 import { V2_19_UpgradeVersionCommandModule } from 'src/database/commands/upgrade-version-command/2-19/2-19-upgrade-version-command.module';
 import { V2_20_UpgradeVersionCommandModule } from 'src/database/commands/upgrade-version-command/2-20/2-20-upgrade-version-command.module';
+import { V2_21_UpgradeVersionCommandModule } from 'src/database/commands/upgrade-version-command/2-21/2-21-upgrade-version-command.module';

@Module({
  imports: [
@@ -45,6 +46,7 @@ import { V2_20_UpgradeVersionCommandModule } from 'src/database/commands/upgrade
    V2_18_UpgradeVersionCommandModule,
    V2_19_UpgradeVersionCommandModule,
    V2_20_UpgradeVersionCommandModule,
+    V2_21_UpgradeVersionCommandModule,
  ],
 })
 export class WorkspaceCommandProviderModule {}