Implement cross version upgrade (#19559)

# Introduction Refactoring the upgrade engine to handle cross version upgrade, completely getting rid of the semver `version` at db and runtime level It remains a visual a listing indicator for or CD process but also during devenv in order to prepare next release Will write a release process runbook documentation on how to handle upgrade step patch, command insertion etc as it needs to be cascaded across all the involved supported version **The upgrade sequence model:** The sequence is a flat, ordered array of upgrade steps (`UpgradeStep[]`), built from the registry by chaining all versions in order, each version contributing its fast-instance → slow-instance → workspace commands sorted by timestamp. Version is metadata for logging, not used in the algorithm. **Segments:** The sequence naturally splits into alternating segments of contiguous instance steps and contiguous workspace steps. The runner processes segments in order: - **Instance segment:** Run sequentially from the instance cursor. Each step runs once globally. - **Workspace segment:** Each workspace independently walks from its own cursor through the end of the segment. Workspaces are independent within a segment — they can be at different positions. - **Synchronization (workspace → instance):** The runner blocks before entering an instance segment. All active/suspended workspaces must have completed the last workspace step of the preceding workspace segment. If any workspace failed, abort. This is the only explicit synchronization point. - Instance → workspace ordering is implicit — the runner processes segments sequentially, so the instance segment naturally completes before the workspace segment begins. full docs https://gist.github.com/prastoin/e62106d455fd72d6b6ebada8351e5492 ## Version constants & type-level deprecation Version management is split into three atomic constants: `TWENTY_PREVIOUS_VERSIONS`, `TWENTY_CURRENT_VERSION`, and `TWENTY_NEXT_VERSIONS`. Two derived constants compose them: `CROSS_UPGRADE_SUPPORTED_VERSIONS` (previous + current — what the engine runs) and `ALL_TWENTY_VERSIONS` (the full ordered tuple including next). The registry service validates at module init that no version is duplicated across constants and that at least one previous version exists. A `DeprecatedSinceVersion<RemoveAtVersion, T>` type utility resolves to `T` while `TWENTY_CURRENT_VERSION` is below `RemoveAtVersion`, and to `never` once it reaches it — turning deprecation into a compile-time guarantee via `IndexOf` and `IsGreaterOrEqual` generics in `twenty-shared`. ### `workspace.version` column deprecation The column is replaced by cursor-based state inference from `UpgradeMigration` records, but cannot be dropped in 1.22: workspaces activated during 1.21 predate the cursor system and need their initial cursor backfilled first (`backfillWorkspaceCreatedIn1_21_0Cursors`). This backfill itself depends on a new `isInitial` column on `UpgradeMigration`, bootstrapped via a targeted TypeORM migration before the upgrade sequence runs. Both functions and the entity field are typed with `DeprecatedSinceVersion<'1.23.0', ...>`. When `TWENTY_CURRENT_VERSION` reaches `1.23.0`, compile errors force their removal — and the pre-declared `DropWorkspaceVersionColumnFastInstanceCommand` takes over to drop the column. ## What's next - ci cross version upgrade ( wip ) - banner asking to contact twenty administrator if workspace is outdated - upgrade healthcheck cli ## New unit/integ test pattern Create a dedicated `createNestApp` that consumes a real database in order not to have to mack any database interaction to the `upgradeMigrations` allowing full coverage of the whole `upgradeRunnerService.run` core logic
2026-04-13 11:42:27 +02:00
parent 7091561489
commit 21142d98fe
52 changed files with 2585 additions and 1359 deletions
@@ -10,7 +10,6 @@ export const WORKSPACE_FIELDS_TO_SEED = [
  'logo',
  'activationStatus',
  'isTwoFactorAuthenticationEnforced',
-  'version',
  'workspaceCustomApplicationId',
 ] as const satisfies (keyof WorkspaceEntity)[];

@@ -48,5 +47,5 @@ export const SEEDER_CREATE_WORKSPACE_INPUT = {
  },
 } as const satisfies Record<
  SeededWorkspacesIds,
-  Omit<CreateWorkspaceInput, 'version' | 'workspaceCustomApplicationId'>
+  Omit<CreateWorkspaceInput, 'workspaceCustomApplicationId'>
 >;
@@ -1,106 +0,0 @@
-import { type DataSource } from 'typeorm';
-import { v4 } from 'uuid';
-
-import { type ApplicationService } from 'src/engine/core-modules/application/application.service';
-import { seedBillingCustomers } from 'src/engine/workspace-manager/dev-seeder/core/billing/utils/seed-billing-customers.util';
-import { seedBillingSubscriptions } from 'src/engine/workspace-manager/dev-seeder/core/billing/utils/seed-billing-subscriptions.util';
-import {
-  type SeededWorkspacesIds,
-  SEEDER_CREATE_WORKSPACE_INPUT,
-} from 'src/engine/workspace-manager/dev-seeder/core/constants/seeder-workspaces.constant';
-import { seedAgents } from 'src/engine/workspace-manager/dev-seeder/core/utils/seed-agents.util';
-import { seedApiKeys } from 'src/engine/workspace-manager/dev-seeder/core/utils/seed-api-keys.util';
-
-import { seedFeatureFlags } from 'src/engine/workspace-manager/dev-seeder/core/utils/seed-feature-flags.util';
-import { seedMetadataEntities } from 'src/engine/workspace-manager/dev-seeder/core/utils/seed-metadata-entities.util';
-import { seedServerId } from 'src/engine/workspace-manager/dev-seeder/core/utils/seed-server-id.util';
-import { seedUserWorkspaces } from 'src/engine/workspace-manager/dev-seeder/core/utils/seed-user-workspaces.util';
-import { seedUsers } from 'src/engine/workspace-manager/dev-seeder/core/utils/seed-users.util';
-import { createWorkspace } from 'src/engine/workspace-manager/dev-seeder/core/utils/seed-workspace.util';
-import { extractVersionMajorMinorPatch } from 'src/utils/version/extract-version-major-minor-patch';
-
-type SeedCoreSchemaArgs = {
-  dataSource: DataSource;
-  workspaceId: SeededWorkspacesIds;
-  appVersion: string | undefined;
-  applicationService: ApplicationService;
-  seedBilling?: boolean;
-  seedFeatureFlags?: boolean;
-};
-
-export const seedCoreSchema = async ({
-  appVersion,
-  dataSource,
-  workspaceId,
-  applicationService,
-  seedBilling = true,
-  seedFeatureFlags: shouldSeedFeatureFlags = true,
-}: SeedCoreSchemaArgs) => {
-  const schemaName = 'core';
-
-  const createWorkspaceStaticInput = SEEDER_CREATE_WORKSPACE_INPUT[workspaceId];
-  const version = extractVersionMajorMinorPatch(appVersion);
-  const queryRunner = dataSource.createQueryRunner();
-
-  await queryRunner.connect();
-  await queryRunner.startTransaction();
-
-  try {
-    const workspaceCustomApplicationId = v4();
-
-    await createWorkspace({
-      queryRunner,
-      schemaName,
-      createWorkspaceInput: {
-        ...createWorkspaceStaticInput,
-        version,
-        workspaceCustomApplicationId,
-      },
-    });
-
-    await applicationService.createWorkspaceCustomApplication(
-      {
-        workspaceId,
-        applicationId: workspaceCustomApplicationId,
-        workspaceDisplayName: createWorkspaceStaticInput.displayName,
-      },
-      queryRunner,
-    );
-
-    await seedServerId({ queryRunner, schemaName });
-
-    await seedUsers({ queryRunner, schemaName });
-
-    await seedUserWorkspaces({ queryRunner, schemaName, workspaceId });
-
-    await applicationService.createTwentyStandardApplication(
-      {
-        workspaceId,
-        skipCacheInvalidation: true,
-      },
-      queryRunner,
-    );
-
-    await seedAgents({ queryRunner, schemaName, workspaceId });
-
-    await seedApiKeys({ queryRunner, schemaName, workspaceId });
-
-    if (shouldSeedFeatureFlags) {
-      await seedFeatureFlags({ queryRunner, schemaName, workspaceId });
-    }
-
-    if (seedBilling) {
-      await seedBillingCustomers({ queryRunner, schemaName, workspaceId });
-      await seedBillingSubscriptions({ queryRunner, schemaName, workspaceId });
-    }
-
-    await seedMetadataEntities({ queryRunner, schemaName, workspaceId });
-
-    await queryRunner.commitTransaction();
-  } catch (error) {
-    await queryRunner.rollbackTransaction();
-    throw error;
-  } finally {
-    await queryRunner.release();
-  }
-};
@@ -17,6 +17,7 @@ import { ObjectPermissionModule } from 'src/engine/metadata-modules/object-permi
 import { RoleTargetModule } from 'src/engine/metadata-modules/role-target/role-target.module';
 import { RoleModule } from 'src/engine/metadata-modules/role/role.module';
 import { UserRoleModule } from 'src/engine/metadata-modules/user-role/user-role.module';
+import { UpgradeModule } from 'src/engine/core-modules/upgrade/upgrade.module';
 import { WorkspaceCacheStorageModule } from 'src/engine/workspace-cache-storage/workspace-cache-storage.module';
 import { WorkspaceCacheModule } from 'src/engine/workspace-cache/workspace-cache.module';
 import { WorkspaceDataSourceModule } from 'src/engine/workspace-datasource/workspace-datasource.module';
@@ -52,6 +53,7 @@ import { WorkspaceMigrationModule } from 'src/engine/workspace-manager/workspace
    WorkspaceMigrationModule,
    TwentyStandardApplicationModule,
    SdkClientModule,
+    UpgradeModule,
  ],
  exports: [DevSeederService],
  providers: [
@@ -3,11 +3,14 @@ import { InjectDataSource, InjectRepository } from '@nestjs/typeorm';

 import { ALL_METADATA_NAME } from 'twenty-shared/metadata';
 import { DataSource, Repository } from 'typeorm';
+import { v4 } from 'uuid';

 import { ApplicationRegistrationService } from 'src/engine/core-modules/application/application-registration/application-registration.service';
 import { ApplicationService } from 'src/engine/core-modules/application/application.service';
 import { SdkClientGenerationService } from 'src/engine/core-modules/sdk-client/sdk-client-generation.service';
 import { TwentyConfigService } from 'src/engine/core-modules/twenty-config/twenty-config.service';
+import { UpgradeMigrationService } from 'src/engine/core-modules/upgrade/services/upgrade-migration.service';
+import { UpgradeSequenceReaderService } from 'src/engine/core-modules/upgrade/services/upgrade-sequence-reader.service';
 import { WorkspaceEntity } from 'src/engine/core-modules/workspace/workspace.entity';
 import { getMetadataFlatEntityMapsKey } from 'src/engine/metadata-modules/flat-entity/utils/get-metadata-flat-entity-maps-key.util';
 import { getMetadataRelatedMetadataNames } from 'src/engine/metadata-modules/flat-entity/utils/get-metadata-related-metadata-names.util';
@@ -15,9 +18,21 @@ import { ObjectMetadataEntity } from 'src/engine/metadata-modules/object-metadat
 import { WorkspaceCacheStorageService } from 'src/engine/workspace-cache-storage/workspace-cache-storage.service';
 import { WorkspaceCacheService } from 'src/engine/workspace-cache/services/workspace-cache.service';
 import { WorkspaceDataSourceService } from 'src/engine/workspace-datasource/workspace-datasource.service';
-import { SeededWorkspacesIds } from 'src/engine/workspace-manager/dev-seeder/core/constants/seeder-workspaces.constant';
+import {
+  type SeededWorkspacesIds,
+  SEEDER_CREATE_WORKSPACE_INPUT,
+} from 'src/engine/workspace-manager/dev-seeder/core/constants/seeder-workspaces.constant';
+import { seedBillingCustomers } from 'src/engine/workspace-manager/dev-seeder/core/billing/utils/seed-billing-customers.util';
+import { seedBillingSubscriptions } from 'src/engine/workspace-manager/dev-seeder/core/billing/utils/seed-billing-subscriptions.util';
 import { DevSeederPermissionsService } from 'src/engine/workspace-manager/dev-seeder/core/services/dev-seeder-permissions.service';
-import { seedCoreSchema } from 'src/engine/workspace-manager/dev-seeder/core/utils/seed-core-schema.util';
+import { seedAgents } from 'src/engine/workspace-manager/dev-seeder/core/utils/seed-agents.util';
+import { seedApiKeys } from 'src/engine/workspace-manager/dev-seeder/core/utils/seed-api-keys.util';
+import { seedFeatureFlags } from 'src/engine/workspace-manager/dev-seeder/core/utils/seed-feature-flags.util';
+import { seedMetadataEntities } from 'src/engine/workspace-manager/dev-seeder/core/utils/seed-metadata-entities.util';
+import { seedServerId } from 'src/engine/workspace-manager/dev-seeder/core/utils/seed-server-id.util';
+import { seedUserWorkspaces } from 'src/engine/workspace-manager/dev-seeder/core/utils/seed-user-workspaces.util';
+import { seedUsers } from 'src/engine/workspace-manager/dev-seeder/core/utils/seed-users.util';
+import { createWorkspace } from 'src/engine/workspace-manager/dev-seeder/core/utils/seed-workspace.util';
 import { seedPageLayoutTabs } from 'src/engine/workspace-manager/dev-seeder/core/utils/seed-page-layout-tabs.util';
 import { seedPageLayoutWidgets } from 'src/engine/workspace-manager/dev-seeder/core/utils/seed-page-layout-widgets.util';
 import { seedPageLayouts } from 'src/engine/workspace-manager/dev-seeder/core/utils/seed-page-layouts.util';
@@ -41,6 +56,8 @@ export class DevSeederService {
    private readonly applicationRegistrationService: ApplicationRegistrationService,
    private readonly workspaceCacheService: WorkspaceCacheService,
    private readonly sdkClientGenerationService: SdkClientGenerationService,
+    private readonly upgradeMigrationService: UpgradeMigrationService,
+    private readonly upgradeSequenceReaderService: UpgradeSequenceReaderService,
    @InjectDataSource()
    private readonly coreDataSource: DataSource,
    @InjectRepository(WorkspaceEntity)
@@ -53,14 +70,16 @@ export class DevSeederService {
  ): Promise<void> {
    const light = options?.light ?? false;
    const isBillingEnabled = this.twentyConfigService.get('IS_BILLING_ENABLED');
-    const appVersion = this.twentyConfigService.get('APP_VERSION');
+    const appVersion = this.twentyConfigService.get('APP_VERSION') ?? 'unknown';

-    await seedCoreSchema({
-      dataSource: this.coreDataSource,
+    const lastWorkspaceCommand =
+      this.upgradeSequenceReaderService.getLastWorkspaceCommand();
+
+    await this.seedCoreSchema({
      workspaceId,
-      applicationService: this.applicationService,
      seedBilling: isBillingEnabled,
      appVersion,
+      lastUpgradeStepName: lastWorkspaceCommand.name,
    });

    await this.applicationRegistrationService.createCliRegistrationIfNotExists();
@@ -171,4 +190,87 @@ export class DevSeederService {

    await this.workspaceCacheStorageService.flush(workspaceId, undefined);
  }
+
+  private async seedCoreSchema({
+    workspaceId,
+    appVersion,
+    lastUpgradeStepName,
+    seedBilling = true,
+  }: {
+    workspaceId: SeededWorkspacesIds;
+    appVersion: string;
+    lastUpgradeStepName: string;
+    seedBilling?: boolean;
+  }): Promise<void> {
+    const schemaName = 'core';
+    const createWorkspaceStaticInput =
+      SEEDER_CREATE_WORKSPACE_INPUT[workspaceId];
+    const queryRunner = this.coreDataSource.createQueryRunner();
+
+    await queryRunner.connect();
+    await queryRunner.startTransaction();
+
+    try {
+      const workspaceCustomApplicationId = v4();
+
+      await createWorkspace({
+        queryRunner,
+        schemaName,
+        createWorkspaceInput: {
+          ...createWorkspaceStaticInput,
+          workspaceCustomApplicationId,
+        },
+      });
+
+      await this.applicationService.createWorkspaceCustomApplication(
+        {
+          workspaceId,
+          applicationId: workspaceCustomApplicationId,
+          workspaceDisplayName: createWorkspaceStaticInput.displayName,
+        },
+        queryRunner,
+      );
+
+      await seedServerId({ queryRunner, schemaName });
+      await seedUsers({ queryRunner, schemaName });
+      await seedUserWorkspaces({ queryRunner, schemaName, workspaceId });
+
+      await this.applicationService.createTwentyStandardApplication(
+        {
+          workspaceId,
+          skipCacheInvalidation: true,
+        },
+        queryRunner,
+      );
+
+      await seedAgents({ queryRunner, schemaName, workspaceId });
+      await seedApiKeys({ queryRunner, schemaName, workspaceId });
+      await seedFeatureFlags({ queryRunner, schemaName, workspaceId });
+
+      if (seedBilling) {
+        await seedBillingCustomers({ queryRunner, schemaName, workspaceId });
+        await seedBillingSubscriptions({
+          queryRunner,
+          schemaName,
+          workspaceId,
+        });
+      }
+
+      await seedMetadataEntities({ queryRunner, schemaName, workspaceId });
+
+      await this.upgradeMigrationService.markAsInitial({
+        name: lastUpgradeStepName,
+        workspaceId,
+        executedByVersion: appVersion,
+        queryRunner,
+      });
+
+      await queryRunner.commitTransaction();
+    } catch (error) {
+      await queryRunner.rollbackTransaction();
+      throw error;
+    } finally {
+      await queryRunner.release();
+    }
+  }
 }