Implement cross version upgrade (#19559)
# Introduction Refactoring the upgrade engine to handle cross version upgrade, completely getting rid of the semver `version` at db and runtime level It remains a visual a listing indicator for or CD process but also during devenv in order to prepare next release Will write a release process runbook documentation on how to handle upgrade step patch, command insertion etc as it needs to be cascaded across all the involved supported version **The upgrade sequence model:** The sequence is a flat, ordered array of upgrade steps (`UpgradeStep[]`), built from the registry by chaining all versions in order, each version contributing its fast-instance → slow-instance → workspace commands sorted by timestamp. Version is metadata for logging, not used in the algorithm. **Segments:** The sequence naturally splits into alternating segments of contiguous instance steps and contiguous workspace steps. The runner processes segments in order: - **Instance segment:** Run sequentially from the instance cursor. Each step runs once globally. - **Workspace segment:** Each workspace independently walks from its own cursor through the end of the segment. Workspaces are independent within a segment — they can be at different positions. - **Synchronization (workspace → instance):** The runner blocks before entering an instance segment. All active/suspended workspaces must have completed the last workspace step of the preceding workspace segment. If any workspace failed, abort. This is the only explicit synchronization point. - Instance → workspace ordering is implicit — the runner processes segments sequentially, so the instance segment naturally completes before the workspace segment begins. full docs https://gist.github.com/prastoin/e62106d455fd72d6b6ebada8351e5492 ## Version constants & type-level deprecation Version management is split into three atomic constants: `TWENTY_PREVIOUS_VERSIONS`, `TWENTY_CURRENT_VERSION`, and `TWENTY_NEXT_VERSIONS`. Two derived constants compose them: `CROSS_UPGRADE_SUPPORTED_VERSIONS` (previous + current — what the engine runs) and `ALL_TWENTY_VERSIONS` (the full ordered tuple including next). The registry service validates at module init that no version is duplicated across constants and that at least one previous version exists. A `DeprecatedSinceVersion<RemoveAtVersion, T>` type utility resolves to `T` while `TWENTY_CURRENT_VERSION` is below `RemoveAtVersion`, and to `never` once it reaches it — turning deprecation into a compile-time guarantee via `IndexOf` and `IsGreaterOrEqual` generics in `twenty-shared`. ### `workspace.version` column deprecation The column is replaced by cursor-based state inference from `UpgradeMigration` records, but cannot be dropped in 1.22: workspaces activated during 1.21 predate the cursor system and need their initial cursor backfilled first (`backfillWorkspaceCreatedIn1_21_0Cursors`). This backfill itself depends on a new `isInitial` column on `UpgradeMigration`, bootstrapped via a targeted TypeORM migration before the upgrade sequence runs. Both functions and the entity field are typed with `DeprecatedSinceVersion<'1.23.0', ...>`. When `TWENTY_CURRENT_VERSION` reaches `1.23.0`, compile errors force their removal — and the pre-declared `DropWorkspaceVersionColumnFastInstanceCommand` takes over to drop the column. ## What's next - ci cross version upgrade ( wip ) - banner asking to contact twenty administrator if workspace is outdated - upgrade healthcheck cli ## New unit/integ test pattern Create a dedicated `createNestApp` that consumes a real database in order not to have to mack any database interaction to the `upgradeMigrations` allowing full coverage of the whole `upgradeRunnerService.run` core logic
This commit is contained in:
Paul Rastoin
@@ -5,9 +5,10 @@ import chalk from 'chalk';
|
||||
import { Command, CommandRunner, Option } from 'nest-commander';
|
||||
import { DataSource } from 'typeorm';
|
||||
|
||||
import { CoreEngineVersionService } from 'src/engine/core-engine-version/services/core-engine-version.service';
|
||||
import { InstanceUpgradeService } from 'src/engine/core-modules/upgrade/services/instance-upgrade.service';
|
||||
import { TWENTY_PREVIOUS_VERSIONS } from 'src/engine/core-modules/upgrade/constants/twenty-previous-versions.constant';
|
||||
import { InstanceCommandRunnerService } from 'src/engine/core-modules/upgrade/services/instance-command-runner.service';
|
||||
import { UpgradeCommandRegistryService } from 'src/engine/core-modules/upgrade/services/upgrade-command-registry.service';
|
||||
import { UpgradeMigrationService } from 'src/engine/core-modules/upgrade/services/upgrade-migration.service';
|
||||
import { WorkspaceVersionService } from 'src/engine/workspace-manager/workspace-version/services/workspace-version.service';
|
||||
|
||||
type RunInstanceCommandsOptions = {
|
||||
@@ -15,6 +16,7 @@ type RunInstanceCommandsOptions = {
|
||||
includeSlow?: boolean;
|
||||
};
|
||||
|
||||
// TODO should be replaced by a specific call to the upgrade
|
||||
@Command({
|
||||
name: 'run-instance-commands',
|
||||
description:
|
||||
@@ -26,10 +28,10 @@ export class RunInstanceCommandsCommand extends CommandRunner {
|
||||
constructor(
|
||||
@InjectDataSource()
|
||||
private readonly dataSource: DataSource,
|
||||
private readonly coreEngineVersionService: CoreEngineVersionService,
|
||||
private readonly workspaceVersionService: WorkspaceVersionService,
|
||||
private readonly upgradeCommandRegistryService: UpgradeCommandRegistryService,
|
||||
private readonly instanceUpgradeService: InstanceUpgradeService,
|
||||
private readonly instanceUpgradeService: InstanceCommandRunnerService,
|
||||
private readonly upgradeMigrationService: UpgradeMigrationService,
|
||||
) {
|
||||
super();
|
||||
}
|
||||
@@ -63,7 +65,7 @@ export class RunInstanceCommandsCommand extends CommandRunner {
|
||||
for (const {
|
||||
command,
|
||||
name,
|
||||
} of this.upgradeCommandRegistryService.getAllFastInstanceCommands()) {
|
||||
} of this.upgradeCommandRegistryService.getCrossUpgradeSupportedFastInstanceCommands()) {
|
||||
const result = await this.instanceUpgradeService.runFastInstanceCommand(
|
||||
{
|
||||
command,
|
||||
@@ -83,7 +85,7 @@ export class RunInstanceCommandsCommand extends CommandRunner {
|
||||
for (const {
|
||||
command,
|
||||
name,
|
||||
} of this.upgradeCommandRegistryService.getAllSlowInstanceCommands()) {
|
||||
} of this.upgradeCommandRegistryService.getCrossUpgradeSupportedSlowInstanceCommands()) {
|
||||
const result =
|
||||
await this.instanceUpgradeService.runSlowInstanceCommand({
|
||||
command,
|
||||
@@ -106,6 +108,52 @@ export class RunInstanceCommandsCommand extends CommandRunner {
|
||||
}
|
||||
}
|
||||
|
||||
private async checkWorkspaceVersionSafety(
|
||||
options: RunInstanceCommandsOptions,
|
||||
): Promise<void> {
|
||||
if (options.force) {
|
||||
this.logger.warn(
|
||||
chalk.yellow('Skipping workspace version check (--force flag used)'),
|
||||
);
|
||||
|
||||
return;
|
||||
}
|
||||
|
||||
const activeWorkspaceIds =
|
||||
await this.workspaceVersionService.getActiveOrSuspendedWorkspaceIds();
|
||||
|
||||
if (activeWorkspaceIds.length === 0) {
|
||||
return;
|
||||
}
|
||||
|
||||
const previousVersion =
|
||||
TWENTY_PREVIOUS_VERSIONS[TWENTY_PREVIOUS_VERSIONS.length - 1];
|
||||
|
||||
const lastWorkspaceCommand =
|
||||
this.upgradeCommandRegistryService.getLastWorkspaceCommandForVersion(
|
||||
previousVersion,
|
||||
);
|
||||
|
||||
if (!lastWorkspaceCommand) {
|
||||
return;
|
||||
}
|
||||
|
||||
const allAtPreviousVersion =
|
||||
await this.upgradeMigrationService.areAllWorkspacesAtCommand({
|
||||
commandName: lastWorkspaceCommand.name,
|
||||
workspaceIds: activeWorkspaceIds,
|
||||
});
|
||||
|
||||
if (!allAtPreviousVersion) {
|
||||
throw new Error(
|
||||
'Unable to run instance commands. Some workspace(s) have not completed ' +
|
||||
`the last workspace command for ${previousVersion} ("${lastWorkspaceCommand.name}").\n` +
|
||||
'Please ensure all workspaces are upgraded to at least the previous version before running migrations.\n' +
|
||||
'Use --force to bypass this check (not recommended).',
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
private async runLegacyPendingTypeOrmMigrations(): Promise<void> {
|
||||
this.logger.log('Running legacy TypeORM migrations...');
|
||||
|
||||
@@ -121,39 +169,4 @@ export class RunInstanceCommandsCommand extends CommandRunner {
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
private async checkWorkspaceVersionSafety(
|
||||
options: RunInstanceCommandsOptions,
|
||||
): Promise<void> {
|
||||
if (options.force) {
|
||||
this.logger.warn(
|
||||
chalk.yellow('Skipping workspace version check (--force flag used)'),
|
||||
);
|
||||
|
||||
return;
|
||||
}
|
||||
|
||||
const previousVersion = this.coreEngineVersionService.getPreviousVersion();
|
||||
|
||||
const workspacesBelow =
|
||||
await this.workspaceVersionService.getWorkspacesBelowVersion(
|
||||
previousVersion.version,
|
||||
);
|
||||
|
||||
if (workspacesBelow.length > 0) {
|
||||
for (const workspace of workspacesBelow) {
|
||||
this.logger.error(
|
||||
chalk.red(
|
||||
`Workspace ${workspace.id} (${workspace.displayName}) is at version ${workspace.version ?? 'undefined'}, which is below the minimum required version.`,
|
||||
),
|
||||
);
|
||||
}
|
||||
|
||||
throw new Error(
|
||||
'Unable to run instance commands. Some workspace(s) are below the minimum required version.\n' +
|
||||
'Please ensure all workspaces are on at least the previous minor version before running migrations.\n' +
|
||||
'Use --force to bypass this check (not recommended).',
|
||||
);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
Reference in New Issue
Block a user