twenty/packages/twenty-server/docs/UPGRADE_COMMANDS.md

Upgrade Commands

The upgrade process relies on two types of commands:

• Instance commands — schema and data migrations that run once at the instance level (replacing raw TypeORM migrations).
• Workspace commands — commands that iterate over all active or suspended workspaces to apply per-workspace changes.

Both are registered via decorators and automatically discovered by the upgrade pipeline.

Instance Commands

Generating an instance command

npx nx run twenty-server:database:migrate:generate --name <name> --type <fast|slow>

This generates a timestamped file and auto-registers it in instance-commands.constant.ts — do not edit that file manually.

Fast instance commands

Fast commands run immediately during the upgrade. They are used for schema changes that could introduce breaking inconsistencies between the database and the server if delayed.

A fast command implements FastInstanceCommand and provides up / down methods:

@RegisteredInstanceCommand('1.22.0', 1775758621017)
export class AddWorkspaceIdToTotoFastInstanceCommand
  implements FastInstanceCommand
{
  public async up(queryRunner: QueryRunner): Promise<void> {
    await queryRunner.query(
      `ALTER TABLE "core"."toto" ADD "workspaceId" uuid`,
    );
  }

  public async down(queryRunner: QueryRunner): Promise<void> {
    await queryRunner.query(
      `ALTER TABLE "core"."toto" DROP COLUMN "workspaceId"`,
    );
  }
}

Slow instance commands

Slow commands are used when a potentially long-running data migration must happen before the schema change. They only run when the --include-slow flag is passed.

A slow command implements SlowInstanceCommand, which extends FastInstanceCommand with an additional runDataMigration method that executes before up:

@RegisteredInstanceCommand('1.22.0', 1775758621018, { type: 'slow' })
export class BackfillWorkspaceIdSlowInstanceCommand
  implements SlowInstanceCommand
{
  async runDataMigration(dataSource: DataSource): Promise<void> {
    // Backfill logic (can be slow — e.g. iterating over workspaces, cache recomputation)
  }

  public async up(queryRunner: QueryRunner): Promise<void> {
    await queryRunner.query(
      `ALTER TABLE "core"."toto" ALTER COLUMN "workspaceId" SET NOT NULL`,
    );
  }

  public async down(queryRunner: QueryRunner): Promise<void> {
    await queryRunner.query(
      `ALTER TABLE "core"."toto" ALTER COLUMN "workspaceId" DROP NOT NULL`,
    );
  }
}

A common pattern is to pair a fast command (add a nullable column) with a slow command (backfill existing rows, then set NOT NULL).

Workspace Commands

Workspace commands run per-workspace logic across all active or suspended workspaces. They are registered with the @RegisteredWorkspaceCommand decorator alongside nest-commander's @Command decorator:

@RegisteredWorkspaceCommand('1.22.0', 1780000002000)
@Command({
  name: 'upgrade:1-22:backfill-standard-skills',
  description:
    'Backfill standard skills for existing workspaces',
})
export class BackfillStandardSkillsCommand
  extends ActiveOrSuspendedWorkspaceCommandRunner
{
  constructor(
    protected readonly workspaceIteratorService: WorkspaceIteratorService,
    // inject any services you need
  ) {
    super(workspaceIteratorService);
  }

  override async runOnWorkspace({
    workspaceId,
    options,
  }: RunOnWorkspaceArgs): Promise<void> {
    // Per-workspace logic goes here
    // options.dryRun, options.verbose are available for free
  }
}

The base class ActiveOrSuspendedWorkspaceCommandRunner handles workspace iteration and provides --dry-run, --verbose, and workspace filter options automatically.

Applying a migration matrix: side-effect vs legacy path

Commands that build a metadata migration go through WorkspaceMigrationValidateBuildAndRunService. Two entry points exist:

• validateBuildAndRunWorkspaceMigration (default): runs the operation matrix through the metadata side-effect engine (expandWithSideEffects) before building. The engine injects and cascades engine-owned companions (system fields and relations, the searchVector field and its GIN index, searchFieldMetadata rows, unique backing indexes). This is what the live API and application manifests rely on, so new commands should use it.
• validateBuildAndRunLegacyWorkspaceMigration: skips side-effect expansion and applies the matrix literally, exactly as it was authored.

The side-effect engine landed in v2.19. Commands authored before then declared their companions explicitly and were never designed to flow through the engine. Running them through it retroactively changes their behavior: it can hard-fail on reserved-identifier collisions (RESERVED_SYSTEM_UNIVERSAL_IDENTIFIER) and silently create rows the command never intended (for example, the deterministic searchFieldMetadata rows that the standalone upgrade:2-16:backfill-search-field-metadata backfill then re-inserts, hitting IDX_SEARCH_FIELD_METADATA_OBJECT_FIELD_UNIQUE).

Rule of thumb:

• Target version < 2.19 → use the legacy method.
• Target version >= 2.19 → use the default side-effect method.

All pre-2.19 commands follow this rule, including upgrade:2-10:sync-call-recording-standard-objects: it builds its create-set from the static twenty-standard definition (which declares all of callRecording's fields, including the searchVector system field) and runs it through the legacy path so nothing is injected on top. Its matrix contains no searchFieldMetadata operations; the deterministic rows are created later in the same upgrade pipeline by upgrade:2-16:backfill-search-field-metadata, which derives them from the standard definition.

Known gap: the static definition does not yet declare callRecording's searchVector GIN index (every other searchable standard object declares its GIN index statically), so workspaces upgrading through 2-10 on the legacy path create the searchVector column unindexed. The static declaration plus a backfill for already-upgraded workspaces land in a follow-up (twentyhq/core-team-issues#2672), which must ship in the same release as this legacy path.

Execution Order

Within a given version of Twenty, the upgrade pipeline runs commands in this order, sorted by timestamp within each group:

1. Instance fast commands
2. Instance slow commands
3. Workspace commands

Workspace commands are executed sequentially across all active/suspended workspaces.

Interrupting a run (Ctrl+C, SIGTERM)

Ctrl+C during an upgrade stops it gracefully: the workspace being processed finishes its commands, then the run stops instead of starting the next one. Ctrl+C again forces an immediate exit, leaving the command in progress unfinished.

Rerun the command to resume. Nothing is rolled back, and the run picks up from the last command recorded in upgradeMigration.

Expect the first Ctrl+C to look like it did nothing while a long step is running: it takes effect once the step ends.

Running detached

A foreground upgrade dies with the shell that started it, so a dropped kubectl exec tunnel, a closed laptop or an expired VPN kills the run. scripts/upgrade-background.sh gives it its own session with no controlling terminal and streams the output from a log file instead. Use it for long upgrades over kubectl exec or SSH; foreground is still right locally and in CI.

yarn upgrade:background [args]   # start detached, then stream the log
yarn upgrade:background:logs     # re-attach from another shell
yarn upgrade:background:stop     # graceful stop; --now immediate, --force SIGKILL

[args] is forwarded verbatim to upgrade: -d/--dry-run, -v/--verbose, -w/--workspace-id <id> (repeatable), --start-from-workspace-id <id>, --workspace-count-limit <n>. The two workspace selectors are mutually exclusive. --include-slow is not among them, it belongs to run-instance-commands.

Ctrl+C detaches the stream only, and logs takes any number of concurrent readers. logs reports whether a run is alive before streaming, and on a finished one prints the verdict and the log tail rather than following a log that will never grow again: a segment can run for minutes without printing, so a silent log alone cannot tell a slow step from a dead process.

stop has three tiers, three explicit invocations with no timed escalation between them, since a segment can take many minutes and a timer would defeat the graceful path. Each prints the log tail after signalling. Only the node process is signalled, never the process group, which would also kill the wrapper that records the exit code.

Invocation	Signal	Effect
stop	one SIGTERM	stops at the next boundary, exit 143
stop --now	two spaced SIGTERMs	immediate exit, step in progress left unfinished
stop --force	SIGKILL	no boundary, a multi-transaction command may leave partial work

The wrapper outlives node and appends an EXIT=<code> line, which is what logs translates:

Log	Meaning
EXIT=0	completed
EXIT=130 / EXIT=143	graceful stop on SIGINT / SIGTERM
EXIT=137	SIGKILLed, from stop --force or an OOM kill
any other EXIT=	failed
no EXIT= line	killed without a graceful stop, wrapper included (pod replaced, host lost)

A graceful stop is always safe to rerun: nothing is rolled back and the run resumes from the last command recorded in upgradeMigration, each workspace either fully done with its segment or untouched. Rerunning after a forced kill relies on commands being idempotent.

Limits of this mode:

• Ctrl+C cannot reach a detached run, so stop is the only graceful entry point and 130 only ever appears on foreground runs.
• Log and PID files live in /tmp in the container (TWENTY_UPGRADE_LOG_FILE, TWENTY_UPGRADE_PID_FILE) and are lost with the pod.
• The start refusal only sees this container. Nothing in upgrade prevents two concurrent sequences either: upgradeMigration has no in-progress state and the sequence runner takes no advisory lock. One upgrade at a time is an operational rule, not an enforced one.
• Needs setsid, present in the runtime image and on Linux, absent on macOS where start refuses and points at the foreground command.
• terminationGracePeriodSeconds does not apply. PID 1 in the command-runner pod is tail -f /dev/null, so on pod deletion the detached process is torn down without ever receiving SIGTERM.

Shipping a command for a future version (deferred drops)

You can write a command for a version listed in TWENTY_NEXT_VERSIONS — typically the second half of a zero-downtime migration, e.g. dropping a column one release after its replacement ships. Pass the target version to the generator:

npx nx run twenty-server:database:migrate:generate --name <name> --type fast --version 2.20.0

It registers and boots (versions are validated against TWENTY_ALL_VERSIONS) but stays dormant — the sequence only runs TWENTY_CROSS_UPGRADE_SUPPORTED_VERSIONS (previous + current). It activates automatically when nx version:bump promotes the version to current.

Caveat: @WasRemovedInUpgrade / @WasIntroducedInUpgrade are validated against the active sequence, so a decorator pointing at a still-dormant next-version command fails boot with unknown-step-name. For a deferred drop, keep the entity's WasRemovedInUpgrade<T> type wrapper now and add the decorator only once the version is current.

See the CI workflows for how upgrade commands are exercised in continuous integration.