## Why App installability can silently regress from two directions, and today CI only covers one of them: 1. A **server** change (about to merge from the monorepo) breaks the ability to install the **current public apps** — a backward-compatibility regression users would hit on upgrade. 2. An **app** change breaks against a server **built from the current monorepo files** (not just the last published image), so the app and the upcoming server drift apart before either ships. Both are compatibility guarantees between the server and the app catalog. Today they are only tested from the app side, against the latest published image. This PR makes CI enforce the contract from both sides: - Any server PR must keep **every** current public app installable. - Any app PR is exercised against both the **released** server (its integration suite — what users run today) and the **upcoming** (monorepo) server (integration plus deploy + install). ## What Shared building blocks so both CIs exercise the same paths instead of duplicating them: - **`spawn-twenty-server`** (composite action) — returns a running server (`server-url` + `api-key`) from either the latest published Docker Hub image or a server built from the monorepo. Both sources expose the same contract, so callers never branch on how the server came up. - **`test-twenty-app`** (composite action) — exercises one app against a given server, delegating deploy + install to the shared `deploy-twenty-app` / `install-twenty-app` actions. - **`discover-apps`** (reusable workflow) — the single source of truth for the app matrix. Parameterized by `scope` (`public` vs `internal-and-public`) and `changed-only`, so both CIs derive their matrix from the filesystem instead of a hand-maintained list. Discovery stays automatic: a newly added public app is picked up with no CI edit, which is what keeps the "every public app" guarantee honest. Wired in: - **CI Server** gains a `server-apps-install-smoke` matrix that installs every public app (`discover-apps` with `scope: public, changed-only: false`) against the about-to-merge server, gated in `ci-server-status-check` so a regression blocks merge. - **CI Twenty Apps** discovers changed apps (`scope: internal-and-public, changed-only: true`) and runs each against both server sources — the released image and the monorepo build. ## Why the coverage differs per side (not "always everything") `test-twenty-app` has three explicit modes — `installation-and-integration-test` (integration + deploy + install), `integration-test-only` (suite only), `installation-only` (deploy + install only) — because the useful signal depends on what actually changed: - **App PR against the monorepo server → `installation-and-integration-test`.** The app changed, so run its whole suite against the upcoming server, install included. - **App PR against the released server → `integration-test-only`.** Checks the app's own suite against what users run today; install against the released image is left to the SDK e2e path. - **Server PR → `installation-only`, across all apps.** The apps did not change; the only question is "can each one still be installed." Running every app's full integration suite on every server PR would be far slower and largely redundant. Installation-only keeps this broad (the whole catalog) and cheap enough to always run and block merge. The tradeoff is deliberate: broad but shallow where nothing in the app changed, deep where it did. ## Notes / trade-offs - On app-only PRs the `local` source pays a full server build per app (the `server-build` cache is only warm on server PRs). Could be optimized later with a shared warm-up job. - SDK-local (Verdaccio) install testing stays in `ci-create-app-e2e-minimal`; this PR's `local` source targets the server build. <a href="https://cubic.dev/pr/twentyhq/twenty/pull/22636?utm_source=github" rel="nofollow noreferrer noopener" target="_blank">``<img alt="Review in cubic" src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg">``</a>
5.0 KiB
Upgrade Commands
The upgrade process relies on two types of commands:
- Instance commands — schema and data migrations that run once at the instance level (replacing raw TypeORM migrations).
- Workspace commands — commands that iterate over all active or suspended workspaces to apply per-workspace changes.
Both are registered via decorators and automatically discovered by the upgrade pipeline.
Instance Commands
Generating an instance command
npx nx run twenty-server:database:migrate:generate --name <name> --type <fast|slow>
This generates a timestamped file and auto-registers it in instance-commands.constant.ts — do not edit that file manually.
Fast instance commands
Fast commands run immediately during the upgrade. They are used for schema changes that could introduce breaking inconsistencies between the database and the server if delayed.
A fast command implements FastInstanceCommand and provides up / down methods:
@RegisteredInstanceCommand('1.22.0', 1775758621017)
export class AddWorkspaceIdToTotoFastInstanceCommand
implements FastInstanceCommand
{
public async up(queryRunner: QueryRunner): Promise<void> {
await queryRunner.query(
`ALTER TABLE "core"."toto" ADD "workspaceId" uuid`,
);
}
public async down(queryRunner: QueryRunner): Promise<void> {
await queryRunner.query(
`ALTER TABLE "core"."toto" DROP COLUMN "workspaceId"`,
);
}
}
Slow instance commands
Slow commands are used when a potentially long-running data migration must happen before the schema change. They only run when the --include-slow flag is passed.
A slow command implements SlowInstanceCommand, which extends FastInstanceCommand with an additional runDataMigration method that executes before up:
@RegisteredInstanceCommand('1.22.0', 1775758621018, { type: 'slow' })
export class BackfillWorkspaceIdSlowInstanceCommand
implements SlowInstanceCommand
{
async runDataMigration(dataSource: DataSource): Promise<void> {
// Backfill logic (can be slow — e.g. iterating over workspaces, cache recomputation)
}
public async up(queryRunner: QueryRunner): Promise<void> {
await queryRunner.query(
`ALTER TABLE "core"."toto" ALTER COLUMN "workspaceId" SET NOT NULL`,
);
}
public async down(queryRunner: QueryRunner): Promise<void> {
await queryRunner.query(
`ALTER TABLE "core"."toto" ALTER COLUMN "workspaceId" DROP NOT NULL`,
);
}
}
A common pattern is to pair a fast command (add a nullable column) with a slow command (backfill existing rows, then set NOT NULL).
Workspace Commands
Workspace commands run per-workspace logic across all active or suspended workspaces. They are registered with the @RegisteredWorkspaceCommand decorator alongside nest-commander's @Command decorator:
@RegisteredWorkspaceCommand('1.22.0', 1780000002000)
@Command({
name: 'upgrade:1-22:backfill-standard-skills',
description:
'Backfill standard skills for existing workspaces',
})
export class BackfillStandardSkillsCommand
extends ActiveOrSuspendedWorkspaceCommandRunner
{
constructor(
protected readonly workspaceIteratorService: WorkspaceIteratorService,
// inject any services you need
) {
super(workspaceIteratorService);
}
override async runOnWorkspace({
workspaceId,
options,
}: RunOnWorkspaceArgs): Promise<void> {
// Per-workspace logic goes here
// options.dryRun, options.verbose are available for free
}
}
The base class ActiveOrSuspendedWorkspaceCommandRunner handles workspace iteration and provides --dry-run, --verbose, and workspace filter options automatically.
Execution Order
Within a given version of Twenty, the upgrade pipeline runs commands in this order, sorted by timestamp within each group:
- Instance fast commands
- Instance slow commands
- Workspace commands
Workspace commands are executed sequentially across all active/suspended workspaces.
Shipping a command for a future version (deferred drops)
You can write a command for a version listed in TWENTY_NEXT_VERSIONS — typically the second half of a zero-downtime migration, e.g. dropping a column one release after its replacement ships. Pass the target version to the generator:
npx nx run twenty-server:database:migrate:generate --name <name> --type fast --version 2.20.0
It registers and boots (versions are validated against TWENTY_ALL_VERSIONS) but stays dormant — the sequence only runs TWENTY_CROSS_UPGRADE_SUPPORTED_VERSIONS (previous + current). It activates automatically when nx version:bump promotes the version to current.
Caveat: @WasRemovedInUpgrade / @WasIntroducedInUpgrade are validated against the active sequence, so a decorator pointing at a still-dormant next-version command fails boot with unknown-step-name. For a deferred drop, keep the entity's WasRemovedInUpgrade<T> type wrapper now and add the decorator only once the version is current.
See the CI workflows for how upgrade commands are exercised in continuous integration.