harden(server): backfill and enforce workspace.databaseSchema invariant with a check constraint (#22855)
## What & why `core.workspace.databaseSchema` is meant to be set for every workspace past the creation phase. It only started being written at creation time in 2.x (dual-write since 2026-03-28, direct write since 2026-04-10); older workspaces relied on the `1-21 backfill-datasource-to-workspace` instance command, which never effectively ran on some instances. On affected rows the column could be left `NULL`. A null value on a post-creation workspace is a real integrity problem — several paths trust the column: - **REST API**: `hydrateRestRequest` throws `No data sources found` for authenticated requests. - **GraphQL API**: `getOrComputeSchemaSDL` returns `null`, so `WorkspaceSchemaFactory` hands back an empty schema. - **GraphQL introspection** (direct execution) returns `null`. This PR makes the invariant impossible to silently violate, and repairs any instance still lagging. ### On the original "No data source, skipping" logs This investigation started from `BackfillActorSourceEnumValuesCommand` logging `No data source for workspace <id>, skipping` at high volume. **That symptom is not explained by this change, and this PR is not a fix for it.** Findings: - The workspace iterator only processes `ACTIVE` + `SUSPENDED` workspaces, and on the affected instance all of those already have `databaseSchema` set (only `PENDING_CREATION` rows are null, and those are never iterated). - `getGlobalWorkspaceDataSource()` never resolves to `undefined` (it returns a value or throws), so a defined-schema workspace should never hit the skip branch. - The upgrade-aware repository proxy was investigated as a possible cause (it can short-circuit `findOne` to `null` for entities marked unavailable during an upgrade) and **exonerated**: `WorkspaceEntity` and its `databaseSchema` column carry no `@WasIntroducedInUpgrade`/`@WasRemovedInUpgrade` decorators, so `resolveEntityShapeAtUpgradeCursor` always reports the entity available and the column visible at every cursor. In other words, current code should emit zero such skips for that instance's data, so the root cause of the observed logs remains undetermined and is tracked separately. See twentyhq/core-team-issues#2666. ## Changes - **Check constraint `workspace_requires_database_schema`** (the core of this PR): enforces `databaseSchema IS NOT NULL` for any workspace past creation (`activationStatus NOT IN ('PENDING_CREATION', 'ONGOING_CREATION')`). Declared on `WorkspaceEntity` and applied in the slow instance command's `up()`. Safe against the creation flow: `databaseSchema` is written in `WorkspaceManagerService.init` (right after schema creation) long before a workspace becomes `ACTIVE`. - **Defensive backfill** (`2-21` slow instance command): repopulates `databaseSchema` where it is `NULL`/empty, deriving the schema name deterministically from the workspace id (`getWorkspaceSchemaName`) and only setting it for workspaces whose schema actually exists in `information_schema.schemata` (so `PENDING_CREATION` rows without a provisioned schema are left untouched, and stay exempt via the constraint). No-op on instances already backfilled. - `runDataMigration` runs before `up()`, so the backfill repairs legacy rows before the constraint is enforced. Keeping both in the same slow command (rather than a standalone fast command) guarantees the constraint is never added ahead of the repair. - `checkSchemaExists` gets an explicit `: Promise<boolean>` return type. ## Notes - Backfill + constraint live in a **slow** instance command, so they only apply on upgrades run with `--include-slow`. - The constraint is added **`NOT VALID`**: the backfill repairs every workspace whose Postgres schema exists, but some legacy active/suspended workspaces (e.g. carried over from very old versions, as reproduced by the cross-version upgrade from v1.22) have a null `databaseSchema` with no schema to point at and are unrepairable. `NOT VALID` enforces the invariant on all future inserts/updates without failing the upgrade on that pre-existing corruption. - No production request path was changed — the iterator and `checkSchemaExists` keep trusting the (now backfilled + constrained) column. ## Test plan - [ ] Run `database:migrate:prod --include-slow` on an instance with null `databaseSchema` rows; verify rows whose schema exists get backfilled and `PENDING_CREATION` rows are left null. - [ ] Verify the `workspace_requires_database_schema` constraint exists on `core.workspace` and rejects nulling `databaseSchema` on an active workspace. - [ ] Verify a fresh workspace creation still succeeds (constraint does not fight the `PENDING_CREATION` → `ACTIVE` transition).
This commit is contained in:
Paul Rastoin
+9
@@ -90,6 +90,15 @@ export class WorkspaceIteratorService {
|
||||
? await this.globalWorkspaceOrmManager.getGlobalWorkspaceDataSource()
|
||||
: undefined;
|
||||
|
||||
if (!isDefined(dataSource)) {
|
||||
this.logger.warn(
|
||||
`Could not retrieve a workspace data source for workspace ${workspaceId} ` +
|
||||
`(index ${index + 1}/${workspaceIdsToProcess.length}): ` +
|
||||
`workspaceRowFound=${isDefined(workspace)}, ` +
|
||||
`databaseSchema=${JSON.stringify(workspace?.databaseSchema ?? null)}`,
|
||||
);
|
||||
}
|
||||
|
||||
await callback({
|
||||
workspaceId,
|
||||
dataSource,
|
||||
|
||||
+55
@@ -0,0 +1,55 @@
|
||||
import { DataSource, QueryRunner } from 'typeorm';
|
||||
|
||||
import { RegisteredInstanceCommand } from 'src/engine/core-modules/upgrade/decorators/registered-instance-command.decorator';
|
||||
import { SlowInstanceCommand } from 'src/engine/core-modules/upgrade/interfaces/slow-instance-command.interface';
|
||||
import { getWorkspaceSchemaName } from 'src/engine/workspace-datasource/utils/get-workspace-schema-name.util';
|
||||
|
||||
@RegisteredInstanceCommand('2.21.0', 1783934147089, { type: 'slow' })
|
||||
export class BackfillWorkspaceDatabaseSchemaSlowInstanceCommand
|
||||
implements SlowInstanceCommand
|
||||
{
|
||||
// Schema names are deterministic from the workspace id, so we backfill them
|
||||
// unconditionally — even if the schema doesn't exist yet in Postgres — which
|
||||
// lets up() add the check constraint fully validated.
|
||||
async runDataMigration(dataSource: DataSource): Promise<void> {
|
||||
const workspacesWithoutSchema: { id: string }[] = await dataSource.query(
|
||||
`SELECT id FROM "core"."workspace"
|
||||
WHERE ("databaseSchema" IS NULL OR "databaseSchema" = '')
|
||||
AND "activationStatus" NOT IN ('PENDING_CREATION', 'ONGOING_CREATION')`,
|
||||
);
|
||||
|
||||
if (workspacesWithoutSchema.length === 0) {
|
||||
return;
|
||||
}
|
||||
|
||||
await dataSource.query(
|
||||
`UPDATE "core"."workspace" AS w
|
||||
SET "databaseSchema" = data.schema_name
|
||||
FROM (
|
||||
SELECT UNNEST($1::uuid[]) AS id, UNNEST($2::text[]) AS schema_name
|
||||
) AS data
|
||||
WHERE w.id = data.id`,
|
||||
[
|
||||
workspacesWithoutSchema.map((workspace) => workspace.id),
|
||||
workspacesWithoutSchema.map((workspace) =>
|
||||
getWorkspaceSchemaName(workspace.id),
|
||||
),
|
||||
],
|
||||
);
|
||||
}
|
||||
|
||||
public async up(queryRunner: QueryRunner): Promise<void> {
|
||||
await queryRunner.query(
|
||||
`ALTER TABLE "core"."workspace" DROP CONSTRAINT IF EXISTS "workspace_requires_database_schema"`,
|
||||
);
|
||||
await queryRunner.query(
|
||||
`ALTER TABLE "core"."workspace" ADD CONSTRAINT "workspace_requires_database_schema" CHECK ("activationStatus" IN ('PENDING_CREATION', 'ONGOING_CREATION') OR ("databaseSchema" IS NOT NULL AND "databaseSchema" <> ''))`,
|
||||
);
|
||||
}
|
||||
|
||||
public async down(queryRunner: QueryRunner): Promise<void> {
|
||||
await queryRunner.query(
|
||||
`ALTER TABLE "core"."workspace" DROP CONSTRAINT IF EXISTS "workspace_requires_database_schema"`,
|
||||
);
|
||||
}
|
||||
}
|
||||
+2
@@ -108,6 +108,7 @@ import { CreateWorkflowCoreTableFastInstanceCommand } from './2-20/2-20-instance
|
||||
import { AddGalleryImagesToApplicationRegistrationFastInstanceCommand } from './2-20/2-20-instance-command-fast-1783615890055-add-gallery-images-to-application-registration';
|
||||
import { BackfillGalleryImagesOnApplicationRegistrationSlowInstanceCommand } from './2-20/2-20-instance-command-slow-1783615890056-backfill-gallery-images-on-application-registration';
|
||||
import { AddWorkflowVersionSyncableColumnsFastInstanceCommand } from './2-20/2-20-instance-command-fast-1783603454480-add-workflow-version-syncable-columns';
|
||||
import { BackfillWorkspaceDatabaseSchemaSlowInstanceCommand } from './2-21/2-21-instance-command-slow-1783934147089-backfill-workspace-database-schema';
|
||||
|
||||
export const INSTANCE_COMMANDS = [
|
||||
AddViewFieldGroupIdIndexOnViewFieldFastInstanceCommand,
|
||||
@@ -218,4 +219,5 @@ export const INSTANCE_COMMANDS = [
|
||||
AddGalleryImagesToApplicationRegistrationFastInstanceCommand,
|
||||
BackfillGalleryImagesOnApplicationRegistrationSlowInstanceCommand,
|
||||
AddWorkflowVersionSyncableColumnsFastInstanceCommand,
|
||||
BackfillWorkspaceDatabaseSchemaSlowInstanceCommand,
|
||||
];
|
||||
|
||||
@@ -67,6 +67,10 @@ registerEnumType(WorkspaceDiscoverability, {
|
||||
'onboarded_workspace_requires_default_role',
|
||||
`"activationStatus" IN ('PENDING_CREATION', 'ONGOING_CREATION') OR "defaultRoleId" IS NOT NULL`,
|
||||
)
|
||||
@Check(
|
||||
'workspace_requires_database_schema',
|
||||
`"activationStatus" IN ('PENDING_CREATION', 'ONGOING_CREATION') OR ("databaseSchema" IS NOT NULL AND "databaseSchema" <> '')`,
|
||||
)
|
||||
@Entity({ name: 'workspace', schema: 'core' })
|
||||
@ObjectType('Workspace')
|
||||
export class WorkspaceEntity {
|
||||
|
||||
+1
-1
@@ -37,7 +37,7 @@ export class WorkspaceDataSourceService {
|
||||
}
|
||||
}
|
||||
|
||||
public async checkSchemaExists(workspaceId: string) {
|
||||
public async checkSchemaExists(workspaceId: string): Promise<boolean> {
|
||||
const workspace = await this.workspaceRepository.findOne({
|
||||
select: ['databaseSchema'],
|
||||
where: { id: workspaceId },
|
||||
|
||||
Reference in New Issue
Block a user