houseassassin/twenty
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

feat(server): support server-scoped files via nullable workspaceId on file table (#22587)

Browse Source 
Part of the app settings architecture cleanup
(twentyhq/core-team-issues#2456) — PR 1 of the server-level documents
plan, reworked after the revert of #22560 (#22579). Same capability,
different shape: **no new entity** — server-level documents live in the
existing `file` table with a nullable `workspaceId`.

## Problem

All file storage is workspace-scoped (`FileEntity.workspaceId NOT NULL`,
`{workspaceId}/{app}/…` storage keys). Server-level data like
application-registration manifests and tarballs for ownerless catalog
registrations has no first-class home, forcing raw-driver bypasses
(`DefaultAiCatalogService`, prototype #22556).

## Changes (core storage layer only — no HTTP serving, no GraphQL
exposure)

**`FileEntity` gains server scope** (mirrors `KeyValuePairEntity`, which
already supports both instance-level and per-workspace rows):
- `workspaceId` uuid becomes **nullable** — NULL means server-scoped;
the entity no longer extends `WorkspaceRelatedEntity` and declares its
columns directly
- `applicationRegistrationId` nullable FK (`onDelete: CASCADE`) —
registration-owned documents follow their registration
- ownership checks: `workspaceId IS NOT NULL OR
applicationRegistrationId IS NOT NULL` and `workspaceId IS NULL OR
applicationRegistrationId IS NULL` — every row has exactly one owner
- `IDX_FILE_APPLICATION_REGISTRATION_ID_PATH_UNIQUE` UNIQUE
(`applicationRegistrationId`, `path`) — mirrors the workspace
unique-constraint pattern; workspace rows are exempt via their NULL
`applicationRegistrationId`

**New `ServerFileStorageService`** (`file-storage/services/`, exported
from the global `FileStorageModule`; `FileStorageService` moved
alongside it):
- storage keys
`server/{fileFolder}/{applicationRegistrationId}/{resourcePath}` — the
registration segment is injected by the service itself, so paths cannot
collide across registrations; scope-validation util mirroring
`validateStoragePathIsWithinWorkspaceOrThrow`; new `ServerFileFolder`
enum in twenty-shared
- `writeServerFile` (upsert on (`applicationRegistrationId`, `path`) +
driver write; throws on failure), `readServerFile`/`readServerFileById`
(missing row or bytes surfaces `FILE_NOT_FOUND`),
`checkServerFileExists`, `deleteServerFile`/`deleteByServerFileId`
(bytes best-effort, row authoritative),
`deleteByApplicationRegistrationId`
- rows are accessed through a plain repository pinned to `workspaceId:
IsNull()` on every query; workspace-file code paths still go through
`WorkspaceScopedRepository`, which never sees NULL rows

**Null-safety ripples** (workspaceId is now `string | null`):
- `WorkspaceScopedEntity` bound widened to `workspaceId: string | null`
(the wrapper always filters with a concrete id)
- `list-and-delete-orphaned-workspace-entities` now skips `workspaceId
IS NULL` rows — previously `NOT EXISTS` would have flagged server rows
as orphans and deleted them
- `PendingFileCleanupService` sweeps only `workspaceId IS NOT NULL`
rows; `application-package-fetcher` pins its tarball lookup to workspace
rows (tarball migration to server scope is a follow-up PR)

**Migration**: `allow-server-scoped-file` ships as a **2-20 fast
instance command** (2.20.0 is current since #22639; re-slotted from 2-19
per review). Command runs are tracked by name, so instances that already
executed the 2-20 `standardOverrides` drop command still pick this one
up. Its realistic timestamp sorts before that drop command's fabricated
`1825000000000`, which the
`ci:allow-upgrade-command-timestamp-exception` label covers.

## Next PRs in the plan

- PR 2: HTTP serving + token type for server files
- PR 3: application-registration manifests stored as versioned server
files (rework of draft #22556)
- PR 4 (optional): registration tarballs migrate to server scope

## Verification

- New spec `server-file-storage.service.spec.ts` (traversal table,
upsert conflict semantics, row-before-bytes reads, best-effort byte
deletion, registration cascade) + scope-validation util spec; affected
suites all green
- Typecheck (server + shared), `lint:diff-with-main`, full `oxfmt
--check src/` on both packages clean
- Fresh `database:reset` on the re-slotted branch: the 2-20 command
executes, generator then reports **no schema drift**; both ownership
checks and the composite unique verified live (dual-owner insert and
duplicate registration+path both rejected)
This commit is contained in:
martmull
2026-07-08 11:56:24 +02:00
committed by GitHub
parent 2f5e9d47ef
commit b733a79821
54 changed files with 987 additions and 53 deletions
Download Patch File Download Diff File Expand all files Collapse all files
packages/twenty-server/src/database/commands/list-and-delete-orphaned-workspace-entities.command.ts
+1
View File
@@ -286,6 +286,7 @@ export class ListOrphanedWorkspaceEntitiesCommand extends MigrationCommandRunner
return `NOT EXISTS ${subQuery}`;
})
.andWhere('entity.workspaceId IS NOT NULL')
.select(['entity.id', 'entity.workspaceId'])
.withDeleted()
.getMany();
packages/twenty-server/src/database/commands/upgrade-version-command/2-20/2-20-instance-command-fast-1783499671541-allow-server-scoped-file.ts
+37
View File
@@ -0,0 +1,37 @@
import { QueryRunner } from 'typeorm';
import { RegisteredInstanceCommand } from 'src/engine/core-modules/upgrade/decorators/registered-instance-command.decorator';
import { FastInstanceCommand } from 'src/engine/core-modules/upgrade/interfaces/fast-instance-command.interface';
@RegisteredInstanceCommand('2.20.0', 1783499671541)
export class AllowServerScopedFileFastInstanceCommand implements FastInstanceCommand {
public async up(queryRunner: QueryRunner): Promise<void> {
await queryRunner.query('ALTER TABLE "core"."file" ADD "applicationRegistrationId" uuid');
await queryRunner.query('ALTER TABLE "core"."file" DROP CONSTRAINT "FK_de468b3d8dcf7e94f7074220929"');
await queryRunner.query('ALTER TABLE "core"."file" DROP CONSTRAINT "IDX_APPLICATION_PATH_WORKSPACE_ID_APPLICATION_ID_UNIQUE"');
await queryRunner.query('ALTER TABLE "core"."file" ALTER COLUMN "workspaceId" DROP NOT NULL');
await queryRunner.query('ALTER TABLE "core"."file" ADD CONSTRAINT "IDX_FILE_APPLICATION_REGISTRATION_ID_PATH_UNIQUE" UNIQUE ("applicationRegistrationId", "path")');
await queryRunner.query('CREATE INDEX "IDX_FILE_APPLICATION_REGISTRATION_ID" ON "core"."file" ("applicationRegistrationId") ');
await queryRunner.query('ALTER TABLE "core"."file" ADD CONSTRAINT "IDX_APPLICATION_PATH_WORKSPACE_ID_APPLICATION_ID_UNIQUE" UNIQUE ("workspaceId", "applicationId", "path")');
await queryRunner.query('ALTER TABLE "core"."file" ADD CONSTRAINT "FK_de468b3d8dcf7e94f7074220929" FOREIGN KEY ("workspaceId") REFERENCES "core"."workspace"("id") ON DELETE CASCADE ON UPDATE NO ACTION');
await queryRunner.query('ALTER TABLE "core"."file" ADD CONSTRAINT "FK_feffd2addf9467be6d7cd51db76" FOREIGN KEY ("applicationRegistrationId") REFERENCES "core"."applicationRegistration"("id") ON DELETE CASCADE ON UPDATE NO ACTION');
await queryRunner.query('ALTER TABLE "core"."file" ADD CONSTRAINT "CHK_FILE_WORKSPACE_ID_OR_APPLICATION_REGISTRATION_ID" CHECK ("workspaceId" IS NOT NULL OR "applicationRegistrationId" IS NOT NULL)');
await queryRunner.query('ALTER TABLE "core"."file" ADD CONSTRAINT "CHK_FILE_WORKSPACE_ID_XOR_APPLICATION_REGISTRATION_ID" CHECK ("workspaceId" IS NULL OR "applicationRegistrationId" IS NULL)');
}
public async down(queryRunner: QueryRunner): Promise<void> {
await queryRunner.query('ALTER TABLE "core"."file" DROP CONSTRAINT "CHK_FILE_WORKSPACE_ID_XOR_APPLICATION_REGISTRATION_ID"');
await queryRunner.query('ALTER TABLE "core"."file" DROP CONSTRAINT "CHK_FILE_WORKSPACE_ID_OR_APPLICATION_REGISTRATION_ID"');
await queryRunner.query('ALTER TABLE "core"."file" DROP CONSTRAINT "FK_feffd2addf9467be6d7cd51db76"');
await queryRunner.query('ALTER TABLE "core"."file" DROP CONSTRAINT "FK_de468b3d8dcf7e94f7074220929"');
await queryRunner.query('ALTER TABLE "core"."file" DROP CONSTRAINT "IDX_APPLICATION_PATH_WORKSPACE_ID_APPLICATION_ID_UNIQUE"');
await queryRunner.query('DROP INDEX "core"."IDX_FILE_APPLICATION_REGISTRATION_ID"');
await queryRunner.query('ALTER TABLE "core"."file" DROP CONSTRAINT "IDX_FILE_APPLICATION_REGISTRATION_ID_PATH_UNIQUE"');
// Server-scoped rows cannot survive the NOT NULL restore.
await queryRunner.query('DELETE FROM "core"."file" WHERE "workspaceId" IS NULL');
await queryRunner.query('ALTER TABLE "core"."file" ALTER COLUMN "workspaceId" SET NOT NULL');
await queryRunner.query('ALTER TABLE "core"."file" ADD CONSTRAINT "IDX_APPLICATION_PATH_WORKSPACE_ID_APPLICATION_ID_UNIQUE" UNIQUE ("workspaceId", "applicationId", "path")');
await queryRunner.query('ALTER TABLE "core"."file" ADD CONSTRAINT "FK_de468b3d8dcf7e94f7074220929" FOREIGN KEY ("workspaceId") REFERENCES "core"."workspace"("id") ON DELETE CASCADE ON UPDATE NO ACTION');
await queryRunner.query('ALTER TABLE "core"."file" DROP COLUMN "applicationRegistrationId"');
}
}
packages/twenty-server/src/database/commands/upgrade-version-command/2-20/allow-server-scoped-file-upgrade-command-name.constant.ts
+4
View File
@@ -0,0 +1,4 @@
// Referenced by @WasIntroducedInUpgrade on the file "applicationRegistrationId"
// column so pre-2.20 upgrade steps don't SELECT it before this command adds it.
export const ALLOW_SERVER_SCOPED_FILE_UPGRADE_COMMAND_NAME =
'2.20.0_AllowServerScopedFileFastInstanceCommand_1783499671541';
packages/twenty-server/src/database/commands/upgrade-version-command/instance-commands.constant.ts
+2
View File
@@ -99,6 +99,7 @@ import { BackfillLogoOnApplicationRegistrationSlowInstanceCommand } from './2-19
import { AddDisplayFieldsToApplicationRegistrationFastInstanceCommand } from './2-19/2-19-instance-command-fast-1783073776590-add-display-fields-to-application-registration';
import { BackfillDisplayFieldsOnApplicationRegistrationSlowInstanceCommand } from './2-19/2-19-instance-command-slow-1783073776591-backfill-display-fields-on-application-registration';
import { BackfillIsFeaturedOnApplicationRegistrationSlowInstanceCommand } from './2-19/2-19-instance-command-slow-1783120000000-backfill-is-featured-on-application-registration';
import { AllowServerScopedFileFastInstanceCommand } from 'src/database/commands/upgrade-version-command/2-20/2-20-instance-command-fast-1783499671541-allow-server-scoped-file';
export const INSTANCE_COMMANDS = [
AddViewFieldGroupIdIndexOnViewFieldFastInstanceCommand,
@@ -200,4 +201,5 @@ export const INSTANCE_COMMANDS = [
AddStatusToFileFastInstanceCommand,
AddPendingMimeCheckToFileFastInstanceCommand,
BackfillIsFeaturedOnApplicationRegistrationSlowInstanceCommand,
AllowServerScopedFileFastInstanceCommand,
];