feat(server): support server-scoped files via nullable workspaceId on file table (#22587)

Part of the app settings architecture cleanup
(twentyhq/core-team-issues#2456) — PR 1 of the server-level documents
plan, reworked after the revert of #22560 (#22579). Same capability,
different shape: **no new entity** — server-level documents live in the
existing `file` table with a nullable `workspaceId`.

## Problem

All file storage is workspace-scoped (`FileEntity.workspaceId NOT NULL`,
`{workspaceId}/{app}/…` storage keys). Server-level data like
application-registration manifests and tarballs for ownerless catalog
registrations has no first-class home, forcing raw-driver bypasses
(`DefaultAiCatalogService`, prototype #22556).

## Changes (core storage layer only — no HTTP serving, no GraphQL
exposure)

**`FileEntity` gains server scope** (mirrors `KeyValuePairEntity`, which
already supports both instance-level and per-workspace rows):
- `workspaceId` uuid becomes **nullable** — NULL means server-scoped;
the entity no longer extends `WorkspaceRelatedEntity` and declares its
columns directly
- `applicationRegistrationId` nullable FK (`onDelete: CASCADE`) —
registration-owned documents follow their registration
- ownership checks: `workspaceId IS NOT NULL OR
applicationRegistrationId IS NOT NULL` and `workspaceId IS NULL OR
applicationRegistrationId IS NULL` — every row has exactly one owner
- `IDX_FILE_APPLICATION_REGISTRATION_ID_PATH_UNIQUE` UNIQUE
(`applicationRegistrationId`, `path`) — mirrors the workspace
unique-constraint pattern; workspace rows are exempt via their NULL
`applicationRegistrationId`

**New `ServerFileStorageService`** (`file-storage/services/`, exported
from the global `FileStorageModule`; `FileStorageService` moved
alongside it):
- storage keys
`server/{fileFolder}/{applicationRegistrationId}/{resourcePath}` — the
registration segment is injected by the service itself, so paths cannot
collide across registrations; scope-validation util mirroring
`validateStoragePathIsWithinWorkspaceOrThrow`; new `ServerFileFolder`
enum in twenty-shared
- `writeServerFile` (upsert on (`applicationRegistrationId`, `path`) +
driver write; throws on failure), `readServerFile`/`readServerFileById`
(missing row or bytes surfaces `FILE_NOT_FOUND`),
`checkServerFileExists`, `deleteServerFile`/`deleteByServerFileId`
(bytes best-effort, row authoritative),
`deleteByApplicationRegistrationId`
- rows are accessed through a plain repository pinned to `workspaceId:
IsNull()` on every query; workspace-file code paths still go through
`WorkspaceScopedRepository`, which never sees NULL rows

**Null-safety ripples** (workspaceId is now `string | null`):
- `WorkspaceScopedEntity` bound widened to `workspaceId: string | null`
(the wrapper always filters with a concrete id)
- `list-and-delete-orphaned-workspace-entities` now skips `workspaceId
IS NULL` rows — previously `NOT EXISTS` would have flagged server rows
as orphans and deleted them
- `PendingFileCleanupService` sweeps only `workspaceId IS NOT NULL`
rows; `application-package-fetcher` pins its tarball lookup to workspace
rows (tarball migration to server scope is a follow-up PR)

**Migration**: `allow-server-scoped-file` ships as a **2-20 fast
instance command** (2.20.0 is current since #22639; re-slotted from 2-19
per review). Command runs are tracked by name, so instances that already
executed the 2-20 `standardOverrides` drop command still pick this one
up. Its realistic timestamp sorts before that drop command's fabricated
`1825000000000`, which the
`ci:allow-upgrade-command-timestamp-exception` label covers.

## Next PRs in the plan

- PR 2: HTTP serving + token type for server files
- PR 3: application-registration manifests stored as versioned server
files (rework of draft #22556)
- PR 4 (optional): registration tarballs migrate to server scope

## Verification

- New spec `server-file-storage.service.spec.ts` (traversal table,
upsert conflict semantics, row-before-bytes reads, best-effort byte
deletion, registration cascade) + scope-validation util spec; affected
suites all green
- Typecheck (server + shared), `lint:diff-with-main`, full `oxfmt
--check src/` on both packages clean
- Fresh `database:reset` on the re-slotted branch: the 2-20 command
executes, generator then reports **no schema drift**; both ownership
checks and the composite unique verified live (dual-owner insert and
duplicate registration+path both rejected)
This commit is contained in:
martmull
2026-07-08 11:56:24 +02:00
committed by GitHub
parent 2f5e9d47ef
commit b733a79821
54 changed files with 987 additions and 53 deletions
@@ -31,7 +31,7 @@ import {
} from 'src/engine/core-modules/application/application.exception';
import { ApplicationService } from 'src/engine/core-modules/application/application.service';
import { CacheLockService } from 'src/engine/core-modules/cache-lock/cache-lock.service';
import { FileStorageService } from 'src/engine/core-modules/file-storage/file-storage.service';
import { FileStorageService } from 'src/engine/core-modules/file-storage/services/file-storage.service';
import { validateFilePath } from 'src/engine/core-modules/file-storage/utils/validate-file-path.util';
import { FileDTO } from 'src/engine/core-modules/file/dtos/file.dto';
import { ResolverValidationPipe } from 'src/engine/core-modules/graphql/pipes/resolver-validation.pipe';
@@ -27,7 +27,7 @@ import {
} from 'src/engine/core-modules/application/application-package/application-version-validation.service';
import { ApplicationSyncService } from 'src/engine/core-modules/application/application-manifest/application-sync.service';
import { CacheLockService } from 'src/engine/core-modules/cache-lock/cache-lock.service';
import { FileStorageService } from 'src/engine/core-modules/file-storage/file-storage.service';
import { FileStorageService } from 'src/engine/core-modules/file-storage/services/file-storage.service';
import {
LogicFunctionTriggerJob,
type LogicFunctionTriggerJobData,
@@ -18,7 +18,7 @@ import {
} from 'src/engine/core-modules/application/application.exception';
import { ApplicationService } from 'src/engine/core-modules/application/application.service';
import { type FlatApplication } from 'src/engine/core-modules/application/types/flat-application.type';
import { FileStorageService } from 'src/engine/core-modules/file-storage/file-storage.service';
import { FileStorageService } from 'src/engine/core-modules/file-storage/services/file-storage.service';
import { LOGIC_FUNCTION_DRIVER_FACTORY_TOKEN } from 'src/engine/core-modules/logic-function/logic-function-drivers/constants/logic-function-driver-factory.token';
import { type LogicFunctionDriverFactory } from 'src/engine/core-modules/logic-function/logic-function-drivers/logic-function-driver.factory';
import { createEmptyAllFlatEntityMaps } from 'src/engine/metadata-modules/flat-entity/constant/create-empty-all-flat-entity-maps.constant';
@@ -10,7 +10,7 @@ import { type Manifest } from 'twenty-shared/application';
import { FileFolder } from 'twenty-shared/types';
import { isDefined } from 'twenty-shared/utils';
import { type PackageJson } from 'type-fest';
import { Repository } from 'typeorm';
import { IsNull, Not, Repository } from 'typeorm';
import { v4 } from 'uuid';
import { ApplicationRegistrationEntity } from 'src/engine/core-modules/application/application-registration/application-registration.entity';
@@ -24,7 +24,7 @@ import { assertValidNpmPackageName } from 'src/engine/core-modules/application/a
import { extractTarballSecurely } from 'src/engine/core-modules/application/application-package/utils/extract-tarball-securely.util';
import { readJsonFileOrThrow } from 'src/engine/core-modules/application/application-package/utils/read-json-file.util';
import { resolvePackageContentDir } from 'src/engine/core-modules/application/application-package/utils/tarball-utils';
import { FileStorageService } from 'src/engine/core-modules/file-storage/file-storage.service';
import { FileStorageService } from 'src/engine/core-modules/file-storage/services/file-storage.service';
import { FileEntity } from 'src/engine/core-modules/file/entities/file.entity';
import { removeFileFolderFromFileEntityPath } from 'src/engine/core-modules/file/utils/remove-file-folder-from-file-entity-path.utils';
import { SecureHttpClientService } from 'src/engine/core-modules/secure-http-client/secure-http-client.service';
@@ -178,13 +178,23 @@ export class ApplicationPackageFetcherService implements OnModuleInit {
try {
const file = await this.fileRepository.findOneOrFail({
where: { id: appRegistration.tarballFileId },
where: {
id: appRegistration.tarballFileId,
workspaceId: Not(IsNull()),
},
});
const application = await this.applicationRepository.findOneOrFail({
where: { id: file.applicationId },
});
if (!isDefined(file.workspaceId)) {
throw new ApplicationException(
`Tarball file ${file.id} for app registration ${appRegistration.id} has no workspaceId`,
ApplicationExceptionCode.TARBALL_EXTRACTION_FAILED,
);
}
const tarballStream = await this.fileStorageService.readFile({
workspaceId: file.workspaceId,
applicationUniversalIdentifier: application.universalIdentifier,
@@ -28,7 +28,7 @@ import {
import { ApplicationRegistrationSourceType } from 'src/engine/core-modules/application/application-registration/enums/application-registration-source-type.enum';
import { fromManifestApplicationToDisplayFields } from 'src/engine/core-modules/application/application-registration/utils/from-manifest-application-to-display-fields.util';
import { ApplicationService } from 'src/engine/core-modules/application/application.service';
import { FileStorageService } from 'src/engine/core-modules/file-storage/file-storage.service';
import { FileStorageService } from 'src/engine/core-modules/file-storage/services/file-storage.service';
import type { ApplicationManifest } from 'twenty-shared/application';
@Injectable()
@@ -18,7 +18,7 @@ import {
} from 'src/engine/core-modules/application/application.exception';
import { WORKSPACE_CUSTOM_APPLICATION_NAME } from 'src/engine/core-modules/application/constants/workspace-custom-application.constant';
import { type FlatApplication } from 'src/engine/core-modules/application/types/flat-application.type';
import { FileStorageService } from 'src/engine/core-modules/file-storage/file-storage.service';
import { FileStorageService } from 'src/engine/core-modules/file-storage/services/file-storage.service';
import { WorkspaceEntity } from 'src/engine/core-modules/workspace/workspace.entity';
import { AgentEntity } from 'src/engine/metadata-modules/ai/ai-agent/entities/agent.entity';
import { CommandMenuItemEntity } from 'src/engine/metadata-modules/command-menu-item/entities/command-menu-item.entity';