feat(server): support server-scoped files via nullable workspaceId on file table (#22587)
Part of the app settings architecture cleanup (twentyhq/core-team-issues#2456) — PR 1 of the server-level documents plan, reworked after the revert of #22560 (#22579). Same capability, different shape: **no new entity** — server-level documents live in the existing `file` table with a nullable `workspaceId`. ## Problem All file storage is workspace-scoped (`FileEntity.workspaceId NOT NULL`, `{workspaceId}/{app}/…` storage keys). Server-level data like application-registration manifests and tarballs for ownerless catalog registrations has no first-class home, forcing raw-driver bypasses (`DefaultAiCatalogService`, prototype #22556). ## Changes (core storage layer only — no HTTP serving, no GraphQL exposure) **`FileEntity` gains server scope** (mirrors `KeyValuePairEntity`, which already supports both instance-level and per-workspace rows): - `workspaceId` uuid becomes **nullable** — NULL means server-scoped; the entity no longer extends `WorkspaceRelatedEntity` and declares its columns directly - `applicationRegistrationId` nullable FK (`onDelete: CASCADE`) — registration-owned documents follow their registration - ownership checks: `workspaceId IS NOT NULL OR applicationRegistrationId IS NOT NULL` and `workspaceId IS NULL OR applicationRegistrationId IS NULL` — every row has exactly one owner - `IDX_FILE_APPLICATION_REGISTRATION_ID_PATH_UNIQUE` UNIQUE (`applicationRegistrationId`, `path`) — mirrors the workspace unique-constraint pattern; workspace rows are exempt via their NULL `applicationRegistrationId` **New `ServerFileStorageService`** (`file-storage/services/`, exported from the global `FileStorageModule`; `FileStorageService` moved alongside it): - storage keys `server/{fileFolder}/{applicationRegistrationId}/{resourcePath}` — the registration segment is injected by the service itself, so paths cannot collide across registrations; scope-validation util mirroring `validateStoragePathIsWithinWorkspaceOrThrow`; new `ServerFileFolder` enum in twenty-shared - `writeServerFile` (upsert on (`applicationRegistrationId`, `path`) + driver write; throws on failure), `readServerFile`/`readServerFileById` (missing row or bytes surfaces `FILE_NOT_FOUND`), `checkServerFileExists`, `deleteServerFile`/`deleteByServerFileId` (bytes best-effort, row authoritative), `deleteByApplicationRegistrationId` - rows are accessed through a plain repository pinned to `workspaceId: IsNull()` on every query; workspace-file code paths still go through `WorkspaceScopedRepository`, which never sees NULL rows **Null-safety ripples** (workspaceId is now `string | null`): - `WorkspaceScopedEntity` bound widened to `workspaceId: string | null` (the wrapper always filters with a concrete id) - `list-and-delete-orphaned-workspace-entities` now skips `workspaceId IS NULL` rows — previously `NOT EXISTS` would have flagged server rows as orphans and deleted them - `PendingFileCleanupService` sweeps only `workspaceId IS NOT NULL` rows; `application-package-fetcher` pins its tarball lookup to workspace rows (tarball migration to server scope is a follow-up PR) **Migration**: `allow-server-scoped-file` ships as a **2-20 fast instance command** (2.20.0 is current since #22639; re-slotted from 2-19 per review). Command runs are tracked by name, so instances that already executed the 2-20 `standardOverrides` drop command still pick this one up. Its realistic timestamp sorts before that drop command's fabricated `1825000000000`, which the `ci:allow-upgrade-command-timestamp-exception` label covers. ## Next PRs in the plan - PR 2: HTTP serving + token type for server files - PR 3: application-registration manifests stored as versioned server files (rework of draft #22556) - PR 4 (optional): registration tarballs migrate to server scope ## Verification - New spec `server-file-storage.service.spec.ts` (traversal table, upsert conflict semantics, row-before-bytes reads, best-effort byte deletion, registration cascade) + scope-validation util spec; affected suites all green - Typecheck (server + shared), `lint:diff-with-main`, full `oxfmt --check src/` on both packages clean - Fresh `database:reset` on the re-slotted branch: the 2-20 command executes, generator then reports **no schema drift**; both ownership checks and the composite unique verified live (dual-owner insert and duplicate registration+path both rejected)
This commit is contained in:
+1
-1
@@ -31,7 +31,7 @@ import {
|
||||
} from 'src/engine/core-modules/application/application.exception';
|
||||
import { ApplicationService } from 'src/engine/core-modules/application/application.service';
|
||||
import { CacheLockService } from 'src/engine/core-modules/cache-lock/cache-lock.service';
|
||||
import { FileStorageService } from 'src/engine/core-modules/file-storage/file-storage.service';
|
||||
import { FileStorageService } from 'src/engine/core-modules/file-storage/services/file-storage.service';
|
||||
import { validateFilePath } from 'src/engine/core-modules/file-storage/utils/validate-file-path.util';
|
||||
import { FileDTO } from 'src/engine/core-modules/file/dtos/file.dto';
|
||||
import { ResolverValidationPipe } from 'src/engine/core-modules/graphql/pipes/resolver-validation.pipe';
|
||||
|
||||
+1
-1
@@ -27,7 +27,7 @@ import {
|
||||
} from 'src/engine/core-modules/application/application-package/application-version-validation.service';
|
||||
import { ApplicationSyncService } from 'src/engine/core-modules/application/application-manifest/application-sync.service';
|
||||
import { CacheLockService } from 'src/engine/core-modules/cache-lock/cache-lock.service';
|
||||
import { FileStorageService } from 'src/engine/core-modules/file-storage/file-storage.service';
|
||||
import { FileStorageService } from 'src/engine/core-modules/file-storage/services/file-storage.service';
|
||||
import {
|
||||
LogicFunctionTriggerJob,
|
||||
type LogicFunctionTriggerJobData,
|
||||
|
||||
+1
-1
@@ -18,7 +18,7 @@ import {
|
||||
} from 'src/engine/core-modules/application/application.exception';
|
||||
import { ApplicationService } from 'src/engine/core-modules/application/application.service';
|
||||
import { type FlatApplication } from 'src/engine/core-modules/application/types/flat-application.type';
|
||||
import { FileStorageService } from 'src/engine/core-modules/file-storage/file-storage.service';
|
||||
import { FileStorageService } from 'src/engine/core-modules/file-storage/services/file-storage.service';
|
||||
import { LOGIC_FUNCTION_DRIVER_FACTORY_TOKEN } from 'src/engine/core-modules/logic-function/logic-function-drivers/constants/logic-function-driver-factory.token';
|
||||
import { type LogicFunctionDriverFactory } from 'src/engine/core-modules/logic-function/logic-function-drivers/logic-function-driver.factory';
|
||||
import { createEmptyAllFlatEntityMaps } from 'src/engine/metadata-modules/flat-entity/constant/create-empty-all-flat-entity-maps.constant';
|
||||
|
||||
+13
-3
@@ -10,7 +10,7 @@ import { type Manifest } from 'twenty-shared/application';
|
||||
import { FileFolder } from 'twenty-shared/types';
|
||||
import { isDefined } from 'twenty-shared/utils';
|
||||
import { type PackageJson } from 'type-fest';
|
||||
import { Repository } from 'typeorm';
|
||||
import { IsNull, Not, Repository } from 'typeorm';
|
||||
import { v4 } from 'uuid';
|
||||
|
||||
import { ApplicationRegistrationEntity } from 'src/engine/core-modules/application/application-registration/application-registration.entity';
|
||||
@@ -24,7 +24,7 @@ import { assertValidNpmPackageName } from 'src/engine/core-modules/application/a
|
||||
import { extractTarballSecurely } from 'src/engine/core-modules/application/application-package/utils/extract-tarball-securely.util';
|
||||
import { readJsonFileOrThrow } from 'src/engine/core-modules/application/application-package/utils/read-json-file.util';
|
||||
import { resolvePackageContentDir } from 'src/engine/core-modules/application/application-package/utils/tarball-utils';
|
||||
import { FileStorageService } from 'src/engine/core-modules/file-storage/file-storage.service';
|
||||
import { FileStorageService } from 'src/engine/core-modules/file-storage/services/file-storage.service';
|
||||
import { FileEntity } from 'src/engine/core-modules/file/entities/file.entity';
|
||||
import { removeFileFolderFromFileEntityPath } from 'src/engine/core-modules/file/utils/remove-file-folder-from-file-entity-path.utils';
|
||||
import { SecureHttpClientService } from 'src/engine/core-modules/secure-http-client/secure-http-client.service';
|
||||
@@ -178,13 +178,23 @@ export class ApplicationPackageFetcherService implements OnModuleInit {
|
||||
|
||||
try {
|
||||
const file = await this.fileRepository.findOneOrFail({
|
||||
where: { id: appRegistration.tarballFileId },
|
||||
where: {
|
||||
id: appRegistration.tarballFileId,
|
||||
workspaceId: Not(IsNull()),
|
||||
},
|
||||
});
|
||||
|
||||
const application = await this.applicationRepository.findOneOrFail({
|
||||
where: { id: file.applicationId },
|
||||
});
|
||||
|
||||
if (!isDefined(file.workspaceId)) {
|
||||
throw new ApplicationException(
|
||||
`Tarball file ${file.id} for app registration ${appRegistration.id} has no workspaceId`,
|
||||
ApplicationExceptionCode.TARBALL_EXTRACTION_FAILED,
|
||||
);
|
||||
}
|
||||
|
||||
const tarballStream = await this.fileStorageService.readFile({
|
||||
workspaceId: file.workspaceId,
|
||||
applicationUniversalIdentifier: application.universalIdentifier,
|
||||
|
||||
+1
-1
@@ -28,7 +28,7 @@ import {
|
||||
import { ApplicationRegistrationSourceType } from 'src/engine/core-modules/application/application-registration/enums/application-registration-source-type.enum';
|
||||
import { fromManifestApplicationToDisplayFields } from 'src/engine/core-modules/application/application-registration/utils/from-manifest-application-to-display-fields.util';
|
||||
import { ApplicationService } from 'src/engine/core-modules/application/application.service';
|
||||
import { FileStorageService } from 'src/engine/core-modules/file-storage/file-storage.service';
|
||||
import { FileStorageService } from 'src/engine/core-modules/file-storage/services/file-storage.service';
|
||||
import type { ApplicationManifest } from 'twenty-shared/application';
|
||||
|
||||
@Injectable()
|
||||
|
||||
@@ -18,7 +18,7 @@ import {
|
||||
} from 'src/engine/core-modules/application/application.exception';
|
||||
import { WORKSPACE_CUSTOM_APPLICATION_NAME } from 'src/engine/core-modules/application/constants/workspace-custom-application.constant';
|
||||
import { type FlatApplication } from 'src/engine/core-modules/application/types/flat-application.type';
|
||||
import { FileStorageService } from 'src/engine/core-modules/file-storage/file-storage.service';
|
||||
import { FileStorageService } from 'src/engine/core-modules/file-storage/services/file-storage.service';
|
||||
import { WorkspaceEntity } from 'src/engine/core-modules/workspace/workspace.entity';
|
||||
import { AgentEntity } from 'src/engine/metadata-modules/ai/ai-agent/entities/agent.entity';
|
||||
import { CommandMenuItemEntity } from 'src/engine/metadata-modules/command-menu-item/entities/command-menu-item.entity';
|
||||
|
||||
Reference in New Issue
Block a user