6c40c7b91a
# Introduction Close twentyhq/core-team-issues#2641 Auto-provisioned field metadata used to get its `universalIdentifier` from three unrelated sources: random `v4()` on the server when creating custom objects, hardcoded values in `STANDARD_OBJECTS`, and an ad-hoc `v5` derivation in the SDK manifest build. This PR unifies all of them behind the shared `getFieldUniversalIdentifier` derivation: ``` universalIdentifier = f(applicationUniversalIdentifier, objectUniversalIdentifier, fieldName) ``` ## Ownership model The rollout is built on an explicit split of who owns a field's universal identifier: - **The 8 system fields** (`id`, `createdAt`, `updatedAt`, `deletedAt`, `createdBy`, `updatedBy`, `position`, `searchVector`) are **server-owned**. Their universal identifiers are always the deterministic derivation, on **every** application (standard, workspace-custom, installed). Clients cannot provide custom values: a temporary check in `validateObjectMetadataSystemFieldsIntegrity` rejects any non-derived system field identifier at migration build time. This check stands in until system fields are generated exclusively server side by the metadata side-effect engine and stripped from client inputs — at which point it becomes structurally impossible to send one. - **`name` is a default field, not a system field**: it is auto-provisioned when absent (server side for custom objects, SDK side for application objects) but authors can define their own. It is only derived where it is guaranteed to be auto-provisioned. In particular, standard objects keep their **historical hardcoded** `name` identifiers: the standard app authors its `name` fields like any installed app would, and moving those identifiers would break every installed application referencing them (e.g. views on `opportunity.name`). - **User-created and author-provided fields** keep random / explicit identifiers, untouched. ## Server - `validateObjectMetadataSystemFieldsIntegrity` now validates, on top of the existing type/`isSystem` checks, that each system field's `universalIdentifier` equals the deterministic derivation. Runs for every object creation going through the migration orchestrator: app sync, custom object creation, standard provisioning - `build-default-flat-field-metadatas-for-custom-object.util.ts` derives the system field identifiers (and the auto-provisioned `name`) with `getFieldUniversalIdentifier` instead of `v4()` - `build-default-relation-flat-field-metadatas-for-custom-object.util.ts` derives both the forward and the reverse default relation field identifiers deterministically - `generateMorphOrRelationFlatFieldMetadataPair` accepts optional `sourceFieldUniversalIdentifier` / `targetFieldUniversalIdentifier` so callers can inject deterministic values; user-created relations still default to `v4()` ## twenty-shared - `STANDARD_OBJECTS` system field identifiers (the 8) are now computed at module load via `buildStandardObjectSystemFields`; `name` and every other identifier keep their hardcoded values - New snapshot test pinning **every** universal identifier of `STANDARD_OBJECTS`: any identifier change now requires an explicit snapshot update and should ship with a coordinated backfill ## SDK (breaking, pre-GA) - `generateDefaultFieldUniversalIdentifier` delegates to `getFieldUniversalIdentifier` and now requires `applicationUniversalIdentifier` - Reverse default relation field identifiers are derived from the field's real coordinates (standard object UID + actual field name, e.g. `targetRocket` on `attachment`) instead of the legacy custom-object UID + synthetic `${fieldName}Inverse` hash input. Field *names* are unchanged - The manifest build threads the application universal identifier through default field injection (two-pass over object configs) - `twenty dev:add` now resolves the application universal identifier upfront and refuses to scaffold anything until `defineApplication` declares one — no more `fill-later` placeholder for the app UID in generated files ## Upgrade A 2.19 **workspace command** backfills existing `fieldMetadata.universalIdentifier` rows to the deterministic derivation. Coverage follows the ownership model: - **The 8 system fields**: taken over for **every application**, whatever value they currently hold. This is both safe and required now that sync rejects non-derived values — leaving a row unconverged would make its application unsyncable - **`name`**: workspace-custom app → always taken over (server-generated, no author to clobber); installed applications → only rows still carrying the legacy SDK derivation are recomputed, author-provided identifiers are never touched; standard app → never touched (hardcoded in `STANDARD_OBJECTS`) - **Default relation fields**: workspace-custom app → forward fields on custom objects and reverse fields on the standard relation objects; installed applications → legacy-derivation probe only All identifiers of a workspace are updated inside a single transaction, then the command flushes the field-metadata-related workspace caches and bumps the metadata version. Stored `applicationRegistration.manifest` snapshots are intentionally **not** rewritten: installs and upgrades always sync from the `manifest.json` inside the resolved package (npm/tarball), the stored column is only used for display/marketplace purposes. ## Breaking behavior for old packages (fail closed) Packages built with an older SDK carry legacy system field identifiers in their tarball `manifest.json`. Installing or upgrading such a package now fails with an explicit `INVALID_SYSTEM_FIELD` validation error ("universal identifier is not deterministic") instead of silently mismatching against the backfilled rows and triggering a destructive delete+create. The remediation is to rebuild the package with the new SDK; the backfill has already converged the installed rows, so the rebuilt manifest syncs cleanly. ## Test plan - [x] `twenty-sdk` unit tests (526 tests) and typecheck - [x] `twenty-shared` unit tests (1635 tests) including the `STANDARD_OBJECTS` snapshot; `name` identifiers verified byte-for-byte identical to `main` - [x] Lint and typecheck clean on all touched packages - [x] Integration: create a custom object and verify system + default relation field identifiers match the deterministic derivation (`create-one-object-metadata-deterministic-field-universal-identifiers`, 13 assertions passing) - [x] Integration: `failing-sync-application-object-system-fields` extended with a non-derived system field identifier case; all identifiers in the spec pinned deterministically so snapshots embedding expected/actual values are stable across runs (verified with a double run) - [x] Integration: all application sync suites pass with the derived system field identifiers now required by the `buildDefaultObjectManifest` test helper (9 suites, 20 tests) - [x] Full test-database reset: standard app provisioning and seeded workspaces pass the new validation - [x] SDK manifest build verified on the postcard example app: all auto-generated default field identifiers match the derivation - [ ] Run `upgrade:2-19:backfill-deterministic-field-universal-identifiers` (dry-run then real) on a seeded workspace and verify identifier convergence with a rebuilt app manifest
179 lines
5.2 KiB
TypeScript
179 lines
5.2 KiB
TypeScript
import { expectOneNotInternalServerErrorSnapshot } from 'test/integration/graphql/utils/expect-one-not-internal-server-error-snapshot.util';
|
|
import { cleanupApplicationAndAppRegistration } from 'test/integration/metadata/suites/application/utils/cleanup-application-and-app-registration.util';
|
|
import { setupApplicationForSync } from 'test/integration/metadata/suites/application/utils/setup-application-for-sync.util';
|
|
import { uploadApplicationFile } from 'test/integration/metadata/suites/application/utils/upload-application-file.util';
|
|
import {
|
|
type EachTestingContext,
|
|
eachTestingContextFilter,
|
|
} from 'twenty-shared/testing';
|
|
import { v4 as uuidv4 } from 'uuid';
|
|
|
|
const TEST_APP_ID = uuidv4();
|
|
const UNKNOWN_APP_ID = uuidv4();
|
|
|
|
type TestContext = {
|
|
applicationUniversalIdentifier: string;
|
|
fileFolder: string;
|
|
filePath: string;
|
|
};
|
|
|
|
const FAILING_TEST_CASES: EachTestingContext<TestContext>[] = [
|
|
{
|
|
title: 'when filePath contains relative path traversal (../)',
|
|
context: {
|
|
applicationUniversalIdentifier: TEST_APP_ID,
|
|
fileFolder: 'BuiltFrontComponent',
|
|
filePath:
|
|
'../../../other-workspace/other-app/BuiltFrontComponent/stolen.mjs',
|
|
},
|
|
},
|
|
{
|
|
title: 'when filePath contains upward traversal (../../)',
|
|
context: {
|
|
applicationUniversalIdentifier: TEST_APP_ID,
|
|
fileFolder: 'BuiltFrontComponent',
|
|
filePath: '../../etc/passwd',
|
|
},
|
|
},
|
|
{
|
|
title: 'when filePath is an absolute path',
|
|
context: {
|
|
applicationUniversalIdentifier: TEST_APP_ID,
|
|
fileFolder: 'BuiltFrontComponent',
|
|
filePath: '/etc/passwd',
|
|
},
|
|
},
|
|
{
|
|
title: 'when filePath contains backslash path traversal',
|
|
context: {
|
|
applicationUniversalIdentifier: TEST_APP_ID,
|
|
fileFolder: 'BuiltFrontComponent',
|
|
filePath: '..\\..\\..\\etc\\passwd',
|
|
},
|
|
},
|
|
{
|
|
title: 'when filePath is empty',
|
|
context: {
|
|
applicationUniversalIdentifier: TEST_APP_ID,
|
|
fileFolder: 'BuiltFrontComponent',
|
|
filePath: '',
|
|
},
|
|
},
|
|
{
|
|
title:
|
|
'when applicationUniversalIdentifier does not match any installed application',
|
|
context: {
|
|
applicationUniversalIdentifier: UNKNOWN_APP_ID,
|
|
fileFolder: 'BuiltFrontComponent',
|
|
filePath: 'src/components/legit.mjs',
|
|
},
|
|
},
|
|
{
|
|
title: 'when applicationUniversalIdentifier is empty',
|
|
context: {
|
|
applicationUniversalIdentifier: '',
|
|
fileFolder: 'BuiltFrontComponent',
|
|
filePath: 'src/components/legit.mjs',
|
|
},
|
|
},
|
|
{
|
|
title: 'when fileFolder is not an allowed application file folder',
|
|
context: {
|
|
applicationUniversalIdentifier: TEST_APP_ID,
|
|
fileFolder: 'CorePicture',
|
|
filePath: 'src/components/legit.mjs',
|
|
},
|
|
},
|
|
{
|
|
title: 'when filePath is a folder path without extension (bare UUID)',
|
|
context: {
|
|
applicationUniversalIdentifier: TEST_APP_ID,
|
|
fileFolder: 'BuiltFrontComponent',
|
|
filePath: '8b2df3cc-23ad-4e1b-87fd-f880d4cefd58',
|
|
},
|
|
},
|
|
{
|
|
title: 'when filePath is a nested folder path without extension',
|
|
context: {
|
|
applicationUniversalIdentifier: TEST_APP_ID,
|
|
fileFolder: 'Source',
|
|
filePath: 'src/logic-functions/my-handler',
|
|
},
|
|
},
|
|
{
|
|
title:
|
|
'when filePath has an invalid extension for BuiltFrontComponent (.js instead of .mjs)',
|
|
context: {
|
|
applicationUniversalIdentifier: TEST_APP_ID,
|
|
fileFolder: 'BuiltFrontComponent',
|
|
filePath: 'src/components/component.js',
|
|
},
|
|
},
|
|
{
|
|
title:
|
|
'when filePath has an invalid extension for BuiltLogicFunction (.html)',
|
|
context: {
|
|
applicationUniversalIdentifier: TEST_APP_ID,
|
|
fileFolder: 'BuiltLogicFunction',
|
|
filePath: 'src/handlers/handler.html',
|
|
},
|
|
},
|
|
{
|
|
title:
|
|
'when filePath has an invalid extension for Source (.js instead of .ts)',
|
|
context: {
|
|
applicationUniversalIdentifier: TEST_APP_ID,
|
|
fileFolder: 'Source',
|
|
filePath: 'src/index.js',
|
|
},
|
|
},
|
|
{
|
|
title:
|
|
'when filePath has an invalid extension for Dependencies (.sh instead of .json/.lock)',
|
|
context: {
|
|
applicationUniversalIdentifier: TEST_APP_ID,
|
|
fileFolder: 'Dependencies',
|
|
filePath: 'install.sh',
|
|
},
|
|
},
|
|
];
|
|
|
|
describe('Upload application file should fail', () => {
|
|
beforeAll(async () => {
|
|
await setupApplicationForSync({
|
|
applicationUniversalIdentifier: TEST_APP_ID,
|
|
name: 'Test Upload Path Traversal App',
|
|
description: 'App for testing path traversal on file upload',
|
|
sourcePath: 'test-upload-path-traversal',
|
|
});
|
|
}, 60000);
|
|
|
|
afterAll(async () => {
|
|
await cleanupApplicationAndAppRegistration({
|
|
applicationUniversalIdentifier: TEST_APP_ID,
|
|
});
|
|
});
|
|
|
|
it.each(eachTestingContextFilter(FAILING_TEST_CASES))(
|
|
'$title',
|
|
async ({ context }) => {
|
|
jest.useRealTimers();
|
|
|
|
const { errors } = await uploadApplicationFile({
|
|
applicationUniversalIdentifier: context.applicationUniversalIdentifier,
|
|
fileFolder: context.fileFolder,
|
|
filePath: context.filePath,
|
|
fileBuffer: Buffer.from('content'),
|
|
filename: 'test-file.mjs',
|
|
contentType: 'application/javascript',
|
|
expectToFail: true,
|
|
});
|
|
|
|
jest.useFakeTimers();
|
|
|
|
expectOneNotInternalServerErrorSnapshot({ errors });
|
|
},
|
|
60000,
|
|
);
|
|
});
|