6c40c7b91a
# Introduction Close twentyhq/core-team-issues#2641 Auto-provisioned field metadata used to get its `universalIdentifier` from three unrelated sources: random `v4()` on the server when creating custom objects, hardcoded values in `STANDARD_OBJECTS`, and an ad-hoc `v5` derivation in the SDK manifest build. This PR unifies all of them behind the shared `getFieldUniversalIdentifier` derivation: ``` universalIdentifier = f(applicationUniversalIdentifier, objectUniversalIdentifier, fieldName) ``` ## Ownership model The rollout is built on an explicit split of who owns a field's universal identifier: - **The 8 system fields** (`id`, `createdAt`, `updatedAt`, `deletedAt`, `createdBy`, `updatedBy`, `position`, `searchVector`) are **server-owned**. Their universal identifiers are always the deterministic derivation, on **every** application (standard, workspace-custom, installed). Clients cannot provide custom values: a temporary check in `validateObjectMetadataSystemFieldsIntegrity` rejects any non-derived system field identifier at migration build time. This check stands in until system fields are generated exclusively server side by the metadata side-effect engine and stripped from client inputs — at which point it becomes structurally impossible to send one. - **`name` is a default field, not a system field**: it is auto-provisioned when absent (server side for custom objects, SDK side for application objects) but authors can define their own. It is only derived where it is guaranteed to be auto-provisioned. In particular, standard objects keep their **historical hardcoded** `name` identifiers: the standard app authors its `name` fields like any installed app would, and moving those identifiers would break every installed application referencing them (e.g. views on `opportunity.name`). - **User-created and author-provided fields** keep random / explicit identifiers, untouched. ## Server - `validateObjectMetadataSystemFieldsIntegrity` now validates, on top of the existing type/`isSystem` checks, that each system field's `universalIdentifier` equals the deterministic derivation. Runs for every object creation going through the migration orchestrator: app sync, custom object creation, standard provisioning - `build-default-flat-field-metadatas-for-custom-object.util.ts` derives the system field identifiers (and the auto-provisioned `name`) with `getFieldUniversalIdentifier` instead of `v4()` - `build-default-relation-flat-field-metadatas-for-custom-object.util.ts` derives both the forward and the reverse default relation field identifiers deterministically - `generateMorphOrRelationFlatFieldMetadataPair` accepts optional `sourceFieldUniversalIdentifier` / `targetFieldUniversalIdentifier` so callers can inject deterministic values; user-created relations still default to `v4()` ## twenty-shared - `STANDARD_OBJECTS` system field identifiers (the 8) are now computed at module load via `buildStandardObjectSystemFields`; `name` and every other identifier keep their hardcoded values - New snapshot test pinning **every** universal identifier of `STANDARD_OBJECTS`: any identifier change now requires an explicit snapshot update and should ship with a coordinated backfill ## SDK (breaking, pre-GA) - `generateDefaultFieldUniversalIdentifier` delegates to `getFieldUniversalIdentifier` and now requires `applicationUniversalIdentifier` - Reverse default relation field identifiers are derived from the field's real coordinates (standard object UID + actual field name, e.g. `targetRocket` on `attachment`) instead of the legacy custom-object UID + synthetic `${fieldName}Inverse` hash input. Field *names* are unchanged - The manifest build threads the application universal identifier through default field injection (two-pass over object configs) - `twenty dev:add` now resolves the application universal identifier upfront and refuses to scaffold anything until `defineApplication` declares one — no more `fill-later` placeholder for the app UID in generated files ## Upgrade A 2.19 **workspace command** backfills existing `fieldMetadata.universalIdentifier` rows to the deterministic derivation. Coverage follows the ownership model: - **The 8 system fields**: taken over for **every application**, whatever value they currently hold. This is both safe and required now that sync rejects non-derived values — leaving a row unconverged would make its application unsyncable - **`name`**: workspace-custom app → always taken over (server-generated, no author to clobber); installed applications → only rows still carrying the legacy SDK derivation are recomputed, author-provided identifiers are never touched; standard app → never touched (hardcoded in `STANDARD_OBJECTS`) - **Default relation fields**: workspace-custom app → forward fields on custom objects and reverse fields on the standard relation objects; installed applications → legacy-derivation probe only All identifiers of a workspace are updated inside a single transaction, then the command flushes the field-metadata-related workspace caches and bumps the metadata version. Stored `applicationRegistration.manifest` snapshots are intentionally **not** rewritten: installs and upgrades always sync from the `manifest.json` inside the resolved package (npm/tarball), the stored column is only used for display/marketplace purposes. ## Breaking behavior for old packages (fail closed) Packages built with an older SDK carry legacy system field identifiers in their tarball `manifest.json`. Installing or upgrading such a package now fails with an explicit `INVALID_SYSTEM_FIELD` validation error ("universal identifier is not deterministic") instead of silently mismatching against the backfilled rows and triggering a destructive delete+create. The remediation is to rebuild the package with the new SDK; the backfill has already converged the installed rows, so the rebuilt manifest syncs cleanly. ## Test plan - [x] `twenty-sdk` unit tests (526 tests) and typecheck - [x] `twenty-shared` unit tests (1635 tests) including the `STANDARD_OBJECTS` snapshot; `name` identifiers verified byte-for-byte identical to `main` - [x] Lint and typecheck clean on all touched packages - [x] Integration: create a custom object and verify system + default relation field identifiers match the deterministic derivation (`create-one-object-metadata-deterministic-field-universal-identifiers`, 13 assertions passing) - [x] Integration: `failing-sync-application-object-system-fields` extended with a non-derived system field identifier case; all identifiers in the spec pinned deterministically so snapshots embedding expected/actual values are stable across runs (verified with a double run) - [x] Integration: all application sync suites pass with the derived system field identifiers now required by the `buildDefaultObjectManifest` test helper (9 suites, 20 tests) - [x] Full test-database reset: standard app provisioning and seeded workspaces pass the new validation - [x] SDK manifest build verified on the postcard example app: all auto-generated default field identifiers match the derivation - [ ] Run `upgrade:2-19:backfill-deterministic-field-universal-identifiers` (dry-run then real) on a seeded workspace and verify identifier convergence with a rebuilt app manifest
461 lines
14 KiB
TypeScript
461 lines
14 KiB
TypeScript
import { expectOneNotInternalServerErrorSnapshot } from 'test/integration/graphql/utils/expect-one-not-internal-server-error-snapshot.util';
|
|
import { buildBaseManifest } from 'test/integration/metadata/suites/application/utils/build-base-manifest.util';
|
|
import { buildDefaultObjectManifest } from 'test/integration/metadata/suites/application/utils/build-default-object-manifest.util';
|
|
import { cleanupApplicationAndAppRegistration } from 'test/integration/metadata/suites/application/utils/cleanup-application-and-app-registration.util';
|
|
import { setupApplicationForSync } from 'test/integration/metadata/suites/application/utils/setup-application-for-sync.util';
|
|
import { syncApplication } from 'test/integration/metadata/suites/application/utils/sync-application.util';
|
|
import { type Manifest, type ObjectManifest } from 'twenty-shared/application';
|
|
import {
|
|
type EachTestingContext,
|
|
eachTestingContextFilter,
|
|
} from 'twenty-shared/testing';
|
|
import { FieldMetadataType } from 'twenty-shared/types';
|
|
import { v4 as uuidv4, v5 as uuidv5 } from 'uuid';
|
|
|
|
// Identifiers are pinned so validation error messages embedding expected and
|
|
// actual universal identifiers stay stable across snapshot runs.
|
|
const TEST_APP_ID = '4e0e42a8-8f9c-4a48-9d43-5e0c5c2f4a10';
|
|
const TEST_ROLE_ID = 'd0a24fbc-4b26-42a5-a4ff-2e142f8e2f6d';
|
|
const TEST_UUID_NAMESPACE = '6a9c8f74-4b7a-4a86-90f4-2f4dbb1c2a30';
|
|
|
|
const computeDeterministicTestUuid = (seed: string) =>
|
|
uuidv5(seed, TEST_UUID_NAMESPACE);
|
|
|
|
type TestContext = {
|
|
manifest: Manifest;
|
|
};
|
|
|
|
type SyncApplicationTestingContext = EachTestingContext<TestContext>[];
|
|
|
|
const buildManifest = (overrides: Pick<Manifest, 'objects' | 'fields'>) =>
|
|
buildBaseManifest({
|
|
appId: TEST_APP_ID,
|
|
roleId: TEST_ROLE_ID,
|
|
overrides,
|
|
});
|
|
|
|
const buildObjectWithLabelField = ({
|
|
nameSingular,
|
|
namePlural,
|
|
labelSingular,
|
|
labelPlural,
|
|
description,
|
|
additionalFields = [],
|
|
}: {
|
|
nameSingular: string;
|
|
namePlural: string;
|
|
labelSingular: string;
|
|
labelPlural: string;
|
|
description: string;
|
|
additionalFields?: ObjectManifest['fields'];
|
|
}): Pick<Manifest, 'objects' | 'fields'> => {
|
|
const objectId = computeDeterministicTestUuid(nameSingular);
|
|
const labelFieldId = computeDeterministicTestUuid(`${nameSingular}-title`);
|
|
|
|
return {
|
|
objects: [
|
|
{
|
|
universalIdentifier: objectId,
|
|
labelIdentifierFieldMetadataUniversalIdentifier: labelFieldId,
|
|
nameSingular,
|
|
namePlural,
|
|
labelSingular,
|
|
labelPlural,
|
|
description,
|
|
icon: 'IconTicket',
|
|
fields: [
|
|
{
|
|
universalIdentifier: labelFieldId,
|
|
type: FieldMetadataType.TEXT,
|
|
name: 'title',
|
|
label: 'Title',
|
|
description: 'Label identifier field',
|
|
icon: 'IconTextCaption',
|
|
},
|
|
...additionalFields,
|
|
],
|
|
},
|
|
],
|
|
fields: [],
|
|
};
|
|
};
|
|
|
|
const buildDefaultObjectWithModifiedSearchVector = ({
|
|
nameSingular,
|
|
namePlural,
|
|
labelSingular,
|
|
labelPlural,
|
|
description,
|
|
searchVectorOverrides,
|
|
}: {
|
|
nameSingular: string;
|
|
namePlural: string;
|
|
labelSingular: string;
|
|
labelPlural: string;
|
|
description: string;
|
|
searchVectorOverrides: Partial<ObjectManifest['fields'][number]>;
|
|
}): Pick<Manifest, 'objects' | 'fields'> => {
|
|
const defaultObject = buildDefaultObjectManifest({
|
|
applicationUniversalIdentifier: TEST_APP_ID,
|
|
universalIdentifier: computeDeterministicTestUuid(nameSingular),
|
|
nameSingular,
|
|
namePlural,
|
|
labelSingular,
|
|
labelPlural,
|
|
description,
|
|
});
|
|
|
|
return {
|
|
objects: [
|
|
{
|
|
...defaultObject,
|
|
fields: defaultObject.fields.map((field) =>
|
|
field.name === 'searchVector'
|
|
? ({
|
|
...field,
|
|
...searchVectorOverrides,
|
|
} as (typeof defaultObject.fields)[number])
|
|
: field,
|
|
),
|
|
},
|
|
],
|
|
fields: [],
|
|
};
|
|
};
|
|
|
|
const failingSyncApplicationSystemFieldsTestCases: SyncApplicationTestingContext =
|
|
[
|
|
{
|
|
title:
|
|
'when object is created without any system fields (missing all 8 system fields)',
|
|
context: {
|
|
manifest: buildManifest(
|
|
buildObjectWithLabelField({
|
|
nameSingular: 'noSystemFieldsObject',
|
|
namePlural: 'noSystemFieldsObjects',
|
|
labelSingular: 'No System Fields Object',
|
|
labelPlural: 'No System Fields Objects',
|
|
description: 'Object with no system fields',
|
|
}),
|
|
),
|
|
},
|
|
},
|
|
{
|
|
title: 'when object has id field with wrong type (TEXT instead of UUID)',
|
|
context: {
|
|
manifest: buildManifest(
|
|
buildObjectWithLabelField({
|
|
nameSingular: 'wrongIdTypeObject',
|
|
namePlural: 'wrongIdTypeObjects',
|
|
labelSingular: 'Wrong Id Type Object',
|
|
labelPlural: 'Wrong Id Type Objects',
|
|
description: 'Object with wrong id field type',
|
|
additionalFields: [
|
|
{
|
|
universalIdentifier:
|
|
computeDeterministicTestUuid('wrongIdTypeObject-id'),
|
|
type: FieldMetadataType.TEXT,
|
|
name: 'id',
|
|
label: 'Id',
|
|
description: 'Id field with wrong type',
|
|
icon: 'IconKey',
|
|
},
|
|
],
|
|
}),
|
|
),
|
|
},
|
|
},
|
|
{
|
|
title: 'when object miss default fields',
|
|
context: {
|
|
manifest: buildManifest(
|
|
buildObjectWithLabelField({
|
|
nameSingular: 'wrongCreatedAtObject',
|
|
namePlural: 'wrongCreatedAtObjects',
|
|
labelSingular: 'Wrong CreatedAt Object',
|
|
labelPlural: 'Wrong CreatedAt Objects',
|
|
description: 'Object with wrong createdAt field type',
|
|
}),
|
|
),
|
|
},
|
|
},
|
|
{
|
|
title:
|
|
'when object has position field with wrong type (TEXT instead of POSITION)',
|
|
context: {
|
|
manifest: buildManifest(
|
|
buildObjectWithLabelField({
|
|
nameSingular: 'wrongPositionObject',
|
|
namePlural: 'wrongPositionObjects',
|
|
labelSingular: 'Wrong Position Object',
|
|
labelPlural: 'Wrong Position Objects',
|
|
description: 'Object with wrong position field type',
|
|
additionalFields: [
|
|
{
|
|
universalIdentifier: computeDeterministicTestUuid(
|
|
'wrongPositionObject-position',
|
|
),
|
|
type: FieldMetadataType.TEXT,
|
|
name: 'position',
|
|
label: 'Position',
|
|
description: 'Position field with wrong type',
|
|
icon: 'IconArrowsSort',
|
|
},
|
|
],
|
|
}),
|
|
),
|
|
},
|
|
},
|
|
{
|
|
title:
|
|
'when object has searchVector field with wrong type (TEXT instead of TS_VECTOR)',
|
|
context: {
|
|
manifest: buildManifest(
|
|
buildDefaultObjectWithModifiedSearchVector({
|
|
nameSingular: 'wrongSearchVectorType',
|
|
namePlural: 'wrongSearchVectorTypes',
|
|
labelSingular: 'Wrong SearchVector Type',
|
|
labelPlural: 'Wrong SearchVector Types',
|
|
description: 'Object with wrong searchVector field type',
|
|
searchVectorOverrides: {
|
|
type: FieldMetadataType.TEXT,
|
|
},
|
|
}),
|
|
),
|
|
},
|
|
},
|
|
{
|
|
title:
|
|
'when object has TS_VECTOR field with wrong name (not searchVector)',
|
|
context: {
|
|
manifest: buildManifest(
|
|
buildDefaultObjectWithModifiedSearchVector({
|
|
nameSingular: 'wrongTsVectorName',
|
|
namePlural: 'wrongTsVectorNames',
|
|
labelSingular: 'Wrong TsVector Name',
|
|
labelPlural: 'Wrong TsVector Names',
|
|
description: 'Object with TS_VECTOR field named incorrectly',
|
|
searchVectorOverrides: {
|
|
name: 'wrongSearchVector',
|
|
},
|
|
}),
|
|
),
|
|
},
|
|
},
|
|
{
|
|
title:
|
|
'when object has a system field with a custom (non-derived) universal identifier',
|
|
context: (() => {
|
|
const defaultObject = buildDefaultObjectManifest({
|
|
applicationUniversalIdentifier: TEST_APP_ID,
|
|
universalIdentifier: computeDeterministicTestUuid(
|
|
'customSystemFieldUidObject',
|
|
),
|
|
nameSingular: 'customSystemFieldUidObject',
|
|
namePlural: 'customSystemFieldUidObjects',
|
|
labelSingular: 'Custom System Field Uid Object',
|
|
labelPlural: 'Custom System Field Uid Objects',
|
|
description:
|
|
'Object with a createdAt system field carrying a custom universal identifier',
|
|
});
|
|
|
|
return {
|
|
manifest: buildManifest({
|
|
objects: [
|
|
{
|
|
...defaultObject,
|
|
fields: defaultObject.fields.map((field) =>
|
|
field.name === 'createdAt'
|
|
? {
|
|
...field,
|
|
universalIdentifier:
|
|
'899ac540-3a1f-42cf-9f99-8a55e79d0d9e',
|
|
}
|
|
: field,
|
|
),
|
|
},
|
|
],
|
|
fields: [],
|
|
}),
|
|
};
|
|
})(),
|
|
},
|
|
{
|
|
title:
|
|
'when label identifier is non-searchable type (searchVector has no expression)',
|
|
context: (() => {
|
|
const nonSearchableFieldId = computeDeterministicTestUuid(
|
|
'noSearchVectorExpression-quantity',
|
|
);
|
|
|
|
return {
|
|
manifest: buildManifest({
|
|
objects: [
|
|
buildDefaultObjectManifest({
|
|
applicationUniversalIdentifier: TEST_APP_ID,
|
|
universalIdentifier: computeDeterministicTestUuid(
|
|
'noSearchVectorExpression',
|
|
),
|
|
nameSingular: 'noSearchVectorExpression',
|
|
namePlural: 'noSearchVectorExpressions',
|
|
labelSingular: 'No SearchVector Expression',
|
|
labelPlural: 'No SearchVector Expressions',
|
|
description:
|
|
'Object whose label identifier is non-searchable, so searchVector has no expression',
|
|
labelIdentifierFieldMetadataUniversalIdentifier:
|
|
nonSearchableFieldId,
|
|
additionalFields: [
|
|
{
|
|
universalIdentifier: nonSearchableFieldId,
|
|
type: FieldMetadataType.NUMBER,
|
|
name: 'quantity',
|
|
label: 'Quantity',
|
|
},
|
|
],
|
|
}),
|
|
],
|
|
fields: [],
|
|
}),
|
|
};
|
|
})(),
|
|
},
|
|
];
|
|
|
|
describe('Sync application should fail due to object system fields integrity', () => {
|
|
beforeAll(async () => {
|
|
await setupApplicationForSync({
|
|
applicationUniversalIdentifier: TEST_APP_ID,
|
|
name: 'Test System Fields App',
|
|
description: 'App for testing system field validation',
|
|
sourcePath: 'test-system-fields',
|
|
});
|
|
}, 60000);
|
|
|
|
afterAll(async () => {
|
|
await cleanupApplicationAndAppRegistration({
|
|
applicationUniversalIdentifier: TEST_APP_ID,
|
|
});
|
|
});
|
|
|
|
it.each(
|
|
eachTestingContextFilter(failingSyncApplicationSystemFieldsTestCases),
|
|
)(
|
|
'$title',
|
|
async ({ context }) => {
|
|
const { errors } = await syncApplication({
|
|
manifest: context.manifest,
|
|
expectToFail: true,
|
|
});
|
|
|
|
expectOneNotInternalServerErrorSnapshot({ errors });
|
|
},
|
|
60000,
|
|
);
|
|
|
|
it('should fail when trying to delete a system field after a successful sync', async () => {
|
|
const labelIdentifierFieldUniversalIdentifier = uuidv4();
|
|
const testObject = buildDefaultObjectManifest({
|
|
applicationUniversalIdentifier: TEST_APP_ID,
|
|
nameSingular: 'deleteSystemFieldObject',
|
|
namePlural: 'deleteSystemFieldObjects',
|
|
labelSingular: 'Delete System Field Object',
|
|
labelPlural: 'Delete System Field Objects',
|
|
description: 'Object for testing system field deletion',
|
|
labelIdentifierFieldMetadataUniversalIdentifier:
|
|
labelIdentifierFieldUniversalIdentifier,
|
|
additionalFields: [
|
|
{
|
|
universalIdentifier: labelIdentifierFieldUniversalIdentifier,
|
|
type: FieldMetadataType.TEXT,
|
|
name: 'labelIdentifierField',
|
|
label: 'Label Identifier Field',
|
|
description: 'Label identifier field',
|
|
icon: 'IconTextCaption',
|
|
},
|
|
],
|
|
});
|
|
|
|
const validManifest = buildManifest({
|
|
objects: [testObject],
|
|
fields: [],
|
|
});
|
|
|
|
await syncApplication({
|
|
manifest: validManifest,
|
|
expectToFail: false,
|
|
});
|
|
|
|
const manifestWithDeletedIdField = buildManifest({
|
|
objects: [
|
|
{
|
|
...testObject,
|
|
fields: testObject.fields.filter((field) => field.name !== 'id'),
|
|
},
|
|
],
|
|
fields: [],
|
|
});
|
|
|
|
const { errors } = await syncApplication({
|
|
manifest: manifestWithDeletedIdField,
|
|
expectToFail: true,
|
|
});
|
|
|
|
expectOneNotInternalServerErrorSnapshot({ errors });
|
|
}, 60000);
|
|
|
|
it('should fail when trying to update a system field after a successful sync', async () => {
|
|
const labelIdentifierFieldUniversalIdentifier = uuidv4();
|
|
const testObject = buildDefaultObjectManifest({
|
|
applicationUniversalIdentifier: TEST_APP_ID,
|
|
nameSingular: 'updateSystemFieldObject',
|
|
namePlural: 'updateSystemFieldObjects',
|
|
labelSingular: 'Update System Field Object',
|
|
labelPlural: 'Update System Field Objects',
|
|
description: 'Object for testing system field update',
|
|
labelIdentifierFieldMetadataUniversalIdentifier:
|
|
labelIdentifierFieldUniversalIdentifier,
|
|
additionalFields: [
|
|
{
|
|
universalIdentifier: labelIdentifierFieldUniversalIdentifier,
|
|
type: FieldMetadataType.TEXT,
|
|
name: 'labelIdentifierField',
|
|
label: 'Label Identifier Field',
|
|
description: 'Label identifier field',
|
|
icon: 'IconTextCaption',
|
|
},
|
|
],
|
|
});
|
|
|
|
const validManifest = buildManifest({
|
|
objects: [testObject],
|
|
fields: [],
|
|
});
|
|
|
|
await syncApplication({
|
|
manifest: validManifest,
|
|
expectToFail: false,
|
|
});
|
|
|
|
const manifestWithUpdatedIdField = buildManifest({
|
|
objects: [
|
|
{
|
|
...testObject,
|
|
fields: testObject.fields.map((field) =>
|
|
field.name === 'id'
|
|
? { ...field, label: 'Modified Id Label' }
|
|
: field,
|
|
),
|
|
},
|
|
],
|
|
fields: [],
|
|
});
|
|
|
|
const { errors } = await syncApplication({
|
|
manifest: manifestWithUpdatedIdField,
|
|
expectToFail: true,
|
|
});
|
|
|
|
expectOneNotInternalServerErrorSnapshot({ errors });
|
|
}, 60000);
|
|
});
|