Files
twenty/packages/twenty-server/test/integration/metadata/suites/application/failing-sync-application-object-system-fields.integration-spec.ts
T
Paul Rastoin 6c40c7b91a Deterministic system field universal identifier (#22565)
# Introduction

Close twentyhq/core-team-issues#2641

Auto-provisioned field metadata used to get its `universalIdentifier`
from three unrelated sources: random `v4()` on the server when creating
custom objects, hardcoded values in `STANDARD_OBJECTS`, and an ad-hoc
`v5` derivation in the SDK manifest build. This PR unifies all of them
behind the shared `getFieldUniversalIdentifier` derivation:

```
universalIdentifier = f(applicationUniversalIdentifier, objectUniversalIdentifier, fieldName)
```

## Ownership model

The rollout is built on an explicit split of who owns a field's
universal identifier:

- **The 8 system fields** (`id`, `createdAt`, `updatedAt`, `deletedAt`,
`createdBy`, `updatedBy`, `position`, `searchVector`) are
**server-owned**. Their universal identifiers are always the
deterministic derivation, on **every** application (standard,
workspace-custom, installed). Clients cannot provide custom values: a
temporary check in `validateObjectMetadataSystemFieldsIntegrity` rejects
any non-derived system field identifier at migration build time. This
check stands in until system fields are generated exclusively server
side by the metadata side-effect engine and stripped from client inputs
— at which point it becomes structurally impossible to send one.
- **`name` is a default field, not a system field**: it is
auto-provisioned when absent (server side for custom objects, SDK side
for application objects) but authors can define their own. It is only
derived where it is guaranteed to be auto-provisioned. In particular,
standard objects keep their **historical hardcoded** `name` identifiers:
the standard app authors its `name` fields like any installed app would,
and moving those identifiers would break every installed application
referencing them (e.g. views on `opportunity.name`).
- **User-created and author-provided fields** keep random / explicit
identifiers, untouched.

## Server

- `validateObjectMetadataSystemFieldsIntegrity` now validates, on top of
the existing type/`isSystem` checks, that each system field's
`universalIdentifier` equals the deterministic derivation. Runs for
every object creation going through the migration orchestrator: app
sync, custom object creation, standard provisioning
- `build-default-flat-field-metadatas-for-custom-object.util.ts` derives
the system field identifiers (and the auto-provisioned `name`) with
`getFieldUniversalIdentifier` instead of `v4()`
-
`build-default-relation-flat-field-metadatas-for-custom-object.util.ts`
derives both the forward and the reverse default relation field
identifiers deterministically
- `generateMorphOrRelationFlatFieldMetadataPair` accepts optional
`sourceFieldUniversalIdentifier` / `targetFieldUniversalIdentifier` so
callers can inject deterministic values; user-created relations still
default to `v4()`

## twenty-shared

- `STANDARD_OBJECTS` system field identifiers (the 8) are now computed
at module load via `buildStandardObjectSystemFields`; `name` and every
other identifier keep their hardcoded values
- New snapshot test pinning **every** universal identifier of
`STANDARD_OBJECTS`: any identifier change now requires an explicit
snapshot update and should ship with a coordinated backfill

## SDK (breaking, pre-GA)

- `generateDefaultFieldUniversalIdentifier` delegates to
`getFieldUniversalIdentifier` and now requires
`applicationUniversalIdentifier`
- Reverse default relation field identifiers are derived from the
field's real coordinates (standard object UID + actual field name, e.g.
`targetRocket` on `attachment`) instead of the legacy custom-object UID
+ synthetic `${fieldName}Inverse` hash input. Field *names* are
unchanged
- The manifest build threads the application universal identifier
through default field injection (two-pass over object configs)
- `twenty dev:add` now resolves the application universal identifier
upfront and refuses to scaffold anything until `defineApplication`
declares one — no more `fill-later` placeholder for the app UID in
generated files

## Upgrade

A 2.19 **workspace command** backfills existing
`fieldMetadata.universalIdentifier` rows to the deterministic
derivation. Coverage follows the ownership model:

- **The 8 system fields**: taken over for **every application**,
whatever value they currently hold. This is both safe and required now
that sync rejects non-derived values — leaving a row unconverged would
make its application unsyncable
- **`name`**: workspace-custom app → always taken over
(server-generated, no author to clobber); installed applications → only
rows still carrying the legacy SDK derivation are recomputed,
author-provided identifiers are never touched; standard app → never
touched (hardcoded in `STANDARD_OBJECTS`)
- **Default relation fields**: workspace-custom app → forward fields on
custom objects and reverse fields on the standard relation objects;
installed applications → legacy-derivation probe only

All identifiers of a workspace are updated inside a single transaction,
then the command flushes the field-metadata-related workspace caches and
bumps the metadata version.

Stored `applicationRegistration.manifest` snapshots are intentionally
**not** rewritten: installs and upgrades always sync from the
`manifest.json` inside the resolved package (npm/tarball), the stored
column is only used for display/marketplace purposes.

## Breaking behavior for old packages (fail closed)

Packages built with an older SDK carry legacy system field identifiers
in their tarball `manifest.json`. Installing or upgrading such a package
now fails with an explicit `INVALID_SYSTEM_FIELD` validation error
("universal identifier is not deterministic") instead of silently
mismatching against the backfilled rows and triggering a destructive
delete+create. The remediation is to rebuild the package with the new
SDK; the backfill has already converged the installed rows, so the
rebuilt manifest syncs cleanly.

## Test plan

- [x] `twenty-sdk` unit tests (526 tests) and typecheck
- [x] `twenty-shared` unit tests (1635 tests) including the
`STANDARD_OBJECTS` snapshot; `name` identifiers verified byte-for-byte
identical to `main`
- [x] Lint and typecheck clean on all touched packages
- [x] Integration: create a custom object and verify system + default
relation field identifiers match the deterministic derivation
(`create-one-object-metadata-deterministic-field-universal-identifiers`,
13 assertions passing)
- [x] Integration: `failing-sync-application-object-system-fields`
extended with a non-derived system field identifier case; all
identifiers in the spec pinned deterministically so snapshots embedding
expected/actual values are stable across runs (verified with a double
run)
- [x] Integration: all application sync suites pass with the derived
system field identifiers now required by the
`buildDefaultObjectManifest` test helper (9 suites, 20 tests)
- [x] Full test-database reset: standard app provisioning and seeded
workspaces pass the new validation
- [x] SDK manifest build verified on the postcard example app: all
auto-generated default field identifiers match the derivation
- [ ] Run
`upgrade:2-19:backfill-deterministic-field-universal-identifiers`
(dry-run then real) on a seeded workspace and verify identifier
convergence with a rebuilt app manifest
2026-07-06 13:34:33 +00:00

461 lines
14 KiB
TypeScript

import { expectOneNotInternalServerErrorSnapshot } from 'test/integration/graphql/utils/expect-one-not-internal-server-error-snapshot.util';
import { buildBaseManifest } from 'test/integration/metadata/suites/application/utils/build-base-manifest.util';
import { buildDefaultObjectManifest } from 'test/integration/metadata/suites/application/utils/build-default-object-manifest.util';
import { cleanupApplicationAndAppRegistration } from 'test/integration/metadata/suites/application/utils/cleanup-application-and-app-registration.util';
import { setupApplicationForSync } from 'test/integration/metadata/suites/application/utils/setup-application-for-sync.util';
import { syncApplication } from 'test/integration/metadata/suites/application/utils/sync-application.util';
import { type Manifest, type ObjectManifest } from 'twenty-shared/application';
import {
type EachTestingContext,
eachTestingContextFilter,
} from 'twenty-shared/testing';
import { FieldMetadataType } from 'twenty-shared/types';
import { v4 as uuidv4, v5 as uuidv5 } from 'uuid';
// Identifiers are pinned so validation error messages embedding expected and
// actual universal identifiers stay stable across snapshot runs.
const TEST_APP_ID = '4e0e42a8-8f9c-4a48-9d43-5e0c5c2f4a10';
const TEST_ROLE_ID = 'd0a24fbc-4b26-42a5-a4ff-2e142f8e2f6d';
const TEST_UUID_NAMESPACE = '6a9c8f74-4b7a-4a86-90f4-2f4dbb1c2a30';
const computeDeterministicTestUuid = (seed: string) =>
uuidv5(seed, TEST_UUID_NAMESPACE);
type TestContext = {
manifest: Manifest;
};
type SyncApplicationTestingContext = EachTestingContext<TestContext>[];
const buildManifest = (overrides: Pick<Manifest, 'objects' | 'fields'>) =>
buildBaseManifest({
appId: TEST_APP_ID,
roleId: TEST_ROLE_ID,
overrides,
});
const buildObjectWithLabelField = ({
nameSingular,
namePlural,
labelSingular,
labelPlural,
description,
additionalFields = [],
}: {
nameSingular: string;
namePlural: string;
labelSingular: string;
labelPlural: string;
description: string;
additionalFields?: ObjectManifest['fields'];
}): Pick<Manifest, 'objects' | 'fields'> => {
const objectId = computeDeterministicTestUuid(nameSingular);
const labelFieldId = computeDeterministicTestUuid(`${nameSingular}-title`);
return {
objects: [
{
universalIdentifier: objectId,
labelIdentifierFieldMetadataUniversalIdentifier: labelFieldId,
nameSingular,
namePlural,
labelSingular,
labelPlural,
description,
icon: 'IconTicket',
fields: [
{
universalIdentifier: labelFieldId,
type: FieldMetadataType.TEXT,
name: 'title',
label: 'Title',
description: 'Label identifier field',
icon: 'IconTextCaption',
},
...additionalFields,
],
},
],
fields: [],
};
};
const buildDefaultObjectWithModifiedSearchVector = ({
nameSingular,
namePlural,
labelSingular,
labelPlural,
description,
searchVectorOverrides,
}: {
nameSingular: string;
namePlural: string;
labelSingular: string;
labelPlural: string;
description: string;
searchVectorOverrides: Partial<ObjectManifest['fields'][number]>;
}): Pick<Manifest, 'objects' | 'fields'> => {
const defaultObject = buildDefaultObjectManifest({
applicationUniversalIdentifier: TEST_APP_ID,
universalIdentifier: computeDeterministicTestUuid(nameSingular),
nameSingular,
namePlural,
labelSingular,
labelPlural,
description,
});
return {
objects: [
{
...defaultObject,
fields: defaultObject.fields.map((field) =>
field.name === 'searchVector'
? ({
...field,
...searchVectorOverrides,
} as (typeof defaultObject.fields)[number])
: field,
),
},
],
fields: [],
};
};
const failingSyncApplicationSystemFieldsTestCases: SyncApplicationTestingContext =
[
{
title:
'when object is created without any system fields (missing all 8 system fields)',
context: {
manifest: buildManifest(
buildObjectWithLabelField({
nameSingular: 'noSystemFieldsObject',
namePlural: 'noSystemFieldsObjects',
labelSingular: 'No System Fields Object',
labelPlural: 'No System Fields Objects',
description: 'Object with no system fields',
}),
),
},
},
{
title: 'when object has id field with wrong type (TEXT instead of UUID)',
context: {
manifest: buildManifest(
buildObjectWithLabelField({
nameSingular: 'wrongIdTypeObject',
namePlural: 'wrongIdTypeObjects',
labelSingular: 'Wrong Id Type Object',
labelPlural: 'Wrong Id Type Objects',
description: 'Object with wrong id field type',
additionalFields: [
{
universalIdentifier:
computeDeterministicTestUuid('wrongIdTypeObject-id'),
type: FieldMetadataType.TEXT,
name: 'id',
label: 'Id',
description: 'Id field with wrong type',
icon: 'IconKey',
},
],
}),
),
},
},
{
title: 'when object miss default fields',
context: {
manifest: buildManifest(
buildObjectWithLabelField({
nameSingular: 'wrongCreatedAtObject',
namePlural: 'wrongCreatedAtObjects',
labelSingular: 'Wrong CreatedAt Object',
labelPlural: 'Wrong CreatedAt Objects',
description: 'Object with wrong createdAt field type',
}),
),
},
},
{
title:
'when object has position field with wrong type (TEXT instead of POSITION)',
context: {
manifest: buildManifest(
buildObjectWithLabelField({
nameSingular: 'wrongPositionObject',
namePlural: 'wrongPositionObjects',
labelSingular: 'Wrong Position Object',
labelPlural: 'Wrong Position Objects',
description: 'Object with wrong position field type',
additionalFields: [
{
universalIdentifier: computeDeterministicTestUuid(
'wrongPositionObject-position',
),
type: FieldMetadataType.TEXT,
name: 'position',
label: 'Position',
description: 'Position field with wrong type',
icon: 'IconArrowsSort',
},
],
}),
),
},
},
{
title:
'when object has searchVector field with wrong type (TEXT instead of TS_VECTOR)',
context: {
manifest: buildManifest(
buildDefaultObjectWithModifiedSearchVector({
nameSingular: 'wrongSearchVectorType',
namePlural: 'wrongSearchVectorTypes',
labelSingular: 'Wrong SearchVector Type',
labelPlural: 'Wrong SearchVector Types',
description: 'Object with wrong searchVector field type',
searchVectorOverrides: {
type: FieldMetadataType.TEXT,
},
}),
),
},
},
{
title:
'when object has TS_VECTOR field with wrong name (not searchVector)',
context: {
manifest: buildManifest(
buildDefaultObjectWithModifiedSearchVector({
nameSingular: 'wrongTsVectorName',
namePlural: 'wrongTsVectorNames',
labelSingular: 'Wrong TsVector Name',
labelPlural: 'Wrong TsVector Names',
description: 'Object with TS_VECTOR field named incorrectly',
searchVectorOverrides: {
name: 'wrongSearchVector',
},
}),
),
},
},
{
title:
'when object has a system field with a custom (non-derived) universal identifier',
context: (() => {
const defaultObject = buildDefaultObjectManifest({
applicationUniversalIdentifier: TEST_APP_ID,
universalIdentifier: computeDeterministicTestUuid(
'customSystemFieldUidObject',
),
nameSingular: 'customSystemFieldUidObject',
namePlural: 'customSystemFieldUidObjects',
labelSingular: 'Custom System Field Uid Object',
labelPlural: 'Custom System Field Uid Objects',
description:
'Object with a createdAt system field carrying a custom universal identifier',
});
return {
manifest: buildManifest({
objects: [
{
...defaultObject,
fields: defaultObject.fields.map((field) =>
field.name === 'createdAt'
? {
...field,
universalIdentifier:
'899ac540-3a1f-42cf-9f99-8a55e79d0d9e',
}
: field,
),
},
],
fields: [],
}),
};
})(),
},
{
title:
'when label identifier is non-searchable type (searchVector has no expression)',
context: (() => {
const nonSearchableFieldId = computeDeterministicTestUuid(
'noSearchVectorExpression-quantity',
);
return {
manifest: buildManifest({
objects: [
buildDefaultObjectManifest({
applicationUniversalIdentifier: TEST_APP_ID,
universalIdentifier: computeDeterministicTestUuid(
'noSearchVectorExpression',
),
nameSingular: 'noSearchVectorExpression',
namePlural: 'noSearchVectorExpressions',
labelSingular: 'No SearchVector Expression',
labelPlural: 'No SearchVector Expressions',
description:
'Object whose label identifier is non-searchable, so searchVector has no expression',
labelIdentifierFieldMetadataUniversalIdentifier:
nonSearchableFieldId,
additionalFields: [
{
universalIdentifier: nonSearchableFieldId,
type: FieldMetadataType.NUMBER,
name: 'quantity',
label: 'Quantity',
},
],
}),
],
fields: [],
}),
};
})(),
},
];
describe('Sync application should fail due to object system fields integrity', () => {
beforeAll(async () => {
await setupApplicationForSync({
applicationUniversalIdentifier: TEST_APP_ID,
name: 'Test System Fields App',
description: 'App for testing system field validation',
sourcePath: 'test-system-fields',
});
}, 60000);
afterAll(async () => {
await cleanupApplicationAndAppRegistration({
applicationUniversalIdentifier: TEST_APP_ID,
});
});
it.each(
eachTestingContextFilter(failingSyncApplicationSystemFieldsTestCases),
)(
'$title',
async ({ context }) => {
const { errors } = await syncApplication({
manifest: context.manifest,
expectToFail: true,
});
expectOneNotInternalServerErrorSnapshot({ errors });
},
60000,
);
it('should fail when trying to delete a system field after a successful sync', async () => {
const labelIdentifierFieldUniversalIdentifier = uuidv4();
const testObject = buildDefaultObjectManifest({
applicationUniversalIdentifier: TEST_APP_ID,
nameSingular: 'deleteSystemFieldObject',
namePlural: 'deleteSystemFieldObjects',
labelSingular: 'Delete System Field Object',
labelPlural: 'Delete System Field Objects',
description: 'Object for testing system field deletion',
labelIdentifierFieldMetadataUniversalIdentifier:
labelIdentifierFieldUniversalIdentifier,
additionalFields: [
{
universalIdentifier: labelIdentifierFieldUniversalIdentifier,
type: FieldMetadataType.TEXT,
name: 'labelIdentifierField',
label: 'Label Identifier Field',
description: 'Label identifier field',
icon: 'IconTextCaption',
},
],
});
const validManifest = buildManifest({
objects: [testObject],
fields: [],
});
await syncApplication({
manifest: validManifest,
expectToFail: false,
});
const manifestWithDeletedIdField = buildManifest({
objects: [
{
...testObject,
fields: testObject.fields.filter((field) => field.name !== 'id'),
},
],
fields: [],
});
const { errors } = await syncApplication({
manifest: manifestWithDeletedIdField,
expectToFail: true,
});
expectOneNotInternalServerErrorSnapshot({ errors });
}, 60000);
it('should fail when trying to update a system field after a successful sync', async () => {
const labelIdentifierFieldUniversalIdentifier = uuidv4();
const testObject = buildDefaultObjectManifest({
applicationUniversalIdentifier: TEST_APP_ID,
nameSingular: 'updateSystemFieldObject',
namePlural: 'updateSystemFieldObjects',
labelSingular: 'Update System Field Object',
labelPlural: 'Update System Field Objects',
description: 'Object for testing system field update',
labelIdentifierFieldMetadataUniversalIdentifier:
labelIdentifierFieldUniversalIdentifier,
additionalFields: [
{
universalIdentifier: labelIdentifierFieldUniversalIdentifier,
type: FieldMetadataType.TEXT,
name: 'labelIdentifierField',
label: 'Label Identifier Field',
description: 'Label identifier field',
icon: 'IconTextCaption',
},
],
});
const validManifest = buildManifest({
objects: [testObject],
fields: [],
});
await syncApplication({
manifest: validManifest,
expectToFail: false,
});
const manifestWithUpdatedIdField = buildManifest({
objects: [
{
...testObject,
fields: testObject.fields.map((field) =>
field.name === 'id'
? { ...field, label: 'Modified Id Label' }
: field,
),
},
],
fields: [],
});
const { errors } = await syncApplication({
manifest: manifestWithUpdatedIdField,
expectToFail: true,
});
expectOneNotInternalServerErrorSnapshot({ errors });
}, 60000);
});