Files
twenty/packages/twenty-server/test/integration/metadata/suites/application/successful-sync-application-workspace-migration.integration-spec.ts
T
Paul Rastoin 6c40c7b91a Deterministic system field universal identifier (#22565)
# Introduction

Close twentyhq/core-team-issues#2641

Auto-provisioned field metadata used to get its `universalIdentifier`
from three unrelated sources: random `v4()` on the server when creating
custom objects, hardcoded values in `STANDARD_OBJECTS`, and an ad-hoc
`v5` derivation in the SDK manifest build. This PR unifies all of them
behind the shared `getFieldUniversalIdentifier` derivation:

```
universalIdentifier = f(applicationUniversalIdentifier, objectUniversalIdentifier, fieldName)
```

## Ownership model

The rollout is built on an explicit split of who owns a field's
universal identifier:

- **The 8 system fields** (`id`, `createdAt`, `updatedAt`, `deletedAt`,
`createdBy`, `updatedBy`, `position`, `searchVector`) are
**server-owned**. Their universal identifiers are always the
deterministic derivation, on **every** application (standard,
workspace-custom, installed). Clients cannot provide custom values: a
temporary check in `validateObjectMetadataSystemFieldsIntegrity` rejects
any non-derived system field identifier at migration build time. This
check stands in until system fields are generated exclusively server
side by the metadata side-effect engine and stripped from client inputs
— at which point it becomes structurally impossible to send one.
- **`name` is a default field, not a system field**: it is
auto-provisioned when absent (server side for custom objects, SDK side
for application objects) but authors can define their own. It is only
derived where it is guaranteed to be auto-provisioned. In particular,
standard objects keep their **historical hardcoded** `name` identifiers:
the standard app authors its `name` fields like any installed app would,
and moving those identifiers would break every installed application
referencing them (e.g. views on `opportunity.name`).
- **User-created and author-provided fields** keep random / explicit
identifiers, untouched.

## Server

- `validateObjectMetadataSystemFieldsIntegrity` now validates, on top of
the existing type/`isSystem` checks, that each system field's
`universalIdentifier` equals the deterministic derivation. Runs for
every object creation going through the migration orchestrator: app
sync, custom object creation, standard provisioning
- `build-default-flat-field-metadatas-for-custom-object.util.ts` derives
the system field identifiers (and the auto-provisioned `name`) with
`getFieldUniversalIdentifier` instead of `v4()`
-
`build-default-relation-flat-field-metadatas-for-custom-object.util.ts`
derives both the forward and the reverse default relation field
identifiers deterministically
- `generateMorphOrRelationFlatFieldMetadataPair` accepts optional
`sourceFieldUniversalIdentifier` / `targetFieldUniversalIdentifier` so
callers can inject deterministic values; user-created relations still
default to `v4()`

## twenty-shared

- `STANDARD_OBJECTS` system field identifiers (the 8) are now computed
at module load via `buildStandardObjectSystemFields`; `name` and every
other identifier keep their hardcoded values
- New snapshot test pinning **every** universal identifier of
`STANDARD_OBJECTS`: any identifier change now requires an explicit
snapshot update and should ship with a coordinated backfill

## SDK (breaking, pre-GA)

- `generateDefaultFieldUniversalIdentifier` delegates to
`getFieldUniversalIdentifier` and now requires
`applicationUniversalIdentifier`
- Reverse default relation field identifiers are derived from the
field's real coordinates (standard object UID + actual field name, e.g.
`targetRocket` on `attachment`) instead of the legacy custom-object UID
+ synthetic `${fieldName}Inverse` hash input. Field *names* are
unchanged
- The manifest build threads the application universal identifier
through default field injection (two-pass over object configs)
- `twenty dev:add` now resolves the application universal identifier
upfront and refuses to scaffold anything until `defineApplication`
declares one — no more `fill-later` placeholder for the app UID in
generated files

## Upgrade

A 2.19 **workspace command** backfills existing
`fieldMetadata.universalIdentifier` rows to the deterministic
derivation. Coverage follows the ownership model:

- **The 8 system fields**: taken over for **every application**,
whatever value they currently hold. This is both safe and required now
that sync rejects non-derived values — leaving a row unconverged would
make its application unsyncable
- **`name`**: workspace-custom app → always taken over
(server-generated, no author to clobber); installed applications → only
rows still carrying the legacy SDK derivation are recomputed,
author-provided identifiers are never touched; standard app → never
touched (hardcoded in `STANDARD_OBJECTS`)
- **Default relation fields**: workspace-custom app → forward fields on
custom objects and reverse fields on the standard relation objects;
installed applications → legacy-derivation probe only

All identifiers of a workspace are updated inside a single transaction,
then the command flushes the field-metadata-related workspace caches and
bumps the metadata version.

Stored `applicationRegistration.manifest` snapshots are intentionally
**not** rewritten: installs and upgrades always sync from the
`manifest.json` inside the resolved package (npm/tarball), the stored
column is only used for display/marketplace purposes.

## Breaking behavior for old packages (fail closed)

Packages built with an older SDK carry legacy system field identifiers
in their tarball `manifest.json`. Installing or upgrading such a package
now fails with an explicit `INVALID_SYSTEM_FIELD` validation error
("universal identifier is not deterministic") instead of silently
mismatching against the backfilled rows and triggering a destructive
delete+create. The remediation is to rebuild the package with the new
SDK; the backfill has already converged the installed rows, so the
rebuilt manifest syncs cleanly.

## Test plan

- [x] `twenty-sdk` unit tests (526 tests) and typecheck
- [x] `twenty-shared` unit tests (1635 tests) including the
`STANDARD_OBJECTS` snapshot; `name` identifiers verified byte-for-byte
identical to `main`
- [x] Lint and typecheck clean on all touched packages
- [x] Integration: create a custom object and verify system + default
relation field identifiers match the deterministic derivation
(`create-one-object-metadata-deterministic-field-universal-identifiers`,
13 assertions passing)
- [x] Integration: `failing-sync-application-object-system-fields`
extended with a non-derived system field identifier case; all
identifiers in the spec pinned deterministically so snapshots embedding
expected/actual values are stable across runs (verified with a double
run)
- [x] Integration: all application sync suites pass with the derived
system field identifiers now required by the
`buildDefaultObjectManifest` test helper (9 suites, 20 tests)
- [x] Full test-database reset: standard app provisioning and seeded
workspaces pass the new validation
- [x] SDK manifest build verified on the postcard example app: all
auto-generated default field identifiers match the derivation
- [ ] Run
`upgrade:2-19:backfill-deterministic-field-universal-identifiers`
(dry-run then real) on a seeded workspace and verify identifier
convergence with a rebuilt app manifest
2026-07-06 13:34:33 +00:00

263 lines
8.1 KiB
TypeScript

import { buildBaseManifest } from 'test/integration/metadata/suites/application/utils/build-base-manifest.util';
import { buildDefaultObjectManifest } from 'test/integration/metadata/suites/application/utils/build-default-object-manifest.util';
import { cleanupApplicationAndAppRegistration } from 'test/integration/metadata/suites/application/utils/cleanup-application-and-app-registration.util';
import { setupApplicationForSync } from 'test/integration/metadata/suites/application/utils/setup-application-for-sync.util';
import { syncApplication } from 'test/integration/metadata/suites/application/utils/sync-application.util';
import { findManyObjectMetadataWithIndexes } from 'test/integration/metadata/suites/object-metadata/utils/find-many-object-metadata-with-indexes.util';
import { findManyObjectMetadata } from 'test/integration/metadata/suites/object-metadata/utils/find-many-object-metadata.util';
import { findRoles } from 'test/integration/metadata/suites/role/utils/find-roles.util';
import { findSkills } from 'test/integration/metadata/suites/skill/utils/find-skills.util';
import { extractRecordIdsAndDatesAsExpectAny } from 'test/utils/extract-record-ids-and-dates-as-expect-any';
import { type FieldManifest, type Manifest } from 'twenty-shared/application';
import { STANDARD_OBJECTS } from 'twenty-shared/metadata';
import { FieldMetadataType } from 'twenty-shared/types';
import { v4 as uuidv4 } from 'uuid';
const TEST_APP_ID = uuidv4();
const TEST_ROLE_ID = uuidv4();
const TEST_FIELD_ID = uuidv4();
const TEST_SKILL_ID = uuidv4();
const TEST_OBJECT = buildDefaultObjectManifest({
applicationUniversalIdentifier: TEST_APP_ID,
nameSingular: 'ticket',
namePlural: 'tickets',
labelSingular: 'Ticket',
labelPlural: 'Tickets',
description: 'A support ticket',
icon: 'IconTicket',
});
const TEST_SKILL = {
universalIdentifier: TEST_SKILL_ID,
name: 'test-skill',
label: 'Test Skill',
description: 'A skill for testing',
icon: 'IconBrain',
content: '# Test Skill\n\nThis is a test skill.',
};
const TEST_FIELD: FieldManifest = {
universalIdentifier: TEST_FIELD_ID,
type: FieldMetadataType.TEXT,
name: 'description',
label: 'Description',
description: 'Ticket description',
icon: 'IconFileDescription',
objectUniversalIdentifier: TEST_OBJECT.universalIdentifier,
};
const buildManifest = (
overrides?: Partial<Pick<Manifest, 'fields' | 'skills' | 'objects'>>,
) =>
buildBaseManifest({
appId: TEST_APP_ID,
roleId: TEST_ROLE_ID,
overrides: {
objects: [TEST_OBJECT],
skills: [TEST_SKILL],
fields: [TEST_FIELD],
...overrides,
},
});
describe('syncApplication', () => {
beforeEach(async () => {
await setupApplicationForSync({
applicationUniversalIdentifier: TEST_APP_ID,
name: 'Test Application',
description: 'A test application',
sourcePath: 'test-sync',
});
}, 60000);
afterEach(async () => {
await cleanupApplicationAndAppRegistration({
applicationUniversalIdentifier: TEST_APP_ID,
});
});
it('should return workspace migration actions on initial sync then on second sync with field rename and new role', async () => {
const { data: firstSyncData } = await syncApplication({
manifest: buildManifest(),
expectToFail: false,
});
expect(firstSyncData).toMatchSnapshot(
extractRecordIdsAndDatesAsExpectAny(firstSyncData),
);
// Verify database state after first sync
const { objects: objectsAfterSync } = await findManyObjectMetadata({
input: {
filter: {},
paging: { first: 100 },
},
gqlFields:
'id nameSingular namePlural labelSingular labelPlural description icon',
expectToFail: false,
});
const ticketObject = objectsAfterSync.find(
(obj) => obj.nameSingular === 'ticket',
);
expect(ticketObject).toBeDefined();
expect(ticketObject).toMatchObject({
nameSingular: 'ticket',
namePlural: 'tickets',
labelSingular: 'Ticket',
labelPlural: 'Tickets',
description: 'A support ticket',
icon: 'IconTicket',
});
const objects = await findManyObjectMetadataWithIndexes({
expectToFail: false,
});
const fieldsAfterSync = objects.find(
(o) => o.universalIdentifier === TEST_OBJECT.universalIdentifier,
)?.fieldsList;
const descriptionField = fieldsAfterSync?.find(
(f) => f.universalIdentifier === TEST_FIELD_ID,
);
expect(descriptionField).toBeDefined();
expect(descriptionField).toMatchObject({
name: 'description',
label: 'Description',
type: FieldMetadataType.TEXT,
description: 'Ticket description',
icon: 'IconFileDescription',
});
const { data: rolesAfterSync } = await findRoles({
gqlFields: 'id label description universalIdentifier',
expectToFail: false,
});
const testRole = rolesAfterSync.getRoles.find(
(role) => role.universalIdentifier === TEST_ROLE_ID,
);
const { data: skillsAfterSync } = await findSkills({
gqlFields: 'id name label description content icon',
expectToFail: false,
input: undefined,
});
const testSkill = skillsAfterSync.skills.find(
(skill) => skill.name === 'test-skill',
);
expect(testRole).toBeDefined();
expect(testRole).toMatchObject({
label: 'Test Role',
description: 'A test role',
});
expect(testSkill).toBeDefined();
expect(testSkill).toMatchObject({
name: 'test-skill',
label: 'Test Skill',
description: 'A skill for testing',
icon: 'IconBrain',
content: '# Test Skill\n\nThis is a test skill.',
});
}, 60000);
it('should delete old field and create equivalent one when field universalIdentifier changes', async () => {
const originalFieldId = uuidv4();
const updatedFieldId = uuidv4();
const testObject = buildDefaultObjectManifest({
applicationUniversalIdentifier: TEST_APP_ID,
nameSingular: 'ticket',
namePlural: 'tickets',
labelSingular: 'Ticket',
labelPlural: 'Tickets',
description: 'A support ticket',
icon: 'IconTicket',
});
const baseField: FieldManifest = {
universalIdentifier: originalFieldId,
type: FieldMetadataType.TEXT,
name: 'description',
label: 'Description',
description: 'Ticket description',
icon: 'IconFileDescription',
objectUniversalIdentifier: testObject.universalIdentifier,
};
const { data: firstSyncData } = await syncApplication({
manifest: buildBaseManifest({
appId: TEST_APP_ID,
roleId: TEST_ROLE_ID,
overrides: {
objects: [testObject],
fields: [baseField],
},
}),
expectToFail: false,
});
expect(firstSyncData).toMatchSnapshot(
extractRecordIdsAndDatesAsExpectAny(firstSyncData),
);
const { data: secondSyncData } = await syncApplication({
manifest: buildBaseManifest({
appId: TEST_APP_ID,
roleId: TEST_ROLE_ID,
overrides: {
objects: [testObject],
fields: [
{
...baseField,
universalIdentifier: updatedFieldId,
},
],
},
}),
expectToFail: false,
});
expect(secondSyncData).toMatchSnapshot(
extractRecordIdsAndDatesAsExpectAny(secondSyncData),
);
}, 60000);
it('should create a TEXT field on the standard Company object', async () => {
const companyFieldId = uuidv4();
const manifest = buildManifest({
skills: [],
objects: [],
fields: [
{
universalIdentifier: companyFieldId,
type: FieldMetadataType.TEXT,
name: 'industry',
label: 'Industry',
description: 'The industry of the company',
icon: 'IconBuildingFactory2',
objectUniversalIdentifier:
STANDARD_OBJECTS.company.universalIdentifier,
},
],
});
const { data: syncData } = await syncApplication({
manifest,
expectToFail: false,
});
expect(syncData).toMatchSnapshot(
extractRecordIdsAndDatesAsExpectAny(syncData),
);
}, 60000);
});