Files
twenty/packages/twenty-server/test/integration/metadata/suites/application/failing-sync-application-unique-index-side-effect-collision.integration-spec.ts
T
Paul Rastoin 6c40c7b91a Deterministic system field universal identifier (#22565)
# Introduction

Close twentyhq/core-team-issues#2641

Auto-provisioned field metadata used to get its `universalIdentifier`
from three unrelated sources: random `v4()` on the server when creating
custom objects, hardcoded values in `STANDARD_OBJECTS`, and an ad-hoc
`v5` derivation in the SDK manifest build. This PR unifies all of them
behind the shared `getFieldUniversalIdentifier` derivation:

```
universalIdentifier = f(applicationUniversalIdentifier, objectUniversalIdentifier, fieldName)
```

## Ownership model

The rollout is built on an explicit split of who owns a field's
universal identifier:

- **The 8 system fields** (`id`, `createdAt`, `updatedAt`, `deletedAt`,
`createdBy`, `updatedBy`, `position`, `searchVector`) are
**server-owned**. Their universal identifiers are always the
deterministic derivation, on **every** application (standard,
workspace-custom, installed). Clients cannot provide custom values: a
temporary check in `validateObjectMetadataSystemFieldsIntegrity` rejects
any non-derived system field identifier at migration build time. This
check stands in until system fields are generated exclusively server
side by the metadata side-effect engine and stripped from client inputs
— at which point it becomes structurally impossible to send one.
- **`name` is a default field, not a system field**: it is
auto-provisioned when absent (server side for custom objects, SDK side
for application objects) but authors can define their own. It is only
derived where it is guaranteed to be auto-provisioned. In particular,
standard objects keep their **historical hardcoded** `name` identifiers:
the standard app authors its `name` fields like any installed app would,
and moving those identifiers would break every installed application
referencing them (e.g. views on `opportunity.name`).
- **User-created and author-provided fields** keep random / explicit
identifiers, untouched.

## Server

- `validateObjectMetadataSystemFieldsIntegrity` now validates, on top of
the existing type/`isSystem` checks, that each system field's
`universalIdentifier` equals the deterministic derivation. Runs for
every object creation going through the migration orchestrator: app
sync, custom object creation, standard provisioning
- `build-default-flat-field-metadatas-for-custom-object.util.ts` derives
the system field identifiers (and the auto-provisioned `name`) with
`getFieldUniversalIdentifier` instead of `v4()`
-
`build-default-relation-flat-field-metadatas-for-custom-object.util.ts`
derives both the forward and the reverse default relation field
identifiers deterministically
- `generateMorphOrRelationFlatFieldMetadataPair` accepts optional
`sourceFieldUniversalIdentifier` / `targetFieldUniversalIdentifier` so
callers can inject deterministic values; user-created relations still
default to `v4()`

## twenty-shared

- `STANDARD_OBJECTS` system field identifiers (the 8) are now computed
at module load via `buildStandardObjectSystemFields`; `name` and every
other identifier keep their hardcoded values
- New snapshot test pinning **every** universal identifier of
`STANDARD_OBJECTS`: any identifier change now requires an explicit
snapshot update and should ship with a coordinated backfill

## SDK (breaking, pre-GA)

- `generateDefaultFieldUniversalIdentifier` delegates to
`getFieldUniversalIdentifier` and now requires
`applicationUniversalIdentifier`
- Reverse default relation field identifiers are derived from the
field's real coordinates (standard object UID + actual field name, e.g.
`targetRocket` on `attachment`) instead of the legacy custom-object UID
+ synthetic `${fieldName}Inverse` hash input. Field *names* are
unchanged
- The manifest build threads the application universal identifier
through default field injection (two-pass over object configs)
- `twenty dev:add` now resolves the application universal identifier
upfront and refuses to scaffold anything until `defineApplication`
declares one — no more `fill-later` placeholder for the app UID in
generated files

## Upgrade

A 2.19 **workspace command** backfills existing
`fieldMetadata.universalIdentifier` rows to the deterministic
derivation. Coverage follows the ownership model:

- **The 8 system fields**: taken over for **every application**,
whatever value they currently hold. This is both safe and required now
that sync rejects non-derived values — leaving a row unconverged would
make its application unsyncable
- **`name`**: workspace-custom app → always taken over
(server-generated, no author to clobber); installed applications → only
rows still carrying the legacy SDK derivation are recomputed,
author-provided identifiers are never touched; standard app → never
touched (hardcoded in `STANDARD_OBJECTS`)
- **Default relation fields**: workspace-custom app → forward fields on
custom objects and reverse fields on the standard relation objects;
installed applications → legacy-derivation probe only

All identifiers of a workspace are updated inside a single transaction,
then the command flushes the field-metadata-related workspace caches and
bumps the metadata version.

Stored `applicationRegistration.manifest` snapshots are intentionally
**not** rewritten: installs and upgrades always sync from the
`manifest.json` inside the resolved package (npm/tarball), the stored
column is only used for display/marketplace purposes.

## Breaking behavior for old packages (fail closed)

Packages built with an older SDK carry legacy system field identifiers
in their tarball `manifest.json`. Installing or upgrading such a package
now fails with an explicit `INVALID_SYSTEM_FIELD` validation error
("universal identifier is not deterministic") instead of silently
mismatching against the backfilled rows and triggering a destructive
delete+create. The remediation is to rebuild the package with the new
SDK; the backfill has already converged the installed rows, so the
rebuilt manifest syncs cleanly.

## Test plan

- [x] `twenty-sdk` unit tests (526 tests) and typecheck
- [x] `twenty-shared` unit tests (1635 tests) including the
`STANDARD_OBJECTS` snapshot; `name` identifiers verified byte-for-byte
identical to `main`
- [x] Lint and typecheck clean on all touched packages
- [x] Integration: create a custom object and verify system + default
relation field identifiers match the deterministic derivation
(`create-one-object-metadata-deterministic-field-universal-identifiers`,
13 assertions passing)
- [x] Integration: `failing-sync-application-object-system-fields`
extended with a non-derived system field identifier case; all
identifiers in the spec pinned deterministically so snapshots embedding
expected/actual values are stable across runs (verified with a double
run)
- [x] Integration: all application sync suites pass with the derived
system field identifiers now required by the
`buildDefaultObjectManifest` test helper (9 suites, 20 tests)
- [x] Full test-database reset: standard app provisioning and seeded
workspaces pass the new validation
- [x] SDK manifest build verified on the postcard example app: all
auto-generated default field identifiers match the derivation
- [ ] Run
`upgrade:2-19:backfill-deterministic-field-universal-identifiers`
(dry-run then real) on a seeded workspace and verify identifier
convergence with a rebuilt app manifest
2026-07-06 13:34:33 +00:00

121 lines
4.3 KiB
TypeScript

import { expectOneNotInternalServerErrorSnapshot } from 'test/integration/graphql/utils/expect-one-not-internal-server-error-snapshot.util';
import { buildBaseManifest } from 'test/integration/metadata/suites/application/utils/build-base-manifest.util';
import { buildDefaultObjectManifest } from 'test/integration/metadata/suites/application/utils/build-default-object-manifest.util';
import { cleanupApplicationAndAppRegistration } from 'test/integration/metadata/suites/application/utils/cleanup-application-and-app-registration.util';
import { setupApplicationForSync } from 'test/integration/metadata/suites/application/utils/setup-application-for-sync.util';
import { syncApplication } from 'test/integration/metadata/suites/application/utils/sync-application.util';
import {
getIndexUniversalIdentifier,
type Manifest,
} from 'twenty-shared/application';
import { FieldMetadataType } from 'twenty-shared/types';
import { v4 as uuidv4 } from 'uuid';
import { generateDeterministicIndexName } from 'src/engine/metadata-modules/index-metadata/utils/generate-deterministic-index-name';
import { type UniversalFlatObjectMetadata } from 'src/engine/workspace-manager/workspace-migration/universal-flat-entity/types/universal-flat-object-metadata.type';
const TEST_APP_ID = uuidv4();
const TEST_ROLE_ID = uuidv4();
const OBJECT_NAME_SINGULAR = 'collidingIndexObject';
const UNIQUE_FIELD_NAME = 'externalId';
const computeSystemUniqueIndexUniversalIdentifier = (
objectUniversalIdentifier: string,
): string => {
const name = generateDeterministicIndexName({
flatObjectMetadata: {
nameSingular: OBJECT_NAME_SINGULAR,
applicationUniversalIdentifier: TEST_APP_ID,
} as UniversalFlatObjectMetadata,
orderedIndexColumnNames: [UNIQUE_FIELD_NAME],
isUnique: true,
indexWhereClause: null,
});
return getIndexUniversalIdentifier({
applicationUniversalIdentifier: TEST_APP_ID,
objectUniversalIdentifier,
name,
});
};
// The object is otherwise fully valid (all system fields present) so the reserved-identifier
// violation is the only validation error surfaced.
const buildCollidingManifest = (): Manifest => {
const objectUniversalIdentifier = uuidv4();
const uniqueFieldUniversalIdentifier = uuidv4();
const object = buildDefaultObjectManifest({
applicationUniversalIdentifier: TEST_APP_ID,
nameSingular: OBJECT_NAME_SINGULAR,
namePlural: 'collidingIndexObjects',
labelSingular: 'Colliding Index Object',
labelPlural: 'Colliding Index Objects',
description: 'Object used to test unique index side-effect collision',
universalIdentifier: objectUniversalIdentifier,
additionalFields: [
{
universalIdentifier: uniqueFieldUniversalIdentifier,
type: FieldMetadataType.TEXT,
name: UNIQUE_FIELD_NAME,
label: 'External ID',
description: 'Unique external identifier',
icon: 'IconId',
isUnique: true,
isNullable: true,
},
],
});
return buildBaseManifest({
appId: TEST_APP_ID,
roleId: TEST_ROLE_ID,
overrides: {
objects: [object],
fields: [],
indexes: [
{
universalIdentifier: computeSystemUniqueIndexUniversalIdentifier(
objectUniversalIdentifier,
),
objectUniversalIdentifier,
isUnique: true,
fields: [
{
universalIdentifier: uuidv4(),
fieldUniversalIdentifier: uniqueFieldUniversalIdentifier,
},
],
},
],
},
});
};
describe('Sync application should fail on unique index side-effect collision', () => {
beforeAll(async () => {
await setupApplicationForSync({
applicationUniversalIdentifier: TEST_APP_ID,
name: 'Test Unique Index Collision App',
description: 'App for testing unique index side-effect collision',
sourcePath: 'test-unique-index-collision',
});
}, 60000);
afterAll(async () => {
await cleanupApplicationAndAppRegistration({
applicationUniversalIdentifier: TEST_APP_ID,
});
});
it('should fail when a declared index reuses the reserved deterministic identifier of the engine-owned unique backing index', async () => {
const { errors } = await syncApplication({
manifest: buildCollidingManifest(),
expectToFail: true,
});
expectOneNotInternalServerErrorSnapshot({ errors });
}, 60000);
});