houseassassin/twenty
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
13,427 Commits 1 Branch 0 Tags
8a4bcd14453eaf2ca25329ea3d9abb11b02aa06b
Commit Graph

7 Commits

Author SHA1 Message Date
Paul Rastoin 6c40c7b91a Deterministic system field universal identifier (#22565) 
# Introduction

Close twentyhq/core-team-issues#2641

Auto-provisioned field metadata used to get its `universalIdentifier`
from three unrelated sources: random `v4()` on the server when creating
custom objects, hardcoded values in `STANDARD_OBJECTS`, and an ad-hoc
`v5` derivation in the SDK manifest build. This PR unifies all of them
behind the shared `getFieldUniversalIdentifier` derivation:

```
universalIdentifier = f(applicationUniversalIdentifier, objectUniversalIdentifier, fieldName)
```

## Ownership model

The rollout is built on an explicit split of who owns a field's
universal identifier:

- **The 8 system fields** (`id`, `createdAt`, `updatedAt`, `deletedAt`,
`createdBy`, `updatedBy`, `position`, `searchVector`) are
**server-owned**. Their universal identifiers are always the
deterministic derivation, on **every** application (standard,
workspace-custom, installed). Clients cannot provide custom values: a
temporary check in `validateObjectMetadataSystemFieldsIntegrity` rejects
any non-derived system field identifier at migration build time. This
check stands in until system fields are generated exclusively server
side by the metadata side-effect engine and stripped from client inputs
— at which point it becomes structurally impossible to send one.
- **`name` is a default field, not a system field**: it is
auto-provisioned when absent (server side for custom objects, SDK side
for application objects) but authors can define their own. It is only
derived where it is guaranteed to be auto-provisioned. In particular,
standard objects keep their **historical hardcoded** `name` identifiers:
the standard app authors its `name` fields like any installed app would,
and moving those identifiers would break every installed application
referencing them (e.g. views on `opportunity.name`).
- **User-created and author-provided fields** keep random / explicit
identifiers, untouched.

## Server

- `validateObjectMetadataSystemFieldsIntegrity` now validates, on top of
the existing type/`isSystem` checks, that each system field's
`universalIdentifier` equals the deterministic derivation. Runs for
every object creation going through the migration orchestrator: app
sync, custom object creation, standard provisioning
- `build-default-flat-field-metadatas-for-custom-object.util.ts` derives
the system field identifiers (and the auto-provisioned `name`) with
`getFieldUniversalIdentifier` instead of `v4()`
-
`build-default-relation-flat-field-metadatas-for-custom-object.util.ts`
derives both the forward and the reverse default relation field
identifiers deterministically
- `generateMorphOrRelationFlatFieldMetadataPair` accepts optional
`sourceFieldUniversalIdentifier` / `targetFieldUniversalIdentifier` so
callers can inject deterministic values; user-created relations still
default to `v4()`

## twenty-shared

- `STANDARD_OBJECTS` system field identifiers (the 8) are now computed
at module load via `buildStandardObjectSystemFields`; `name` and every
other identifier keep their hardcoded values
- New snapshot test pinning **every** universal identifier of
`STANDARD_OBJECTS`: any identifier change now requires an explicit
snapshot update and should ship with a coordinated backfill

## SDK (breaking, pre-GA)

- `generateDefaultFieldUniversalIdentifier` delegates to
`getFieldUniversalIdentifier` and now requires
`applicationUniversalIdentifier`
- Reverse default relation field identifiers are derived from the
field's real coordinates (standard object UID + actual field name, e.g.
`targetRocket` on `attachment`) instead of the legacy custom-object UID
+ synthetic `${fieldName}Inverse` hash input. Field *names* are
unchanged
- The manifest build threads the application universal identifier
through default field injection (two-pass over object configs)
- `twenty dev:add` now resolves the application universal identifier
upfront and refuses to scaffold anything until `defineApplication`
declares one — no more `fill-later` placeholder for the app UID in
generated files

## Upgrade

A 2.19 **workspace command** backfills existing
`fieldMetadata.universalIdentifier` rows to the deterministic
derivation. Coverage follows the ownership model:

- **The 8 system fields**: taken over for **every application**,
whatever value they currently hold. This is both safe and required now
that sync rejects non-derived values — leaving a row unconverged would
make its application unsyncable
- **`name`**: workspace-custom app → always taken over
(server-generated, no author to clobber); installed applications → only
rows still carrying the legacy SDK derivation are recomputed,
author-provided identifiers are never touched; standard app → never
touched (hardcoded in `STANDARD_OBJECTS`)
- **Default relation fields**: workspace-custom app → forward fields on
custom objects and reverse fields on the standard relation objects;
installed applications → legacy-derivation probe only

All identifiers of a workspace are updated inside a single transaction,
then the command flushes the field-metadata-related workspace caches and
bumps the metadata version.

Stored `applicationRegistration.manifest` snapshots are intentionally
**not** rewritten: installs and upgrades always sync from the
`manifest.json` inside the resolved package (npm/tarball), the stored
column is only used for display/marketplace purposes.

## Breaking behavior for old packages (fail closed)

Packages built with an older SDK carry legacy system field identifiers
in their tarball `manifest.json`. Installing or upgrading such a package
now fails with an explicit `INVALID_SYSTEM_FIELD` validation error
("universal identifier is not deterministic") instead of silently
mismatching against the backfilled rows and triggering a destructive
delete+create. The remediation is to rebuild the package with the new
SDK; the backfill has already converged the installed rows, so the
rebuilt manifest syncs cleanly.

## Test plan

- [x] `twenty-sdk` unit tests (526 tests) and typecheck
- [x] `twenty-shared` unit tests (1635 tests) including the
`STANDARD_OBJECTS` snapshot; `name` identifiers verified byte-for-byte
identical to `main`
- [x] Lint and typecheck clean on all touched packages
- [x] Integration: create a custom object and verify system + default
relation field identifiers match the deterministic derivation
(`create-one-object-metadata-deterministic-field-universal-identifiers`,
13 assertions passing)
- [x] Integration: `failing-sync-application-object-system-fields`
extended with a non-derived system field identifier case; all
identifiers in the spec pinned deterministically so snapshots embedding
expected/actual values are stable across runs (verified with a double
run)
- [x] Integration: all application sync suites pass with the derived
system field identifiers now required by the
`buildDefaultObjectManifest` test helper (9 suites, 20 tests)
- [x] Full test-database reset: standard app provisioning and seeded
workspaces pass the new validation
- [x] SDK manifest build verified on the postcard example app: all
auto-generated default field identifiers match the derivation
- [ ] Run
`upgrade:2-19:backfill-deterministic-field-universal-identifiers`
(dry-run then real) on a seeded workspace and verify identifier
convergence with a rebuilt app manifest
 2026-07-06 13:34:33 +00:00
Paul Rastoin f0a625c3f8 Cleanup application and app registration test util (#19981) 
## Introduction
Centralizing integ test app cleanup
Role will be deleted by the app uninstall if exists
 2026-04-22 14:39:58 +00:00
Marie 2d6c8be7df [Apps] Fix - app-synced object should be searchable (#19206) 
## Summary

- **Make app-synced objects searchable**: `isSearchable` was hardcoded
to `false` and the `searchVector` field was missing the `GENERATED
ALWAYS AS (...)` expression, causing all records to have a `NULL` search
vector and be excluded from search results. Fixed by defaulting
`isSearchable` to `true` (configurable via the object manifest),
computing the `asExpression` from the label identifier field, and
allowing the update-field-action-handler to handle the `null` → defined
`asExpression` transition.
- **Make `isSearchable` updatable on an object**: The property had
`toCompare: false` in the entity properties configuration, so updates
via the API were silently ignored and never persisted. Fixed by setting
`toCompare: true`.
 2026-04-02 17:14:37 +00:00
Félix Malfait 514d0017ea Refactor application module architecture for clarity and explicitness (#18432) 
## Summary

- **Module reorganization**: Moved `ApplicationUpgradeService` and cron
jobs to `application-upgrade/`, `ApplicationSyncService` to
`application-manifest/`, and
`runWorkspaceMigration`/`uninstallApplication` mutations to the manifest
resolver — each module now has a single clear responsibility.
- **Explicit install flow**: Removed implicit `ApplicationEntity`
creation from `ApplicationSyncService`. The install service and dev
resolver now explicitly create the `ApplicationEntity` before syncing.
npm packages are resolved at registration time to extract manifest
metadata (universalIdentifier, name, description, etc.), eliminating the
`reconcileUniversalIdentifier` hack.
- **Better error handling**: Frontend hooks now surface actual server
error messages in snackbars instead of swallowing them. Replaced the
ugly `ConfirmationModal` for transfer ownership with a proper form
modal. Fixed `SettingsAdminTableCard` row height overflow and corrected
the `yarn-engine` asset path.

## Test plan
- [ ] Register an npm package — verify manifest metadata (name,
description, universalIdentifier) is extracted correctly
- [ ] Install a registered npm app on a workspace — verify
ApplicationEntity is created and sync succeeds
- [ ] Test `app:dev` CLI flow — verify local app registration and sync
work
- [ ] Upload a tarball — verify registration and install flow
- [ ] Transfer ownership — verify the new modal UX works
- [ ] Verify error messages appear correctly in snackbars when
operations fail


Made with [Cursor](https://cursor.com)
 2026-03-06 08:45:08 +01:00
martmull f080b952eb Add manifest integration tests (#18203) 
- add integration test on sync manifest endpoint
- fix invalid standard uuid + migration command
 2026-02-25 10:37:50 +01:00
Paul Rastoin 175df59c21 Support Skill in manifest (#18092) 
# Introduction
Support skill in manifest, pre-requisite for the twenty standard app
migration
 2026-02-20 11:45:42 +01:00
Paul Rastoin 88424611ec Refactor and standardize isSystem field and object (#17992) 
# Introduction

## Centralize system field definitions
- Extract a single `PARTIAL_SYSTEM_FLAT_FIELD_METADATAS` constant as the
source of truth for all 8 system fields (`id`, `createdAt`, `updatedAt`,
`deletedAt`, `createdBy`, `updatedBy`, `position`, `searchVector`),
eliminating duplication across custom object and standard app field
builders
- Refactor `buildDefaultFlatFieldMetadatasForCustomObject` to use the
shared constant via a new `buildObjectSystemFlatFieldMetadatas` helper

## Mark system fields as `isSystem: true`
- Fields `id`, `createdAt`, `updatedAt`, `deletedAt`, `createdBy`,
`updatedBy`, `position`, `searchVector` are now properly flagged as
system fields across all standard objects and custom object creation
- Standard app field builders for all ~30 standard objects updated to
set `isSystem: true` on `createdAt`, `updatedAt`, `deletedAt`,
`createdBy`, `updatedBy`
- System-only standard objects (blocklist, calendar channels, message
threads, etc.) now also include `createdBy`, `updatedBy`, `position`,
`searchVector` field definitions that were previously missing

## Validate system fields on object creation
- New transversal validation (`crossEntityTransversalValidation`) runs
after all atomic entity validations in the build orchestrator, ensuring
all 8 system fields are present with correct `type` and `isSystem: true`
when an object is created
- New `buildUniversalFlatObjectFieldByNameAndJoinColumnMaps` utility to
resolve field names to universal identifiers for a given object
- New exception codes: `MISSING_SYSTEM_FIELD` and `INVALID_SYSTEM_FIELD`
on `ObjectMetadataExceptionCode`

## Protect system fields and objects from mutation
- Field validators now block update/delete of `isSystem` fields by
non-system callers (`FIELD_MUTATION_NOT_ALLOWED`)
- Object validators now block update/delete of `isSystem` objects by
non-system callers
- `POSITION` and `TS_VECTOR` field type validators replaced: instead of
rejecting creation outright, they now validate that the field is named
correctly (`position` / `searchVector`) and has `isSystem: true`

## Distinguish `isSystemBuild` from `isCallerTwentyStandardApp`
- New `isCallerTwentyStandardApp` utility checks whether the caller's
`applicationUniversalIdentifier` matches the twenty standard app
- Name-sync logic (`isFlatFieldMetadataNameSyncedWithLabel`,
`areFlatObjectMetadataNamesSyncedWithLabels`) refactored to use
`isCallerTwentyStandardApp` for custom suffix decisions, keeping
`isSystemBuild` for mutation permission checks
- `WorkspaceMigrationBuilderOptions` type updated to include
`applicationUniversalIdentifier`

## Adapt frontend filtering
- New `HIDDEN_SYSTEM_FIELD_NAMES` constant (`id`, `position`,
`searchVector`) and `isHiddenSystemField` utility to only hide truly
internal fields while keeping user-facing system fields (`createdAt`,
`updatedAt`, `deletedAt`, `createdBy`, `updatedBy`) visible in the UI
- ~20 frontend files updated to replace `!field.isSystem` checks with
`!isHiddenSystemField(field)` across record index, settings, data model,
charts, workflows, spreadsheet import, aggregations, and role
permissions

## Add 1.19 upgrade commands
- **`backfill-system-fields-is-system`**: Raw SQL command to set
`isSystem = true` on existing workspace fields matching system field
names, and fix `position` field type from `NUMBER` to `POSITION` for
`favorite`/`favoriteFolder` objects. Includes proper cache invalidation.
- **`add-missing-system-fields-to-standard-objects`**: Codegen'd
workspace migration to create missing `position`, `searchVector`,
`createdBy`, `updatedBy` fields on standard objects that didn't
previously have them. Runs via `WorkspaceMigrationRunnerService` in a
single transaction with idempotency check. **Known limitation**: assumes
all standard objects exist and are valid in the target workspace.

## Add `universalIdentifier` for system fields in standard object
constants
- `standard-object.constant.ts` updated to include `universalIdentifier`
for `createdBy`, `updatedBy`, `position`, and `searchVector` across all
standard objects
- `fieldManifestType.ts` updated to support the new field manifest shape

## System relation
Completely removed and backfilled all `isSystem` relation to be false
false
As we won't require an object to have any relation system fields

## Add integration tests
- New test suite `failing-sync-application-object-system-fields`
covering: missing system fields, wrong field types (`id` as TEXT,
`createdAt` as TEXT, `position` as TEXT), system field deletion
attempts, and system field update attempts
- New test utilities: `buildDefaultObjectManifest` (builds an object
manifest with all 8 system fields) and `setupApplicationForSync`
(centralizes application setup)
- Existing successful sync test updated to verify system fields are
created with correct properties

## Next step
Make the builder scope the compared entity to be the currently built app
+ nor twenty standard app
 2026-02-19 10:13:50 +00:00