System side effect relations (#22882)
Closes twentyhq/core-team-issues#2667 ## What Default relations to the standard relation objects (`timelineActivities`, `attachments`, `noteTargets`, `taskTargets`) are now fully owned by the **metadata side-effect engine**. Neither the API transpilers nor the SDK manifest builder provision them anymore: any object creation, rename or deletion — regardless of the caller — goes through the same engine handlers. ## Why - Provisioning was duplicated across the API path and the SDK manifest builder, with diverging behavior. - Universal identifiers of relation fields were derived from object **names**, so renaming an object mutated them and forced lossy delete+create cycles on manifest sync. ## How ### Engine-owned lifecycle (side-effect handlers) - `objectSystemRelationsOnCreate`: provisions the 8 forward/reverse relation fields (+ join column indexes) when an object is created. - `objectSystemRelationsOnUpdate`: renames the reverse morph fields (`target<ObjectName>`) when their host object is renamed — a lossless `fieldMetadata.update`. - `objectSystemSideEffectsOnDelete`: cascades deletion of engine-owned fields/indexes when the object is deleted. - The API transpilers and the SDK `buildManifest` no longer inject these fields; `isSystemSideEffect: true` marks engine-owned entities, guarded by a granular property allowlist (only `isActive` is user-editable) and excluded from manifest deletion inference. ### Name-free deterministic universal identifiers New `getSystemRelationFieldUniversalIdentifier({ applicationUniversalIdentifier, objectUniversalIdentifier, relationTargetObjectUniversalIdentifier })` in `twenty-shared`, exported from `twenty-sdk/define`. The identifier is keyed on the two **object** identifiers instead of field names (direction encoded by argument order), so object renames never mutate relation field identifiers. It cannot collide with the name-based `getFieldUniversalIdentifier` derivation (field names cannot contain `:`). ### twenty-standard re-owned All 48 forward/reverse system relation field declarations in `STANDARD_OBJECTS` now pin the derived name-free identifiers (computed inline via the shared util) and carry `isSystemSideEffect: true`, with labels/icons declared explicitly (translated via `msg`). `twenty-standard` is projected as if the engine had generated these fields itself. ### 2.23 upgrade commands - `reconcile-system-relation-field-universal-identifier`: structurally matches existing default relation fields per workspace and backfills the derived universal identifiers, `isSystemSideEffect` flags, and standard labels/icons. - `upgrade-people-data-labs-application`: upgrades installed PDL apps to `1.0.7` right after the backfill to close the desync window (its views reference the re-derived identifiers). ### Misc - `people-data-labs` `1.0.7`: views temporarily pin the new derived identifiers (TODO: import from the next released `twenty-sdk`). - `UpgradeStatusModule` split out of `UpgradeModule` so the application module cluster can consume upgrade status/migration services without importing the versioned command bundles (fixes a require cycle that crashed boot). - Docs: `system-fields.mdx` documents the system relation fields and their resolver; `sync-and-recovery.mdx` plan example no longer shows auto-injected relations. ## Known red CI `people-data-labs (dockerhub-latest)` fails by design until the 2.23 server image is published: the app pins the new identifiers which only exist on a 2.23 server. The `local` leg (server built from this branch) is green. ## System fields are no longer manifest-authorable (accepted regression) The manifest converter no longer derives `isSystem` / `isSystemSideEffect` from field names. Reserved-system-named manifest fields (`id`, `createdAt`, `updatedAt`, `deletedAt`, `createdBy`, `updatedBy`, `position`, `searchVector`) are now skipped at conversion time when they carry the exact derived universal identifier (keeps manifests built with older SDKs installable), and rejected with `INVALID_INPUT` when they pin any other identifier. System fields are therefore fully engine-canonical: nothing a manifest carries can produce a system-flagged entity anymore. **Accepted regression**: a manifest can no longer influence system field properties at all. Previously a (legacy) re-declaration could shape them at creation — which actually produced broken system fields, e.g. a nullable, non-unique `id` — and could still toggle the allowlisted `isActive` / `universalSettings` afterwards. We consider this acceptable for now: per-app granularity over system fields will be reintroduced later through the **override framework**, which will also settle update semantics by forbidding direct updates over `isSystemSideEffect: true` entities and expressing divergence as overrides. `isSystemSideEffect`-only entities (the default relation fields provisioned by this PR) still have no engine-level update guard (see Follow-up below); that part is unchanged and also lands with the overrides refactor. ## Follow-up `isSystemSideEffect` field update/delete guards intentionally live at the API layer (`sanitize-raw-update-field-input.ts`, `from-delete-field-input-...util.ts`) rather than in the engine-level `FlatFieldMetadataValidatorService`. Moving them into the validator requires threading operation-origin (direct field mutation vs engine cascade) through the migration matrix, otherwise legitimate object rename/delete cascades (which carry `isSystemBuild=false`) would be rejected. Tracked in twentyhq/core-team-issues#2671. <!-- This is an auto-generated description by cubic. --> <a href="https://cubic.dev/pr/twentyhq/twenty/pull/22882?utm_source=github" target="_blank" rel="noopener noreferrer" data-no-image-dialog="true"><picture><source media="(prefers-color-scheme: dark)" srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source media="(prefers-color-scheme: light)" srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img alt="Review in cubic" src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a> <!-- End of auto-generated description by cubic. -->
This commit is contained in:
Paul Rastoin
+17
@@ -106,6 +106,23 @@ describe('fromFieldManifestToUniversalFlatFieldMetadata', () => {
|
||||
});
|
||||
});
|
||||
|
||||
describe('system flags', () => {
|
||||
it('never derives isSystem or isSystemSideEffect, even for reserved system field names', () => {
|
||||
const result = fromFieldManifestToUniversalFlatFieldMetadata({
|
||||
fieldManifest: buildFieldManifest({
|
||||
type: FieldMetadataType.DATE_TIME,
|
||||
name: 'createdAt',
|
||||
label: 'Creation date',
|
||||
}),
|
||||
applicationUniversalIdentifier: APP_UID,
|
||||
now: NOW,
|
||||
});
|
||||
|
||||
expect(result.isSystem).toBe(false);
|
||||
expect(result.isSystemSideEffect).toBe(false);
|
||||
});
|
||||
});
|
||||
|
||||
describe('isUIEditable', () => {
|
||||
it('defaults to true when omitted from the manifest', () => {
|
||||
const result = fromFieldManifestToUniversalFlatFieldMetadata({
|
||||
|
||||
+2
-7
@@ -12,7 +12,6 @@ import { type CompositeFieldMetadataType } from 'src/engine/metadata-modules/fie
|
||||
import { generateDefaultValue } from 'src/engine/metadata-modules/field-metadata/utils/generate-default-value';
|
||||
import { isCompositeFieldMetadataType } from 'src/engine/metadata-modules/field-metadata/utils/is-composite-field-metadata-type.util';
|
||||
import { nullifyEmptyCompositeDefaultValue } from 'src/engine/metadata-modules/flat-field-metadata/utils/nullify-empty-composite-default-value.util';
|
||||
import { PARTIAL_SYSTEM_FLAT_FIELD_METADATAS } from 'src/engine/metadata-modules/object-metadata/constants/partial-system-flat-field-metadatas.constant';
|
||||
import { isMorphOrRelationFieldMetadataType } from 'src/engine/utils/is-morph-or-relation-field-metadata-type.util';
|
||||
import { type UniversalFlatFieldMetadata } from 'src/engine/workspace-manager/workspace-migration/universal-flat-entity/types/universal-flat-field-metadata.type';
|
||||
|
||||
@@ -68,9 +67,6 @@ export const fromFieldManifestToUniversalFlatFieldMetadata = ({
|
||||
relationTargetObjectMetadataUniversalIdentifier,
|
||||
} = getRelationTargetUniversalIdentifiers(fieldManifest);
|
||||
|
||||
// TODO: generate system fields server-side from the object manifest
|
||||
// so the converter doesn't need to re-normalize composite defaults
|
||||
// that the SDK couldn't have known the canonical shape of.
|
||||
const rawDefaultValue =
|
||||
fieldManifest.defaultValue ?? generateDefaultValue(fieldManifest.type);
|
||||
const defaultValue = isCompositeFieldMetadataType(fieldManifest.type)
|
||||
@@ -93,9 +89,8 @@ export const fromFieldManifestToUniversalFlatFieldMetadata = ({
|
||||
defaultValue,
|
||||
universalSettings: fieldManifest.universalSettings ?? null,
|
||||
isActive: true,
|
||||
isSystem: fieldManifest.name in PARTIAL_SYSTEM_FLAT_FIELD_METADATAS,
|
||||
isSystemSideEffect:
|
||||
fieldManifest.name in PARTIAL_SYSTEM_FLAT_FIELD_METADATAS,
|
||||
isSystem: false,
|
||||
isSystemSideEffect: false,
|
||||
isUIEditable: fieldManifest.isUIEditable ?? true,
|
||||
isNullable: fieldManifest.isNullable ?? true,
|
||||
isUnique: fieldManifest.isUnique ?? false,
|
||||
|
||||
+2
-2
@@ -8,14 +8,14 @@ import { FileStorageModule } from 'src/engine/core-modules/file-storage/file-sto
|
||||
import { FileEntity } from 'src/engine/core-modules/file/entities/file.entity';
|
||||
import { SecureHttpClientModule } from 'src/engine/core-modules/secure-http-client/secure-http-client.module';
|
||||
import { TwentyConfigModule } from 'src/engine/core-modules/twenty-config/twenty-config.module';
|
||||
import { UpgradeModule } from 'src/engine/core-modules/upgrade/upgrade.module';
|
||||
import { UpgradeStatusModule } from 'src/engine/core-modules/upgrade/upgrade-status.module';
|
||||
|
||||
@Module({
|
||||
imports: [
|
||||
FileStorageModule,
|
||||
SecureHttpClientModule,
|
||||
TwentyConfigModule,
|
||||
UpgradeModule,
|
||||
UpgradeStatusModule,
|
||||
TypeOrmModule.forFeature([FileEntity, ApplicationEntity]),
|
||||
],
|
||||
providers: [
|
||||
|
||||
@@ -0,0 +1,35 @@
|
||||
import { Module } from '@nestjs/common';
|
||||
import { DiscoveryModule } from '@nestjs/core';
|
||||
import { TypeOrmModule } from '@nestjs/typeorm';
|
||||
|
||||
import { CoreEntityCacheModule } from 'src/engine/core-entity-cache/core-entity-cache.module';
|
||||
import { UpgradeCommandRegistryService } from 'src/engine/core-modules/upgrade/services/upgrade-command-registry.service';
|
||||
import { UpgradeMigrationService } from 'src/engine/core-modules/upgrade/services/upgrade-migration.service';
|
||||
import { UpgradeSequenceReaderService } from 'src/engine/core-modules/upgrade/services/upgrade-sequence-reader.service';
|
||||
import { UpgradeStatusCacheService } from 'src/engine/core-modules/upgrade/services/upgrade-status-cache.service';
|
||||
import { UpgradeStatusService } from 'src/engine/core-modules/upgrade/services/upgrade-status.service';
|
||||
import { UpgradeMigrationEntity } from 'src/engine/core-modules/upgrade/upgrade-migration.entity';
|
||||
import { WorkspaceEntity } from 'src/engine/core-modules/workspace/workspace.entity';
|
||||
|
||||
@Module({
|
||||
imports: [
|
||||
CoreEntityCacheModule,
|
||||
DiscoveryModule,
|
||||
TypeOrmModule.forFeature([UpgradeMigrationEntity, WorkspaceEntity]),
|
||||
],
|
||||
providers: [
|
||||
UpgradeCommandRegistryService,
|
||||
UpgradeMigrationService,
|
||||
UpgradeSequenceReaderService,
|
||||
UpgradeStatusCacheService,
|
||||
UpgradeStatusService,
|
||||
],
|
||||
exports: [
|
||||
UpgradeCommandRegistryService,
|
||||
UpgradeMigrationService,
|
||||
UpgradeSequenceReaderService,
|
||||
UpgradeStatusCacheService,
|
||||
UpgradeStatusService,
|
||||
],
|
||||
})
|
||||
export class UpgradeStatusModule {}
|
||||
@@ -1,59 +1,39 @@
|
||||
import { Module } from '@nestjs/common';
|
||||
import { DiscoveryModule } from '@nestjs/core';
|
||||
import { TypeOrmModule } from '@nestjs/typeorm';
|
||||
|
||||
import { WorkspaceIteratorModule } from 'src/database/commands/command-runners/workspace-iterator.module';
|
||||
import { InstanceCommandProviderModule } from 'src/database/commands/upgrade-version-command/instance-command-provider.module';
|
||||
import { WorkspaceCommandProviderModule } from 'src/database/commands/upgrade-version-command/workspace-command-provider.module';
|
||||
import { CoreEntityCacheModule } from 'src/engine/core-entity-cache/core-entity-cache.module';
|
||||
import { MetricsModule } from 'src/engine/core-modules/metrics/metrics.module';
|
||||
import { InstanceCommandRunnerService } from 'src/engine/core-modules/upgrade/services/instance-command-runner.service';
|
||||
import { UpgradeCommandRegistryService } from 'src/engine/core-modules/upgrade/services/upgrade-command-registry.service';
|
||||
import { UpgradeMigrationService } from 'src/engine/core-modules/upgrade/services/upgrade-migration.service';
|
||||
import { UpgradeSequenceReaderService } from 'src/engine/core-modules/upgrade/services/upgrade-sequence-reader.service';
|
||||
import { UpgradeSequenceRunnerService } from 'src/engine/core-modules/upgrade/services/upgrade-sequence-runner.service';
|
||||
import { UpgradeStatusCacheService } from 'src/engine/core-modules/upgrade/services/upgrade-status-cache.service';
|
||||
import { UpgradeStatusService } from 'src/engine/core-modules/upgrade/services/upgrade-status.service';
|
||||
import { WorkspaceCommandRunnerService } from 'src/engine/core-modules/upgrade/services/workspace-command-runner.service';
|
||||
import { UpgradeGaugeService } from 'src/engine/core-modules/upgrade/upgrade-gauge.service';
|
||||
import { UpgradeMigrationEntity } from 'src/engine/core-modules/upgrade/upgrade-migration.entity';
|
||||
import { WorkspaceEntity } from 'src/engine/core-modules/workspace/workspace.entity';
|
||||
import { UpgradeStatusModule } from 'src/engine/core-modules/upgrade/upgrade-status.module';
|
||||
import { UpgradeAwareEntityMetadataAdapter } from 'src/engine/twenty-orm/upgrade-aware/upgrade-aware-entity-metadata.adapter';
|
||||
import { WorkspaceVersionModule } from 'src/engine/workspace-manager/workspace-version/workspace-version.module';
|
||||
|
||||
@Module({
|
||||
imports: [
|
||||
CoreEntityCacheModule,
|
||||
DiscoveryModule,
|
||||
InstanceCommandProviderModule,
|
||||
MetricsModule,
|
||||
UpgradeStatusModule,
|
||||
WorkspaceCommandProviderModule,
|
||||
WorkspaceIteratorModule,
|
||||
WorkspaceVersionModule,
|
||||
TypeOrmModule.forFeature([UpgradeMigrationEntity, WorkspaceEntity]),
|
||||
],
|
||||
providers: [
|
||||
UpgradeMigrationService,
|
||||
InstanceCommandRunnerService,
|
||||
WorkspaceCommandRunnerService,
|
||||
UpgradeCommandRegistryService,
|
||||
UpgradeAwareEntityMetadataAdapter,
|
||||
UpgradeSequenceReaderService,
|
||||
UpgradeSequenceRunnerService,
|
||||
UpgradeStatusService,
|
||||
UpgradeStatusCacheService,
|
||||
UpgradeGaugeService,
|
||||
],
|
||||
exports: [
|
||||
UpgradeMigrationService,
|
||||
UpgradeStatusModule,
|
||||
InstanceCommandRunnerService,
|
||||
WorkspaceCommandRunnerService,
|
||||
UpgradeCommandRegistryService,
|
||||
UpgradeAwareEntityMetadataAdapter,
|
||||
UpgradeSequenceReaderService,
|
||||
UpgradeSequenceRunnerService,
|
||||
UpgradeStatusService,
|
||||
UpgradeStatusCacheService,
|
||||
],
|
||||
})
|
||||
export class UpgradeModule {}
|
||||
|
||||
Reference in New Issue
Block a user