Fix and refactor all metadata relation (#17978)
# Introduction The initial motivation was that in the workspace migration create action some universal foreign key aggregators weren't correctly deleted before returned due to constant missconfiguration <img width="2300" height="972" alt="image" src="https://github.com/user-attachments/assets/9401eb02-2bb2-4e69-9c5f-9a354ff61079" /> It also meant that under the hood some optimistic behavior wasn't correctly rendered for some aggregators ## Solution Refactored the `ALL_METADATA_RELATIONS` as follows: This way we can infer the FK and transpile it to a universalFK, also the aggregators are one to one instead of one versus all available Making the only manual configuration to be defined the `foreignKey` and `inverseOneToManyProperty` ``` ┌──────────────────────────────────────┐ ┌─────────────────────────────────────────────┐ │ ALL_MANY_TO_ONE_METADATA_FOREIGN_KEY│ │ ALL_ONE_TO_MANY_METADATA_RELATIONS │ │──────────────────────────────────────│ │─────────────────────────────────────────────│ │ Derived from: Entity types │ │ Derived from: Entity types │ │ │ │ │ │ Provides: │ │ Provides: │ │ • foreignKey │ │ • metadataName │ │ │ │ • flatEntityForeignKeyAggregator │ │ Standalone low-level primitive │ │ • universalFlatEntityForeignKeyAggregator │ └──────────────┬───────────────────────┘ └──────────────┬──────────────────────────────┘ │ │ │ foreignKey type + │ inverseOneToManyProperty │ universalForeignKey derivation │ keys (type constraint) │ │ ▼ ▼ ┌───────────────────────────────────────────────────────────────┐ │ ALL_MANY_TO_ONE_METADATA_RELATIONS │ │───────────────────────────────────────────────────────────────│ │ Derived from: │ │ • Entity types (metadataName, isNullable) │ │ • ALL_MANY_TO_ONE_METADATA_FOREIGN_KEY (FK → universalFK) │ │ • ALL_ONE_TO_MANY_METADATA_RELATIONS (inverse keys) │ │ │ │ Provides: │ │ • metadataName │ │ • foreignKey (replicated from FK constant) │ │ • inverseOneToManyProperty │ │ • isNullable │ │ • universalForeignKey │ └──────────────────────────┬────────────────────────────────────┘ │ ┌──────────────────┼──────────────────┐ │ │ │ ▼ ▼ ▼ ┌───────────────────┐ ┌────────────────┐ ┌──────────────────────┐ │ Type consumers │ │ Atomic utils │ │ Optimistic utils │ │───────────────────│ │────────────────│ │──────────────────────│ │ • JoinColumn │ │ • resolve-* │ │ • add/delete flat │ │ • RelatedNames │ │ • get-* │ │ entity maps │ │ • UniversalFlat │ │ │ │ • add/delete │ │ EntityFrom │ │ │ │ universal flat │ │ │ │ │ │ entity maps │ └───────────────────┘ └────────────────┘ │ │ │ (bridge via │ │ inverseOneToMany │ │ Property → │ │ ONE_TO_MANY for │ │ aggregator lookup) │ └──────────────────────┘ ``` ### Previously ``` ┌─────────────────────────────────────────────────────────────────────┐ │ ALL_METADATA_RELATIONS │ │─────────────────────────────────────────────────────────────────────│ │ Derived from: Entity types │ │ │ │ Structure: { [metadataName]: { manyToOne: {...}, oneToMany: {...},│ │ serializedRelations?: {...} } } │ │ │ │ manyToOne provides: │ │ • metadataName │ │ • foreignKey │ │ • flatEntityForeignKeyAggregator (nullable, often wrong/null) │ │ • isNullable │ │ │ │ oneToMany provides: │ │ • metadataName │ │ │ │ Monolithic single source of truth │ └──────────────────────────┬──────────────────────────────────────────┘ │ │ manyToOne entries transformed via │ ToUniversalMetadataManyToOneRelationConfiguration │ ▼ ┌─────────────────────────────────────────────────────────────────────┐ │ ALL_UNIVERSAL_METADATA_RELATIONS │ │─────────────────────────────────────────────────────────────────────│ │ Derived from: ALL_METADATA_RELATIONS (type-level transform) │ │ │ │ Structure: { [metadataName]: { manyToOne: {...}, oneToMany: {...} │ │ } } │ │ │ │ manyToOne provides: │ │ • metadataName │ │ • foreignKey │ │ • universalForeignKey (derived: FK → replace Id → UniversalId) │ │ • universalFlatEntityForeignKeyAggregator (derived from │ │ flatEntityForeignKeyAggregator → replace Ids → UniversalIds) │ │ • isNullable │ │ │ │ oneToMany: passthrough from ALL_METADATA_RELATIONS │ │ │ │ Duplicated monolith with universal key transforms │ └──────────────────────────┬──────────────────────────────────────────┘ │ ┌──────────────────┼──────────────────────┐ │ │ │ ▼ ▼ ▼ ┌───────────────┐ ┌────────────────────┐ ┌──────────────────────┐ │ Type consumers│ │ Atomic utils │ │ Optimistic utils │ │───────────────│ │────────────────────│ │──────────────────────│ │ • JoinColumn │ │ • resolve-entity- │ │ • add/delete flat │ │ • RelatedNames│ │ relation-univ-id │ │ entity maps │ │ • Universal │ │ (ALL_METADATA_ │ │ (ALL_METADATA_ │ │ FlatEntity │ │ RELATIONS │ │ RELATIONS │ │ From │ │ .manyToOne) │ │ .manyToOne) │ │ │ │ │ │ │ │ Mixed usage │ │ • resolve-univ- │ │ • add/delete univ │ │ of both │ │ relation-ids │ │ flat entity maps │ │ constants │ │ (ALL_UNIVERSAL_ │ │ (ALL_UNIVERSAL_ │ │ │ │ METADATA_REL │ │ METADATA_REL │ │ │ │ .manyToOne) │ │ .manyToOne) │ │ │ │ │ │ │ │ │ │ • resolve-univ- │ │ universalFlatEntity │ │ │ │ update-rel-ids │ │ ForeignKeyAggregator │ │ │ │ (ALL_UNIVERSAL_ │ │ read directly from │ │ │ │ METADATA_REL │ │ the constant │ │ │ │ .manyToOne) │ │ │ │ │ │ │ │ │ │ │ │ • regex hack: │ │ │ │ │ │ foreignKey │ │ │ │ │ │ .replace(/Id$/, │ │ │ │ │ │ 'UniversalId') │ │ │ └───────────────┘ └────────────────────┘ └──────────────────────┘ ```
This commit is contained in:
Paul Rastoin
@@ -56,7 +56,7 @@ Follow these skills in order:
|
||||
- Create TypeORM entity (extends `SyncableEntity`)
|
||||
- Define flat entity types
|
||||
- Define action types (universal + flat)
|
||||
- Register in 4 central constants
|
||||
- Register in 5 central constants
|
||||
|
||||
**Why first:** Everything else depends on these types
|
||||
|
||||
@@ -177,9 +177,12 @@ packages/twenty-server/src/engine/metadata-modules/
|
||||
│ ├── services/
|
||||
│ └── utils/
|
||||
└── flat-entity/constant/ # Step 1 (central registries)
|
||||
├── all-entity-properties-configuration-by-metadata-name.constant.ts
|
||||
├── all-one-to-many-metadata-relations.constant.ts
|
||||
├── all-many-to-one-metadata-foreign-key.constant.ts
|
||||
└── all-many-to-one-metadata-relations.constant.ts
|
||||
|
||||
packages/twenty-server/src/engine/workspace-manager/workspace-migration/
|
||||
├── universal-flat-entity/constants/ # Step 1
|
||||
├── workspace-migration-builder/ # Step 3
|
||||
│ ├── builders/my-entity/
|
||||
│ └── validators/services/
|
||||
@@ -192,7 +195,7 @@ packages/twenty-server/src/engine/workspace-manager/workspace-migration/
|
||||
Before considering complete:
|
||||
- [ ] All 6 guides completed
|
||||
- [ ] TypeORM entity extends `SyncableEntity`
|
||||
- [ ] All constants registered (4 central registries)
|
||||
- [ ] All constants registered (5 central registries)
|
||||
- [ ] Cache service with correct decorator
|
||||
- [ ] Transform utils return universal flat entities
|
||||
- [ ] Validator never throws/mutates
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
---
|
||||
name: syncable-entity-types-and-constants
|
||||
description: Define types, entities, and central constant registrations for syncable entities in Twenty's workspace migration system. Use when creating new syncable entities, defining TypeORM entities, flat entity types, or registering in central constants (ALL_ENTITY_PROPERTIES_CONFIGURATION_BY_METADATA_NAME, ALL_METADATA_RELATIONS, ALL_UNIVERSAL_METADATA_RELATIONS).
|
||||
description: Define types, entities, and central constant registrations for syncable entities in Twenty's workspace migration system. Use when creating new syncable entities, defining TypeORM entities, flat entity types, or registering in central constants (ALL_ENTITY_PROPERTIES_CONFIGURATION_BY_METADATA_NAME, ALL_ONE_TO_MANY_METADATA_RELATIONS, ALL_MANY_TO_ONE_METADATA_FOREIGN_KEY, ALL_MANY_TO_ONE_METADATA_RELATIONS).
|
||||
---
|
||||
|
||||
# Syncable Entity: Types & Constants (Step 1/6)
|
||||
@@ -18,7 +18,7 @@ This step creates:
|
||||
2. TypeORM entity (extends `SyncableEntity`)
|
||||
3. Flat entity types
|
||||
4. Action types (universal + flat)
|
||||
5. Central constant registrations (4 constants)
|
||||
5. Central constant registrations (5 constants)
|
||||
|
||||
---
|
||||
|
||||
@@ -225,61 +225,91 @@ export const ALL_ENTITY_PROPERTIES_CONFIGURATION_BY_METADATA_NAME = {
|
||||
- `toStringify: true` → JSONB/object property (needs JSON serialization)
|
||||
- `universalProperty` → Maps to universal version (for foreign keys & JSONB with `SerializedRelation`)
|
||||
|
||||
### 6c. ALL_METADATA_RELATIONS
|
||||
### 6c. ALL_ONE_TO_MANY_METADATA_RELATIONS
|
||||
|
||||
**File**: `src/engine/metadata-modules/flat-entity/constant/all-metadata-relations.constant.ts`
|
||||
**File**: `src/engine/metadata-modules/flat-entity/constant/all-one-to-many-metadata-relations.constant.ts`
|
||||
|
||||
This constant is **type-checked** — values for `metadataName`, `flatEntityForeignKeyAggregator`, and `universalFlatEntityForeignKeyAggregator` are derived from entity type definitions. The aggregator names follow the pattern: remove trailing `'s'` from the relation property name, then append `Ids` or `UniversalIdentifiers`.
|
||||
|
||||
```typescript
|
||||
export const ALL_METADATA_RELATIONS = {
|
||||
export const ALL_ONE_TO_MANY_METADATA_RELATIONS = {
|
||||
// ... existing entries
|
||||
myEntity: {
|
||||
manyToOne: {
|
||||
workspace: null,
|
||||
application: null,
|
||||
parentEntity: {
|
||||
metadataName: 'parentEntity',
|
||||
flatEntityForeignKeyAggregator: 'myEntityIds',
|
||||
foreignKey: 'parentEntityId',
|
||||
isNullable: false,
|
||||
},
|
||||
// If myEntity has a `childEntities: ChildEntityEntity[]` property:
|
||||
childEntities: {
|
||||
metadataName: 'childEntity',
|
||||
flatEntityForeignKeyAggregator: 'childEntityIds',
|
||||
universalFlatEntityForeignKeyAggregator: 'childEntityUniversalIdentifiers',
|
||||
},
|
||||
oneToMany: {
|
||||
childEntities: { metadataName: 'childEntity' },
|
||||
},
|
||||
// Only if JSONB contains SerializedRelation fields
|
||||
serializedRelations: {
|
||||
fieldMetadata: true,
|
||||
// null for relations to non-syncable entities
|
||||
someNonSyncableRelation: null,
|
||||
},
|
||||
} as const;
|
||||
```
|
||||
|
||||
### 6d. ALL_MANY_TO_ONE_METADATA_FOREIGN_KEY
|
||||
|
||||
**File**: `src/engine/metadata-modules/flat-entity/constant/all-many-to-one-metadata-foreign-key.constant.ts`
|
||||
|
||||
Low-level primitive constant. Only contains `foreignKey` — the column name ending in `Id` that stores the foreign key. Type-checked against entity properties.
|
||||
|
||||
```typescript
|
||||
export const ALL_MANY_TO_ONE_METADATA_FOREIGN_KEY = {
|
||||
// ... existing entries
|
||||
myEntity: {
|
||||
workspace: null,
|
||||
application: null,
|
||||
parentEntity: {
|
||||
foreignKey: 'parentEntityId',
|
||||
},
|
||||
},
|
||||
} as const;
|
||||
```
|
||||
|
||||
### 6d. ALL_UNIVERSAL_METADATA_RELATIONS
|
||||
### 6e. ALL_MANY_TO_ONE_METADATA_RELATIONS
|
||||
|
||||
**File**: `src/engine/workspace-manager/workspace-migration/universal-flat-entity/constants/all-universal-metadata-relations.constant.ts`
|
||||
**File**: `src/engine/metadata-modules/flat-entity/constant/all-many-to-one-metadata-relations.constant.ts`
|
||||
|
||||
Derived from both `ALL_MANY_TO_ONE_METADATA_FOREIGN_KEY` (for `foreignKey` type and `universalForeignKey` derivation) and `ALL_ONE_TO_MANY_METADATA_RELATIONS` (for `inverseOneToManyProperty` key constraint). This is the main constant consumed by utils and optimistic tooling.
|
||||
|
||||
```typescript
|
||||
export const ALL_UNIVERSAL_METADATA_RELATIONS = {
|
||||
export const ALL_MANY_TO_ONE_METADATA_RELATIONS = {
|
||||
// ... existing entries
|
||||
myEntity: {
|
||||
manyToOne: {
|
||||
workspace: null,
|
||||
application: null,
|
||||
parentEntity: {
|
||||
metadataName: 'parentEntity',
|
||||
foreignKey: 'parentEntityId',
|
||||
universalForeignKey: 'parentEntityUniversalIdentifier',
|
||||
universalFlatEntityForeignKeyAggregator: 'myEntityUniversalIdentifiers',
|
||||
isNullable: false,
|
||||
},
|
||||
},
|
||||
oneToMany: {
|
||||
childEntities: { metadataName: 'childEntity' },
|
||||
workspace: null,
|
||||
application: null,
|
||||
parentEntity: {
|
||||
metadataName: 'parentEntity',
|
||||
foreignKey: 'parentEntityId',
|
||||
inverseOneToManyProperty: 'myEntities', // key in ALL_ONE_TO_MANY_METADATA_RELATIONS['parentEntity'], or null if no inverse
|
||||
isNullable: false,
|
||||
universalForeignKey: 'parentEntityUniversalIdentifier',
|
||||
},
|
||||
},
|
||||
} as const;
|
||||
```
|
||||
|
||||
**Derivation dependency graph**:
|
||||
|
||||
```
|
||||
ALL_MANY_TO_ONE_METADATA_FOREIGN_KEY ALL_ONE_TO_MANY_METADATA_RELATIONS
|
||||
(foreignKey only) (metadataName, aggregators)
|
||||
│ │
|
||||
│ FK type + universalFK derivation │ inverseOneToManyProperty keys
|
||||
│ │
|
||||
└────────────────┬───────────────────────┘
|
||||
▼
|
||||
ALL_MANY_TO_ONE_METADATA_RELATIONS
|
||||
(metadataName, foreignKey, inverseOneToManyProperty,
|
||||
isNullable, universalForeignKey)
|
||||
```
|
||||
|
||||
**Rules**:
|
||||
- `workspace: null`, `application: null` — always present, always null (non-syncable relations)
|
||||
- `inverseOneToManyProperty` — must be a key in `ALL_ONE_TO_MANY_METADATA_RELATIONS[targetMetadataName]`, or `null` if the target entity doesn't expose an inverse one-to-many relation
|
||||
- `universalForeignKey` — derived from `foreignKey` by replacing the `Id` suffix with `UniversalIdentifier`
|
||||
- Optimistic utils resolve `flatEntityForeignKeyAggregator` / `universalFlatEntityForeignKeyAggregator` at runtime by looking up `inverseOneToManyProperty` in `ALL_ONE_TO_MANY_METADATA_RELATIONS`
|
||||
|
||||
---
|
||||
|
||||
## Checklist
|
||||
@@ -295,8 +325,9 @@ Before moving to Step 2:
|
||||
- [ ] Universal and flat action types defined
|
||||
- [ ] Registered in `AllFlatEntityTypesByMetadataName`
|
||||
- [ ] Registered in `ALL_ENTITY_PROPERTIES_CONFIGURATION_BY_METADATA_NAME`
|
||||
- [ ] Registered in `ALL_METADATA_RELATIONS`
|
||||
- [ ] Registered in `ALL_UNIVERSAL_METADATA_RELATIONS`
|
||||
- [ ] Registered in `ALL_ONE_TO_MANY_METADATA_RELATIONS` (if entity has one-to-many relations)
|
||||
- [ ] Registered in `ALL_MANY_TO_ONE_METADATA_FOREIGN_KEY`
|
||||
- [ ] Registered in `ALL_MANY_TO_ONE_METADATA_RELATIONS`
|
||||
- [ ] TypeScript compiles without errors
|
||||
|
||||
---
|
||||
|
||||
Reference in New Issue
Block a user