867b03393bc60ebc65d5bc3556d5501b58c523f6
8 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Paul Rastoin
|
476bdf764c |
Refactor flat entity maps to be universal oriented (#17665)
# Introduction
In preparation of the workspace agnostic builder, we're migrating
`FlatEntityMaps` to be universal identifier oriented and based
As in the builder context there're won't be any ids at all
Please also note that the FlatEntity is a UniversalFlatEntity superset
From
```ts
import { type SyncableFlatEntity } from 'src/engine/metadata-modules/flat-entity/types/flat-entity-from.type';
export type FlatEntityMaps<T extends SyncableFlatEntity> = {
byId: Partial<Record<string, T>>;
idByUniversalIdentifier: Partial<Record<string, string>>;
universalIdentifiersByApplicationId: Partial<Record<string, string[]>>;
};
```
To
```ts
export type FlatEntityMaps<
T extends SyncableFlatEntity | UniversalSyncableFlatEntity,
> = {
byUniversalIdentifier: Partial<Record<string, T>>;
universalIdentifierById: Partial<Record<string, string>>;
universalIdentifiersByApplicationId: Partial<Record<string, string[]>>; // this might make more sense to be migrated to universalIdentifiersByApplicationUniversalIdentifier but it's the main topic of this PR
};
```
## Low level maps tools
Had to refactor find | create | delete | replace | find-many | get-sub
tools ( through mutations and or throw equivalent )
|
||
|
Paul Rastoin
|
bd9688421f |
ObjectMetadata and FieldMetadata agnostic workspace migration runner (#17572)
# Introduction
Important note: This PR officially deprecates the `standardId`, about to
drop col and entity property after this has been merged
Important note2: Haven't updated the optimistic tool to also update the
universal identifier aggregators only the ids one, they should not be
consumed in the runner context -> need to improve typing or either the
optimistic tooling
In this PR we're introducing all the devxp allowing future metadata
incremental universal migration -> this has an impact on all existing
metadata actions handler ( explaining its size )
This PR also introduce workspace agnostic create update actions runner
for both field and object metadata in order to battle test the described
above devxp
Noting that these two metadata are the most complex to handle
Notes:
- A workspace migration is now highly bind to a
`applicationUniversalIdentifier`. Though we don't strictly validate
application scope for the moment
## Next
Migrate both object and field builder to universal comparison
## Universal Actions vs Flat Actions Architecture
### Concept
The migration system uses a two-phase action model:
1. **Universal Actions** - Actions defined using `universalIdentifier`
(stable, portable identifiers like `standardId` + `applicationId`)
2. **Flat Actions** - Actions defined using database `entityId` (UUIDs
specific to a workspace)
### Why This Separation?
- **Universal actions are portable**: They can be serialized, stored,
and replayed across different workspaces
- **Flat actions are executable**: They contain the actual database IDs
needed to perform operations
- **Decoupling**: The builder produces universal actions; the runner
transpiles them to flat actions at execution time
### Transpiler Pattern
Each action handler must implement
`transpileUniversalActionToFlatAction()`:
```typescript
@Injectable()
export class CreateFieldActionHandlerService extends WorkspaceMigrationRunnerActionHandler(
'create',
'fieldMetadata',
) {
override async transpileUniversalActionToFlatAction(
context: WorkspaceMigrationActionRunnerArgs<UniversalCreateFieldAction>,
): Promise<FlatCreateFieldAction> {
// Resolve universal identifiers to database IDs
const flatObjectMetadata = findFlatEntityByUniversalIdentifierOrThrow({
flatEntityMaps: allFlatEntityMaps.flatObjectMetadataMaps,
universalIdentifier: action.objectMetadataUniversalIdentifier,
});
return {
type: action.type,
metadataName: action.metadataName,
objectMetadataId: flatObjectMetadata.id, // Resolved ID
flatFieldMetadatas: /* ... transpiled entities ... */,
};
}
}
```
### Action Handler Base Class
`BaseWorkspaceMigrationRunnerActionHandlerService<TActionType,
TMetadataName>` provides:
- **`transpileUniversalActionToFlatAction()`** - Abstract method each
handler must implement
- **`transpileUniversalDeleteActionToFlatDeleteAction()`** - Shared
helper for delete actions
## FlatEntityMaps custom properties
Introduced a `TWithCustomMapsProperties` generic parameter to control
whether custom indexing structures are included:
- **`false` (default)**: Returns `FlatEntityMaps<MetadataFlatEntity<T>>`
- used in builder/runner contexts
- **`true`**: Returns the full maps type with custom properties (e.g.,
`byUserWorkspaceIdAndFolderId`) - used in cache contexts
## Create Field Actions Refactor
Refactored create-field actions to support relation field pairs
bundling.
**Problem:** Relation fields (e.g., `Attachment.targetTask` ↔
`Task.attachments`) couldn't resolve each other's IDs during
transpilation because they were in separate actions with independent
`fieldIdByUniversalIdentifier` maps.
**Solution:**
- Removed `objectMetadataUniversalIdentifier` from
`UniversalCreateFieldAction` and `objectMetadataId` from
`FlatCreateFieldAction` - each field now carries its own
- Runner groups fields by object internally and processes each table
separately
- Split aggregator into two focused utilities:
- `aggregateNonRelationFieldsIntoObjectActions` - merges non-relation
fields into object actions
- `aggregateRelationFieldPairs` - bundles relation pairs with shared
`fieldIdByUniversalIdentifier`
|
||
|
Marie
|
0001c2e7d0 |
[Apps] Apps marketplace (first draft) (#17562)
https://github.com/user-attachments/assets/c4e63edb-6e98-44bc-841f-ee110ae712d4 How it works - `manifest.json` files are now committed when apps are published to our repo - to display available apps, from the server, we read into our github repo, using a pod-scoped cache - feature flagged - app installation will be behind permission gate MARKETPLACE_APPS Limitations and what is yet to develop - content and settings tabs - installed apps tab - app installation - test and potentially fix reading from .manifest.json once [Reshape manifest structure](https://github.com/twentyhq/core-team-issues/issues/2183) is done. additional work is expected on assets notably. (couldnt properly do it here as manifest.json will only be committed after this pr) - we only read in community/ folder for now - we may want tochange that - the cache is rather artisanal for now and scoped by pod - we may want to change that |
||
|
Abdul Rahman
|
51a2a90b0a |
feat: CommandMenuItem entity and FrontComponent support in command menu (#17555)
https://github.com/user-attachments/assets/6f172ffc-d43d-42fd-a26b-94f591fb767e |
||
|
Paul Rastoin
|
2a28c34a37 |
Refactor workspace migration runner exception handling (#17310)
# Introduction
In this PR we catch all the runner errors coming from a single workspace
migration action execution.
**1. `WorkspaceMigrationActionExecutionException`** (low-level,
action-specific)
- Thrown from action handlers, utils, and helper functions
- Contains specific error codes: `FIELD_METADATA_NOT_FOUND`,
`OBJECT_METADATA_NOT_FOUND`, `ENUM_OPERATION_FAILED`, `NOT_SUPPORTED`,
etc.
- Simple structure: `message`, `code`, `userFriendlyMessage`
- No action context - just describes what went wrong
**2. `WorkspaceMigrationRunnerException`** (high-level, runner-scoped)
- Only two codes: `INTERNAL_SERVER_ERROR` and `EXECUTION_FAILED`
- `EXECUTION_FAILED` **requires** `action` + `errors` (contains the
action context)
- `INTERNAL_SERVER_ERROR` **requires** `message` (no action context)
## Refactor
- Removed the `relatedFlatEntityMapsKeys` from the `WorkspaceMigration`
type as they're directly inferred from passed actions
- Swallowing actions rollbacks errors in order to iterate over all of
them
## Testing
Created a very straigthforward install application from workspace
migration endpoint in order to start testing the introduced
`WorkspaceMigrationActionExecutionException`
Introduced a feature flag that stop the access to the endpoint if not
enabled
Whole taken direction are totally subjective and highly prone to
mutations ( endpoint location, naming and input schema see
`ts-expect-error` comment )
cc @martmull
## Response error
```ts
{
"eventId": "evt_a1b2c3d4-5678-90ab-cdef-1234567890ab",
"extensions": {
"action": {
"metadataName": "fieldMetadata",
"type": "delete",
"universalIdentifier": "20202020-6110-4547-9fd0-2525257a2c3f"
},
"code": "APPLICATION_INSTALLATION_FAILED",
"errors": {
"metadata": {
"code": "ENTITY_NOT_FOUND",
"message": "Could not find flat entity with universal identifier 20202020-6110-4547-9fd0-2525257a2c3f"
},
"workspaceSchema": {
"code": "ENTITY_NOT_FOUND",
"message": "Could not find flat entity in maps"
}
},
"exceptionEventId": "exc_f9e8d7c6-5432-10ba-fedc-ba0987654321",
"userFriendlyMessage": "Migration execution failed."
},
"message": "Migration action 'delete' for 'fieldMetadata' failed",
"name": "GraphQLError"
}
```
|
||
|
Paul Rastoin
|
596b7cc62d |
Deprecate nullable syncableEntity (#17279)
# Introduction As we've been identifying both standard and custom entities for all the metadata that had standard We now still need to identify all custom entities enforcing them to have an `applicationId` and `universalIdentifier` In this PR we've removed the `SyncableEntityRequired` in favor requiring props directly in the `SyncableEntity` Which means that all metadata in db will now expect non nullable applicationId and universalIdentifier across the whole application Will add some type cleanup later in https://github.com/twentyhq/twenty/pull/17277 |
||
|
Paul Rastoin
|
fae6d0e262 |
Improve cleaning job (#17208)
# Introduction Refactored the workspace deletion to dynamically iterate over all known v2 syncable entities repos and delete all of them from child to parent Exception for field metadata that we chunk delete in order to avoid locking the core schema too long, it does not have an impact on perfs at all ( neither plus or less ) Chunking by constraint within a transaction is not necessary both does not cost more ## From 30s for a workspace complete deletion ```ts [Nest] 93244 - 01/16/2026, 10:24:52 PM LOG [WorkspaceService] workspace WS_ID cache flushed [Runner] Total execution: 26.290s // ( deleteAllObjectMetadatas v2 ) [Nest] 93244 - 01/16/2026, 10:25:22 PM LOG [WorkspaceService] workspace WS_ID hard deleted ``` ## To 3s ! ```ts [Nest] 65112 - 01/18/2026, 4:37:38 PM LOG [DatabaseConfigDriver] [INIT] Config variables loaded: 0 values found in DB, 69 falling to env vars/defaults [Nest] 65112 - 01/18/2026, 4:37:38 PM LOG [CleanSuspendedWorkspacesCommand] IGNORING GRACE PERIOD - Cleaning 1 suspended workspaces [Nest] 65112 - 01/18/2026, 4:37:38 PM LOG [CleanerWorkspaceService] batchWarnOrCleanSuspendedWorkspaces running... [Nest] 65112 - 01/18/2026, 4:37:38 PM LOG [CleanerWorkspaceService] Processing workspace - 1/1 [Nest] 65112 - 01/18/2026, 4:37:38 PM LOG [CleanerWorkspaceService] Destroying workspace Twenty Eng [Nest] 65112 - 01/18/2026, 4:37:38 PM LOG [WorkspaceService] workspace user workspaces deleted [Nest] 65112 - 01/18/2026, 4:37:38 PM LOG [WorkspaceService] workspace cache flushed [Nest] 65112 - 01/18/2026, 4:37:38 PM LOG [WorkspaceService] workspace: deleted 80 viewFilter record(s) [Nest] 65112 - 01/18/2026, 4:37:38 PM LOG [WorkspaceService] workspace: deleted 21 pageLayoutWidget record(s) [Nest] 65112 - 01/18/2026, 4:37:38 PM LOG [WorkspaceService] workspace: deleted 1515 viewField record(s) [Nest] 65112 - 01/18/2026, 4:37:38 PM LOG [WorkspaceService] workspace: deleted 91 index record(s) [Nest] 65112 - 01/18/2026, 4:37:38 PM LOG [WorkspaceService] workspace: deleted 66 roleTarget record(s) [Nest] 65112 - 01/18/2026, 4:37:38 PM LOG [WorkspaceService] workspace: deleted 174 viewGroup record(s) [Nest] 65112 - 01/18/2026, 4:37:38 PM LOG [WorkspaceService] workspace: deleted 1 agent record(s) [Nest] 65112 - 01/18/2026, 4:37:38 PM LOG [WorkspaceService] workspace: deleted 7 pageLayout record(s) [Nest] 65112 - 01/18/2026, 4:37:38 PM LOG [WorkspaceService] workspace: deleted 111 view record(s) [Nest] 65112 - 01/18/2026, 4:37:38 PM LOG [WorkspaceService] workspace: fieldMetadata chunk 1/15 - deleted 50 record(s) [Nest] 65112 - 01/18/2026, 4:37:38 PM LOG [WorkspaceService] workspace: fieldMetadata chunk 2/15 - deleted 50 record(s) [Nest] 65112 - 01/18/2026, 4:37:38 PM LOG [WorkspaceService] workspace: fieldMetadata chunk 3/15 - deleted 50 record(s) [Nest] 65112 - 01/18/2026, 4:37:38 PM LOG [WorkspaceService] workspace: fieldMetadata chunk 4/15 - deleted 50 record(s) [Nest] 65112 - 01/18/2026, 4:37:38 PM LOG [WorkspaceService] workspace: fieldMetadata chunk 5/15 - deleted 50 record(s) [Nest] 65112 - 01/18/2026, 4:37:38 PM LOG [WorkspaceService] workspace: fieldMetadata chunk 6/15 - deleted 50 record(s) [Nest] 65112 - 01/18/2026, 4:37:38 PM LOG [WorkspaceService] workspace: fieldMetadata chunk 7/15 - deleted 50 record(s) [Nest] 65112 - 01/18/2026, 4:37:38 PM LOG [WorkspaceService] workspace: fieldMetadata chunk 8/15 - deleted 50 record(s) [Nest] 65112 - 01/18/2026, 4:37:38 PM LOG [WorkspaceService] workspace: fieldMetadata chunk 9/15 - deleted 50 record(s) [Nest] 65112 - 01/18/2026, 4:37:38 PM LOG [WorkspaceService] workspace: fieldMetadata chunk 10/15 - deleted 50 record(s) [Nest] 65112 - 01/18/2026, 4:37:38 PM LOG [WorkspaceService] workspace: fieldMetadata chunk 11/15 - deleted 50 record(s) [Nest] 65112 - 01/18/2026, 4:37:38 PM LOG [WorkspaceService] workspace: fieldMetadata chunk 12/15 - deleted 51 record(s) [Nest] 65112 - 01/18/2026, 4:37:38 PM LOG [WorkspaceService] workspace: fieldMetadata chunk 13/15 - deleted 50 record(s) [Nest] 65112 - 01/18/2026, 4:37:38 PM LOG [WorkspaceService] workspace: fieldMetadata chunk 14/15 - deleted 50 record(s) [Nest] 65112 - 01/18/2026, 4:37:38 PM LOG [WorkspaceService] workspace: fieldMetadata chunk 15/15 - deleted 36 record(s) [Nest] 65112 - 01/18/2026, 4:37:38 PM LOG [WorkspaceService] workspace: deleted 737 fieldMetadata record(s) [Nest] 65112 - 01/18/2026, 4:37:38 PM LOG [WorkspaceService] workspace: deleted 6 role record(s) [Nest] 65112 - 01/18/2026, 4:37:38 PM LOG [WorkspaceService] workspace: deleted 78 serverlessFunction record(s) [Nest] 65112 - 01/18/2026, 4:37:39 PM LOG [WorkspaceService] workspace: deleted 43 objectMetadata record(s) [Nest] 65112 - 01/18/2026, 4:37:41 PM LOG [WorkspaceService] workspace hard deleted [Nest] 65112 - 01/18/2026, 4:37:41 PM LOG [CleanerWorkspaceService] Destroyed 1 workspaces on 5 limit durings this execution [Nest] 65112 - 01/18/2026, 4:37:41 PM LOG [CleanerWorkspaceService] batchWarnOrCleanSuspendedWorkspaces done! [Nest] 65112 - 01/18/2026, 4:37:41 PM LOG [CleanSuspendedWorkspacesCommand] Command completed! ``` ## Update Discussed with @charlesBochet ended debugging and analyzing sql query operations He discovered that we were not indexing foreignKey effectively We've ended up fixing all the FK indeces coverage leading to ## Cleaning Removed the ```sh npx nx run twenty-server:command workspace:clean-soft-deleted-suspended-workspaces --ignore-grace-period ``` In favor of ```sh npx nx run twenty-server:command workspace:clean --only-operation destroy --ignore-destroy-grace-period ``` ## Conclusion Not that crazy but still worth it and could demultiply in production |
||
|
Abdul Rahman
|
5a617e4718 | Add command menu item entity (#17181) |