e8d4f11d8e04eb798945a84d434e268ae15eb55f
106 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Paul Rastoin
|
4996f3dd28 |
Finalize twenty standard app as workspace migration object and fields (#16353)
# Introduction Related to https://github.com/twentyhq/core-team-issues/issues/1995 In this PR we're fixing the remaining object/fields validation errors resulting from standard objects and fields now passing a validation that wasn't when using the sync metadata ## Key Changes - **Field naming**: Renamed `iCalUID` to `iCalUid` for consistent camelCase convention across calendar events - **Enum standardization**: Uppercased enum values for message channels (email→EMAIL), message participants (from→FROM, to→TO, cc→CC, bcc→BCC), and message direction (incoming→INCOMING, outgoing→OUTGOING) - **Label simplification**: Removed example values from workspace member number format labels for cleaner UI - **Migration infrastructure**: Added `isSystemBuild` flag throughout field metadata service pipeline to allow system-level updates of standard fields that bypass normal restrictions ## Migrating the existing data We've created an upgrade command that will identify using the existing object and field standard id field that needs to be updated, even though the sync metadata still in usage could have fix them ( and the goal is to deprecate it by the end of the sprint ) We will call the updateOneField for each of them, we're passing by the field service in order to battle test what are going to be the temporary way to handle standard migrations when we will start deprecating the sync metadata but haven't still refactored the v2 workspace migration to be workspace agnostic ## Twenty eng migration Tested the whole migration + upgrade on twenty eng Here are generated workspace migration Records are handled natively gracefully too ### ICalUid ```json { "status": "success", "workspaceMigration": { "relatedFlatEntityMapsKeys": [ "flatFieldMetadataMaps", "flatIndexMaps", "flatViewFilterMaps", "flatViewGroupMaps", "flatViewMaps", "flatViewFieldMaps", "flatObjectMetadataMaps" ], "actions": [ { "type": "update_field", "fieldMetadataId": "", "objectMetadataId": "", "updates": [ { "from": "iCalUID", "to": "iCalUid", "property": "name" } ] } ], "workspaceId": "" } } ``` ### Incoming Outgoing None as already caps in database somehow ```json { "status": "success", "workspaceMigration": { "relatedFlatEntityMapsKeys": [ "flatFieldMetadataMaps", "flatIndexMaps", "flatViewFilterMaps", "flatViewGroupMaps", "flatViewMaps", "flatViewFieldMaps", "flatObjectMetadataMaps" ], "actions": [], "workspaceId": "" } } ``` ### EMAIL ```json { "status": "success", "workspaceMigration": { "relatedFlatEntityMapsKeys": [ "flatFieldMetadataMaps", "flatIndexMaps", "flatViewFilterMaps", "flatViewGroupMaps", "flatViewMaps", "flatViewFieldMaps", "flatObjectMetadataMaps" ], "actions": [ { "type": "update_field", "fieldMetadataId": "", "objectMetadataId": "", "updates": [ { "from": "'email'", "to": "'EMAIL'", "property": "defaultValue" }, { "from": [ { "color": "green", "id": "", "label": "Email", "position": 0, "value": "email" }, { "color": "blue", "id": "", "label": "SMS", "position": 1, "value": "sms" } ], "to": [ { "color": "green", "id": "", "label": "Email", "position": 0, "value": "EMAIL" }, { "color": "blue", "id": "", "label": "SMS", "position": 1, "value": "SMS" } ], "property": "options" } ] } ], "workspaceId": "e" } } ``` ### MessageParticipantRole ```json { "status": "success", "workspaceMigration": { "relatedFlatEntityMapsKeys": [ "flatFieldMetadataMaps", "flatIndexMaps", "flatViewFilterMaps", "flatViewGroupMaps", "flatViewMaps", "flatViewFieldMaps", "flatObjectMetadataMaps" ], "actions": [ { "type": "update_field", "fieldMetadataId": "", "objectMetadataId": "", "updates": [ { "from": "'from'", "to": "'FROM'", "property": "defaultValue" }, { "from": [ { "color": "green", "id": "", "label": "From", "position": 0, "value": "from" }, { "color": "blue", "id": "", "label": "To", "position": 1, "value": "to" }, { "color": "orange", "id": "", "label": "Cc", "position": 2, "value": "cc" }, { "color": "red", "id": "", "label": "Bcc", "position": 3, "value": "bcc" } ], "to": [ { "color": "green", "id": "", "label": "From", "position": 0, "value": "FROM" }, { "color": "blue", "id": "", "label": "To", "position": 1, "value": "TO" }, { "color": "orange", "id": "", "label": "Cc", "position": 2, "value": "CC" }, { "color": "red", "id": "", "label": "Bcc", "position": 3, "value": "BCC" } ], "property": "options" } ] } ], "workspaceId": "" } } ``` ### Workspace member number format labels ```json { "status": "success", "workspaceMigration": { "relatedFlatEntityMapsKeys": [ "flatFieldMetadataMaps", "flatIndexMaps", "flatViewFilterMaps", "flatViewGroupMaps", "flatViewMaps", "flatViewFieldMaps", "flatObjectMetadataMaps" ], "actions": [ { "type": "update_field", "fieldMetadataId": "", "objectMetadataId": "", "updates": [ { "from": [ { "color": "turquoise", "id": "", "label": "System", "position": 0, "value": "SYSTEM" }, { "color": "blue", "id": "", "label": "Commas and dot (1,234.56)", "position": 1, "value": "COMMAS_AND_DOT" }, { "color": "green", "id": "", "label": "Spaces and comma (1 234,56)", "position": 2, "value": "SPACES_AND_COMMA" }, { "color": "orange", "id": "", "label": "Dots and comma (1.234,56)", "position": 3, "value": "DOTS_AND_COMMA" }, { "color": "purple", "id": "", "label": "Apostrophe and dot (1'234.56)", "position": 4, "value": "APOSTROPHE_AND_DOT" } ], "to": [ { "color": "turquoise", "id": "", "label": "System", "position": 0, "value": "SYSTEM" }, { "color": "blue", "id": "", "label": "Commas and dot", "position": 1, "value": "COMMAS_AND_DOT" }, { "color": "green", "id": "", "label": "Spaces and comma", "position": 2, "value": "SPACES_AND_COMMA" }, { "color": "orange", "id": "", "label": "Dots and comma", "position": 3, "value": "DOTS_AND_COMMA" }, { "color": "purple", "id": "", "label": "Apostrophe and dot", "position": 4, "value": "APOSTROPHE_AND_DOT" } ], "property": "options" } ] } ], "workspaceId": "" } } ``` |
||
|
Raphaël Bosi
|
5fb7e76005 | Migrate page layout to v2 (#16364) | ||
|
martmull
|
7f1e69740a |
1895 extensibility v1 application tokens (#16365)
First PR to implement application tokens - add new application role in twenty-server - move duplicated constants and types to twenty-shared - will add role configuration utils into twenty-sdk in another PR |
||
|
Paul Rastoin
|
c8f541618d |
Refactor validate build and run for configuration to be less verbose and more reliable (#16343)
# Introduction Refactored the api `validateBuildAndRunWorkspaceMigration` to be require less configuration but to infer required args dynamically depending on provided metadata maps to compare ## `inferDeletionFromMissingEntities` Is not dynamically computed avoiding any miss configuration issue and any missleading devxp ## Maps computation Making only one call to redis to build both dependency and to be compared entity maps. It does not matter to avoid passing a about to compared flat entity maps to could also be a depedency, it's handled directly in the builder setup optimistic cache logic Please note that the flat maps used for the service input transpilation might differ from the one that we will dynamically compute and inject in the builder. Leading to do 2 redis calls but also race condition prone validation error We prefer that this occurs at the builder rather than at the runner level as the pg instance is not cache and reflect the real state of a given workspace In a nutshell, there's a possible race condition between cache invalidation and computation in both service input transpilers and builder but we're totally ok with that |
||
|
Marie
|
7ce22d5c7e |
breaking (soft) - Migrate viewGroup.fieldMetadataId -> view.mainGroupByFieldMetadataId (2/3) (#16277)
Should be merged once https://github.com/twentyhq/twenty/pull/16206 has been released + command run to prod In this PR - Remove usage of viewGroup.fieldMetadataId, both in BE and FE states. - But we still need to properly populate it until we fully remove viewGroup.fieldMetadataId from db and ORM entity (upcoming 3rd PR out of 3). fieldMetadataId was removed from CoreViewGroup type and CreateViewGroupInput and is determined BE-side based on the associated view's mainGroupByFieldMetadataId. **I expect this means a downtime on viewGroup creation, until both FE and BE are deployed and cache is flushed.** This seems acceptable to me as it only regards viewGroup creation. - this information is replaced by view.mainGroupByFieldMetadataID - Handle view group creation, update and deletion in the BE as a side-effect of a view creation, update or deletion. Optimistic effects are still used - Add validation at view creation or update regarding mainGroupByFieldMetadata Left to do in 3rd PR - Remove viewGroup.fieldMetadataId from db and ORM entity - Restore feature allowing to update an existing grouped view's group by field (already OK on BE side but need to rebuild FE optimistic) |
||
|
Raphaël Bosi
|
077be7644c | Migrate page layout widget to v2 of the API (#16323) | ||
|
Paul Rastoin
|
28cdb02fbb |
Twenty standard application Objects and fields as allFlatEntityMaps ID non-agnostic (#16298)
# Introduction Related to https://github.com/twentyhq/core-team-issues/issues/1995 This PR introduces the basis of the `twentyStandard` application as code on demand, it's highly tied to `ids` where it will becomes workspace agnostic following the builder and runner `universalIdentifier` refactor later. The goal here to allow computing the `allFlatEntityMaps` `to` of the `twentyStandard` application on a empty workspace ( workspace creation ). Allowing installing the twenty standard app through a workspace migration instead of passing by the sync metadata Nothing done will be run in production for the moment if it's not the small validation refactor we've introduced Please note that everything introduced here will be replaced at some point by a twenty app instance when the twenty sdk is mature enough to handle of the edge cases we need here ## How we've proceeded We've been iterating over every workspace entity both objects and their fields, and transpiled them to flatEntity. Being sure we migrate the defaultValue, settings and so on accordingly. We've also compute all the ids in prior of the whole entities computation so we don't face any hoisting issue. ## Current state At the moment only handling all of the 29 standard objects and their fields Settings a unique universalIdentifier for all of them Will come views, agent role targets and so on later ## `workspace:compute-twenty-standard-migration` command This command allow generating a workspace migration that will result in installing the twenty standard app in an empty workspace It's temporary and aims to allow debugging for the moment we might not keep it in the future as it is right now It contains debug writeFileSync which is expected no worries greptile ## `LabelFieldMetadataIdentifierId` Small refactor allowing defining the label identifier field metadata id of a uuid field metadata type for system object, as some of our standard object don't have a name field and don't aim to Also please note that we might remove this build options later in the sake of the currently installed universal identifier application that we could compare with the deterministic twenty standard one ## `runFlatFieldMetadataValidators` Deprecated this pattern which was redundant and not v2 friendly pattern ## Current errors that will address in upcoming PR Current standard objects and fields metadata does not pass the validation that we have in place, as historically the sync metadata would directly consume the repositories and would just ignore the validation. This is about to change. Will handle the below errors in dedicated PRs as they will required upgrade commands in order to migrate the data, or will handle that from the sync metadata instead still to be determined but nothing critical here - camel case field metadata name - options label invalid format ```json { "status": "fail", "report": { "fieldMetadata": [ { "status": "fail", "errors": [ { "code": "INVALID_FIELD_INPUT", "message": "Name should be in camelCase", "userFriendlyMessage": { "id": "P+jdmX", "message": "Name should be in camelCase" }, "value": "iCalUID" } ], "flatEntityMinimalInformation": { "id": "68dd83cd-92c8-4233-bb28-47939bab6124", "name": "iCalUID", "objectMetadataId": "11c16ab6-9176-439e-a2db-a12c5a58a524" }, "type": "create_field" }, { "status": "fail", "errors": [ { "code": "INVALID_FIELD_INPUT", "message": "Value must be in UPPER_CASE and follow snake_case \"email\"", "userFriendlyMessage": { "id": "UBPzFQ", "message": "Value must be in UPPER_CASE and follow snake_case \"{sanitizedValue}\"", "values": { "sanitizedValue": "email" } }, "value": "email" }, { "code": "INVALID_FIELD_INPUT", "message": "Value must be in UPPER_CASE and follow snake_case \"sms\"", "userFriendlyMessage": { "id": "UBPzFQ", "message": "Value must be in UPPER_CASE and follow snake_case \"{sanitizedValue}\"", "values": { "sanitizedValue": "sms" } }, "value": "sms" } ], "flatEntityMinimalInformation": { "id": "e3caaf2a-e07d-4146-8dfc-9eef904e82c9", "name": "type", "objectMetadataId": "4b777de5-4c7b-4af4-9b92-655c0f87512b" }, "type": "create_field" }, { "status": "fail", "errors": [ { "code": "INVALID_FIELD_INPUT", "message": "Value must be in UPPER_CASE and follow snake_case \"incoming\"", "userFriendlyMessage": { "id": "UBPzFQ", "message": "Value must be in UPPER_CASE and follow snake_case \"{sanitizedValue}\"", "values": { "sanitizedValue": "incoming" } }, "value": "incoming" }, { "code": "INVALID_FIELD_INPUT", "message": "Value must be in UPPER_CASE and follow snake_case \"outgoing\"", "userFriendlyMessage": { "id": "UBPzFQ", "message": "Value must be in UPPER_CASE and follow snake_case \"{sanitizedValue}\"", "values": { "sanitizedValue": "outgoing" } }, "value": "outgoing" } ], "flatEntityMinimalInformation": { "id": "d96233a4-93be-45ea-9548-3b50f3c700cf", "name": "direction", "objectMetadataId": "480a648a-d2e5-482a-992f-ef053e1b4bb0" }, "type": "create_field" }, { "status": "fail", "errors": [ { "code": "INVALID_FIELD_INPUT", "message": "Value must be in UPPER_CASE and follow snake_case \"from\"", "userFriendlyMessage": { "id": "UBPzFQ", "message": "Value must be in UPPER_CASE and follow snake_case \"{sanitizedValue}\"", "values": { "sanitizedValue": "from" } }, "value": "from" }, { "code": "INVALID_FIELD_INPUT", "message": "Value must be in UPPER_CASE and follow snake_case \"to\"", "userFriendlyMessage": { "id": "UBPzFQ", "message": "Value must be in UPPER_CASE and follow snake_case \"{sanitizedValue}\"", "values": { "sanitizedValue": "to" } }, "value": "to" }, { "code": "INVALID_FIELD_INPUT", "message": "Value must be in UPPER_CASE and follow snake_case \"cc\"", "userFriendlyMessage": { "id": "UBPzFQ", "message": "Value must be in UPPER_CASE and follow snake_case \"{sanitizedValue}\"", "values": { "sanitizedValue": "cc" } }, "value": "cc" }, { "code": "INVALID_FIELD_INPUT", "message": "Value must be in UPPER_CASE and follow snake_case \"bcc\"", "userFriendlyMessage": { "id": "UBPzFQ", "message": "Value must be in UPPER_CASE and follow snake_case \"{sanitizedValue}\"", "values": { "sanitizedValue": "bcc" } }, "value": "bcc" } ], "flatEntityMinimalInformation": { "id": "961c598e-67c3-452d-8bb2-b92c0bc64404", "name": "role", "objectMetadataId": "8af8a13c-ff97-4cd3-b70d-52a7dc2924b4" }, "type": "create_field" }, { "status": "fail", "errors": [ { "code": "INVALID_FIELD_INPUT", "message": "Label must not contain a comma", "userFriendlyMessage": { "id": "k731jp", "message": "Label must not contain a comma" }, "value": "Commas and dot (1,234.56)" }, { "code": "INVALID_FIELD_INPUT", "message": "Label must not contain a comma", "userFriendlyMessage": { "id": "k731jp", "message": "Label must not contain a comma" }, "value": "Spaces and comma (1 234,56)" }, { "code": "INVALID_FIELD_INPUT", "message": "Label must not contain a comma", "userFriendlyMessage": { "id": "k731jp", "message": "Label must not contain a comma" }, "value": "Dots and comma (1.234,56)" } ], "flatEntityMinimalInformation": { "id": "7fa20caf-2597-42e3-84e5-15a91b125b9b", "name": "numberFormat", "objectMetadataId": "a6974302-9e72-461c-aa09-9390f4ff16fc" }, "type": "create_field" } ], "objectMetadata": [], "view": [], "viewField": [], "viewGroup": [], "index": [], "serverlessFunction": [], "cronTrigger": [], "databaseEventTrigger": [], "routeTrigger": [], "viewFilter": [], "role": [], "roleTarget": [], "agent": [] } } ``` |
||
|
Paul Rastoin
|
b2d785de4b |
Page layout tab v2 (#16319)
# Introduction Migrating `pageLayoutTab` to the v2 engine - Types and constants - Builder and validate - Runner Introduced a new `StrictSyncableEntity` that enforces that `universalIdentifier` and `applicationId` are defined As these entities are brand new we could enforce this rule already This still requires a migration command to associate the existing entities to custom workspace application instance and define universalIdentifier Handled retro-comp of the migration through a migration as upgrade command fallback --------- Co-authored-by: bosiraphael <raphael.bosi@gmail.com> |
||
|
Weiko
|
3d95d6ca00 |
Add local only cache to cache service and cache typeorm entity metadata (#16287)
## Problem buildEntityMetadatas in GlobalWorkspaceOrmManager is computationally expensive and was running on every executeInWorkspaceContext call. This method uses TypeORM's EntitySchemaTransformer and EntityMetadataBuilder to build metadata for all workspace entities (30-50+ objects with many fields each). The resulting EntityMetadata[] is not serialisable which means it cannot be cached in Redis because they contain: - Circular references - Functions/methods - References to the DataSource instance ## Solution Extended the workspace cache system to support local-only caching, then created a cache provider for entityMetadatas. ## Implementation details Updated @WorkspaceCache decorator (workspace-cache.decorator.ts) - Added localOnly?: boolean option to skip Redis storage for non-serializable data Created WorkspaceEntityMetadatasCacheService - Computes entity metadatas from DB to avoid race condition, this is acceptable Simplified GlobalWorkspaceOrmManager - Now fetches entityMetadatas from cache instead of rebuilding on every call Updated Workspace migration runner - the only entry point where metadata can change - Now invalidate the new 'entityMetadata' local cache when shouldIncrementMetadataGraphqlSchemaVersion is true (== field/object mutations) |
||
|
Paul Rastoin
|
59672e3e34 |
Migrate agent v2 (#16214)
# Introduction Closes https://github.com/twentyhq/core-team-issues/issues/1980 In this PR we migrate the agent from v1 to v2. ## New FlatRoleTargetByAgentIdMaps Derivated the `flatRoleTargetMaps` to be building a `flatRoleTargetByAgentIdMaps` to ease retrieving a roleId to associate to an agent ## Coverage Added strong coverage on both failing and successful CRU agents operations --------- Co-authored-by: Weiko <corentin@twenty.com> |
||
|
Marie
|
77409b6eb2 |
[Requires "warm" cache flush (no immediate downtime before flush)] Migrate viewGroup.fieldMetadataId -> view.mainGroupByFieldMetadataId (1/3) (#16206)
In this PR (1/3) - introduce view.mainGroupByFieldMetadataId as the new reference determining which fieldMetadataId is used in a grouped view, in order to deprecate viewGroup.fieldMetadataId which creates inconsistencies. view.mainGroupByFieldMetadataId is now filled at every view creation, though not in use yet. - Introduce a command to backfill view.mainGroupByFieldMetadataId for existing views + delete all viewGroup.fieldMetadataId with a fieldMetadataId that is not view.mainGroupByFieldMetadataId. (It should concern 37 active workspaces) - Temporarily disable the option to change a grouped view's fieldMetadataId as for now it creates inconsistencies. This feature can be reintroduced when we have done the full migration. In a next PR - (2/3) use view.mainGroupByFieldMetadataId instead of viewGroup.fieldMetadataId. In FE we may keep viewGroup.fieldMetadataId as a state (TBD). View groups will now be created / deleted as a side effect of view's mainGroupByFieldMetadataId update. - (3/3) remove viewGroup.fieldMetadataId --------- Co-authored-by: Charles Bochet <charles@twenty.com> |
||
|
Weiko
|
13e283fc3a | Rename roleTargets -> roleTarget (#16247) | ||
|
Weiko
|
1eb2e44058 |
Refactor workspace cache service (#16208)
## Context We've recently introduced a new workspace cache service which now acts as a cache access and local storage for all workspace related data, deprecating the individual specific services. - Better performance through multiple caching/fetching strategies - Consistent data access patterns across the codebase - Reduced redis queries through MGET/MSET/PIPELINE with multiple cache keys |
||
|
Abdullah.
|
ee08060798 |
Improve deactivated objects & fields behaviors. (#16090)
Closes [1918](https://github.com/twentyhq/core-team-issues/issues/1918). - For the first point in the issue, we just show the deactivated entries along with the deactivated text. --- - For the second point, we show a banner and control the enabled/disabled state of save button depending on whether we're allowing the user to create table with the typed name. - For example, we do not want to allow the user to create a table with reserved name, so we disable the save button without showing a banner. - Similarly, we do not want the user to create a table with a name that already exists in the database. In this case, we show a banner and we also disable the save button. - Finally, we do not want to allow the user to create a table where singular and plural name are the same. Therefore, we disable the save button for names like `works`. --- - For the third point, if we add the delete button, it logically means that we allow the user to delete a custom object/field even it has not been deactivated yet, so did that. - Upon deleting the object/field, if we wait for the metadata to refetch before we navigate, this is what we see because the path does not exist any longer after deletion and we're waiting for refetch on the path until we navigate away. https://github.com/user-attachments/assets/dbe0569c-db88-4285-851f-22551b1ca81e - To avoid this page from appearing, I replaced awaiting refetch to not awaiting refetch and redirecting while the refetch happens in the background. - Therefore, when we delete something, there is a slight delay for when it is actually cleared out from the list, but the Not Found view does not appear on the screen. https://github.com/user-attachments/assets/47f49579-ce51-4d6a-b857-72046247bb4b - I tried optimistically removing the object/field from the metadata, but it leads to some issues (crashes the app) and I have not been able to find a solution for it yet. - Therefore, instead of getting stuck at perfection and blocking myself, I stopped getting into the issue further and created this PR by ensuring that the desired functionality works. <!-- CURSOR_SUMMARY --> --- > [!NOTE] > Display deactivated objects/fields by default, add delete actions with confirmation, and unify metadata name computation (auto-suffix reserved keywords) across front/back with conflict checks in object creation. > > - **Frontend (Settings/Data Model)**: > - **Visibility/UX**: Show `Deactivated` labels for objects/fields; filters default to include inactive (`showDeactivated`/`showInactive` true); replace field action dropdown with chevron link. > - **Delete flows**: Add delete buttons for custom objects/fields with confirmation modals and background refetch to avoid Not Found flashes. > - **Creation/Edit validation**: Add name conflict detection banner in `SettingsDataModelObjectAboutForm` and disable Save on conflicts; simplify `metadataLabelSchema` to use computed name; form fields validate on change and sync API names. > - **Shared (twenty-shared/metadata)**: > - Add `computeMetadataNameFromLabel` util (slugify+camelCase) and `RESERVED_METADATA_NAME_KEYWORDS`; auto-append `Custom` to reserved names; export constants/utilities. > - **Backend**: > - Migrate to shared `computeMetadataNameFromLabel`; update validators to use shared reserved keywords with new messages; allow deletion of active custom fields/objects (keep standard guards); adjust services/decorators accordingly. > - **Tests/Stories**: > - Update unit/integration snapshots for new reserved-name messages and behaviors; add missing i18n/router decorators in stories. > > <sup>Written by [Cursor Bugbot](https://cursor.com/dashboard?tab=bugbot) for commit 5b126155606f6dbc8f7f91e2192cffb7bd2ebd2c. This will update automatically on new commits. Configure [here](https://cursor.com/dashboard?tab=bugbot).</sup> <!-- /CURSOR_SUMMARY --> --------- Co-authored-by: Félix Malfait <felix.malfait@gmail.com> Co-authored-by: Félix Malfait <felix@twenty.com> |
||
|
Paul Rastoin
|
ea3c5d2d45 |
Migrate role and role target to v2 (#16009)
# Introduction close https://github.com/twentyhq/core-team-issues/issues/1930 close https://github.com/twentyhq/core-team-issues/issues/1929 Migrating role and roleTarget entities to the v2 core engine, allowing v2 caching leverage and allow migrating agent to v2 that needs role target in prior After agent we should be able to pass twenty standard app totally though workspace migration ## Role target assignation Please note that role target have 3 creation entrypoints: - Agent - User workspace - ApiKey Refactored all 3 of them to pass through a new role-target.service.ts that consumes the v2 under the hood. --------- Co-authored-by: Weiko <corentin@twenty.com> |
||
|
Guillim
|
1a45576990 |
Morph-add-new-object-destination (#16027)
Add new object target to an existing morph relation (backend only) Fixes https://github.com/twentyhq/core-team-issues/issues/1898 --------- Co-authored-by: prastoin <paul@twenty.com> |
||
|
Paul Rastoin
|
996ccd8353 | Non composite and non morph or relation field update fix (#16091) | ||
|
Weiko
|
04562b11fb |
Migrate metadata cache (#16030)
## Context Deprecating legacy ObjectMetadata from cache in favor of flat entities. Introducing utils to build byName/byNameSingular/byNamePlural in isolated cases ## Next - I had to introduce a util to build from flat to legacy objectMetadataMaps, we should instead use flat maps directly when needed (datasource, schema generation, etc) - Deprecate metadata version in the cache - Use the new cache strategy for flat entities with permissions and feature flags and inject in the global datasource context |
||
|
Etienne
|
208c0857ee |
common api - null equivalence (#15926)
closes https://github.com/twentyhq/core-team-issues/issues/1629 To do before requesting review : - filter update Migration to come in an other PR Strat : 1/ Null transformation - [x] Transform NULL equivalent value to NULL in field validation in common api - pre-query - with feature flag - [ ] Same logic in ORM (Not done, complex to handle feature flag here) - [x] Transform NULL value to equivalent in data formatting in ORM - post-query 2/ Migration (in other PR) for fieldMetadata not nullable with default defaultValue (empty string, ...) - [ ] Remove NOT NULL db constraint - [ ] Update record value to NULL - [ ] Update field metadata : isNullable:true - [ ] Update uniqueIndex whereClause (also for standard uniqueIndex) - [ ] Activate feature flag 3/ Update metadata creation - [x] No more default default value - [x] Update standard field nullability - [x] Remove index default whereClause for standard field 4/ Update filter - [x] When filtering on NULL or empty string, be sure all records are returned (the one with NULL + the one with "") 5/ Test - [ ] Strat. to do |
||
|
Paul Rastoin
|
0be228fc85 |
Attest standard object isActive update regression + TDD tests (#15976)
# Introduction Related https://github.com/twentyhq/twenty/issues/15846 The root cause is that universalIdentifier is still optional in database and fallbacked when extracted out of database to standardId. But all `BaseWorkspaceEntity` and `CustomWorkspaceEntity` share the same standardId for their default standard fields `createdAt` `deletedAt` resulting in such compare result in dispatcher ```ts { "initialDispatcher": { "createdFlatEntityMaps": { "byId": {}, "idByUniversalIdentifier": {}, "universalIdentifiersByApplicationId": {} }, "deletedFlatEntityMaps": { "byId": {}, "idByUniversalIdentifier": {}, "universalIdentifiersByApplicationId": {} }, "updatedFlatEntityMaps": { "byId": { "55e1568c-eb87-4b8a-9f1b-19bbf6042f3e": { "updates": [ { "from": "Deletion date", "to": "Date when the record was deleted", "property": "description" }, { "from": "IconCalendarClock", "to": "IconCalendarMinus", "property": "icon" }, { "from": false, "to": true, "property": "isLabelSyncedWithName" }, { "from": null, "to": { "displayFormat": "RELATIVE" }, "property": "settings" } ] } } } }, "fromFlatEntity": { "universalIdentifier": "20202020-b9a7-48d8-8387-b9a3090a50ec", "applicationId": null, "id": "9c97c8bf-1f64-463c-915c-f68f41d3cd60", "standardId": "20202020-b9a7-48d8-8387-b9a3090a50ec", "objectMetadataId": "e9565126-8351-457b-b003-3ea4c6d253bc", "type": "DATE_TIME", "name": "deletedAt", "label": "Deleted at", "defaultValue": null, "description": "Deletion date", "icon": "IconCalendarClock", "standardOverrides": null, "options": null, "settings": null, "isCustom": false, "isActive": true, "isSystem": false, "isUIReadOnly": true, "isNullable": true, "isUnique": false, "workspaceId": "20202020-1c25-4d02-bf25-6aeccf7ea419", "isLabelSyncedWithName": false, "relationTargetFieldMetadataId": null, "relationTargetObjectMetadataId": null, "morphId": null, "createdAt": "2025-11-20T17:28:45.474Z", "updatedAt": "2025-11-20T17:28:45.474Z", "kanbanAggregateOperationViewIds": [], "calendarViewIds": [], "viewGroupIds": [], "viewFieldIds": [], "viewFilterIds": [] }, "toFlatEntity": { "universalIdentifier": "20202020-b9a7-48d8-8387-b9a3090a50ec", "applicationId": null, "id": "55e1568c-eb87-4b8a-9f1b-19bbf6042f3e", "standardId": "20202020-b9a7-48d8-8387-b9a3090a50ec", "objectMetadataId": "37263f48-6858-4d28-a6e1-5f7321e49c24", "type": "DATE_TIME", "name": "deletedAt", "label": "Deleted at", "defaultValue": null, "description": "Date when the record was deleted", "icon": "IconCalendarMinus", "standardOverrides": null, "options": null, "settings": { "displayFormat": "RELATIVE" }, "isCustom": false, "isActive": true, "isSystem": false, "isUIReadOnly": true, "isNullable": true, "isUnique": false, "workspaceId": "20202020-1c25-4d02-bf25-6aeccf7ea419", "isLabelSyncedWithName": true, "relationTargetFieldMetadataId": null, "relationTargetObjectMetadataId": null, "morphId": null, "createdAt": "2025-11-20T17:28:44.267Z", "updatedAt": "2025-11-21T17:17:55.057Z", "kanbanAggregateOperationViewIds": [], "calendarViewIds": [], "viewGroupIds": [], "viewFieldIds": [], "viewFilterIds": [] } } ``` ## Impact - This might be corrupting label and description of an other standard field of an other object - Race condition on latest universalIdentifier assigned in cache making the update sometime accurate sometimes not ## Fix Will be fixed by the in coming work on applicationId and universalIdentifier as required in database + upgrade command that will handle retro-comp. ( won't handle description corruption though, should be anecdotical ) https://github.com/twentyhq/twenty/pull/15911 ( handling this only for new workspace, retro comp upgrade command will be coming just after ) ## PR scope - Introduce TDD integration tests as failing - Added unit test to critical methods that might have been involved in the root cause ( still worth it to keep ) --------- Co-authored-by: guillim <guigloo@msn.com> |
||
|
Guillim
|
4848bc03f3 |
Update Name of relation fieldMetadata (#15749)
UpdateOne of a Relation that involves a CustomObject, because the nameSingular needs to be updated in the fieldMetadata - nameSingular and namePlural must be provided since they are necessary for morph name computation - label sync should be false Interesting files to look at: - packages/twenty-server/src/engine/metadata-modules/object-metadata/utils/build-default-relation-flat-field-metadatas-for-custom-object.util.ts - UPDATE => packages/twenty-server/src/engine/metadata-modules/flat-object-metadata/utils/rename-related-morph-field-on-object-names-update.util.ts ( also update relation indexes ) needs v2 refactor to handle field relation name update - CREATE => packages/twenty-server/src/engine/metadata-modules/object-metadata/utils/build-default-relation-flat-field-metadatas-for-custom-object.util.ts ( handle morph instead of previous classic relation ) - DELETE => DONE Edit: closes https://github.com/twentyhq/core-team-issues/issues/1897 --------- Co-authored-by: prastoin <paul@twenty.com> |
||
|
Paul Rastoin
|
e5255df1a1 |
Fix and refactor relation field name collision validation (#15920)
# Introduction Fixes https://github.com/twentyhq/private-issues/issues/371 We weren't strictly validating relation field collision on join column name availability of the target field object ## Refactor Extracted morph or relation specific condition out of the common flat field metadata name validate availability to be located in the dedicated morph or relation flat field validator ## Tests Added two tests, ONE_TO_MANY and MANY_TO_ONE in order to cover the use case |
||
|
Paul Rastoin
|
48031a7ce2 |
Runner v2 on cascade delete_field/object + refactor workspace deletion side effect (#15830)
# Introduction While removing the v1 https://github.com/twentyhq/twenty/pull/15823 I've encountered the method `objectMetadataService.deleteObjectsMetadata` that I didn't wanted to migrate as it is and if it's not challenging its existence legitimacy. ## Motivations In a nutshell on a workspace deletion object metadata and field metadata cascading deletion is correclty handled But that's not the case for all of a workspaces entities ( roles, workspaceMigrations ) I suspect that we did not defined the foreignKey explicitly through `typeorm` ## Battle testing v2 I still decided to give a try to a complex operation in the v2 such as a workspace all object metadata deletion. Spoiler it failed due to object being interdependent between them and not being topologically sorted ( morph relation can introduce circular dep anw ). Note: Even after removing the delete field on delete object aggregator we end up with an equivalent circular dep error which an object and its field metadata identifier connection. ## Elegant `DEFERRED` and `DEFERRABLE` foreign keys The most safe, low level solution would be to make all field relations deferrable and start the runner transaction as deferred. But this requires a quite invasive migration of existing FK ## On cascade solution I've opted for the quick fix, on object or field deletion spread cacasde. It's pretty safe as the builder priorly validates the deletion and and its related entities integrity Only for both field and object deletion action types in v2 runner ## Integration coverage Added a test that will scan a new workspace database core schema tables and expect now result after workspace deletion through its only user deletion |
||
|
Weiko
|
998365457e |
Improve infer deletion from entities (#15807)
## Context inferDeletionFromEntities only accepts a set of keys for each entities that needs deletion. This could be error-prone if tmr we want to add a new side effect and forget to add the entity when in practice you want to delete all entities that are in the fromToAllFlatEntityMaps (this is the case for applications for example) |
||
|
martmull
|
9880f192a5 | Move composite types to twenty-shared (#15741) | ||
|
Paul Rastoin
|
3514054235 |
V2 centralize relation optimistic logic (#15552)
# Introduction
This PR aims to deprecate having to manually handle optimistic side
effect foreign key addition in the whole v2 experience.
This PR implements the strong basis + builder refactor of the optimistic
computation of a given flat entity maps with its related flat entity
maps ( runner needs a small refactor on actions type definition first )
Flat entity maps updates through mutations are now only scoped to the
generic entity builder ( very isolated )
## What's next
- Refactor actions v2 type definition to gain grain over `metadataName`
and action operation ( `create` `delete` `update` ).
from `{type: 'create_view_field'}` to `{metadataName: 'view_field',
type: 'create' }`
- Use new optimistic tool computation tools
- Only invalidate impacted flat maps cache
## New tools
Strictly dynamically typed new flat entity maps tools
- `addFlatEntityToFlatEntityAndRelatedEntityMapsThroughMutationOrThrow`
-
`deleteFlatEntityFromFlatEntityAndRelatedEntityMapsThroughMutationOrThrow`
## Unit test
Adding basic unit testing coverage to introduced tools
## `FlatEntityValidationArgs`
From
```ts
export type FlatEntityValidationArgs<T extends AllMetadataName> = {
flatEntityToValidate: MetadataFlatEntity<T>;
optimisticFlatEntityMaps: MetadataFlatEntityMaps<T>;
mutableDependencyOptimisticFlatEntityMaps: MetadataValidationRelatedFlatEntityMaps<T>;
workspaceId: string;
remainingFlatEntityMapsToValidate: MetadataFlatEntityMaps<T>;
buildOptions: WorkspaceMigrationBuilderOptions;
};
```
To
```ts
export type FlatEntityValidationArgs<T extends AllMetadataName> = {
flatEntityToValidate: MetadataFlatEntity<T>;
optimisticFlatEntityMapsAndRelatedFlatEntityMaps: MetadataFlatEntityAndRelatedFlatEntityMapsForValidation<T>;
workspaceId: string;
remainingFlatEntityMapsToValidate: MetadataFlatEntityMaps<T>;
buildOptions: WorkspaceMigrationBuilderOptions;
};
```
|
||
|
Paul Rastoin
|
d640b93096 |
Improve v2 and cache invalidation perfs (#15467)
# Introduction Log are debug logs of `packages/twenty-server/test/integration/metadata/suites/object-metadata/create-delete-and-create-object-metadata-v2.integration-spec.ts`run ten times in a row on clean db reset ## Next Will improve cache computation to lighter invalidation. RelationLoad `query` does not seem to work with typeorm so I'll continue the custom integration i've started in https://github.com/twentyhq/twenty/tree/optimize-cache-read-v2 ## Integration tests duration Significant test duration improvement too ### Before <img width="2632" height="1402" alt="image" src="https://github.com/user-attachments/assets/2f1f0ccf-44de-4856-bfe1-4f45a351763a" /> ### After <img width="2632" height="1402" alt="image" src="https://github.com/user-attachments/assets/4bc9e6db-3046-48b9-b903-1464053936c4" /> ## What's next - The legacy cache invalidation removal - Factorizing redis calls in only one operation ## Autogenerated performance comparison ( including mutation refactor too ) [Before](https://gist.github.com/prastoin/3c1e21fa9e3b3ce4b0716902ff4a2dd6) [After](https://gist.github.com/prastoin/7bfddd14bfded2e4991a9378970a026d) The optimized implementation shows **dramatic performance improvements** across all metrics: - 🚀 **Cache Invalidation**: 156.3ms → 76.8ms (**50.9% faster**) - 🚀 **Builder Operations**: 21.3ms → 14.2ms (**33.3% faster**) - ⚡ **Consistency**: 16.4% more predictable performance --- ## 1. Overall Performance Summary | Component | Before (avg) | After (avg) | Best (After) | Worst (After) | Improvement | |-----------|-------------|-------------|--------------|---------------|-------------| | **Total Execution Time** | 180.2ms | 110.5ms | 52.3ms | 585.9ms | **38.7% faster** ⚡⚡ | | **Cache Invalidation** | 156.3ms | 76.8ms | 47.8ms | 285.1ms | **50.9% faster** ⚡⚡⚡ | | **Transaction Execution** | 22.4ms | 22.1ms | 0.99ms | 314.2ms | Similar | | **Initial Cache Retrieval** | 0.81ms | 2.08ms | 0.21ms | 8.24ms | Similar | | **Entity Builder (total)** | 21.3ms | 14.2ms | 0.36ms | 42.5ms | **33.3% faster** ⚡ | ### Total Execution Time Distribution #### Before (Legacy Sequential) ``` Time (ms) Count Percentage Visualization < 150 18 16% ████ 150-180 32 29% ███████ 180-210 35 32% ████████ 210-250 17 15% ████ 250-350 6 5% █ > 350 2 2% ▌ ``` #### After (Optimized Parallel) ``` Time (ms) Count Percentage Visualization < 70 28 31% ████████ 70-100 31 34% █████████ 100-150 18 20% █████ 150-200 8 9% ██ 200-300 4 4% █ > 300 2 2% ▌ ``` --- ## 2. Builder Performance Breakdown ### Field Metadata Builder | Operation | Before (avg) | After (avg) | Improvement | |-----------|-------------|-------------|-------------| | Matrix computation | 3.5ms | 3.4ms | Similar | | Creation validation | 15.2ms | 2.1ms | **86% faster** ⚡⚡⚡ | | Deletion validation | 0.08ms | 0.06ms | Similar | | Update validation | 1.3ms | 0.09ms | **93% faster** ⚡⚡⚡ | | Entity processing | 18.6ms | 11.8ms | **37% faster** ⚡ | | **Total validateAndBuild** | **21.3ms** | **14.2ms** | **33% faster** ⚡ | #### Performance Distribution ``` Before: ▁▂▄█████▆▄▂▁ (wide spread, 15-28ms range) After: ▁▁▃█████▃▁▁ (tight clustering, 10-18ms range) ``` ## 4. Cache Invalidation Performance Breakdown ### Cache Invalidation Summary | Metric | Before (Legacy) | After (Optimized) | Improvement | |--------|-----------------|-------------------|-------------| | **Best Time** | 131.965ms | 47.833ms | **63.7% faster** ⚡⚡⚡ | | **10th Percentile** | 140.2ms | 51.7ms | **63.1% faster** ⚡⚡⚡ | | **25th Percentile** | 146.1ms | 54.4ms | **62.7% faster** ⚡⚡⚡ | | **Median (50th)** | 155.1ms | 63.4ms | **59.1% faster** ⚡⚡⚡ | | **Average** | 156.3ms | 76.8ms | **50.9% faster** ⚡⚡⚡ | | **75th Percentile** | 160.2ms | 90.2ms | **43.7% faster** ⚡⚡ | | **90th Percentile** | 191.7ms | 100.2ms | **47.7% faster** ⚡⚡ | | **95th Percentile** | 235.6ms | 110.3ms | **53.2% faster** ⚡⚡⚡ | | **99th Percentile** | 278.2ms | 224.9ms | **19.1% faster** ⚡ | | **Worst Time** | 383.914ms | 285.102ms | **25.7% faster** ⚡ | ### Cache Invalidation Time Distribution #### Before (Legacy Sequential) ``` Time (ms) Count Percentage Visualization 130-140 3 3% ▊ 140-150 15 14% ████ 150-160 48 44% ███████████ 160-180 31 28% ███████ 180-220 8 7% ██ 220-280 3 3% ▊ > 280 2 2% ▌ ``` #### After (Optimized Parallel + Intersection) ``` Time (ms) Count Percentage Visualization < 50 3 3% ▊ 50-60 27 25% ███████ 60-70 25 23% ██████ 70-90 25 23% ██████ 90-100 15 14% ████ 100-120 8 7% ██ 120-150 3 3% ▊ > 150 4 4% █ ``` ## 6. Performance Consistency Analysis ### Standard Deviation & Variance | Metric | Before | After | Improvement | |--------|--------|-------|-------------| | **Cache Invalidation Std Dev** | 42.1ms | 35.2ms | **16.4% more consistent** ⚡ | | **Total Execution Std Dev** | 68.3ms | 89.1ms | Slightly more variable | | **Coefficient of Variation (Cache)** | 26.9% | 45.8% | More variance | | **Outliers (> 2σ)** | 5 cases | 3 cases | **40% fewer outliers** ⚡ | --- |
||
|
martmull
|
a6cc80eedd |
1751 extensibility twenty sdk v2 use twenty sdk to define a serverless function trigger (#15347)
This PR adds 2 columns handlerPath and handlerName in serverlessFunction
to locate the entrypoint of a serverless in a codebase
It adds the following decorators in twenty-sdk:
- ServerlessFunction
- DatabaseEventTrigger
- RouteTrigger
- CronTrigger
- ApplicationVariable
It still supports deprecated entity.manifest.jsonc
Overall code needs to be cleaned a little bit, but it should work
properly so you can try to test if the DEVX fits your needs
See updates in hello-world application
```typescript
import axios from 'axios';
import {
DatabaseEventTrigger,
ServerlessFunction,
RouteTrigger,
CronTrigger,
ApplicationVariable,
} from 'twenty-sdk';
@ApplicationVariable({
universalIdentifier: 'dedc53eb-9c12-4fe2-ba86-4a2add19d305',
key: 'TWENTY_API_KEY',
description: 'Twenty API Key',
isSecret: true,
})
@DatabaseEventTrigger({
universalIdentifier: '203f1df3-4a82-4d06-a001-b8cf22a31156',
eventName: 'person.created',
})
@RouteTrigger({
universalIdentifier: 'c9f84c8d-b26d-40d1-95dd-4f834ae5a2c6',
path: '/post-card/create',
httpMethod: 'GET',
isAuthRequired: false,
})
@CronTrigger({
universalIdentifier: 'dd802808-0695-49e1-98c9-d5c9e2704ce2',
pattern: '0 0 1 1 *', // Every year 1st of January
})
@ServerlessFunction({
universalIdentifier: 'e56d363b-0bdc-4d8a-a393-6f0d1c75bdcf',
})
class CreateNewPostCard {
main = async (params: { recipient: string }): Promise<string> => {
const { recipient } = params;
const options = {
method: 'POST',
url: 'http://localhost:3000/rest/postCards',
headers: {
'Content-Type': 'application/json',
Authorization: `Bearer ${process.env.TWENTY_API_KEY}`,
},
data: { name: recipient ?? 'Unknown' },
};
try {
const { data } = await axios.request(options);
console.log(`New post card to "${recipient}" created`);
return data;
} catch (error) {
console.error(error);
throw error;
}
};
}
export const createNewPostCardHandler = new CreateNewPostCard().main;
```
### [edit] V2
After the v1 proposal, I see that using a class method to define the
serverless function handler is pretty confusing. Lets leave
serverlessFunction configuration decorators on the class, but move the
handler like before. Here is the v2 hello-world serverless function:
```typescript
import axios from 'axios';
import {
DatabaseEventTrigger,
ServerlessFunction,
RouteTrigger,
CronTrigger,
ApplicationVariable,
} from 'twenty-sdk';
@ApplicationVariable({
universalIdentifier: 'dedc53eb-9c12-4fe2-ba86-4a2add19d305',
key: 'TWENTY_API_KEY',
description: 'Twenty API Key',
isSecret: true,
})
@DatabaseEventTrigger({
universalIdentifier: '203f1df3-4a82-4d06-a001-b8cf22a31156',
eventName: 'person.created',
})
@RouteTrigger({
universalIdentifier: 'c9f84c8d-b26d-40d1-95dd-4f834ae5a2c6',
path: '/post-card/create',
httpMethod: 'GET',
isAuthRequired: false,
})
@CronTrigger({
universalIdentifier: 'dd802808-0695-49e1-98c9-d5c9e2704ce2',
pattern: '0 0 1 1 *', // Every year 1st of January
})
@ServerlessFunction({
universalIdentifier: 'e56d363b-0bdc-4d8a-a393-6f0d1c75bdcf',
})
export class ServerlessFunctionDefinition {}
export const main = async (params: { recipient: string }): Promise<string> => {
const { recipient } = params;
const options = {
method: 'POST',
url: 'http://localhost:3000/rest/postCards',
headers: {
'Content-Type': 'application/json',
Authorization: `Bearer ${process.env.TWENTY_API_KEY}`,
},
data: { name: recipient ?? 'Unknown' },
};
try {
const { data } = await axios.request(options);
console.log(`New post card to "${recipient}" created`);
return data;
} catch (error) {
console.error(error);
throw error;
}
};
```
### [edit] V3
After the v2 proposal, we don't really like decorators on empty classes.
We decided to go with a Vercel approach with a config constant
```typescript
import axios from 'axios';
import { ServerlessFunctionConfig } from 'twenty-sdk';
export const main = async (params: { recipient: string }): Promise<string> => {
const { recipient } = params;
const options = {
method: 'POST',
url: 'http://localhost:3000/rest/postCards',
headers: {
'Content-Type': 'application/json',
Authorization: `Bearer ${process.env.TWENTY_API_KEY}`,
},
data: { name: recipient ?? 'Unknown' },
};
try {
const { data } = await axios.request(options);
console.log(`New post card to "${recipient}" created`);
return data;
} catch (error) {
console.error(error);
throw error;
}
};
export const config: ServerlessFunctionConfig = {
universalIdentifier: 'e56d363b-0bdc-4d8a-a393-6f0d1c75bdcf',
routeTriggers: [
{
universalIdentifier: 'c9f84c8d-b26d-40d1-95dd-4f834ae5a2c6',
path: '/post-card/create',
httpMethod: 'GET',
isAuthRequired: false,
}
],
cronTriggers: [
{
universalIdentifier: 'dd802808-0695-49e1-98c9-d5c9e2704ce2',
pattern: '0 0 1 1 *', // Every year 1st of January
}
],
databaseEventTriggers: [
{
universalIdentifier: '203f1df3-4a82-4d06-a001-b8cf22a31156',
eventName: 'person.created',
}
]
}
```
|
||
|
Paul Rastoin
|
267af42412 |
Centralize v2 errors types in twenty-shared (#15358)
# Introduction Followup of https://github.com/twentyhq/twenty/pull/15331 ( Reducing size by concerns ) This PR centralizes v2 format error types in `twenty-shared` and consuming them in the existing v2 error format logic in `twenty-server` ## Next This https://github.com/twentyhq/twenty/pull/15360 handles the frontend v2 format error refactor ## Conclusion Related to https://github.com/twentyhq/core-team-issues/issues/1776 |
||
|
Paul Rastoin
|
7edfe4bc7a |
[Followup] Fix defaultValue with enum options update in migration v2 integration test + polish (#15300)
# Introduction Followup of https://github.com/twentyhq/twenty/pull/15286 - Polish replacing sorting by conditional filter and push - Integration test |
||
|
Paul Rastoin
|
0780380fe5 |
Fix composite field update v2 (#15290)
# Introduction Fixing composite field update by computing field column type for each of its properties instead of globally ## Coverage Added integration tests for each composite field on both successful `create` and `update` ```ts Test Suites: 17 passed, 17 total Tests: 104 passed, 104 total Snapshots: 14 passed, 14 total Time: 135.431 s, estimated 143 s ``` ## Conlusion Related to https://github.com/twentyhq/core-team-issues/issues/1753 |
||
|
Weiko
|
3f5efdbabc |
Fix defaultValue with enum options update in migration v2 (#15286)
## Context When updating both enum options and defaultValue, the old default might not be in the new options (or vice versa), causing PostgreSQL constraint violations regardless of update order. ## Solution Sort updates to process defaultValue last; before updating options, temporarily set the new defaultValue in metadata so alterEnumValues creates the column with the correct default, then skip the redundant defaultValue update handler. |
||
|
Paul Rastoin
|
1cf442966d |
FindAllCoreViews graphql cache operation invalidation in view related v2 action run (#15285)
# Introduction
Adding a view field to a view in v2 would be optimistically rendered by
the front but on refresh would not get persisted.
That's because we cache both:
```ts
useCachedMetadata({
cacheGetter: cacheStorageService.get.bind(cacheStorageService),
cacheSetter: cacheStorageService.set.bind(cacheStorageService),
operationsToCache: ['ObjectMetadataItems', 'FindAllCoreViews'],
}),
```
With keys that look like:
```ts
return `graphql:operations:${operationName}:${workspace.id}:${workspaceMetadataVersion}:${locale}:${queryHash}`;
```
It was functional in v1 as we would be incrementing metadata version
often.
In v2 it gets incremented only if implies an interaction to metadata
object or fields ( will be deprecated in the future though, until we
finish the // run )
The fix was to check if an `view` or related has been processed in the
workspace migration or if we incremented the metadata in order to
manually flush the `findAllCoreViews` redis cache.
|
||
|
Paul Rastoin
|
2e84c11eae |
[v2_FIX] Update standard object/field (#15233)
# Introduction Refactoring the standard overrides dispatcher to only pass over fields to has to be dispatched in the standardOverrides entry and let the other side effects resulting from out of standard overrides mutation trigger Related to https://github.com/twentyhq/core-team-issues/issues/1753 ## This allows - standard field settings, options etc updates and so on ## Remark - Determine what we should do on object deactivation ( right now in production we can still access deactivated object relation properties and so on e.g deactivate opportunities still accessible from a view field on company ( still have to re-create it as it has been deleted ) => decided to leave as it is right now, `isActive` could be considered as uiDeactivated in the end - We should also add forbidden standard field mutations validation inside the builder itself ( here we want to early return in the api input transpiler too as we don't want to spread invalid side effects ) => or in the end we could just centralize both but it will generate several errors ## Coverage ```ts PASS test/integration/metadata/suites/object-metadata/successful-update-one-standard-object-metadata.integration-spec.ts PASS test/integration/metadata/suites/field-metadata/successful-update-one-standard-field-metadata.integration-spec.ts PASS test/integration/metadata/suites/object-metadata/failing-update-one-standard-object-metadata.integration-spec.ts PASS test/integration/metadata/suites/field-metadata/failing-update-one-standard-field-metadata.integration-spec.ts Test Suites: 4 passed, 4 total Tests: 18 passed, 18 total Snapshots: 16 passed, 16 total Time: 8.721 s, estimated 10 s ``` ## Update post review Faced a behavior where updating back the company label to its original value would result in storing this value in the standard overrides Refactored both field and object transpilation behavior to rather remove the standard override value instead and let fallback on original value Yes it's quite duplicated will factorize once we move this inside the builder |
||
|
Félix Malfait
|
c5564d9bd0 |
[BREAKING CHANGE] refactor: Add Entity suffix to TypeORM entity classes (#15239)
## Summary This PR refactors all TypeORM entity classes in the Twenty codebase to include an 'Entity' suffix (e.g., User → UserEntity, Workspace → WorkspaceEntity) to improve code clarity and follow TypeORM naming conventions. ## Changes ### Entity Renaming - ✅ Renamed **57 core TypeORM entities** with 'Entity' suffix - ✅ Updated all related imports, decorators, and type references - ✅ Fixed Repository<T>, @InjectRepository(), and TypeOrmModule.forFeature() patterns - ✅ Fixed @ManyToOne/@OneToMany/@OneToOne decorator references ### Backward Compatibility - ✅ Preserved GraphQL schema names using @ObjectType('OriginalName') decorators - ✅ **No breaking changes** to GraphQL API - ✅ **No database migrations** required - ✅ File names unchanged (user.entity.ts remains as-is) ### Code Quality - ✅ Fixed **497 TypeScript errors** (82% reduction from 606 to 109) - ✅ **All linter checks passing** - ✅ Improved type safety across the codebase ## Entities Renamed ``` User → UserEntity Workspace → WorkspaceEntity ApiKey → ApiKeyEntity AppToken → AppTokenEntity UserWorkspace → UserWorkspaceEntity Webhook → WebhookEntity FeatureFlag → FeatureFlagEntity ApprovedAccessDomain → ApprovedAccessDomainEntity TwoFactorAuthenticationMethod → TwoFactorAuthenticationMethodEntity WorkspaceSSOIdentityProvider → WorkspaceSSOIdentityProviderEntity EmailingDomain → EmailingDomainEntity KeyValuePair → KeyValuePairEntity PublicDomain → PublicDomainEntity PostgresCredentials → PostgresCredentialsEntity ...and 43 more entities ``` ## Impact ### Files Changed - **400 files** modified - **2,575 insertions**, **2,191 deletions** ### Progress - ✅ **82% complete** (497/606 errors fixed) - ⚠️ **109 TypeScript errors** remain (18% of original) ## Remaining Work The 109 remaining TypeScript errors are primarily: 1. **Function signature mismatches** (~15 errors) - Test mocks with incorrect parameter counts 2. **Entity type mismatches** (~25 errors) - UserEntity vs UserWorkspaceEntity confusion 3. **Pre-existing issues** (~50 errors) - Null safety and DTO compatibility (unrelated to refactoring) 4. **Import type issues** (~10 errors) - Entities imported with 'import type' but used as values 5. **Minor decorator issues** (~9 errors) - onDelete property configurations These can be addressed in follow-up PRs without blocking this refactoring. ## Testing Checklist - [x] Linter passing - [ ] Unit tests should be run (CI will verify) - [ ] Integration tests should be run (CI will verify) - [ ] Manual testing recommended for critical user flows ## Breaking Changes **None** - This is a pure refactoring with full backward compatibility: - GraphQL API unchanged (uses original entity names) - Database schema unchanged - External APIs unchanged ## Notes - Created comprehensive `REFACTORING_STATUS.md` documenting the entire process - All temporary scripts have been cleaned up - Branch: `refactor/add-entity-suffix-to-typeorm-entities` ## Reviewers Please review especially: - Entity renaming patterns - GraphQL backward compatibility - Any areas where entity types are confused (UserEntity vs UserWorkspaceEntity) --------- Co-authored-by: Charles Bochet <charles@twenty.com> |
||
|
Paul Rastoin
|
45473218d3 |
Field deactivation side effect views calendar kanban viewFields (#15180)
# Introduction
Handling both:
- field deactivation side effect on view fields, view filters and views
- field deactivation side effect on view that targets it as
`kanbanAggregateFieldMetadataId`
- field deactivation side effect on view that targets it as
`calendarFieldMetadataId`
## Coverage
added coverage
```ts
PASS test/integration/metadata/suites/field-metadata/kanban-aggregate-field-deactivation-deletes-views.integration-spec.ts (13.132 s)
kanban-aggregate-field-deactivation-nullifies-kanban-properties
✓ should nullify kanban properties when field used as kanbanAggregateOperationFieldMetadataId is deactivated (3923 ms)
✓ should not modify views when field not used as kanbanAggregateOperationFieldMetadataId is deactivated (2958 ms)
✓ should nullify kanban properties on multiple views when they all use the same field as kanbanAggregateOperationFieldMetadataId (2542 ms)
✓ should nullify kanban properties when views have different aggregate operations on same field (3380 ms)
Test Suites: 1 passed, 1 total
Tests: 4 passed, 4 total
Snapshots: 0 total
Time: 13.154 s
```
```ts
PASS test/integration/metadata/suites/field-metadata/view-group-field-deactivation-deletes-views.integration-spec.ts (12.639 s)
view-group-field-deactivation-deletes-views
✓ should delete view when field used in view group is deactivated (3469 ms)
✓ should not delete view when field not used in view group is deactivated (3109 ms)
✓ should delete multiple views when they all use the same field in view groups (2741 ms)
✓ should handle deactivation when view has multiple view groups with different fields (3008 ms)
Test Suites: 1 passed, 1 total
Tests: 4 passed, 4 total
Snapshots: 0 total
Time: 12.664 s
```
```ts
PASS test/integration/metadata/suites/field-metadata/calendar-field-deactivation-deletes-views.integration-spec.ts (14.579 s)
calendar-field-deactivation-deletes-views
✓ should delete view when field used as calendarFieldMetadataId is deactivated (3388 ms)
✓ should not delete view when field not used as calendarFieldMetadataId is deactivated (2438 ms)
✓ should delete multiple views when they all use the same field as calendarFieldMetadataId (2635 ms)
✓ should handle deactivation when views have different calendar layouts on same field (3195 ms)
✓ should delete calendar view but not other view types when calendar field is deactivated (2682 ms)
Test Suites: 1 passed, 1 total
Tests: 5 passed, 5 total
Snapshots: 0 total
Time: 14.601 s, estimated 15 s
```
## View soft deletion
We decided to remove the soft deletion grain on all the views, in this
PR context we've only removed soft deleted validation requirement on any
view entities
## Conclusion
close https://github.com/twentyhq/core-team-issues/issues/1754
|
||
|
Weiko
|
0122f805a4 |
Fix serverless update not saving new code with migration v2 (#15221)
## Context Regression introduced in https://github.com/twentyhq/twenty/pull/15032 With the new code, we don't have access to the from/to from the specialised builder anymore and we now rely on diffing result and cache to create the action which broke serverless update because "code" is not part of the cache nor part of the diffing (checksum is). To maintain the existing architecture and keep it generic (by only modifying the specialized builder), the serverless builder overrides the parent validateAndBuild method |
||
|
Weiko
|
cceeb6ed4d |
Add applicationId to syncableEntity and fix syncApp deletion (#15170)
## Context - All flatEntity should extend SyncableEntity - SyncableEntity should now have applicationId and application relation - Fix syncApp deletion, should now properly use migration v2 to delete syncable entities |
||
|
Paul Rastoin
|
3462a2e288 |
ViewGroup and ViewFilters side effect in v2 (#15096)
# Introduction ### Summary Implements side effect handling for `ViewGroup` and `ViewFilters` when field metadata is updated in the v2 architecture. This ensures that view-related records are properly maintained when enum field options are modified, deleted, or created. ### Side effects - **Side Effect System**: Added side effect handling for field metadata updates that manages related view groups and view filters - **Enum Field Updates**: When enum field options are modified, the system now: - **View Groups**: Creates new groups for added options, updates existing groups for modified options, and deletes groups for removed options - **View Filters**: Updates filter values to reflect option changes and removes filters that reference deleted options ### Enum runner fix Update now works for both atomic enum and array enum ( multi select for instance ) ### Compute flat entity maps from to Standardized this method usage across v2 services Next step is to require dependencies dynamically ## Conclusion closes https://github.com/twentyhq/core-team-issues/issues/1649 |
||
|
Weiko
|
c0ed246a03 |
Add is unique in migration v2 + refactor inferDeletionFromMissingEntities (#15070)
## Goal - inferDeletionFromMissingEntities is now a map instead of a single bool, allowing us to parameterise it based on the entity we want to compare - Adding index creation/update when field isUnique is set to true and index deletion when isUnique is false (for now) close https://github.com/twentyhq/core-team-issues/issues/1346 |
||
|
Paul Rastoin
|
7d747c9876 |
[REQUIRES_CACHE_FLUSH][GQL_VIEW_GROUP_API_BREAKING_CHANGE] ViewGroup in v2 (#15052)
# Introduction Adding view-group to core engine v2 Following https://github.com/twentyhq/twenty/pull/15010 ( same pattern ) ## What's done - Created flat-view-group - flat view group runner - flat view group builder - create view group service v2 and input transpilers - refactor the existing view group resolver to fix standard ( BREAKING_CHANGE on graphql api update especially ) REST stays the same - refactored the front to consume the mutations autogenerated close https://github.com/twentyhq/core-team-issues/issues/1665 |
||
|
Paul Rastoin
|
bce83c52f0 |
Increment metadata version only if schema changes v2 (#15064)
# Introduction After updating a `viewField` in a view the frontend receives a missmatch metadata version. That's because the `flatFieldMetadata` needs to be invalidated on a viewField addition as it contains its primary key in its cache. Before we would be checking updated flat entity maps, meaning that on a view field update the flat field metadata maps would also get updated, but we also invalidate the old v1 cache at the same time. Resuling in a metadata version missmatch that's not really relevant for the gql schema integrity Now we only check if a object or field actions has been processed, if yes increment the metadata version. We should deco-relate the v1 object and fields cache from the metadata version that should only serve as a "Please refresh browser state because gql schema has mutated" |
||
|
Paul Rastoin
|
6188c72f74 |
Simplify and enhance v2 type devxp (#15032)
# Introduction This PR introduces a huge type refactor that will leverage dynamic intra entity optimistic flat maps update in the future and also a more granular cache invalidation enhancing performances close https://github.com/twentyhq/core-team-issues/issues/1717 close https://github.com/twentyhq/core-team-issues/issues/1716 close https://github.com/twentyhq/core-team-issues/issues/1643 ## What's done ### Comparators centralization Comparator is now done through global configuration as const for each metadata names Thanks to Note: Definition of standard is evolving, standard is now scoped to an app. Meaning that a manifest should be able to update its own standards objects but on other app standards ones ? Each synchronizable entities will have a standardOverrides ? ## Typing refactor ### `AllFlatEntityTypesByMetadataName` **Single source of truth for the complete type ecosystem**, mapping each metadata name to its entity types, flat entities, and migration actions: ```typescript export type AllFlatEntityTypesByMetadataName = { fieldMetadata: { actions: { created: CreateFieldAction; updated: UpdateFieldAction; deleted: DeleteFieldAction; }; flatEntity: FlatFieldMetadata; entity: FieldMetadataEntity; }; objectMetadata: { /* ... */ }; // ... all 10 metadata types }; ``` ### `ALL_METADATA_NAME_MANY_TO_ONE_RELATIONS` **Explicitly declares database relationships** between entities with compile-time validation: ```typescript export const ALL_METADATA_NAME_MANY_TO_ONE_RELATIONS = { viewField: { view: 'viewId', fieldMetadata: 'fieldMetadataId', }, cronTrigger: { serverlessFunction: 'serverlessFunctionId', }, // ... all relations } as const satisfies MetadataNameAndRelations; ``` ### `ALL_FLAT_ENTITY_CONFIGURATION` **Centralizes comparison and serialization logic** for each metadata type: ```typescript export const ALL_FLAT_ENTITY_CONFIGURATION = { fieldMetadata: { propertiesToCompare: ['name', 'type', 'label', 'defaultValue', /* ... */], propertiesToStringify: ['options', 'settings', 'defaultValue'], }, objectMetadata: { propertiesToCompare: ['nameSingular', 'namePlural', 'isActive', /* ... */], propertiesToStringify: [], }, // ... all metadata types } as const satisfies AllFlatEntityConfiguration; ``` ## Combined Impact These three configurations work together to create a **strongly-typed, centrally-managed metadata system**: 1. **`AllFlatEntityTypesByMetadataName`** defines *what exists* 2. **`ALL_METADATA_NAME_MANY_TO_ONE_RELATIONS`** defines *how they relate* 3. **`ALL_FLAT_ENTITY_CONFIGURATION`** defines *how to compare and serialize them* **Result:** Builders and validators become thin wrappers around type-safe, configuration-driven logic instead of containing scattered, error-prone manual implementations. ## What's next ### StandardOverrides standardization Every metadata entity can be a standard one for a workspace if it's an installed app, which means it might not expose the whole entity api to be editable through an import dynamically The standard overrides logic should not be applied to Fields and Objects but to every entities At the moment we have a logic of `EDITABLE_PROPERTIES` through the api, and also `STANDARD_OVERRIDEDABLE_PROPERTIES` This should be configuration centered like `propertiesToCompare` and `propertiesToStringify`. Scoping this PR to two last for the moment. As update dispatch to standardOverrides could be considered as a side effect prefer waiting to start the side effect refactor ### Granular Optimistic deprecation With this new grain at runtime we will be able to add a flat entity and dispatch its addition to related flat maps, so we don't have to describe an optimistic method for each flat entity operations See `addFlatEntityToFlatEntityAndRelatedEntityMapsOrThrow` Note: Still in wip and included in this PR but about to create a new one to integrate these utils and remove existing methods ### ValidateBuildAndRun dynamic args typed defintion We should restrain the devxp to send expected flat maps entity as at least from to or dependency as we now have the grain both a type lvl and runtime to do so It should not be possible in the devxp to forgot adding the views to the v2 builder when passing the view field anymore ( that would lead to permanent validation error in view field integrity checks ) ## Conclusion Thanks for reading and reviewing ! Any suggestions are more than welcomed ! ( same as for questions too ! ) |
||
|
Paul Rastoin
|
651ab184a7 |
[GQL_VIEW_FILTER_API_BREAKING_CHANGE][WHEN_RELEASED_REQUIRES_CACHE_FLUSH] ViewFilter migration to workspace migration v2 (#15010)
# Introduction Migrating `viewFilter` to v2 in order to migrate later the field update side effect on view to v2 too ## What's done - Created flat-view-filter - flat view filter runner - flat view filter builder - create view filter service v2 and input transpilers - refactor the existing view filter resolver to fix standard ( BREAKING_CHANGE on graphql api update especially ) REST stays the same - refactored the front to consume the mutations autogenerated ## New generic tools ### Compare two flat entity Introducing a new util to compare two flat entity, it's strictly typed and will be added to the generic builder in a following PR This will ease flat entity addition as won't required to create a specific abstraction for comparison Generic builder will expect specific constant: properties to compare and properties to stringify ### Transform flat entity for comparison Forked and refactor the initial existing method for flat entity business scope and type safety ## Coverage Migrated existing integration tests to fit new contract API This PR does not add strong coverage on validation exceptions Deadlines are too short close https://github.com/twentyhq/core-team-issues/issues/1666 |
||
|
Félix Malfait
|
e577c2d746 |
Update user friendly errors for translations (#15000)
Force msg typing instead of string for user friendly errors |
||
|
Paul Rastoin
|
59fbe35a8c |
Move view in metadata-modules/ and create atomic folder + module for each view entity (#14990)
# Introduction Preparing view-filter and view-group introduction in v2 core engine Moving view from `core-modules` to `metadata-modules` ## What happened ### Created dedicated modules for each view entity: - ViewFieldModule - ViewFilterModule - ViewFilterGroupModule - ViewGroupModule - ViewSortModule ### Each module is now completely independent with its own: - Controller - Resolver - Service - Entity ### Created dedicated abstraction metadata module folder for: - flat-view-field - flat-view ### Dependencies - Eleminated circular dep on ViewModule to all others ones - Granular import not importing the whole viewModule anymore everywhere close https://github.com/twentyhq/core-team-issues/issues/1703 |
||
|
Paul Rastoin
|
4ecc9c622d |
[WHEN_RELEASED_REQUIRES_CACHE_FLUSH] Object related record logic in v2 (#14937)
# Introduction
Initial motivation here was to migrate the object related records logic
from v1 to v2, please note that now in v2 views aren't records anymore
but core engine entities
## What's done
- Added specific label identifier targeting view field logic
- Handled side effects on viewField creation with lowest position on
object label identifier mutation
- Added viewField relations in field metadate entity + handled
optimistic in builder v2
- Added view relations in object metadata entity + handled optimistic in
builder v2
- Added integration tests covering the side effects and new validation
exceptions
- Sandardized cache computation
- Coverage on object metadata creation side effect on views and view
fields
## Coverage
```ts
PASS test/integration/graphql/suites/view/view-field/object-identifier-update-side-effect-on-view-field.integration-spec.ts
View Field Resolver - Successful object metadata identifier update side effect on view field
✓ should create a view field on label identifier object metadata update if it does not exist on view (7 ms)
✓ Should not allow deleting a label identifier view field (17 ms)
✓ Should not allow destroying a label identifier view field (6 ms)
✓ Should not allow updating a label identifier view field visibility to false (8 ms)
✓ Should not allow creating a view field with a position lower than the label idenfitier view field (180 ms)
✓ Should not allow updated labelIdentifier view field with a position higher than existing other view field (346 ms)
✓ Should allow updated labelIdentifier view field with a position higher than existing other view field (434 ms)
Test Suites: 1 passed, 1 total
Tests: 7 passed, 7 total
Snapshots: 5 passed, 5 total
Time: 4.571 s, estimated 5 s
```
close https://github.com/twentyhq/core-team-issues/issues/1664
|
||
|
Paul Rastoin
|
23e21cbeea |
Label identifier validation v2 (#14867)
# Introduction Adding a hacky way to validate object against fields before fields validation ( bi-directional validation process ) If you encounter an identical setup we will add a specific devXp as cleanup validation but for the moment this seems enough close https://github.com/twentyhq/core-team-issues/issues/1639 |
||
|
Paul Rastoin
|
d3a7241b6f |
Cache flush after database reset (#14873)
When starting the app on a fresh database reset the cache would be
filled with empty flat field metadata maps
Because the reset command hack through the repository directly in order
to create views and stuff
implemented an iso flush as the one existing initially
added it to a workspace deletion tambien
```
{
byId: {],
universalIdById: {}
}
```
|
||
|
Weiko
|
0b60aa4249 |
Add custom routes to migration v2 (#14846)
## Context Add routes to migration V2 - Resolvers - Service v2 - Builder - Validator - Action runner Next PR: Add to twenty-cli to sync routes with serverless |