1d9eb07ee0f320e72d2297de7a3f20406bef552a
93 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Paul Rastoin
|
1d9eb07ee0 |
Relation field metadata strict validation (#16890)
# Introduction Now validating relation related object and field foreignkey integrity |
||
|
Paul Rastoin
|
1128331cc1 |
Fix field item type tag (#16860)
# Introduction This has been fixed on main already 1 hour ago, this PR now only passes the field application id instead of the object applicationId that could be different for example when creating a custom field on a standard object For the moment not introducing any logic around application integrity directly and still relying on the isCustom and standardId definition This will have to be refactored once we deprecate the standardId |
||
|
Paul Rastoin
|
e3ffdb0c2b |
[BREAKING_CHANGE_NESTED_WORKSPACE]Refactor FlatEntity typing in aim of introducing UniversalFlatEntity (#16701)
# Introduction
Added a `WorkspaceRelated` and `AllNonWorkspaceRelatedEntity` to
simplify the `FlatEntityFrom` that now do not expect a string literal to
omit and itself builds the related many to one entities foreign key
aggregators
We now have the type grain over relation to syncable or just workspace
related entities
Added a migrations that sets the fk on missing entities
## Next
In upcoming PR we will be able to introduce such below type
```ts
import { type CastRecordTypeOrmDatePropertiesToString } from 'src/engine/metadata-modules/flat-entity/types/cast-record-typeorm-date-properties-to-string.type';
import { type ExtractEntityManyToOneEntityRelationProperties } from 'src/engine/metadata-modules/flat-entity/types/extract-entity-many-to-one-entity-relation-properties.type';
import { type ExtractEntityOneToManyEntityRelationProperties } from 'src/engine/metadata-modules/flat-entity/types/extract-entity-one-to-many-entity-relation-properties.type';
import { type ExtractEntityRelatedEntityProperties } from 'src/engine/metadata-modules/flat-entity/types/extract-entity-related-entity-properties.type';
import { type RemoveSuffix } from 'src/engine/workspace-manager/workspace-migration-v2/workspace-migration-builder-v2/types/remove-suffix.type';
import { type SyncableEntity } from 'src/engine/workspace-manager/workspace-sync/types/syncable-entity.interface';
export type UniversalFlatEntityFrom<TEntity extends SyncableEntity> = Omit<
TEntity,
| `${ExtractEntityManyToOneEntityRelationProperties<TEntity> & string}Id`
| ExtractEntityRelatedEntityProperties<TEntity>
| 'application'
| 'workspaceId'
| 'applicationId'
| keyof CastRecordTypeOrmDatePropertiesToString<TEntity>
> &
CastRecordTypeOrmDatePropertiesToString<TEntity> & {
[P in ExtractEntityManyToOneEntityRelationProperties<TEntity> &
string as `${RemoveSuffix<P, 's'>}UniversalIdentifier`]: string;
} & {
[P in ExtractEntityOneToManyEntityRelationProperties<
TEntity,
SyncableEntity
> &
string as `${RemoveSuffix<P, 's'>}UniversalIdentifiers`]: string[];
};
```
|
||
|
Abdul Rahman
|
720348c583 |
Add upgrade commands to backfill updatedBy field and view fields (#16845)
--- prastoin edit ## Introduction As we're deprecating the sync metadata we cannot rely on it anymore in order to create the new standard updatedBy field In this PR we introduce a command that will backfill the dynamic field on both workspace standard and custom objects, for standard object it will create a standard fields ( twenty-standard app scoped ) and for custom one it will create a custom field ( worksapce-custom-app scoped ) ## Testing method Checkout `1.14.0` `yarn` and `npx nx database:reset twenty-server` checkout PR and `yarn` `npx nx build twenty-server` and `yarn database:migrate:prod` finally running the command #### Before <img width="2244" height="1464" alt="image" src="https://github.com/user-attachments/assets/2fb866cd-13b8-4152-99b8-1fcc813b1d46" /> #### After <img width="2244" height="1464" alt="image" src="https://github.com/user-attachments/assets/b836ac06-c7b8-4add-bee1-bc1963418f29" /> #### Logs ```ts [Nest] 20678 - 12/30/2025, 1:47:35 PM LOG [BackfillUpdatedByFieldCommand] Found 12 objects that need updatedBy field [EntityBuilder fieldMetadata] matrix computation: 3.994ms [EntityBuilder fieldMetadata] creation validation: 1.822ms [EntityBuilder fieldMetadata] deletion validation: 0.016ms [EntityBuilder fieldMetadata] update validation: 0.009ms [EntityBuilder fieldMetadata] entity processing: 6.098ms [EntityBuilder fieldMetadata] validateAndBuild: 10.217ms [EntityBuilder index] matrix computation: 0.857ms [EntityBuilder index] creation validation: 0.003ms [EntityBuilder index] deletion validation: 0.017ms [EntityBuilder index] update validation: 0.008ms [EntityBuilder index] entity processing: 4.721ms [EntityBuilder index] validateAndBuild: 5.632ms [Runner] Initial cache retrieval: 0.07ms [BaseWorkspaceMigrationRunnerActionHandlerService] create_field executeForMetadata: 7.69ms [BaseWorkspaceMigrationRunnerActionHandlerService] create_field executeForWorkspaceSchema: 10.531ms [BaseWorkspaceMigrationRunnerActionHandlerService] create_field executeForMetadata: 5.354ms [BaseWorkspaceMigrationRunnerActionHandlerService] create_field executeForWorkspaceSchema: 6.134ms [BaseWorkspaceMigrationRunnerActionHandlerService] create_field executeForMetadata: 2.507ms [BaseWorkspaceMigrationRunnerActionHandlerService] create_field executeForWorkspaceSchema: 3.745ms [BaseWorkspaceMigrationRunnerActionHandlerService] create_field executeForMetadata: 1.113ms [BaseWorkspaceMigrationRunnerActionHandlerService] create_field executeForWorkspaceSchema: 1.585ms [BaseWorkspaceMigrationRunnerActionHandlerService] create_field executeForMetadata: 1.223ms [BaseWorkspaceMigrationRunnerActionHandlerService] create_field executeForWorkspaceSchema: 2.221ms [BaseWorkspaceMigrationRunnerActionHandlerService] create_field executeForMetadata: 5.375ms [BaseWorkspaceMigrationRunnerActionHandlerService] create_field executeForWorkspaceSchema: 5.943ms [BaseWorkspaceMigrationRunnerActionHandlerService] create_field executeForMetadata: 1.363ms [BaseWorkspaceMigrationRunnerActionHandlerService] create_field executeForWorkspaceSchema: 1.997ms [BaseWorkspaceMigrationRunnerActionHandlerService] create_field executeForMetadata: 2.03ms [BaseWorkspaceMigrationRunnerActionHandlerService] create_field executeForWorkspaceSchema: 2.982ms [BaseWorkspaceMigrationRunnerActionHandlerService] create_field executeForMetadata: 0.88ms [BaseWorkspaceMigrationRunnerActionHandlerService] create_field executeForWorkspaceSchema: 2.977ms [BaseWorkspaceMigrationRunnerActionHandlerService] create_field executeForMetadata: 9.638ms [BaseWorkspaceMigrationRunnerActionHandlerService] create_field executeForWorkspaceSchema: 11.297ms [BaseWorkspaceMigrationRunnerActionHandlerService] create_field executeForMetadata: 1.581ms [BaseWorkspaceMigrationRunnerActionHandlerService] create_field executeForWorkspaceSchema: 2.248ms [BaseWorkspaceMigrationRunnerActionHandlerService] create_field executeForMetadata: 0.847ms [BaseWorkspaceMigrationRunnerActionHandlerService] create_field executeForWorkspaceSchema: 1.732ms [Runner] Transaction execution: 82.555ms [Runner] Cache invalidation flatFieldMetadataMaps,flatIndexMaps,flatObjectMetadataMaps: 89.247ms [Runner] Total execution: 171.963ms [Nest] 20678 - 12/30/2025, 1:47:35 PM LOG [BackfillUpdatedByFieldCommand] Successfully backfilled updatedBy field for 12 objects in workspace 20202020-1c25-4d02-bf25-6aeccf7ea419 [Nest] 20678 - 12/30/2025, 1:47:35 PM LOG [BackfillUpdatedByFieldCommand] Running command on workspace 3b8e6458-5fc1-4e63-8563-008ccddaa6db 2/2 [Nest] 20678 - 12/30/2025, 1:47:35 PM LOG [BackfillUpdatedByFieldCommand] Starting backfill of updatedBy field for workspace 3b8e6458-5fc1-4e63-8563-008ccddaa6db [Nest] 20678 - 12/30/2025, 1:47:35 PM LOG [BackfillUpdatedByFieldCommand] Found 10 objects that need updatedBy field [EntityBuilder fieldMetadata] matrix computation: 3.224ms [EntityBuilder fieldMetadata] creation validation: 0.969ms [EntityBuilder fieldMetadata] deletion validation: 0.014ms [EntityBuilder fieldMetadata] update validation: 0.002ms [EntityBuilder fieldMetadata] entity processing: 5.149ms [EntityBuilder fieldMetadata] validateAndBuild: 8.472ms [EntityBuilder index] matrix computation: 0.785ms [EntityBuilder index] creation validation: 0.002ms [EntityBuilder index] deletion validation: 0.014ms [EntityBuilder index] update validation: 0.002ms [EntityBuilder index] entity processing: 3.818ms [EntityBuilder index] validateAndBuild: 4.637ms [Runner] Initial cache retrieval: 0.05ms [BaseWorkspaceMigrationRunnerActionHandlerService] create_field executeForMetadata: 2.964ms [BaseWorkspaceMigrationRunnerActionHandlerService] create_field executeForWorkspaceSchema: 3.902ms [BaseWorkspaceMigrationRunnerActionHandlerService] create_field executeForMetadata: 1.502ms [BaseWorkspaceMigrationRunnerActionHandlerService] create_field executeForWorkspaceSchema: 2.039ms [BaseWorkspaceMigrationRunnerActionHandlerService] create_field executeForMetadata: 2.673ms [BaseWorkspaceMigrationRunnerActionHandlerService] create_field executeForWorkspaceSchema: 3.403ms [BaseWorkspaceMigrationRunnerActionHandlerService] create_field executeForMetadata: 1.143ms [BaseWorkspaceMigrationRunnerActionHandlerService] create_field executeForWorkspaceSchema: 2.123ms [BaseWorkspaceMigrationRunnerActionHandlerService] create_field executeForMetadata: 2.033ms [BaseWorkspaceMigrationRunnerActionHandlerService] create_field executeForWorkspaceSchema: 2.954ms [BaseWorkspaceMigrationRunnerActionHandlerService] create_field executeForMetadata: 1.826ms [BaseWorkspaceMigrationRunnerActionHandlerService] create_field executeForWorkspaceSchema: 2.903ms [BaseWorkspaceMigrationRunnerActionHandlerService] create_field executeForMetadata: 1.495ms [BaseWorkspaceMigrationRunnerActionHandlerService] create_field executeForWorkspaceSchema: 2.106ms [BaseWorkspaceMigrationRunnerActionHandlerService] create_field executeForMetadata: 14.014ms [BaseWorkspaceMigrationRunnerActionHandlerService] create_field executeForWorkspaceSchema: 14.373ms [BaseWorkspaceMigrationRunnerActionHandlerService] create_field executeForMetadata: 1.669ms [BaseWorkspaceMigrationRunnerActionHandlerService] create_field executeForWorkspaceSchema: 2.48ms [BaseWorkspaceMigrationRunnerActionHandlerService] create_field executeForMetadata: 0.62ms [BaseWorkspaceMigrationRunnerActionHandlerService] create_field executeForWorkspaceSchema: 1.048ms [Runner] Transaction execution: 73.222ms [Runner] Cache invalidation flatFieldMetadataMaps,flatIndexMaps,flatObjectMetadataMaps: 37.703ms [Runner] Total execution: 111.071ms ``` --- --------- Co-authored-by: prastoin <paul@twenty.com> |
||
|
Marie
|
a0b963ef86 |
Remove viewGroup.fieldMetadataId (#16571)
Final step of https://github.com/orgs/twentyhq/projects/1/views/8?pane=issue&itemId=142348748&issue=twentyhq%7Ccore-team-issues%7C1965 Removing viewGroup.fieldMetadataId. It's already not used in FE anymore |
||
|
Guillim
|
999bc84b17 |
fix(server): Favortites, attachments, timeline... (#16509)
fix(server): system objects like Favortites, attachments, timeline... should be system fields |
||
|
Guillim
|
b83dc3aaff |
TimelineActivity migration to morph (#15652)
# TimelineActivity migration to morph - Creates `timelineActivities2` relations on Company, Dashboard, Note, Opportunity, Person, Task, Workflow, WorkflowRun, and WorkflowVersion entities with proper metadata and cascade delete behavior. It was required to create standard fields as well since the mapObjectMetadataByUniqueIdentifier needs it. otherwise the fields won't be considered - Feature Flag `IS_TIMELINE_ACTIVITY_MIGRATED` necessary to have the two states in parallel. It is used as a stamp once the migration has been run - Migration is done using the coreDataSource. Why ? even though is unsafe to use, the first implementation of the migration took forever on each workspace. See [this commit](https://github.com/twentyhq/twenty/pull/15652/commits/477011e8d7d4c580f79ba7ec4a8fb002a3ec86b2) The plan for this complex migration is as follows :  Note: we will need to rename fields in the release 1.12 (there is no easy way to do all this in one release) |
||
|
Paul Rastoin
|
7b98804ec1 |
Fix flat field metadata date type (#16445)
# Introduction Following https://github.com/twentyhq/twenty/pull/16420 |
||
|
Charles Bochet
|
a18203934c |
Fix flat entity maps date serialization (#16420)
Changes: - as we store date in redis as serialized, let's make all flatEntity dates as string. This requires changing FlatEntity types and making sure that entity are converted to flatEntity and flatEntity to dtos |
||
|
Paul Rastoin
|
4996f3dd28 |
Finalize twenty standard app as workspace migration object and fields (#16353)
# Introduction Related to https://github.com/twentyhq/core-team-issues/issues/1995 In this PR we're fixing the remaining object/fields validation errors resulting from standard objects and fields now passing a validation that wasn't when using the sync metadata ## Key Changes - **Field naming**: Renamed `iCalUID` to `iCalUid` for consistent camelCase convention across calendar events - **Enum standardization**: Uppercased enum values for message channels (email→EMAIL), message participants (from→FROM, to→TO, cc→CC, bcc→BCC), and message direction (incoming→INCOMING, outgoing→OUTGOING) - **Label simplification**: Removed example values from workspace member number format labels for cleaner UI - **Migration infrastructure**: Added `isSystemBuild` flag throughout field metadata service pipeline to allow system-level updates of standard fields that bypass normal restrictions ## Migrating the existing data We've created an upgrade command that will identify using the existing object and field standard id field that needs to be updated, even though the sync metadata still in usage could have fix them ( and the goal is to deprecate it by the end of the sprint ) We will call the updateOneField for each of them, we're passing by the field service in order to battle test what are going to be the temporary way to handle standard migrations when we will start deprecating the sync metadata but haven't still refactored the v2 workspace migration to be workspace agnostic ## Twenty eng migration Tested the whole migration + upgrade on twenty eng Here are generated workspace migration Records are handled natively gracefully too ### ICalUid ```json { "status": "success", "workspaceMigration": { "relatedFlatEntityMapsKeys": [ "flatFieldMetadataMaps", "flatIndexMaps", "flatViewFilterMaps", "flatViewGroupMaps", "flatViewMaps", "flatViewFieldMaps", "flatObjectMetadataMaps" ], "actions": [ { "type": "update_field", "fieldMetadataId": "", "objectMetadataId": "", "updates": [ { "from": "iCalUID", "to": "iCalUid", "property": "name" } ] } ], "workspaceId": "" } } ``` ### Incoming Outgoing None as already caps in database somehow ```json { "status": "success", "workspaceMigration": { "relatedFlatEntityMapsKeys": [ "flatFieldMetadataMaps", "flatIndexMaps", "flatViewFilterMaps", "flatViewGroupMaps", "flatViewMaps", "flatViewFieldMaps", "flatObjectMetadataMaps" ], "actions": [], "workspaceId": "" } } ``` ### EMAIL ```json { "status": "success", "workspaceMigration": { "relatedFlatEntityMapsKeys": [ "flatFieldMetadataMaps", "flatIndexMaps", "flatViewFilterMaps", "flatViewGroupMaps", "flatViewMaps", "flatViewFieldMaps", "flatObjectMetadataMaps" ], "actions": [ { "type": "update_field", "fieldMetadataId": "", "objectMetadataId": "", "updates": [ { "from": "'email'", "to": "'EMAIL'", "property": "defaultValue" }, { "from": [ { "color": "green", "id": "", "label": "Email", "position": 0, "value": "email" }, { "color": "blue", "id": "", "label": "SMS", "position": 1, "value": "sms" } ], "to": [ { "color": "green", "id": "", "label": "Email", "position": 0, "value": "EMAIL" }, { "color": "blue", "id": "", "label": "SMS", "position": 1, "value": "SMS" } ], "property": "options" } ] } ], "workspaceId": "e" } } ``` ### MessageParticipantRole ```json { "status": "success", "workspaceMigration": { "relatedFlatEntityMapsKeys": [ "flatFieldMetadataMaps", "flatIndexMaps", "flatViewFilterMaps", "flatViewGroupMaps", "flatViewMaps", "flatViewFieldMaps", "flatObjectMetadataMaps" ], "actions": [ { "type": "update_field", "fieldMetadataId": "", "objectMetadataId": "", "updates": [ { "from": "'from'", "to": "'FROM'", "property": "defaultValue" }, { "from": [ { "color": "green", "id": "", "label": "From", "position": 0, "value": "from" }, { "color": "blue", "id": "", "label": "To", "position": 1, "value": "to" }, { "color": "orange", "id": "", "label": "Cc", "position": 2, "value": "cc" }, { "color": "red", "id": "", "label": "Bcc", "position": 3, "value": "bcc" } ], "to": [ { "color": "green", "id": "", "label": "From", "position": 0, "value": "FROM" }, { "color": "blue", "id": "", "label": "To", "position": 1, "value": "TO" }, { "color": "orange", "id": "", "label": "Cc", "position": 2, "value": "CC" }, { "color": "red", "id": "", "label": "Bcc", "position": 3, "value": "BCC" } ], "property": "options" } ] } ], "workspaceId": "" } } ``` ### Workspace member number format labels ```json { "status": "success", "workspaceMigration": { "relatedFlatEntityMapsKeys": [ "flatFieldMetadataMaps", "flatIndexMaps", "flatViewFilterMaps", "flatViewGroupMaps", "flatViewMaps", "flatViewFieldMaps", "flatObjectMetadataMaps" ], "actions": [ { "type": "update_field", "fieldMetadataId": "", "objectMetadataId": "", "updates": [ { "from": [ { "color": "turquoise", "id": "", "label": "System", "position": 0, "value": "SYSTEM" }, { "color": "blue", "id": "", "label": "Commas and dot (1,234.56)", "position": 1, "value": "COMMAS_AND_DOT" }, { "color": "green", "id": "", "label": "Spaces and comma (1 234,56)", "position": 2, "value": "SPACES_AND_COMMA" }, { "color": "orange", "id": "", "label": "Dots and comma (1.234,56)", "position": 3, "value": "DOTS_AND_COMMA" }, { "color": "purple", "id": "", "label": "Apostrophe and dot (1'234.56)", "position": 4, "value": "APOSTROPHE_AND_DOT" } ], "to": [ { "color": "turquoise", "id": "", "label": "System", "position": 0, "value": "SYSTEM" }, { "color": "blue", "id": "", "label": "Commas and dot", "position": 1, "value": "COMMAS_AND_DOT" }, { "color": "green", "id": "", "label": "Spaces and comma", "position": 2, "value": "SPACES_AND_COMMA" }, { "color": "orange", "id": "", "label": "Dots and comma", "position": 3, "value": "DOTS_AND_COMMA" }, { "color": "purple", "id": "", "label": "Apostrophe and dot", "position": 4, "value": "APOSTROPHE_AND_DOT" } ], "property": "options" } ] } ], "workspaceId": "" } } ``` |
||
|
Etienne
|
07cfaa78ef |
Fix unique standard field (#16371)
Fixes https://github.com/twentyhq/twenty/issues/15925 - update field metadata update logic - uniformize the way index are named - command to migrate v1-named unique index - add integration testing --------- Co-authored-by: prastoin <paul@twenty.com> |
||
|
Marie
|
7ce22d5c7e |
breaking (soft) - Migrate viewGroup.fieldMetadataId -> view.mainGroupByFieldMetadataId (2/3) (#16277)
Should be merged once https://github.com/twentyhq/twenty/pull/16206 has been released + command run to prod In this PR - Remove usage of viewGroup.fieldMetadataId, both in BE and FE states. - But we still need to properly populate it until we fully remove viewGroup.fieldMetadataId from db and ORM entity (upcoming 3rd PR out of 3). fieldMetadataId was removed from CoreViewGroup type and CreateViewGroupInput and is determined BE-side based on the associated view's mainGroupByFieldMetadataId. **I expect this means a downtime on viewGroup creation, until both FE and BE are deployed and cache is flushed.** This seems acceptable to me as it only regards viewGroup creation. - this information is replaced by view.mainGroupByFieldMetadataID - Handle view group creation, update and deletion in the BE as a side-effect of a view creation, update or deletion. Optimistic effects are still used - Add validation at view creation or update regarding mainGroupByFieldMetadata Left to do in 3rd PR - Remove viewGroup.fieldMetadataId from db and ORM entity - Restore feature allowing to update an existing grouped view's group by field (already OK on BE side but need to rebuild FE optimistic) |
||
|
Paul Rastoin
|
28cdb02fbb |
Twenty standard application Objects and fields as allFlatEntityMaps ID non-agnostic (#16298)
# Introduction Related to https://github.com/twentyhq/core-team-issues/issues/1995 This PR introduces the basis of the `twentyStandard` application as code on demand, it's highly tied to `ids` where it will becomes workspace agnostic following the builder and runner `universalIdentifier` refactor later. The goal here to allow computing the `allFlatEntityMaps` `to` of the `twentyStandard` application on a empty workspace ( workspace creation ). Allowing installing the twenty standard app through a workspace migration instead of passing by the sync metadata Nothing done will be run in production for the moment if it's not the small validation refactor we've introduced Please note that everything introduced here will be replaced at some point by a twenty app instance when the twenty sdk is mature enough to handle of the edge cases we need here ## How we've proceeded We've been iterating over every workspace entity both objects and their fields, and transpiled them to flatEntity. Being sure we migrate the defaultValue, settings and so on accordingly. We've also compute all the ids in prior of the whole entities computation so we don't face any hoisting issue. ## Current state At the moment only handling all of the 29 standard objects and their fields Settings a unique universalIdentifier for all of them Will come views, agent role targets and so on later ## `workspace:compute-twenty-standard-migration` command This command allow generating a workspace migration that will result in installing the twenty standard app in an empty workspace It's temporary and aims to allow debugging for the moment we might not keep it in the future as it is right now It contains debug writeFileSync which is expected no worries greptile ## `LabelFieldMetadataIdentifierId` Small refactor allowing defining the label identifier field metadata id of a uuid field metadata type for system object, as some of our standard object don't have a name field and don't aim to Also please note that we might remove this build options later in the sake of the currently installed universal identifier application that we could compare with the deterministic twenty standard one ## `runFlatFieldMetadataValidators` Deprecated this pattern which was redundant and not v2 friendly pattern ## Current errors that will address in upcoming PR Current standard objects and fields metadata does not pass the validation that we have in place, as historically the sync metadata would directly consume the repositories and would just ignore the validation. This is about to change. Will handle the below errors in dedicated PRs as they will required upgrade commands in order to migrate the data, or will handle that from the sync metadata instead still to be determined but nothing critical here - camel case field metadata name - options label invalid format ```json { "status": "fail", "report": { "fieldMetadata": [ { "status": "fail", "errors": [ { "code": "INVALID_FIELD_INPUT", "message": "Name should be in camelCase", "userFriendlyMessage": { "id": "P+jdmX", "message": "Name should be in camelCase" }, "value": "iCalUID" } ], "flatEntityMinimalInformation": { "id": "68dd83cd-92c8-4233-bb28-47939bab6124", "name": "iCalUID", "objectMetadataId": "11c16ab6-9176-439e-a2db-a12c5a58a524" }, "type": "create_field" }, { "status": "fail", "errors": [ { "code": "INVALID_FIELD_INPUT", "message": "Value must be in UPPER_CASE and follow snake_case \"email\"", "userFriendlyMessage": { "id": "UBPzFQ", "message": "Value must be in UPPER_CASE and follow snake_case \"{sanitizedValue}\"", "values": { "sanitizedValue": "email" } }, "value": "email" }, { "code": "INVALID_FIELD_INPUT", "message": "Value must be in UPPER_CASE and follow snake_case \"sms\"", "userFriendlyMessage": { "id": "UBPzFQ", "message": "Value must be in UPPER_CASE and follow snake_case \"{sanitizedValue}\"", "values": { "sanitizedValue": "sms" } }, "value": "sms" } ], "flatEntityMinimalInformation": { "id": "e3caaf2a-e07d-4146-8dfc-9eef904e82c9", "name": "type", "objectMetadataId": "4b777de5-4c7b-4af4-9b92-655c0f87512b" }, "type": "create_field" }, { "status": "fail", "errors": [ { "code": "INVALID_FIELD_INPUT", "message": "Value must be in UPPER_CASE and follow snake_case \"incoming\"", "userFriendlyMessage": { "id": "UBPzFQ", "message": "Value must be in UPPER_CASE and follow snake_case \"{sanitizedValue}\"", "values": { "sanitizedValue": "incoming" } }, "value": "incoming" }, { "code": "INVALID_FIELD_INPUT", "message": "Value must be in UPPER_CASE and follow snake_case \"outgoing\"", "userFriendlyMessage": { "id": "UBPzFQ", "message": "Value must be in UPPER_CASE and follow snake_case \"{sanitizedValue}\"", "values": { "sanitizedValue": "outgoing" } }, "value": "outgoing" } ], "flatEntityMinimalInformation": { "id": "d96233a4-93be-45ea-9548-3b50f3c700cf", "name": "direction", "objectMetadataId": "480a648a-d2e5-482a-992f-ef053e1b4bb0" }, "type": "create_field" }, { "status": "fail", "errors": [ { "code": "INVALID_FIELD_INPUT", "message": "Value must be in UPPER_CASE and follow snake_case \"from\"", "userFriendlyMessage": { "id": "UBPzFQ", "message": "Value must be in UPPER_CASE and follow snake_case \"{sanitizedValue}\"", "values": { "sanitizedValue": "from" } }, "value": "from" }, { "code": "INVALID_FIELD_INPUT", "message": "Value must be in UPPER_CASE and follow snake_case \"to\"", "userFriendlyMessage": { "id": "UBPzFQ", "message": "Value must be in UPPER_CASE and follow snake_case \"{sanitizedValue}\"", "values": { "sanitizedValue": "to" } }, "value": "to" }, { "code": "INVALID_FIELD_INPUT", "message": "Value must be in UPPER_CASE and follow snake_case \"cc\"", "userFriendlyMessage": { "id": "UBPzFQ", "message": "Value must be in UPPER_CASE and follow snake_case \"{sanitizedValue}\"", "values": { "sanitizedValue": "cc" } }, "value": "cc" }, { "code": "INVALID_FIELD_INPUT", "message": "Value must be in UPPER_CASE and follow snake_case \"bcc\"", "userFriendlyMessage": { "id": "UBPzFQ", "message": "Value must be in UPPER_CASE and follow snake_case \"{sanitizedValue}\"", "values": { "sanitizedValue": "bcc" } }, "value": "bcc" } ], "flatEntityMinimalInformation": { "id": "961c598e-67c3-452d-8bb2-b92c0bc64404", "name": "role", "objectMetadataId": "8af8a13c-ff97-4cd3-b70d-52a7dc2924b4" }, "type": "create_field" }, { "status": "fail", "errors": [ { "code": "INVALID_FIELD_INPUT", "message": "Label must not contain a comma", "userFriendlyMessage": { "id": "k731jp", "message": "Label must not contain a comma" }, "value": "Commas and dot (1,234.56)" }, { "code": "INVALID_FIELD_INPUT", "message": "Label must not contain a comma", "userFriendlyMessage": { "id": "k731jp", "message": "Label must not contain a comma" }, "value": "Spaces and comma (1 234,56)" }, { "code": "INVALID_FIELD_INPUT", "message": "Label must not contain a comma", "userFriendlyMessage": { "id": "k731jp", "message": "Label must not contain a comma" }, "value": "Dots and comma (1.234,56)" } ], "flatEntityMinimalInformation": { "id": "7fa20caf-2597-42e3-84e5-15a91b125b9b", "name": "numberFormat", "objectMetadataId": "a6974302-9e72-461c-aa09-9390f4ff16fc" }, "type": "create_field" } ], "objectMetadata": [], "view": [], "viewField": [], "viewGroup": [], "index": [], "serverlessFunction": [], "cronTrigger": [], "databaseEventTrigger": [], "routeTrigger": [], "viewFilter": [], "role": [], "roleTarget": [], "agent": [] } } ``` |
||
|
Paul Rastoin
|
59672e3e34 |
Migrate agent v2 (#16214)
# Introduction Closes https://github.com/twentyhq/core-team-issues/issues/1980 In this PR we migrate the agent from v1 to v2. ## New FlatRoleTargetByAgentIdMaps Derivated the `flatRoleTargetMaps` to be building a `flatRoleTargetByAgentIdMaps` to ease retrieving a roleId to associate to an agent ## Coverage Added strong coverage on both failing and successful CRU agents operations --------- Co-authored-by: Weiko <corentin@twenty.com> |
||
|
Marie
|
77409b6eb2 |
[Requires "warm" cache flush (no immediate downtime before flush)] Migrate viewGroup.fieldMetadataId -> view.mainGroupByFieldMetadataId (1/3) (#16206)
In this PR (1/3) - introduce view.mainGroupByFieldMetadataId as the new reference determining which fieldMetadataId is used in a grouped view, in order to deprecate viewGroup.fieldMetadataId which creates inconsistencies. view.mainGroupByFieldMetadataId is now filled at every view creation, though not in use yet. - Introduce a command to backfill view.mainGroupByFieldMetadataId for existing views + delete all viewGroup.fieldMetadataId with a fieldMetadataId that is not view.mainGroupByFieldMetadataId. (It should concern 37 active workspaces) - Temporarily disable the option to change a grouped view's fieldMetadataId as for now it creates inconsistencies. This feature can be reintroduced when we have done the full migration. In a next PR - (2/3) use view.mainGroupByFieldMetadataId instead of viewGroup.fieldMetadataId. In FE we may keep viewGroup.fieldMetadataId as a state (TBD). View groups will now be created / deleted as a side effect of view's mainGroupByFieldMetadataId update. - (3/3) remove viewGroup.fieldMetadataId --------- Co-authored-by: Charles Bochet <charles@twenty.com> |
||
|
Weiko
|
1eb2e44058 |
Refactor workspace cache service (#16208)
## Context We've recently introduced a new workspace cache service which now acts as a cache access and local storage for all workspace related data, deprecating the individual specific services. - Better performance through multiple caching/fetching strategies - Consistent data access patterns across the codebase - Reduced redis queries through MGET/MSET/PIPELINE with multiple cache keys |
||
|
Abdullah.
|
ee08060798 |
Improve deactivated objects & fields behaviors. (#16090)
Closes [1918](https://github.com/twentyhq/core-team-issues/issues/1918). - For the first point in the issue, we just show the deactivated entries along with the deactivated text. --- - For the second point, we show a banner and control the enabled/disabled state of save button depending on whether we're allowing the user to create table with the typed name. - For example, we do not want to allow the user to create a table with reserved name, so we disable the save button without showing a banner. - Similarly, we do not want the user to create a table with a name that already exists in the database. In this case, we show a banner and we also disable the save button. - Finally, we do not want to allow the user to create a table where singular and plural name are the same. Therefore, we disable the save button for names like `works`. --- - For the third point, if we add the delete button, it logically means that we allow the user to delete a custom object/field even it has not been deactivated yet, so did that. - Upon deleting the object/field, if we wait for the metadata to refetch before we navigate, this is what we see because the path does not exist any longer after deletion and we're waiting for refetch on the path until we navigate away. https://github.com/user-attachments/assets/dbe0569c-db88-4285-851f-22551b1ca81e - To avoid this page from appearing, I replaced awaiting refetch to not awaiting refetch and redirecting while the refetch happens in the background. - Therefore, when we delete something, there is a slight delay for when it is actually cleared out from the list, but the Not Found view does not appear on the screen. https://github.com/user-attachments/assets/47f49579-ce51-4d6a-b857-72046247bb4b - I tried optimistically removing the object/field from the metadata, but it leads to some issues (crashes the app) and I have not been able to find a solution for it yet. - Therefore, instead of getting stuck at perfection and blocking myself, I stopped getting into the issue further and created this PR by ensuring that the desired functionality works. <!-- CURSOR_SUMMARY --> --- > [!NOTE] > Display deactivated objects/fields by default, add delete actions with confirmation, and unify metadata name computation (auto-suffix reserved keywords) across front/back with conflict checks in object creation. > > - **Frontend (Settings/Data Model)**: > - **Visibility/UX**: Show `Deactivated` labels for objects/fields; filters default to include inactive (`showDeactivated`/`showInactive` true); replace field action dropdown with chevron link. > - **Delete flows**: Add delete buttons for custom objects/fields with confirmation modals and background refetch to avoid Not Found flashes. > - **Creation/Edit validation**: Add name conflict detection banner in `SettingsDataModelObjectAboutForm` and disable Save on conflicts; simplify `metadataLabelSchema` to use computed name; form fields validate on change and sync API names. > - **Shared (twenty-shared/metadata)**: > - Add `computeMetadataNameFromLabel` util (slugify+camelCase) and `RESERVED_METADATA_NAME_KEYWORDS`; auto-append `Custom` to reserved names; export constants/utilities. > - **Backend**: > - Migrate to shared `computeMetadataNameFromLabel`; update validators to use shared reserved keywords with new messages; allow deletion of active custom fields/objects (keep standard guards); adjust services/decorators accordingly. > - **Tests/Stories**: > - Update unit/integration snapshots for new reserved-name messages and behaviors; add missing i18n/router decorators in stories. > > <sup>Written by [Cursor Bugbot](https://cursor.com/dashboard?tab=bugbot) for commit 5b126155606f6dbc8f7f91e2192cffb7bd2ebd2c. This will update automatically on new commits. Configure [here](https://cursor.com/dashboard?tab=bugbot).</sup> <!-- /CURSOR_SUMMARY --> --------- Co-authored-by: Félix Malfait <felix.malfait@gmail.com> Co-authored-by: Félix Malfait <felix@twenty.com> |
||
|
Weiko
|
1607aebcc6 |
Deprecate object metadata maps in favor of flat entities (#16080)
## Context Deprecating the old objectMetadataMap type in favour of split flat entities to match with our new caching. In the long run, trying to achieve: - Better performance through caching - Consistent data access patterns across the codebase - Reduced database queries Now that everything is based on flat entities, which are cached, we can finish the refactoring of workspace context cache which should already improve performances. Then the last step will be to consume that new cache in the new global datasource to get rid of the many workspace datasources stored in the server |
||
|
Guillim
|
1a45576990 |
Morph-add-new-object-destination (#16027)
Add new object target to an existing morph relation (backend only) Fixes https://github.com/twentyhq/core-team-issues/issues/1898 --------- Co-authored-by: prastoin <paul@twenty.com> |
||
|
neo773
|
316aec3c40 |
fix: prevent NUMERIC field type creation via API (#16038)
Block users from creating NUMERIC, POSITION, and TS_VECTOR fields via the API as these are system-only types. Users should use NUMBER instead of NUMERIC. /closes #16023 |
||
|
Weiko
|
04562b11fb |
Migrate metadata cache (#16030)
## Context Deprecating legacy ObjectMetadata from cache in favor of flat entities. Introducing utils to build byName/byNameSingular/byNamePlural in isolated cases ## Next - I had to introduce a util to build from flat to legacy objectMetadataMaps, we should instead use flat maps directly when needed (datasource, schema generation, etc) - Deprecate metadata version in the cache - Use the new cache strategy for flat entities with permissions and feature flags and inject in the global datasource context |
||
|
Etienne
|
208c0857ee |
common api - null equivalence (#15926)
closes https://github.com/twentyhq/core-team-issues/issues/1629 To do before requesting review : - filter update Migration to come in an other PR Strat : 1/ Null transformation - [x] Transform NULL equivalent value to NULL in field validation in common api - pre-query - with feature flag - [ ] Same logic in ORM (Not done, complex to handle feature flag here) - [x] Transform NULL value to equivalent in data formatting in ORM - post-query 2/ Migration (in other PR) for fieldMetadata not nullable with default defaultValue (empty string, ...) - [ ] Remove NOT NULL db constraint - [ ] Update record value to NULL - [ ] Update field metadata : isNullable:true - [ ] Update uniqueIndex whereClause (also for standard uniqueIndex) - [ ] Activate feature flag 3/ Update metadata creation - [x] No more default default value - [x] Update standard field nullability - [x] Remove index default whereClause for standard field 4/ Update filter - [x] When filtering on NULL or empty string, be sure all records are returned (the one with NULL + the one with "") 5/ Test - [ ] Strat. to do |
||
|
Paul Rastoin
|
f9ab09c404 |
Metadata api create entity in workspace custom app (#15911)
# Introduction Cleaner and fewer scope version of https://github.com/twentyhq/twenty/pull/15745 ( removed sync-metadata hack through, too ambitious migration and upgrade ) Please note that this PR won't have any interaction with the existing sync-metadata Which mean that the sync metadata does not update the standard entities applicationId and universalIdentifier, and it won't we will deprecate it on favor of a workspace migration aka twenty-standard app installation ## API Metadata Any operation going through the api metadata nows automatically scope the related entity to the workspace custom application instance. ( optionally passing an applicationId to allow current hacky implem of app sync service ) We need to either ignore the tests or remove the cli status check from the blocking status badges for a PR to be merged ## New workspace Already handled in previous https://github.com/twentyhq/twenty/pull/15625, when a workspace is created it gets created a twenty standard and custom workspace instance All his views and permissions will be prefilled to the its twenty standard app instance with a specific universalIdentifier ## New universalIdentifier At the contrary as before with standardIds, universalIdentifier are unique for a given workspace This means that createdAt field of both object company and opportunity will have a unique universalIdentifier whereas they share the same standardId ## FlatApplication Introduced the flatApplication and cache. Will migrate existing `MetadataName` to be `SyncableMetadataName` in a following PR ## What's next Next we will describe a twenty standard app configuration as json that will be used to generate a workspace migration that will be run instead of the sync metadata, in a nutshell we aim to deprecated the sync metadata So we can standardize any entity to have a non nullable applicationId and universalIdentifier ## Upgrade command Introduced an upgrade command that will create a custom workspace instance for any workspace that do not have one in order to align with the new behavior when creating a new workspace |
||
|
Guillim
|
4848bc03f3 |
Update Name of relation fieldMetadata (#15749)
UpdateOne of a Relation that involves a CustomObject, because the nameSingular needs to be updated in the fieldMetadata - nameSingular and namePlural must be provided since they are necessary for morph name computation - label sync should be false Interesting files to look at: - packages/twenty-server/src/engine/metadata-modules/object-metadata/utils/build-default-relation-flat-field-metadatas-for-custom-object.util.ts - UPDATE => packages/twenty-server/src/engine/metadata-modules/flat-object-metadata/utils/rename-related-morph-field-on-object-names-update.util.ts ( also update relation indexes ) needs v2 refactor to handle field relation name update - CREATE => packages/twenty-server/src/engine/metadata-modules/object-metadata/utils/build-default-relation-flat-field-metadatas-for-custom-object.util.ts ( handle morph instead of previous classic relation ) - DELETE => DONE Edit: closes https://github.com/twentyhq/core-team-issues/issues/1897 --------- Co-authored-by: prastoin <paul@twenty.com> |
||
|
Paul Rastoin
|
e5255df1a1 |
Fix and refactor relation field name collision validation (#15920)
# Introduction Fixes https://github.com/twentyhq/private-issues/issues/371 We weren't strictly validating relation field collision on join column name availability of the target field object ## Refactor Extracted morph or relation specific condition out of the common flat field metadata name validate availability to be located in the dedicated morph or relation flat field validator ## Tests Added two tests, ONE_TO_MANY and MANY_TO_ONE in order to cover the use case |
||
|
Paul Rastoin
|
a39efeb1ab |
[BREAKING_CHANGE/GRAPHQL/OBJECT_METADATA_CREATE_ONE] Remove object/fields/view-fields v1 implementation (#15823)
# Introduction Remove the v2 feature flag for view-field field-metadata and object-metadata metadata entities ## Some details - Disabled nestjs-query for object metadata creation and explicitly calling it - removed all v1 integration tests files ## Remarks Not remove v2 referencing in both filenaming right now will handle that globally later ## Breaking change Due to object metadata resolver createOne standardization had to rename the input from `CreateObjectInput` to `CreateOneObjectInput` |
||
|
Guillim
|
0389fcf00d |
removing feature flag IS_MORPH_RELATION_ENABLED (#15783)
releasing the morph feature for all workspaces |
||
|
martmull
|
9880f192a5 | Move composite types to twenty-shared (#15741) | ||
|
martmull
|
abde3c04ac |
1630 extensibility twenty cli ability to create edit and delete fields (#15501)
As title - adds decorators in twenty-sdk - update twenty-cli load-manifest to it gets @FieldMetadata infos + testing - update twenty-server so it CRUD fields properly, using universalIdentifier - Fix UI so we can update managed objects records - move FieldMetadata items from twenty-server to twenty-shared |
||
|
Paul Rastoin
|
3514054235 |
V2 centralize relation optimistic logic (#15552)
# Introduction
This PR aims to deprecate having to manually handle optimistic side
effect foreign key addition in the whole v2 experience.
This PR implements the strong basis + builder refactor of the optimistic
computation of a given flat entity maps with its related flat entity
maps ( runner needs a small refactor on actions type definition first )
Flat entity maps updates through mutations are now only scoped to the
generic entity builder ( very isolated )
## What's next
- Refactor actions v2 type definition to gain grain over `metadataName`
and action operation ( `create` `delete` `update` ).
from `{type: 'create_view_field'}` to `{metadataName: 'view_field',
type: 'create' }`
- Use new optimistic tool computation tools
- Only invalidate impacted flat maps cache
## New tools
Strictly dynamically typed new flat entity maps tools
- `addFlatEntityToFlatEntityAndRelatedEntityMapsThroughMutationOrThrow`
-
`deleteFlatEntityFromFlatEntityAndRelatedEntityMapsThroughMutationOrThrow`
## Unit test
Adding basic unit testing coverage to introduced tools
## `FlatEntityValidationArgs`
From
```ts
export type FlatEntityValidationArgs<T extends AllMetadataName> = {
flatEntityToValidate: MetadataFlatEntity<T>;
optimisticFlatEntityMaps: MetadataFlatEntityMaps<T>;
mutableDependencyOptimisticFlatEntityMaps: MetadataValidationRelatedFlatEntityMaps<T>;
workspaceId: string;
remainingFlatEntityMapsToValidate: MetadataFlatEntityMaps<T>;
buildOptions: WorkspaceMigrationBuilderOptions;
};
```
To
```ts
export type FlatEntityValidationArgs<T extends AllMetadataName> = {
flatEntityToValidate: MetadataFlatEntity<T>;
optimisticFlatEntityMapsAndRelatedFlatEntityMaps: MetadataFlatEntityAndRelatedFlatEntityMapsForValidation<T>;
workspaceId: string;
remainingFlatEntityMapsToValidate: MetadataFlatEntityMaps<T>;
buildOptions: WorkspaceMigrationBuilderOptions;
};
```
|
||
|
Paul Rastoin
|
d640b93096 |
Improve v2 and cache invalidation perfs (#15467)
# Introduction Log are debug logs of `packages/twenty-server/test/integration/metadata/suites/object-metadata/create-delete-and-create-object-metadata-v2.integration-spec.ts`run ten times in a row on clean db reset ## Next Will improve cache computation to lighter invalidation. RelationLoad `query` does not seem to work with typeorm so I'll continue the custom integration i've started in https://github.com/twentyhq/twenty/tree/optimize-cache-read-v2 ## Integration tests duration Significant test duration improvement too ### Before <img width="2632" height="1402" alt="image" src="https://github.com/user-attachments/assets/2f1f0ccf-44de-4856-bfe1-4f45a351763a" /> ### After <img width="2632" height="1402" alt="image" src="https://github.com/user-attachments/assets/4bc9e6db-3046-48b9-b903-1464053936c4" /> ## What's next - The legacy cache invalidation removal - Factorizing redis calls in only one operation ## Autogenerated performance comparison ( including mutation refactor too ) [Before](https://gist.github.com/prastoin/3c1e21fa9e3b3ce4b0716902ff4a2dd6) [After](https://gist.github.com/prastoin/7bfddd14bfded2e4991a9378970a026d) The optimized implementation shows **dramatic performance improvements** across all metrics: - 🚀 **Cache Invalidation**: 156.3ms → 76.8ms (**50.9% faster**) - 🚀 **Builder Operations**: 21.3ms → 14.2ms (**33.3% faster**) - ⚡ **Consistency**: 16.4% more predictable performance --- ## 1. Overall Performance Summary | Component | Before (avg) | After (avg) | Best (After) | Worst (After) | Improvement | |-----------|-------------|-------------|--------------|---------------|-------------| | **Total Execution Time** | 180.2ms | 110.5ms | 52.3ms | 585.9ms | **38.7% faster** ⚡⚡ | | **Cache Invalidation** | 156.3ms | 76.8ms | 47.8ms | 285.1ms | **50.9% faster** ⚡⚡⚡ | | **Transaction Execution** | 22.4ms | 22.1ms | 0.99ms | 314.2ms | Similar | | **Initial Cache Retrieval** | 0.81ms | 2.08ms | 0.21ms | 8.24ms | Similar | | **Entity Builder (total)** | 21.3ms | 14.2ms | 0.36ms | 42.5ms | **33.3% faster** ⚡ | ### Total Execution Time Distribution #### Before (Legacy Sequential) ``` Time (ms) Count Percentage Visualization < 150 18 16% ████ 150-180 32 29% ███████ 180-210 35 32% ████████ 210-250 17 15% ████ 250-350 6 5% █ > 350 2 2% ▌ ``` #### After (Optimized Parallel) ``` Time (ms) Count Percentage Visualization < 70 28 31% ████████ 70-100 31 34% █████████ 100-150 18 20% █████ 150-200 8 9% ██ 200-300 4 4% █ > 300 2 2% ▌ ``` --- ## 2. Builder Performance Breakdown ### Field Metadata Builder | Operation | Before (avg) | After (avg) | Improvement | |-----------|-------------|-------------|-------------| | Matrix computation | 3.5ms | 3.4ms | Similar | | Creation validation | 15.2ms | 2.1ms | **86% faster** ⚡⚡⚡ | | Deletion validation | 0.08ms | 0.06ms | Similar | | Update validation | 1.3ms | 0.09ms | **93% faster** ⚡⚡⚡ | | Entity processing | 18.6ms | 11.8ms | **37% faster** ⚡ | | **Total validateAndBuild** | **21.3ms** | **14.2ms** | **33% faster** ⚡ | #### Performance Distribution ``` Before: ▁▂▄█████▆▄▂▁ (wide spread, 15-28ms range) After: ▁▁▃█████▃▁▁ (tight clustering, 10-18ms range) ``` ## 4. Cache Invalidation Performance Breakdown ### Cache Invalidation Summary | Metric | Before (Legacy) | After (Optimized) | Improvement | |--------|-----------------|-------------------|-------------| | **Best Time** | 131.965ms | 47.833ms | **63.7% faster** ⚡⚡⚡ | | **10th Percentile** | 140.2ms | 51.7ms | **63.1% faster** ⚡⚡⚡ | | **25th Percentile** | 146.1ms | 54.4ms | **62.7% faster** ⚡⚡⚡ | | **Median (50th)** | 155.1ms | 63.4ms | **59.1% faster** ⚡⚡⚡ | | **Average** | 156.3ms | 76.8ms | **50.9% faster** ⚡⚡⚡ | | **75th Percentile** | 160.2ms | 90.2ms | **43.7% faster** ⚡⚡ | | **90th Percentile** | 191.7ms | 100.2ms | **47.7% faster** ⚡⚡ | | **95th Percentile** | 235.6ms | 110.3ms | **53.2% faster** ⚡⚡⚡ | | **99th Percentile** | 278.2ms | 224.9ms | **19.1% faster** ⚡ | | **Worst Time** | 383.914ms | 285.102ms | **25.7% faster** ⚡ | ### Cache Invalidation Time Distribution #### Before (Legacy Sequential) ``` Time (ms) Count Percentage Visualization 130-140 3 3% ▊ 140-150 15 14% ████ 150-160 48 44% ███████████ 160-180 31 28% ███████ 180-220 8 7% ██ 220-280 3 3% ▊ > 280 2 2% ▌ ``` #### After (Optimized Parallel + Intersection) ``` Time (ms) Count Percentage Visualization < 50 3 3% ▊ 50-60 27 25% ███████ 60-70 25 23% ██████ 70-90 25 23% ██████ 90-100 15 14% ████ 100-120 8 7% ██ 120-150 3 3% ▊ > 150 4 4% █ ``` ## 6. Performance Consistency Analysis ### Standard Deviation & Variance | Metric | Before | After | Improvement | |--------|--------|-------|-------------| | **Cache Invalidation Std Dev** | 42.1ms | 35.2ms | **16.4% more consistent** ⚡ | | **Total Execution Std Dev** | 68.3ms | 89.1ms | Slightly more variable | | **Coefficient of Variation (Cache)** | 26.9% | 45.8% | More variance | | **Outliers (> 2σ)** | 5 cases | 3 cases | **40% fewer outliers** ⚡ | --- |
||
|
Paul Rastoin
|
1bf40d9dca |
Fix v2 self relation field creation (#15382)
# Introduction Fixing self relation field creation in v2 - When computing related flat field to delete on object metadata deletion that has self relation fields - On self relation creation validation name availability not searching for the relation target field of the current object field if it's the field being validated ## Coverage - added CUD integration testing on self relation fields close https://github.com/twentyhq/twenty/issues/15153 |
||
|
Paul Rastoin
|
267af42412 |
Centralize v2 errors types in twenty-shared (#15358)
# Introduction Followup of https://github.com/twentyhq/twenty/pull/15331 ( Reducing size by concerns ) This PR centralizes v2 format error types in `twenty-shared` and consuming them in the existing v2 error format logic in `twenty-server` ## Next This https://github.com/twentyhq/twenty/pull/15360 handles the frontend v2 format error refactor ## Conclusion Related to https://github.com/twentyhq/core-team-issues/issues/1776 |
||
|
Paul Rastoin
|
2e84c11eae |
[v2_FIX] Update standard object/field (#15233)
# Introduction Refactoring the standard overrides dispatcher to only pass over fields to has to be dispatched in the standardOverrides entry and let the other side effects resulting from out of standard overrides mutation trigger Related to https://github.com/twentyhq/core-team-issues/issues/1753 ## This allows - standard field settings, options etc updates and so on ## Remark - Determine what we should do on object deactivation ( right now in production we can still access deactivated object relation properties and so on e.g deactivate opportunities still accessible from a view field on company ( still have to re-create it as it has been deleted ) => decided to leave as it is right now, `isActive` could be considered as uiDeactivated in the end - We should also add forbidden standard field mutations validation inside the builder itself ( here we want to early return in the api input transpiler too as we don't want to spread invalid side effects ) => or in the end we could just centralize both but it will generate several errors ## Coverage ```ts PASS test/integration/metadata/suites/object-metadata/successful-update-one-standard-object-metadata.integration-spec.ts PASS test/integration/metadata/suites/field-metadata/successful-update-one-standard-field-metadata.integration-spec.ts PASS test/integration/metadata/suites/object-metadata/failing-update-one-standard-object-metadata.integration-spec.ts PASS test/integration/metadata/suites/field-metadata/failing-update-one-standard-field-metadata.integration-spec.ts Test Suites: 4 passed, 4 total Tests: 18 passed, 18 total Snapshots: 16 passed, 16 total Time: 8.721 s, estimated 10 s ``` ## Update post review Faced a behavior where updating back the company label to its original value would result in storing this value in the standard overrides Refactored both field and object transpilation behavior to rather remove the standard override value instead and let fallback on original value Yes it's quite duplicated will factorize once we move this inside the builder |
||
|
Paul Rastoin
|
45473218d3 |
Field deactivation side effect views calendar kanban viewFields (#15180)
# Introduction
Handling both:
- field deactivation side effect on view fields, view filters and views
- field deactivation side effect on view that targets it as
`kanbanAggregateFieldMetadataId`
- field deactivation side effect on view that targets it as
`calendarFieldMetadataId`
## Coverage
added coverage
```ts
PASS test/integration/metadata/suites/field-metadata/kanban-aggregate-field-deactivation-deletes-views.integration-spec.ts (13.132 s)
kanban-aggregate-field-deactivation-nullifies-kanban-properties
✓ should nullify kanban properties when field used as kanbanAggregateOperationFieldMetadataId is deactivated (3923 ms)
✓ should not modify views when field not used as kanbanAggregateOperationFieldMetadataId is deactivated (2958 ms)
✓ should nullify kanban properties on multiple views when they all use the same field as kanbanAggregateOperationFieldMetadataId (2542 ms)
✓ should nullify kanban properties when views have different aggregate operations on same field (3380 ms)
Test Suites: 1 passed, 1 total
Tests: 4 passed, 4 total
Snapshots: 0 total
Time: 13.154 s
```
```ts
PASS test/integration/metadata/suites/field-metadata/view-group-field-deactivation-deletes-views.integration-spec.ts (12.639 s)
view-group-field-deactivation-deletes-views
✓ should delete view when field used in view group is deactivated (3469 ms)
✓ should not delete view when field not used in view group is deactivated (3109 ms)
✓ should delete multiple views when they all use the same field in view groups (2741 ms)
✓ should handle deactivation when view has multiple view groups with different fields (3008 ms)
Test Suites: 1 passed, 1 total
Tests: 4 passed, 4 total
Snapshots: 0 total
Time: 12.664 s
```
```ts
PASS test/integration/metadata/suites/field-metadata/calendar-field-deactivation-deletes-views.integration-spec.ts (14.579 s)
calendar-field-deactivation-deletes-views
✓ should delete view when field used as calendarFieldMetadataId is deactivated (3388 ms)
✓ should not delete view when field not used as calendarFieldMetadataId is deactivated (2438 ms)
✓ should delete multiple views when they all use the same field as calendarFieldMetadataId (2635 ms)
✓ should handle deactivation when views have different calendar layouts on same field (3195 ms)
✓ should delete calendar view but not other view types when calendar field is deactivated (2682 ms)
Test Suites: 1 passed, 1 total
Tests: 5 passed, 5 total
Snapshots: 0 total
Time: 14.601 s, estimated 15 s
```
## View soft deletion
We decided to remove the soft deletion grain on all the views, in this
PR context we've only removed soft deleted validation requirement on any
view entities
## Conclusion
close https://github.com/twentyhq/core-team-issues/issues/1754
|
||
|
Weiko
|
cceeb6ed4d |
Add applicationId to syncableEntity and fix syncApp deletion (#15170)
## Context - All flatEntity should extend SyncableEntity - SyncableEntity should now have applicationId and application relation - Fix syncApp deletion, should now properly use migration v2 to delete syncable entities |
||
|
Paul Rastoin
|
3462a2e288 |
ViewGroup and ViewFilters side effect in v2 (#15096)
# Introduction ### Summary Implements side effect handling for `ViewGroup` and `ViewFilters` when field metadata is updated in the v2 architecture. This ensures that view-related records are properly maintained when enum field options are modified, deleted, or created. ### Side effects - **Side Effect System**: Added side effect handling for field metadata updates that manages related view groups and view filters - **Enum Field Updates**: When enum field options are modified, the system now: - **View Groups**: Creates new groups for added options, updates existing groups for modified options, and deletes groups for removed options - **View Filters**: Updates filter values to reflect option changes and removes filters that reference deleted options ### Enum runner fix Update now works for both atomic enum and array enum ( multi select for instance ) ### Compute flat entity maps from to Standardized this method usage across v2 services Next step is to require dependencies dynamically ## Conclusion closes https://github.com/twentyhq/core-team-issues/issues/1649 |
||
|
Weiko
|
c0ed246a03 |
Add is unique in migration v2 + refactor inferDeletionFromMissingEntities (#15070)
## Goal - inferDeletionFromMissingEntities is now a map instead of a single bool, allowing us to parameterise it based on the entity we want to compare - Adding index creation/update when field isUnique is set to true and index deletion when isUnique is false (for now) close https://github.com/twentyhq/core-team-issues/issues/1346 |
||
|
Paul Rastoin
|
7d747c9876 |
[REQUIRES_CACHE_FLUSH][GQL_VIEW_GROUP_API_BREAKING_CHANGE] ViewGroup in v2 (#15052)
# Introduction Adding view-group to core engine v2 Following https://github.com/twentyhq/twenty/pull/15010 ( same pattern ) ## What's done - Created flat-view-group - flat view group runner - flat view group builder - create view group service v2 and input transpilers - refactor the existing view group resolver to fix standard ( BREAKING_CHANGE on graphql api update especially ) REST stays the same - refactored the front to consume the mutations autogenerated close https://github.com/twentyhq/core-team-issues/issues/1665 |
||
|
Paul Rastoin
|
6188c72f74 |
Simplify and enhance v2 type devxp (#15032)
# Introduction This PR introduces a huge type refactor that will leverage dynamic intra entity optimistic flat maps update in the future and also a more granular cache invalidation enhancing performances close https://github.com/twentyhq/core-team-issues/issues/1717 close https://github.com/twentyhq/core-team-issues/issues/1716 close https://github.com/twentyhq/core-team-issues/issues/1643 ## What's done ### Comparators centralization Comparator is now done through global configuration as const for each metadata names Thanks to Note: Definition of standard is evolving, standard is now scoped to an app. Meaning that a manifest should be able to update its own standards objects but on other app standards ones ? Each synchronizable entities will have a standardOverrides ? ## Typing refactor ### `AllFlatEntityTypesByMetadataName` **Single source of truth for the complete type ecosystem**, mapping each metadata name to its entity types, flat entities, and migration actions: ```typescript export type AllFlatEntityTypesByMetadataName = { fieldMetadata: { actions: { created: CreateFieldAction; updated: UpdateFieldAction; deleted: DeleteFieldAction; }; flatEntity: FlatFieldMetadata; entity: FieldMetadataEntity; }; objectMetadata: { /* ... */ }; // ... all 10 metadata types }; ``` ### `ALL_METADATA_NAME_MANY_TO_ONE_RELATIONS` **Explicitly declares database relationships** between entities with compile-time validation: ```typescript export const ALL_METADATA_NAME_MANY_TO_ONE_RELATIONS = { viewField: { view: 'viewId', fieldMetadata: 'fieldMetadataId', }, cronTrigger: { serverlessFunction: 'serverlessFunctionId', }, // ... all relations } as const satisfies MetadataNameAndRelations; ``` ### `ALL_FLAT_ENTITY_CONFIGURATION` **Centralizes comparison and serialization logic** for each metadata type: ```typescript export const ALL_FLAT_ENTITY_CONFIGURATION = { fieldMetadata: { propertiesToCompare: ['name', 'type', 'label', 'defaultValue', /* ... */], propertiesToStringify: ['options', 'settings', 'defaultValue'], }, objectMetadata: { propertiesToCompare: ['nameSingular', 'namePlural', 'isActive', /* ... */], propertiesToStringify: [], }, // ... all metadata types } as const satisfies AllFlatEntityConfiguration; ``` ## Combined Impact These three configurations work together to create a **strongly-typed, centrally-managed metadata system**: 1. **`AllFlatEntityTypesByMetadataName`** defines *what exists* 2. **`ALL_METADATA_NAME_MANY_TO_ONE_RELATIONS`** defines *how they relate* 3. **`ALL_FLAT_ENTITY_CONFIGURATION`** defines *how to compare and serialize them* **Result:** Builders and validators become thin wrappers around type-safe, configuration-driven logic instead of containing scattered, error-prone manual implementations. ## What's next ### StandardOverrides standardization Every metadata entity can be a standard one for a workspace if it's an installed app, which means it might not expose the whole entity api to be editable through an import dynamically The standard overrides logic should not be applied to Fields and Objects but to every entities At the moment we have a logic of `EDITABLE_PROPERTIES` through the api, and also `STANDARD_OVERRIDEDABLE_PROPERTIES` This should be configuration centered like `propertiesToCompare` and `propertiesToStringify`. Scoping this PR to two last for the moment. As update dispatch to standardOverrides could be considered as a side effect prefer waiting to start the side effect refactor ### Granular Optimistic deprecation With this new grain at runtime we will be able to add a flat entity and dispatch its addition to related flat maps, so we don't have to describe an optimistic method for each flat entity operations See `addFlatEntityToFlatEntityAndRelatedEntityMapsOrThrow` Note: Still in wip and included in this PR but about to create a new one to integrate these utils and remove existing methods ### ValidateBuildAndRun dynamic args typed defintion We should restrain the devxp to send expected flat maps entity as at least from to or dependency as we now have the grain both a type lvl and runtime to do so It should not be possible in the devxp to forgot adding the views to the v2 builder when passing the view field anymore ( that would lead to permanent validation error in view field integrity checks ) ## Conclusion Thanks for reading and reviewing ! Any suggestions are more than welcomed ! ( same as for questions too ! ) |
||
|
Paul Rastoin
|
651ab184a7 |
[GQL_VIEW_FILTER_API_BREAKING_CHANGE][WHEN_RELEASED_REQUIRES_CACHE_FLUSH] ViewFilter migration to workspace migration v2 (#15010)
# Introduction Migrating `viewFilter` to v2 in order to migrate later the field update side effect on view to v2 too ## What's done - Created flat-view-filter - flat view filter runner - flat view filter builder - create view filter service v2 and input transpilers - refactor the existing view filter resolver to fix standard ( BREAKING_CHANGE on graphql api update especially ) REST stays the same - refactored the front to consume the mutations autogenerated ## New generic tools ### Compare two flat entity Introducing a new util to compare two flat entity, it's strictly typed and will be added to the generic builder in a following PR This will ease flat entity addition as won't required to create a specific abstraction for comparison Generic builder will expect specific constant: properties to compare and properties to stringify ### Transform flat entity for comparison Forked and refactor the initial existing method for flat entity business scope and type safety ## Coverage Migrated existing integration tests to fit new contract API This PR does not add strong coverage on validation exceptions Deadlines are too short close https://github.com/twentyhq/core-team-issues/issues/1666 |
||
|
Félix Malfait
|
e577c2d746 |
Update user friendly errors for translations (#15000)
Force msg typing instead of string for user friendly errors |
||
|
Paul Rastoin
|
4ecc9c622d |
[WHEN_RELEASED_REQUIRES_CACHE_FLUSH] Object related record logic in v2 (#14937)
# Introduction
Initial motivation here was to migrate the object related records logic
from v1 to v2, please note that now in v2 views aren't records anymore
but core engine entities
## What's done
- Added specific label identifier targeting view field logic
- Handled side effects on viewField creation with lowest position on
object label identifier mutation
- Added viewField relations in field metadate entity + handled
optimistic in builder v2
- Added view relations in object metadata entity + handled optimistic in
builder v2
- Added integration tests covering the side effects and new validation
exceptions
- Sandardized cache computation
- Coverage on object metadata creation side effect on views and view
fields
## Coverage
```ts
PASS test/integration/graphql/suites/view/view-field/object-identifier-update-side-effect-on-view-field.integration-spec.ts
View Field Resolver - Successful object metadata identifier update side effect on view field
✓ should create a view field on label identifier object metadata update if it does not exist on view (7 ms)
✓ Should not allow deleting a label identifier view field (17 ms)
✓ Should not allow destroying a label identifier view field (6 ms)
✓ Should not allow updating a label identifier view field visibility to false (8 ms)
✓ Should not allow creating a view field with a position lower than the label idenfitier view field (180 ms)
✓ Should not allow updated labelIdentifier view field with a position higher than existing other view field (346 ms)
✓ Should allow updated labelIdentifier view field with a position higher than existing other view field (434 ms)
Test Suites: 1 passed, 1 total
Tests: 7 passed, 7 total
Snapshots: 5 passed, 5 total
Time: 4.571 s, estimated 5 s
```
close https://github.com/twentyhq/core-team-issues/issues/1664
|
||
|
Guillim
|
f60817a1e1 |
Morph many to one picker (#14155)
This PR follows the multiSelect PR merged previously. It will enable morph relation Many to One to be handled from the table, using a singleSelect picker Main point : I decided to change the singleSelect API to take an array of **objectMetadataName** instead of only one to deal with both our usecases. --------- Co-authored-by: Charles Bochet <charles@twenty.com> |
||
|
Paul Rastoin
|
23e21cbeea |
Label identifier validation v2 (#14867)
# Introduction Adding a hacky way to validate object against fields before fields validation ( bi-directional validation process ) If you encounter an identical setup we will add a specific devXp as cleanup validation but for the moment this seems enough close https://github.com/twentyhq/core-team-issues/issues/1639 |
||
|
Weiko
|
0b60aa4249 |
Add custom routes to migration v2 (#14846)
## Context Add routes to migration V2 - Resolvers - Service v2 - Builder - Validator - Action runner Next PR: Add to twenty-cli to sync routes with serverless |
||
|
Paul Rastoin
|
170f6d27c5 |
Extract field metadata build out of object metadata build (#14763)
# Introduction Extracting the legacy fields build and dispatch out of the object one to follow the generic flat entity build, also update caches entries for object ## Main tasks - flat field map cache - flat field builder - refactored the dispatch matrix to return flat entity maps instead of flat entity arrays - orchestrator aggregator - removing legacy code - making universal identifier aka standardId of standard field for custom object deterministically dynamic - perfs debug logs for v2 ## TODO - [x] Refactor the generic entity builder to be dependency flat maps in order to main foreign keys list in flat parent - [x] Refactor the flat object metadata to contain the array of related fields and avoid costy find object fields - [ ] Improve the create field handler to handle multiple field at once - [ ] Refactor the dispatch to embbed the comparison - [ ] Improve perf by extracting from elements out of existing - [ ] Fix the labelIdentifierId validators on object before field creation ( integ tests are in failing mode ) ## Debug logs snippet ```ts [EntityBuilder fieldMetadata] matrix computation: 0.027ms [EntityBuilder fieldMetadata] creation validation: 0.001ms [EntityBuilder fieldMetadata] deletion validation: 0.293ms [EntityBuilder fieldMetadata] update validation: 0.006ms [EntityBuilder fieldMetadata] entity processing: 0.363ms [EntityBuilder fieldMetadata] validateAndBuild: 0.455ms [EntityBuilder index] matrix computation: 0.005ms [EntityBuilder index] creation validation: 0.001ms [EntityBuilder index] deletion validation: 0.146ms [EntityBuilder index] update validation: 0.004ms [EntityBuilder index] entity processing: 0.199ms [EntityBuilder index] validateAndBuild: 0.228ms [Runner] Initial cache retrieval: 0.549ms [BaseWorkspaceMigrationRunnerActionHandlerService] delete_index executeForWorkspaceSchema: 11.665ms [BaseWorkspaceMigrationRunnerActionHandlerService] delete_index executeForMetadata: 12.864ms [BaseWorkspaceMigrationRunnerActionHandlerService] delete_field executeForWorkspaceSchema: 1.476ms [BaseWorkspaceMigrationRunnerActionHandlerService] delete_field executeForMetadata: 6.816ms [BaseWorkspaceMigrationRunnerActionHandlerService] delete_field executeForWorkspaceSchema: 0.062ms [BaseWorkspaceMigrationRunnerActionHandlerService] delete_field executeForMetadata: 0.889ms [Runner] Transaction execution: 23.434ms [Runner] Cache invalidation: 316.662ms [Runner] Total execution: 340.767ms ``` As you can see cache invalidation is way to long, we could replace the cache by the optimistic in the end |
||
|
Weiko
|
68b21d2942 |
Add db event trigger and cron trigger to migration v2 (#14772)
## Context Add DB events triggers and Cron triggers to migration v2 builder/runner (and adding corresponding services/resolvers) |
||
|
Paul Rastoin
|
4fbdfb6abc |
Activate v2 default seed (#14660)
## Introduction After enabling flag by default got following errors: ```ts Test Suites: 48 failed, 1 skipped, 97 passed, 145 of 146 total Tests: 499 failed, 1 skipped, 644 passed, 1144 total Snapshots: 61 failed, 133 passed, 194 total Time: 363.226 s Ran all test suites. ``` ## From <img width="2952" height="1510" alt="image" src="https://github.com/user-attachments/assets/7e3b20c6-2552-40a7-90bb-2d7b3002c895" /> ## To <img width="3134" height="1510" alt="image" src="https://github.com/user-attachments/assets/4fc9ada4-3c14-4333-a1db-11daf87db8d6" /> There's a huge test bundle in the latest shard that we could split up ## Notes - Set as failing morph relation field rename as for the moment we do not handle relation field mutation - fixed the object update and creation validation adding label identifier field metadata id checks - and more Some integrations tests are still on the v1 ( they have before and after all disabling and re-enabling the flat ) but mainly we now have more coverage on the v2 than the v1. Mainly related records, uniqueness have to be migrated the v2 and so tests too |
||
|
Weiko
|
188d66909d | Add ServerlessFunction to migration v2 (#14698) |