fbbd8fe967183babdafd1b50da697429f67804a6
31 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
|
|
7d69000ab7 |
Update pageLayout* data models for backend recordPageLayout refactor (#17446)
<img width="814" height="356" alt="Screenshot 2026-01-26 at 16 09 27" src="https://github.com/user-attachments/assets/91d95a1d-cc33-4bf0-a63e-4d1815030983" /> |
||
|
|
da6f1bbef3 |
Rename serverlessFunction to logicFunction (#17494)
## Summary Rename "Serverless Function" to "Logic Function" across the codebase for clearer naming. ### Environment Variable Changes | Old | New | |-----|-----| | `SERVERLESS_TYPE` | `LOGIC_FUNCTION_TYPE` | | `SERVERLESS_LAMBDA_REGION` | `LOGIC_FUNCTION_LAMBDA_REGION` | | `SERVERLESS_LAMBDA_ROLE` | `LOGIC_FUNCTION_LAMBDA_ROLE` | | `SERVERLESS_LAMBDA_SUBHOSTING_URL` | `LOGIC_FUNCTION_LAMBDA_SUBHOSTING_URL` | | `SERVERLESS_LAMBDA_ACCESS_KEY_ID` | `LOGIC_FUNCTION_LAMBDA_ACCESS_KEY_ID` | | `SERVERLESS_LAMBDA_SECRET_ACCESS_KEY` | `LOGIC_FUNCTION_LAMBDA_SECRET_ACCESS_KEY` | ### Breaking Changes - Environment variables must be updated in production deployments - Database migration renames `serverlessFunction` → `logicFunction` tables |
||
|
|
a4499d21bb |
Migrate cron, databaseEventTrigger, httpRoute triggers to serverless functions (#17488)
## Summary Migrates trigger entities (`CronTriggerEntity`, `DatabaseEventTriggerEntity`, `RouteTriggerEntity`) into `ServerlessFunctionEntity` by storing trigger settings as JSONB columns directly on the serverless function. This simplifies the architecture since these relationships were effectively one-to-one. ## Changes ### Schema Changes - Added three new nullable JSONB columns to `ServerlessFunctionEntity`: - `cronTriggerSettings` - stores cron pattern - `databaseEventTriggerSettings` - stores event name and updated fields filter - `httpRouteTriggerSettings` - stores path, HTTP method, auth requirements, and forwarded headers ### Core Logic Updates - `CronTriggerCronJob` - now queries `ServerlessFunctionEntity` directly instead of `CronTriggerEntity` - `CallDatabaseEventTriggerJobsJob` - now queries `ServerlessFunctionEntity` directly - `RouteTriggerService` - now queries `ServerlessFunctionEntity` directly - `ApplicationSyncService` - extracts trigger settings from manifest and writes to serverless function |
||
|
|
bc7791871f |
Introduce webhook v2 (#17456)
Migrate webhook to v2 entity |
||
|
|
2a8f834377 |
Integrate NavigationMenuItem with feature flag support (#17268)
## Implement Navigation Menu Items Frontend Implements the frontend for navigation menu items, the new system replacing favorites. ### Changes - Added GraphQL fragments and queries for navigation menu items - Added hooks for managing navigation menu items (create, update, delete, sorting, filtering) - Updated components to use navigation menu items instead of favorites - Added test coverage for utility functions ### Migration Note The favorites and navigation menu item modules currently exist in parallel. The favorites code will be removed once all data has been migrated to navigation menu items. <!-- CURSOR_SUMMARY --> --- > [!NOTE] > Replaces Favorites with feature-flagged `NavigationMenuItem` across frontend and backend, while keeping Favorites as fallback until migration completes. > > - UI: new `navigation-menu-item` components (folders, orphan items, drag provider/droppable, icons, skeleton), dispatcher components to switch from Favorites, and updated “Add to favorites” action to create `NavigationMenuItem` when `IS_NAVIGATION_MENU_ITEM_ENABLED` > - DnD: shared `validateAndExtractFolderId` and droppable id utils moved to `ui/layout/draggable-list`; favorites DnD updated to use shared utils > - GraphQL (client): add fragments, queries, mutations, hooks (create/update/delete/find), and generated types; added `RecordIdentifier` and `targetRecordIdentifier` on `NavigationMenuItem` > - Prefetch: new prefetch state/effect for navigation menu items; skip favorites prefetch when flag enabled > - Backend: add DTOs (`NavigationMenuItem`, `RecordIdentifier`), resolver `targetRecordIdentifier` field, service logic to fetch record identifiers with permission-aware access and image signing, `getRecordImageIdentifier` util, entity relation to `view`, and migration adding FK on `viewId` > - Feature flags & seeding: add `IS_NAVIGATION_MENU_ITEM_ENABLED` to enums, dev seeder enables it; standard app seeds workspace navigation menu items instead of favorites when flag on > - Tests: add unit tests for sorting/labels/folder id and related utils > > <sup>Written by [Cursor Bugbot](https://cursor.com/dashboard?tab=bugbot) for commit c99746f08b9f84fc8cec4fcc3a7d7afb8ea92db7. This will update automatically on new commits. Configure [here](https://cursor.com/dashboard?tab=bugbot).</sup> <!-- /CURSOR_SUMMARY --> --------- Co-authored-by: Félix Malfait <felix.malfait@gmail.com> Co-authored-by: Aman Raj <92664006+araj00@users.noreply.github.com> Co-authored-by: Félix Malfait <felix@twenty.com> Co-authored-by: Paul Rastoin <45004772+prastoin@users.noreply.github.com> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: github-actions <github-actions@twenty.com> |
||
|
|
d0bc9a94c0 |
[OBJECT_CACHE_FLUSH_REQUIRED_WHEN_RELEASED] Remove FlatObjectMetadata custom fieldMetadataIds fk aggregator property (#17438)
# Introduction Currently refactoring `flatEntity` typing, encountering some tsc errors due to this fk aggregator custom override It shall now follow generic pattern leading to be named `fieldIds` Needs to flush object cache when released |
||
|
|
4c93ab5259 |
Introduce UniversalFlatEntityFrom (#17367)
# Introduction
Creating a `UniversalFlatEntityFrom` that strips out all the relation
and foreignKey properties in order to replace them with
`UniversalIdentifier` suffix
This data type will be major for the workspace migration workspace
agnostic refactor
## Chore
- renamed `flat-entity.type` to `flat-entity-from.type.ts` ( more
accurate to exported module )
- create static test type over the field metadata entity on quite
complex utils as both coverage and documentation
## Example
Here's an example of a `UniversalFlatEntityFrom<FieldMetadataEntity>`
```ts
const universalFlatFieldMetadata: UniversalFlatFieldMetadata<FieldMetadataType.RELATION> = {
// Base properties (from FieldMetadataEntity, excluding relations and applicationId)
universalIdentifier: '550e8400-e29b-41d4-a716-446655440001',
applicationUniversalIdentifier: '5800681c-088e-4e2b-9fc3-bcf6e8ec2051',
type: FieldMetadataType.RELATION,
name: 'firstName',
label: 'First Name',
defaultValue: null,
description: 'The first name of the person',
icon: 'IconUser',
standardOverrides: null,
options: null,
settings: {
relationType: RelationType.ONE_TO_MANY,
},
isCustom: false,
isActive: true,
isSystem: false,
isUIReadOnly: false,
isNullable: true,
isUnique: false,
isLabelSyncedWithName: true,
morphId: null,
// Date properties cast to string
createdAt: '2024-01-15T10:30:00.000Z',
updatedAt: '2024-01-15T10:30:00.000Z',
// ManyToOne relation universal identifiers (from FieldMetadataEntity relations)
relationTargetFieldMetadataUniversalIdentifier:
'550e8400-e29b-41d4-a716-446655440012',
relationTargetObjectMetadataUniversalIdentifier:
'550e8400-e29b-41d4-a716-446655440013',
// Join column universal identifiers (foreignKey -> universalIdentifier)
objectMetadataUniversalIdentifier: '550e8400-e29b-41d4-a716-446655440010',
// OneToMany relation universal identifiers (array of related entity identifiers)
viewFieldUniversalIdentifiers: [
'550e8400-e29b-41d4-a716-446655440020',
'550e8400-e29b-41d4-a716-446655440021',
],
viewFilterUniversalIdentifiers: ['550e8400-e29b-41d4-a716-446655440030'],
kanbanAggregateOperationViewUniversalIdentifiers: [],
calendarViewUniversalIdentifiers: [],
mainGroupByFieldMetadataViewUniversalIdentifiers: [],
};
```
## Settings
Will hop on the settings typing next. Might not be dynamic but
declarative though
|
||
|
|
aff577689f |
Add syncable NavigationMenuItem entity to core schema (#17232)
Implements a new syncable `navigationMenuItem` entity in the core schema to replace the workspace `favorite` entity. ## Next Steps - Frontend integration ([separate PR](https://github.com/twentyhq/twenty/pull/17268)) - Data migration (separate PR) |
||
|
|
fae6d0e262 |
Improve cleaning job (#17208)
# Introduction Refactored the workspace deletion to dynamically iterate over all known v2 syncable entities repos and delete all of them from child to parent Exception for field metadata that we chunk delete in order to avoid locking the core schema too long, it does not have an impact on perfs at all ( neither plus or less ) Chunking by constraint within a transaction is not necessary both does not cost more ## From 30s for a workspace complete deletion ```ts [Nest] 93244 - 01/16/2026, 10:24:52 PM LOG [WorkspaceService] workspace WS_ID cache flushed [Runner] Total execution: 26.290s // ( deleteAllObjectMetadatas v2 ) [Nest] 93244 - 01/16/2026, 10:25:22 PM LOG [WorkspaceService] workspace WS_ID hard deleted ``` ## To 3s ! ```ts [Nest] 65112 - 01/18/2026, 4:37:38 PM LOG [DatabaseConfigDriver] [INIT] Config variables loaded: 0 values found in DB, 69 falling to env vars/defaults [Nest] 65112 - 01/18/2026, 4:37:38 PM LOG [CleanSuspendedWorkspacesCommand] IGNORING GRACE PERIOD - Cleaning 1 suspended workspaces [Nest] 65112 - 01/18/2026, 4:37:38 PM LOG [CleanerWorkspaceService] batchWarnOrCleanSuspendedWorkspaces running... [Nest] 65112 - 01/18/2026, 4:37:38 PM LOG [CleanerWorkspaceService] Processing workspace - 1/1 [Nest] 65112 - 01/18/2026, 4:37:38 PM LOG [CleanerWorkspaceService] Destroying workspace Twenty Eng [Nest] 65112 - 01/18/2026, 4:37:38 PM LOG [WorkspaceService] workspace user workspaces deleted [Nest] 65112 - 01/18/2026, 4:37:38 PM LOG [WorkspaceService] workspace cache flushed [Nest] 65112 - 01/18/2026, 4:37:38 PM LOG [WorkspaceService] workspace: deleted 80 viewFilter record(s) [Nest] 65112 - 01/18/2026, 4:37:38 PM LOG [WorkspaceService] workspace: deleted 21 pageLayoutWidget record(s) [Nest] 65112 - 01/18/2026, 4:37:38 PM LOG [WorkspaceService] workspace: deleted 1515 viewField record(s) [Nest] 65112 - 01/18/2026, 4:37:38 PM LOG [WorkspaceService] workspace: deleted 91 index record(s) [Nest] 65112 - 01/18/2026, 4:37:38 PM LOG [WorkspaceService] workspace: deleted 66 roleTarget record(s) [Nest] 65112 - 01/18/2026, 4:37:38 PM LOG [WorkspaceService] workspace: deleted 174 viewGroup record(s) [Nest] 65112 - 01/18/2026, 4:37:38 PM LOG [WorkspaceService] workspace: deleted 1 agent record(s) [Nest] 65112 - 01/18/2026, 4:37:38 PM LOG [WorkspaceService] workspace: deleted 7 pageLayout record(s) [Nest] 65112 - 01/18/2026, 4:37:38 PM LOG [WorkspaceService] workspace: deleted 111 view record(s) [Nest] 65112 - 01/18/2026, 4:37:38 PM LOG [WorkspaceService] workspace: fieldMetadata chunk 1/15 - deleted 50 record(s) [Nest] 65112 - 01/18/2026, 4:37:38 PM LOG [WorkspaceService] workspace: fieldMetadata chunk 2/15 - deleted 50 record(s) [Nest] 65112 - 01/18/2026, 4:37:38 PM LOG [WorkspaceService] workspace: fieldMetadata chunk 3/15 - deleted 50 record(s) [Nest] 65112 - 01/18/2026, 4:37:38 PM LOG [WorkspaceService] workspace: fieldMetadata chunk 4/15 - deleted 50 record(s) [Nest] 65112 - 01/18/2026, 4:37:38 PM LOG [WorkspaceService] workspace: fieldMetadata chunk 5/15 - deleted 50 record(s) [Nest] 65112 - 01/18/2026, 4:37:38 PM LOG [WorkspaceService] workspace: fieldMetadata chunk 6/15 - deleted 50 record(s) [Nest] 65112 - 01/18/2026, 4:37:38 PM LOG [WorkspaceService] workspace: fieldMetadata chunk 7/15 - deleted 50 record(s) [Nest] 65112 - 01/18/2026, 4:37:38 PM LOG [WorkspaceService] workspace: fieldMetadata chunk 8/15 - deleted 50 record(s) [Nest] 65112 - 01/18/2026, 4:37:38 PM LOG [WorkspaceService] workspace: fieldMetadata chunk 9/15 - deleted 50 record(s) [Nest] 65112 - 01/18/2026, 4:37:38 PM LOG [WorkspaceService] workspace: fieldMetadata chunk 10/15 - deleted 50 record(s) [Nest] 65112 - 01/18/2026, 4:37:38 PM LOG [WorkspaceService] workspace: fieldMetadata chunk 11/15 - deleted 50 record(s) [Nest] 65112 - 01/18/2026, 4:37:38 PM LOG [WorkspaceService] workspace: fieldMetadata chunk 12/15 - deleted 51 record(s) [Nest] 65112 - 01/18/2026, 4:37:38 PM LOG [WorkspaceService] workspace: fieldMetadata chunk 13/15 - deleted 50 record(s) [Nest] 65112 - 01/18/2026, 4:37:38 PM LOG [WorkspaceService] workspace: fieldMetadata chunk 14/15 - deleted 50 record(s) [Nest] 65112 - 01/18/2026, 4:37:38 PM LOG [WorkspaceService] workspace: fieldMetadata chunk 15/15 - deleted 36 record(s) [Nest] 65112 - 01/18/2026, 4:37:38 PM LOG [WorkspaceService] workspace: deleted 737 fieldMetadata record(s) [Nest] 65112 - 01/18/2026, 4:37:38 PM LOG [WorkspaceService] workspace: deleted 6 role record(s) [Nest] 65112 - 01/18/2026, 4:37:38 PM LOG [WorkspaceService] workspace: deleted 78 serverlessFunction record(s) [Nest] 65112 - 01/18/2026, 4:37:39 PM LOG [WorkspaceService] workspace: deleted 43 objectMetadata record(s) [Nest] 65112 - 01/18/2026, 4:37:41 PM LOG [WorkspaceService] workspace hard deleted [Nest] 65112 - 01/18/2026, 4:37:41 PM LOG [CleanerWorkspaceService] Destroyed 1 workspaces on 5 limit durings this execution [Nest] 65112 - 01/18/2026, 4:37:41 PM LOG [CleanerWorkspaceService] batchWarnOrCleanSuspendedWorkspaces done! [Nest] 65112 - 01/18/2026, 4:37:41 PM LOG [CleanSuspendedWorkspacesCommand] Command completed! ``` ## Update Discussed with @charlesBochet ended debugging and analyzing sql query operations He discovered that we were not indexing foreignKey effectively We've ended up fixing all the FK indeces coverage leading to ## Cleaning Removed the ```sh npx nx run twenty-server:command workspace:clean-soft-deleted-suspended-workspaces --ignore-grace-period ``` In favor of ```sh npx nx run twenty-server:command workspace:clean --only-operation destroy --ignore-destroy-grace-period ``` ## Conclusion Not that crazy but still worth it and could demultiply in production |
||
|
|
5a617e4718 | Add command menu item entity (#17181) | ||
|
|
5fc4e810f7 |
Front Extensibility: Introduce Front Component Entity (#17175)
As part of the extensibility effort, we are introducing a new engine entity called "Front Component". This represents a dynamic react component that will be rendered in CommandMenu actions or in PageLayout widgets This PR introduce the entity and all the necessary boilerplate to make it syncable and cachable in the engine |
||
|
|
8413c6f3dd |
Fix insert new record with RLS (#17164)
## Context Now that RLS predicates are applied, creating a record through the FE (which is empty by default) is failing if your role has predicates and your input does not respect them (which will always be true since, as said above, input will be pretty much empty) ## Implementation - Moved isMatching* filters to twenty-shared - Implemented isMatchingRlsPredicates utils in the backend (ORM) to check before insertion/update if the record is matching the current user role Rls predicates, reusing the isMatching* filters utils moved to twenty-shared - Frontend now applies RLS predicates before creating a new record (similarly to what we do with view filters) Note: It seems composite were not properly handled with view-filter insertion logic, since I'm reusing the util for now, the issue remains for RLS and will need to be addressed |
||
|
|
2c8d3f02e1 |
feat: upgrade to Storybook version 10 (#17110)
Upgraded to Storybook 10. We still use `@storybook/test-runner` for testing since it appears it'd require more work to move from Jest to Vitest than I initially anticipated, but I completed this PR to fix `storybook:serve:dev` - it takes time to load, but it works the way it used to with Storybook 8. https://github.com/user-attachments/assets/7afc32c6-4bcf-4b37-b83b-8d00d28dda15 |
||
|
|
c795b8c52a |
RLS FE implementation (#17062)
## Summary This PR introduces row-level security (RLS) permissions for roles in the frontend, allowing fine-grained access control at the record level. Users can now define permission rules that determine which specific records a role can access based on dynamic conditions and filters. ## What's Changed Implemented UI for configuring record-level permissions on object permissions screens Added support for defining permission predicates using filter conditions (similar to advanced filters) Introduced variable picker for dynamic permission rules (e.g., "me" context for user-specific access) Built predicate conversion layer to sync UI state with backend permission structure Extended GraphQL schema with mutations for upserting row-level permission predicates Fixed handling of orphaned RLS groups to prevent data inconsistencies Added enterprise key validation for RLS features The implementation enables scenarios like "users can only see their own records" or "users can access records associated with their team." <img width="687" height="702" alt="Screenshot 2026-01-09 at 23 07 02" src="https://github.com/user-attachments/assets/33fe736e-6cbf-40bd-b2eb-c8a90c8d21bc" /> --------- Co-authored-by: Charles Bochet <charles@twenty.com> |
||
|
|
46e5420d20 |
dashboard workspace standard seeds (#16962)
# Introduction In this pull request we're introducing new standard page layout, tabs and widget ( 1 page layout, 1 tab and 8 widgets ) and also a new opportunity field Also now prefilling new records, 6 opportunities and a dashboard. ## Standard declaration ### New workspace creation Relies on existing standard declaration builder ### Backfill command We've been hacking through the standard builder in order to extract only the standard page layout entities, updated their entity dependencies to match the workspace ids so the validation passes ## Remark - Refactored the `PageLayoutWidget` configuration type to be dynamically typed through a generic discriminated union --------- Co-authored-by: prastoin <paul@twenty.com> |
||
|
|
942d2fef83 |
Remove sync-metadata and IS_WORKSPACE_CREATION_V2_ENABLED feature flag (#16997)
# Introduction Followup of https://github.com/twentyhq/twenty/pull/17001#pullrequestreview-3638508738 close https://github.com/twentyhq/core-team-issues/issues/1910 We've completely decom the `sync-metadata` in production. We're now then removing its implementation in favor of the v2. ## TODO: - [x] Remove sync-metadata implem and commands - [x] Remove workspace decorators - [x] Type each deprecated field to deprecated on their workspaceEntity - [x] Remove the `workspace-sync-metadata` folder entirely - [x] remove workspace migration - [x] workspace migration removal migration - [x] remove the `v2` references from workspace manager file names - [x] remove the `v2` references from workspace manager modules - [ ] Double check impact on translation file path updates ## Note - Removed the gate logic - Remains some service v2 naming, serverless needs to be migrated on v2 fully - Removed workspaceMigration service app health consumption, making it always returning up ( no more down ) cc @FelixMalfait ( quite obsolete health check now, will require complete refactor once we introduce inter app dependency etc ) |
||
|
|
9b38254256 |
Refactor runner dynamic cache invalidation (#16913)
# Introduction closes https://github.com/twentyhq/core-team-issues/issues/1792 Refactoring the cache to invalidate to be more precise and prevent any corrupted cache occurrences ## Next The load dependency cache from the workspace migration, that's not optimal it should be smart enough to infer it from the actions themselves. For the moment their definition isn't granular enough and inferring such info would be very dirty |
||
|
|
42c9ae1ebc |
Centralize metadata relations constant + simplification (#16901)
# Introduction As we introduced a new grain on relation extraction thanks to low level `SyncableEntity` and `WorkspaceRelatedEntity` we're able to strictly typesafe extract metadata entity The new constant centralizes both many to one and one to many constants metadata entity constants in a more strictly typesafe way. Remains only the flatEntityForeignKey aggregator which has to be chosen manually across all available targeted flat entity ids properties |
||
|
|
38785cd4e9 |
Refactor seed to use twenty-standard application (#16598)
# Introduction In this pull-request we introduce a service dedicated to the twenty-standard app installation, we will later be able to re-use existing logic to be more generic and allow any app installation. For the moment sticking to this usage https://github.com/twentyhq/core-team-issues/issues/1995 ## Encountered issues - We decided not to migrate deprecated fields ( also they will become custom field for any existing workspace having them in the future ) - duplicate criteria - wrong search index declaration - forgotten isSearchable - Attachement seed - Restored standardId ## Note For the moment we're still searching through standardId for code that run on both existing and new workspaces. For code running on new workspace exclusively we're searching using universalIdentifier We will standardize universalIdentifier usage later when we've migratred all the existing workspaces ## Workspace creation Will handle workspace creation the same way in another PR Related https://github.com/twentyhq/twenty/pull/15065 ## TODO - [ ] Double all frontend hardcoded queries to not refer to deprecated fields especially attachments |
||
|
|
a18203934c |
Fix flat entity maps date serialization (#16420)
Changes: - as we store date in redis as serialized, let's make all flatEntity dates as string. This requires changing FlatEntity types and making sure that entity are converted to flatEntity and flatEntity to dtos |
||
|
|
7f1e69740a |
1895 extensibility v1 application tokens (#16365)
First PR to implement application tokens - add new application role in twenty-server - move duplicated constants and types to twenty-shared - will add role configuration utils into twenty-sdk in another PR |
||
|
|
c8f541618d |
Refactor validate build and run for configuration to be less verbose and more reliable (#16343)
# Introduction Refactored the api `validateBuildAndRunWorkspaceMigration` to be require less configuration but to infer required args dynamically depending on provided metadata maps to compare ## `inferDeletionFromMissingEntities` Is not dynamically computed avoiding any miss configuration issue and any missleading devxp ## Maps computation Making only one call to redis to build both dependency and to be compared entity maps. It does not matter to avoid passing a about to compared flat entity maps to could also be a depedency, it's handled directly in the builder setup optimistic cache logic Please note that the flat maps used for the service input transpilation might differ from the one that we will dynamically compute and inject in the builder. Leading to do 2 redis calls but also race condition prone validation error We prefer that this occurs at the builder rather than at the runner level as the pg instance is not cache and reflect the real state of a given workspace In a nutshell, there's a possible race condition between cache invalidation and computation in both service input transpilers and builder but we're totally ok with that |
||
|
|
7ce22d5c7e |
breaking (soft) - Migrate viewGroup.fieldMetadataId -> view.mainGroupByFieldMetadataId (2/3) (#16277)
Should be merged once https://github.com/twentyhq/twenty/pull/16206 has been released + command run to prod In this PR - Remove usage of viewGroup.fieldMetadataId, both in BE and FE states. - But we still need to properly populate it until we fully remove viewGroup.fieldMetadataId from db and ORM entity (upcoming 3rd PR out of 3). fieldMetadataId was removed from CoreViewGroup type and CreateViewGroupInput and is determined BE-side based on the associated view's mainGroupByFieldMetadataId. **I expect this means a downtime on viewGroup creation, until both FE and BE are deployed and cache is flushed.** This seems acceptable to me as it only regards viewGroup creation. - this information is replaced by view.mainGroupByFieldMetadataID - Handle view group creation, update and deletion in the BE as a side-effect of a view creation, update or deletion. Optimistic effects are still used - Add validation at view creation or update regarding mainGroupByFieldMetadata Left to do in 3rd PR - Remove viewGroup.fieldMetadataId from db and ORM entity - Restore feature allowing to update an existing grouped view's group by field (already OK on BE side but need to rebuild FE optimistic) |
||
|
|
0be228fc85 |
Attest standard object isActive update regression + TDD tests (#15976)
# Introduction Related https://github.com/twentyhq/twenty/issues/15846 The root cause is that universalIdentifier is still optional in database and fallbacked when extracted out of database to standardId. But all `BaseWorkspaceEntity` and `CustomWorkspaceEntity` share the same standardId for their default standard fields `createdAt` `deletedAt` resulting in such compare result in dispatcher ```ts { "initialDispatcher": { "createdFlatEntityMaps": { "byId": {}, "idByUniversalIdentifier": {}, "universalIdentifiersByApplicationId": {} }, "deletedFlatEntityMaps": { "byId": {}, "idByUniversalIdentifier": {}, "universalIdentifiersByApplicationId": {} }, "updatedFlatEntityMaps": { "byId": { "55e1568c-eb87-4b8a-9f1b-19bbf6042f3e": { "updates": [ { "from": "Deletion date", "to": "Date when the record was deleted", "property": "description" }, { "from": "IconCalendarClock", "to": "IconCalendarMinus", "property": "icon" }, { "from": false, "to": true, "property": "isLabelSyncedWithName" }, { "from": null, "to": { "displayFormat": "RELATIVE" }, "property": "settings" } ] } } } }, "fromFlatEntity": { "universalIdentifier": "20202020-b9a7-48d8-8387-b9a3090a50ec", "applicationId": null, "id": "9c97c8bf-1f64-463c-915c-f68f41d3cd60", "standardId": "20202020-b9a7-48d8-8387-b9a3090a50ec", "objectMetadataId": "e9565126-8351-457b-b003-3ea4c6d253bc", "type": "DATE_TIME", "name": "deletedAt", "label": "Deleted at", "defaultValue": null, "description": "Deletion date", "icon": "IconCalendarClock", "standardOverrides": null, "options": null, "settings": null, "isCustom": false, "isActive": true, "isSystem": false, "isUIReadOnly": true, "isNullable": true, "isUnique": false, "workspaceId": "20202020-1c25-4d02-bf25-6aeccf7ea419", "isLabelSyncedWithName": false, "relationTargetFieldMetadataId": null, "relationTargetObjectMetadataId": null, "morphId": null, "createdAt": "2025-11-20T17:28:45.474Z", "updatedAt": "2025-11-20T17:28:45.474Z", "kanbanAggregateOperationViewIds": [], "calendarViewIds": [], "viewGroupIds": [], "viewFieldIds": [], "viewFilterIds": [] }, "toFlatEntity": { "universalIdentifier": "20202020-b9a7-48d8-8387-b9a3090a50ec", "applicationId": null, "id": "55e1568c-eb87-4b8a-9f1b-19bbf6042f3e", "standardId": "20202020-b9a7-48d8-8387-b9a3090a50ec", "objectMetadataId": "37263f48-6858-4d28-a6e1-5f7321e49c24", "type": "DATE_TIME", "name": "deletedAt", "label": "Deleted at", "defaultValue": null, "description": "Date when the record was deleted", "icon": "IconCalendarMinus", "standardOverrides": null, "options": null, "settings": { "displayFormat": "RELATIVE" }, "isCustom": false, "isActive": true, "isSystem": false, "isUIReadOnly": true, "isNullable": true, "isUnique": false, "workspaceId": "20202020-1c25-4d02-bf25-6aeccf7ea419", "isLabelSyncedWithName": true, "relationTargetFieldMetadataId": null, "relationTargetObjectMetadataId": null, "morphId": null, "createdAt": "2025-11-20T17:28:44.267Z", "updatedAt": "2025-11-21T17:17:55.057Z", "kanbanAggregateOperationViewIds": [], "calendarViewIds": [], "viewGroupIds": [], "viewFieldIds": [], "viewFilterIds": [] } } ``` ## Impact - This might be corrupting label and description of an other standard field of an other object - Race condition on latest universalIdentifier assigned in cache making the update sometime accurate sometimes not ## Fix Will be fixed by the in coming work on applicationId and universalIdentifier as required in database + upgrade command that will handle retro-comp. ( won't handle description corruption though, should be anecdotical ) https://github.com/twentyhq/twenty/pull/15911 ( handling this only for new workspace, retro comp upgrade command will be coming just after ) ## PR scope - Introduce TDD integration tests as failing - Added unit test to critical methods that might have been involved in the root cause ( still worth it to keep ) --------- Co-authored-by: guillim <guigloo@msn.com> |
||
|
|
3514054235 |
V2 centralize relation optimistic logic (#15552)
# Introduction
This PR aims to deprecate having to manually handle optimistic side
effect foreign key addition in the whole v2 experience.
This PR implements the strong basis + builder refactor of the optimistic
computation of a given flat entity maps with its related flat entity
maps ( runner needs a small refactor on actions type definition first )
Flat entity maps updates through mutations are now only scoped to the
generic entity builder ( very isolated )
## What's next
- Refactor actions v2 type definition to gain grain over `metadataName`
and action operation ( `create` `delete` `update` ).
from `{type: 'create_view_field'}` to `{metadataName: 'view_field',
type: 'create' }`
- Use new optimistic tool computation tools
- Only invalidate impacted flat maps cache
## New tools
Strictly dynamically typed new flat entity maps tools
- `addFlatEntityToFlatEntityAndRelatedEntityMapsThroughMutationOrThrow`
-
`deleteFlatEntityFromFlatEntityAndRelatedEntityMapsThroughMutationOrThrow`
## Unit test
Adding basic unit testing coverage to introduced tools
## `FlatEntityValidationArgs`
From
```ts
export type FlatEntityValidationArgs<T extends AllMetadataName> = {
flatEntityToValidate: MetadataFlatEntity<T>;
optimisticFlatEntityMaps: MetadataFlatEntityMaps<T>;
mutableDependencyOptimisticFlatEntityMaps: MetadataValidationRelatedFlatEntityMaps<T>;
workspaceId: string;
remainingFlatEntityMapsToValidate: MetadataFlatEntityMaps<T>;
buildOptions: WorkspaceMigrationBuilderOptions;
};
```
To
```ts
export type FlatEntityValidationArgs<T extends AllMetadataName> = {
flatEntityToValidate: MetadataFlatEntity<T>;
optimisticFlatEntityMapsAndRelatedFlatEntityMaps: MetadataFlatEntityAndRelatedFlatEntityMapsForValidation<T>;
workspaceId: string;
remainingFlatEntityMapsToValidate: MetadataFlatEntityMaps<T>;
buildOptions: WorkspaceMigrationBuilderOptions;
};
```
|
||
|
|
d640b93096 |
Improve v2 and cache invalidation perfs (#15467)
# Introduction Log are debug logs of `packages/twenty-server/test/integration/metadata/suites/object-metadata/create-delete-and-create-object-metadata-v2.integration-spec.ts`run ten times in a row on clean db reset ## Next Will improve cache computation to lighter invalidation. RelationLoad `query` does not seem to work with typeorm so I'll continue the custom integration i've started in https://github.com/twentyhq/twenty/tree/optimize-cache-read-v2 ## Integration tests duration Significant test duration improvement too ### Before <img width="2632" height="1402" alt="image" src="https://github.com/user-attachments/assets/2f1f0ccf-44de-4856-bfe1-4f45a351763a" /> ### After <img width="2632" height="1402" alt="image" src="https://github.com/user-attachments/assets/4bc9e6db-3046-48b9-b903-1464053936c4" /> ## What's next - The legacy cache invalidation removal - Factorizing redis calls in only one operation ## Autogenerated performance comparison ( including mutation refactor too ) [Before](https://gist.github.com/prastoin/3c1e21fa9e3b3ce4b0716902ff4a2dd6) [After](https://gist.github.com/prastoin/7bfddd14bfded2e4991a9378970a026d) The optimized implementation shows **dramatic performance improvements** across all metrics: - 🚀 **Cache Invalidation**: 156.3ms → 76.8ms (**50.9% faster**) - 🚀 **Builder Operations**: 21.3ms → 14.2ms (**33.3% faster**) - ⚡ **Consistency**: 16.4% more predictable performance --- ## 1. Overall Performance Summary | Component | Before (avg) | After (avg) | Best (After) | Worst (After) | Improvement | |-----------|-------------|-------------|--------------|---------------|-------------| | **Total Execution Time** | 180.2ms | 110.5ms | 52.3ms | 585.9ms | **38.7% faster** ⚡⚡ | | **Cache Invalidation** | 156.3ms | 76.8ms | 47.8ms | 285.1ms | **50.9% faster** ⚡⚡⚡ | | **Transaction Execution** | 22.4ms | 22.1ms | 0.99ms | 314.2ms | Similar | | **Initial Cache Retrieval** | 0.81ms | 2.08ms | 0.21ms | 8.24ms | Similar | | **Entity Builder (total)** | 21.3ms | 14.2ms | 0.36ms | 42.5ms | **33.3% faster** ⚡ | ### Total Execution Time Distribution #### Before (Legacy Sequential) ``` Time (ms) Count Percentage Visualization < 150 18 16% ████ 150-180 32 29% ███████ 180-210 35 32% ████████ 210-250 17 15% ████ 250-350 6 5% █ > 350 2 2% ▌ ``` #### After (Optimized Parallel) ``` Time (ms) Count Percentage Visualization < 70 28 31% ████████ 70-100 31 34% █████████ 100-150 18 20% █████ 150-200 8 9% ██ 200-300 4 4% █ > 300 2 2% ▌ ``` --- ## 2. Builder Performance Breakdown ### Field Metadata Builder | Operation | Before (avg) | After (avg) | Improvement | |-----------|-------------|-------------|-------------| | Matrix computation | 3.5ms | 3.4ms | Similar | | Creation validation | 15.2ms | 2.1ms | **86% faster** ⚡⚡⚡ | | Deletion validation | 0.08ms | 0.06ms | Similar | | Update validation | 1.3ms | 0.09ms | **93% faster** ⚡⚡⚡ | | Entity processing | 18.6ms | 11.8ms | **37% faster** ⚡ | | **Total validateAndBuild** | **21.3ms** | **14.2ms** | **33% faster** ⚡ | #### Performance Distribution ``` Before: ▁▂▄█████▆▄▂▁ (wide spread, 15-28ms range) After: ▁▁▃█████▃▁▁ (tight clustering, 10-18ms range) ``` ## 4. Cache Invalidation Performance Breakdown ### Cache Invalidation Summary | Metric | Before (Legacy) | After (Optimized) | Improvement | |--------|-----------------|-------------------|-------------| | **Best Time** | 131.965ms | 47.833ms | **63.7% faster** ⚡⚡⚡ | | **10th Percentile** | 140.2ms | 51.7ms | **63.1% faster** ⚡⚡⚡ | | **25th Percentile** | 146.1ms | 54.4ms | **62.7% faster** ⚡⚡⚡ | | **Median (50th)** | 155.1ms | 63.4ms | **59.1% faster** ⚡⚡⚡ | | **Average** | 156.3ms | 76.8ms | **50.9% faster** ⚡⚡⚡ | | **75th Percentile** | 160.2ms | 90.2ms | **43.7% faster** ⚡⚡ | | **90th Percentile** | 191.7ms | 100.2ms | **47.7% faster** ⚡⚡ | | **95th Percentile** | 235.6ms | 110.3ms | **53.2% faster** ⚡⚡⚡ | | **99th Percentile** | 278.2ms | 224.9ms | **19.1% faster** ⚡ | | **Worst Time** | 383.914ms | 285.102ms | **25.7% faster** ⚡ | ### Cache Invalidation Time Distribution #### Before (Legacy Sequential) ``` Time (ms) Count Percentage Visualization 130-140 3 3% ▊ 140-150 15 14% ████ 150-160 48 44% ███████████ 160-180 31 28% ███████ 180-220 8 7% ██ 220-280 3 3% ▊ > 280 2 2% ▌ ``` #### After (Optimized Parallel + Intersection) ``` Time (ms) Count Percentage Visualization < 50 3 3% ▊ 50-60 27 25% ███████ 60-70 25 23% ██████ 70-90 25 23% ██████ 90-100 15 14% ████ 100-120 8 7% ██ 120-150 3 3% ▊ > 150 4 4% █ ``` ## 6. Performance Consistency Analysis ### Standard Deviation & Variance | Metric | Before | After | Improvement | |--------|--------|-------|-------------| | **Cache Invalidation Std Dev** | 42.1ms | 35.2ms | **16.4% more consistent** ⚡ | | **Total Execution Std Dev** | 68.3ms | 89.1ms | Slightly more variable | | **Coefficient of Variation (Cache)** | 26.9% | 45.8% | More variance | | **Outliers (> 2σ)** | 5 cases | 3 cases | **40% fewer outliers** ⚡ | --- |
||
|
|
267af42412 |
Centralize v2 errors types in twenty-shared (#15358)
# Introduction Followup of https://github.com/twentyhq/twenty/pull/15331 ( Reducing size by concerns ) This PR centralizes v2 format error types in `twenty-shared` and consuming them in the existing v2 error format logic in `twenty-server` ## Next This https://github.com/twentyhq/twenty/pull/15360 handles the frontend v2 format error refactor ## Conclusion Related to https://github.com/twentyhq/core-team-issues/issues/1776 |
||
|
|
45473218d3 |
Field deactivation side effect views calendar kanban viewFields (#15180)
# Introduction
Handling both:
- field deactivation side effect on view fields, view filters and views
- field deactivation side effect on view that targets it as
`kanbanAggregateFieldMetadataId`
- field deactivation side effect on view that targets it as
`calendarFieldMetadataId`
## Coverage
added coverage
```ts
PASS test/integration/metadata/suites/field-metadata/kanban-aggregate-field-deactivation-deletes-views.integration-spec.ts (13.132 s)
kanban-aggregate-field-deactivation-nullifies-kanban-properties
✓ should nullify kanban properties when field used as kanbanAggregateOperationFieldMetadataId is deactivated (3923 ms)
✓ should not modify views when field not used as kanbanAggregateOperationFieldMetadataId is deactivated (2958 ms)
✓ should nullify kanban properties on multiple views when they all use the same field as kanbanAggregateOperationFieldMetadataId (2542 ms)
✓ should nullify kanban properties when views have different aggregate operations on same field (3380 ms)
Test Suites: 1 passed, 1 total
Tests: 4 passed, 4 total
Snapshots: 0 total
Time: 13.154 s
```
```ts
PASS test/integration/metadata/suites/field-metadata/view-group-field-deactivation-deletes-views.integration-spec.ts (12.639 s)
view-group-field-deactivation-deletes-views
✓ should delete view when field used in view group is deactivated (3469 ms)
✓ should not delete view when field not used in view group is deactivated (3109 ms)
✓ should delete multiple views when they all use the same field in view groups (2741 ms)
✓ should handle deactivation when view has multiple view groups with different fields (3008 ms)
Test Suites: 1 passed, 1 total
Tests: 4 passed, 4 total
Snapshots: 0 total
Time: 12.664 s
```
```ts
PASS test/integration/metadata/suites/field-metadata/calendar-field-deactivation-deletes-views.integration-spec.ts (14.579 s)
calendar-field-deactivation-deletes-views
✓ should delete view when field used as calendarFieldMetadataId is deactivated (3388 ms)
✓ should not delete view when field not used as calendarFieldMetadataId is deactivated (2438 ms)
✓ should delete multiple views when they all use the same field as calendarFieldMetadataId (2635 ms)
✓ should handle deactivation when views have different calendar layouts on same field (3195 ms)
✓ should delete calendar view but not other view types when calendar field is deactivated (2682 ms)
Test Suites: 1 passed, 1 total
Tests: 5 passed, 5 total
Snapshots: 0 total
Time: 14.601 s, estimated 15 s
```
## View soft deletion
We decided to remove the soft deletion grain on all the views, in this
PR context we've only removed soft deleted validation requirement on any
view entities
## Conclusion
close https://github.com/twentyhq/core-team-issues/issues/1754
|
||
|
|
cceeb6ed4d |
Add applicationId to syncableEntity and fix syncApp deletion (#15170)
## Context - All flatEntity should extend SyncableEntity - SyncableEntity should now have applicationId and application relation - Fix syncApp deletion, should now properly use migration v2 to delete syncable entities |
||
|
|
3462a2e288 |
ViewGroup and ViewFilters side effect in v2 (#15096)
# Introduction ### Summary Implements side effect handling for `ViewGroup` and `ViewFilters` when field metadata is updated in the v2 architecture. This ensures that view-related records are properly maintained when enum field options are modified, deleted, or created. ### Side effects - **Side Effect System**: Added side effect handling for field metadata updates that manages related view groups and view filters - **Enum Field Updates**: When enum field options are modified, the system now: - **View Groups**: Creates new groups for added options, updates existing groups for modified options, and deletes groups for removed options - **View Filters**: Updates filter values to reflect option changes and removes filters that reference deleted options ### Enum runner fix Update now works for both atomic enum and array enum ( multi select for instance ) ### Compute flat entity maps from to Standardized this method usage across v2 services Next step is to require dependencies dynamically ## Conclusion closes https://github.com/twentyhq/core-team-issues/issues/1649 |
||
|
|
6188c72f74 |
Simplify and enhance v2 type devxp (#15032)
# Introduction This PR introduces a huge type refactor that will leverage dynamic intra entity optimistic flat maps update in the future and also a more granular cache invalidation enhancing performances close https://github.com/twentyhq/core-team-issues/issues/1717 close https://github.com/twentyhq/core-team-issues/issues/1716 close https://github.com/twentyhq/core-team-issues/issues/1643 ## What's done ### Comparators centralization Comparator is now done through global configuration as const for each metadata names Thanks to Note: Definition of standard is evolving, standard is now scoped to an app. Meaning that a manifest should be able to update its own standards objects but on other app standards ones ? Each synchronizable entities will have a standardOverrides ? ## Typing refactor ### `AllFlatEntityTypesByMetadataName` **Single source of truth for the complete type ecosystem**, mapping each metadata name to its entity types, flat entities, and migration actions: ```typescript export type AllFlatEntityTypesByMetadataName = { fieldMetadata: { actions: { created: CreateFieldAction; updated: UpdateFieldAction; deleted: DeleteFieldAction; }; flatEntity: FlatFieldMetadata; entity: FieldMetadataEntity; }; objectMetadata: { /* ... */ }; // ... all 10 metadata types }; ``` ### `ALL_METADATA_NAME_MANY_TO_ONE_RELATIONS` **Explicitly declares database relationships** between entities with compile-time validation: ```typescript export const ALL_METADATA_NAME_MANY_TO_ONE_RELATIONS = { viewField: { view: 'viewId', fieldMetadata: 'fieldMetadataId', }, cronTrigger: { serverlessFunction: 'serverlessFunctionId', }, // ... all relations } as const satisfies MetadataNameAndRelations; ``` ### `ALL_FLAT_ENTITY_CONFIGURATION` **Centralizes comparison and serialization logic** for each metadata type: ```typescript export const ALL_FLAT_ENTITY_CONFIGURATION = { fieldMetadata: { propertiesToCompare: ['name', 'type', 'label', 'defaultValue', /* ... */], propertiesToStringify: ['options', 'settings', 'defaultValue'], }, objectMetadata: { propertiesToCompare: ['nameSingular', 'namePlural', 'isActive', /* ... */], propertiesToStringify: [], }, // ... all metadata types } as const satisfies AllFlatEntityConfiguration; ``` ## Combined Impact These three configurations work together to create a **strongly-typed, centrally-managed metadata system**: 1. **`AllFlatEntityTypesByMetadataName`** defines *what exists* 2. **`ALL_METADATA_NAME_MANY_TO_ONE_RELATIONS`** defines *how they relate* 3. **`ALL_FLAT_ENTITY_CONFIGURATION`** defines *how to compare and serialize them* **Result:** Builders and validators become thin wrappers around type-safe, configuration-driven logic instead of containing scattered, error-prone manual implementations. ## What's next ### StandardOverrides standardization Every metadata entity can be a standard one for a workspace if it's an installed app, which means it might not expose the whole entity api to be editable through an import dynamically The standard overrides logic should not be applied to Fields and Objects but to every entities At the moment we have a logic of `EDITABLE_PROPERTIES` through the api, and also `STANDARD_OVERRIDEDABLE_PROPERTIES` This should be configuration centered like `propertiesToCompare` and `propertiesToStringify`. Scoping this PR to two last for the moment. As update dispatch to standardOverrides could be considered as a side effect prefer waiting to start the side effect refactor ### Granular Optimistic deprecation With this new grain at runtime we will be able to add a flat entity and dispatch its addition to related flat maps, so we don't have to describe an optimistic method for each flat entity operations See `addFlatEntityToFlatEntityAndRelatedEntityMapsOrThrow` Note: Still in wip and included in this PR but about to create a new one to integrate these utils and remove existing methods ### ValidateBuildAndRun dynamic args typed defintion We should restrain the devxp to send expected flat maps entity as at least from to or dependency as we now have the grain both a type lvl and runtime to do so It should not be possible in the devxp to forgot adding the views to the v2 builder when passing the view field anymore ( that would lead to permanent validation error in view field integrity checks ) ## Conclusion Thanks for reading and reviewing ! Any suggestions are more than welcomed ! ( same as for questions too ! ) |