Create the necessary tooling to listen to metadata events and plug it to
the front components. Now we have a hot reload like experience when we
edit a component in an app.
## Backend
- Split `EventWithQueryIds` into `ObjectRecordEventWithQueryIds` and
`MetadataEventWithQueryIds`
- Publish metadata event batches to active SSE streams in
`MetadataEventsToDbListener`
## Frontend
- Create a metadata event dispatching pipeline: SSE metadata events are
grouped by metadata name, transformed into
`MetadataOperationBrowserEventDetail` objects, and dispatched as browser
`CustomEvents`
- Add `useListenToMetadataOperationBrowserEvent` hook for consuming
metadata operation events filtered by metadata name and operation type
- Rename `useListenToObjectRecordEventsForQuery` to
`useListenToEventsForQuery`, now accepting both
`RecordGqlOperationSignature` and `MetadataGqlOperationSignature`
- Implement `useOnFrontComponentUpdated` which subscribes to front
component metadata events and updates the Apollo cache when the
component is modified
- Add `builtComponentChecksum` to the front component query and appends
it to the component URL for browser cache invalidation
## Reduce type leakage between GraphQL schemas
### Why
Twenty runs two separate GraphQL schemas: **core** and **metadata**.
NestJS's `@nestjs/graphql` uses a global `TypeMetadataStorage` that
accumulates all decorated types across all modules. When each schema is
built, every registered type leaks into both schemas regardless of which
module it belongs to.
This means the core schema's generated TypeScript
(`generated/graphql.ts`) contained ~2,700 lines of types that only
belong to the metadata schema (and vice versa). This creates confusion
about type ownership, inflates generated code, and makes it harder to
reason about which API surface each schema actually exposes.
### How
**1. Patch `@nestjs/graphql` to support schema-scoped type resolution**
- **(Already done)** Added a `resolverSchemaScope` option to
`GqlModuleOptions`, allowing each schema to declare a scope (e.g.
`'metadata'`)
- `ResolversExplorerService` now filters resolvers by a
`RESOLVER_SCHEMA_SCOPE` metadata key, so each schema only sees its own
resolvers
- `GraphQLSchemaFactory` now performs a **reachability walk**
(`computeReachableTypes`) starting from scoped resolver return types and
arguments, only including types that are transitively referenced —
handling unions, interfaces, and prototype chains
- Type definition storage and orphaned reference registry are cleared
between schema builds to prevent cross-contamination
**2. Register `ClientConfig` as orphaned type in metadata schema**
Since `ClientConfig` is needed in the metadata schema but not directly
returned by a resolver, it's explicitly declared via
`buildSchemaOptions.orphanedTypes`.
**3. Regenerate frontend types and fix imports**
- `generated/graphql.ts` shrank by ~2,700 lines (types moved to where
they belong)
- `generated-metadata/graphql.ts` gained types like `ClientConfig` that
were previously missing
- ~500 frontend files updated to import from the correct generated file
This PR adds what is required to handle soft-delete and restore SSE
events in virtualized table and board.
Restore is not handled in board for now as it requires respecting sorts
when inserting record ids.
Since virtualized table is refetching small chunks, we just refetch for
now.
The long term goal is to handle event handling without refetching in all
main components, and also handle SSE events that have the same origin
that the current tab. But for now we implement what is easily doable.
# QA
Delete between table and board (delete only) :
https://github.com/user-attachments/assets/715dd44a-007a-44ab-bf49-5ef039cd57c3
Delete and restore between table and table :
https://github.com/user-attachments/assets/f2122519-e969-491f-b71c-018d0f85bd86
## Implement Navigation Menu Items Frontend
Implements the frontend for navigation menu items, the new system
replacing favorites.
### Changes
- Added GraphQL fragments and queries for navigation menu items
- Added hooks for managing navigation menu items (create, update,
delete, sorting, filtering)
- Updated components to use navigation menu items instead of favorites
- Added test coverage for utility functions
### Migration Note
The favorites and navigation menu item modules currently exist in
parallel. The favorites code will be removed once all data has been
migrated to navigation menu items.
<!-- CURSOR_SUMMARY -->
---
> [!NOTE]
> Replaces Favorites with feature-flagged `NavigationMenuItem` across
frontend and backend, while keeping Favorites as fallback until
migration completes.
>
> - UI: new `navigation-menu-item` components (folders, orphan items,
drag provider/droppable, icons, skeleton), dispatcher components to
switch from Favorites, and updated “Add to favorites” action to create
`NavigationMenuItem` when `IS_NAVIGATION_MENU_ITEM_ENABLED`
> - DnD: shared `validateAndExtractFolderId` and droppable id utils
moved to `ui/layout/draggable-list`; favorites DnD updated to use shared
utils
> - GraphQL (client): add fragments, queries, mutations, hooks
(create/update/delete/find), and generated types; added
`RecordIdentifier` and `targetRecordIdentifier` on `NavigationMenuItem`
> - Prefetch: new prefetch state/effect for navigation menu items; skip
favorites prefetch when flag enabled
> - Backend: add DTOs (`NavigationMenuItem`, `RecordIdentifier`),
resolver `targetRecordIdentifier` field, service logic to fetch record
identifiers with permission-aware access and image signing,
`getRecordImageIdentifier` util, entity relation to `view`, and
migration adding FK on `viewId`
> - Feature flags & seeding: add `IS_NAVIGATION_MENU_ITEM_ENABLED` to
enums, dev seeder enables it; standard app seeds workspace navigation
menu items instead of favorites when flag on
> - Tests: add unit tests for sorting/labels/folder id and related utils
>
> <sup>Written by [Cursor
Bugbot](https://cursor.com/dashboard?tab=bugbot) for commit
c99746f08b9f84fc8cec4fcc3a7d7afb8ea92db7. This will update automatically
on new commits. Configure
[here](https://cursor.com/dashboard?tab=bugbot).</sup>
<!-- /CURSOR_SUMMARY -->
---------
Co-authored-by: Félix Malfait <felix.malfait@gmail.com>
Co-authored-by: Aman Raj <92664006+araj00@users.noreply.github.com>
Co-authored-by: Félix Malfait <felix@twenty.com>
Co-authored-by: Paul Rastoin <45004772+prastoin@users.noreply.github.com>
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: github-actions <github-actions@twenty.com>
fixes#16340
we are updating the cache partially to prevent the flow of the corrupted
fields into the cache
the fields get corrupted during the mutation process potentially
overriding the recoil cache as we are passing the `newRecordCache`
directly in `upsertRecordsInStore`, the newRecordCache is thin and hence
the recoil wipes out the fields that are undefined or empty
we prevent this by extracting the partial data ( only the data which is
being updated in the field ) and passing it to `upsertRecordsInStore`,
as it only touched the specific fields and updates the data, leaving the
other fields untouched.
https://github.com/user-attachments/assets/5256bef7-70c3-47b3-b2ce-dd02ee1a2de8
---------
Co-authored-by: Arun kumar <arunkumar@Aruns-MacBook-Air.local>
Co-authored-by: Lucas Bordeau <bordeau.lucas@gmail.com>
Previously we introduced useUpdateOneRecordV2 to handle morph relations.
In this PR we are removing it to only use useUpdateOneRecord which has
been adapted not to requires objectMetadataNameSingular in its props.
This PR handles SSE create event.
It also fixes a regression on board, which was making it flash with
skeletons on any update / create.
This PR was a good test case to experience how each main components of
the app : table, board, calendar, reacts to a create event.
It is not straightforward for now, because each component handles
records with its own state management to turn those records into a
coherent display of rows or cards.
The requests for each component are also different : fetch more, group
by, multiple requests in parallel, so the cleanest way to handle
optimistic effect requires to create one small optimistic engine per
component tuned for its internal data logic.
For now we've decided to implement what's doable in a reasonable amount
of time and that includes not handling table with groups for now.
Creating a clean optimistic logic for each component will be done later.
# QA
## Importing 100 records via the API (could be a script, a workflow, an
AI calling tools, a CSV import, etc.)
https://github.com/user-attachments/assets/d38a3770-8b0a-4f83-8275-5e7d1be0a5c6
## Creating from different components and seeing the SSE event being
processed by other components
https://github.com/user-attachments/assets/106b2f49-19cb-4190-92b7-0653c8373366
## Context
buildValueFromFilter was not handling composite filters which was needed
for RLS. This PR implements that and fix some issues with RLS
Tested with a few composite + relation fields
<img width="559" height="206" alt="Screenshot 2026-01-19 at 15 38 42"
src="https://github.com/user-attachments/assets/d64afa6e-3e12-4843-a215-a693665112c5"
/>
This PR implements SSE update events across the main components of the
application : tables, boards, calendars, show pages.
There is still work to do on other event type and on making sure
everything works fine, but this first implementation should be robust
enough to start with.
Some problems encountered along the way :
- Events are returning raw Postgres output, because they are not
normalized by the GraphQL layer, so we ended up with amountMicros as
string values, which the frontend does not like, so I implemented a
small util in our `formatResult` generic pipeline to turn amountMicros
to a number if it's a string value. We could implement other formatters
for composite fields if we see problems with events.
-`action` property was missing in SSE events, which is required by the
frontend to know what kind of event it is.
# QA
https://github.com/user-attachments/assets/393b45ac-59d2-48b0-855b-2ce9c4b8ae57https://github.com/user-attachments/assets/cb214e7a-1595-4b85-bdb6-87630993d2e2
## Context
Now that RLS predicates are applied, creating a record through the FE
(which is empty by default) is failing if your role has predicates and
your input does not respect them (which will always be true since, as
said above, input will be pretty much empty)
## Implementation
- Moved isMatching* filters to twenty-shared
- Implemented isMatchingRlsPredicates utils in the backend (ORM) to
check before insertion/update if the record is matching the current user
role Rls predicates, reusing the isMatching* filters utils moved to
twenty-shared
- Frontend now applies RLS predicates before creating a new record
(similarly to what we do with view filters)
Note:
It seems composite were not properly handled with view-filter insertion
logic, since I'm reusing the util for now, the issue remains for RLS and
will need to be addressed
This PR refactors object record operation dispatch through a browser
event instead of a state that was registering all operations.
This is a cleaner pattern as it is a synchronous event code path instead
of relying on a useEffect to watch state change, which is not ideal.
We introduce this change first to then rely on this new pattern to
dispatch SSE events in a following PR.
Upgraded to Storybook 10. We still use `@storybook/test-runner` for
testing since it appears it'd require more work to move from Jest to
Vitest than I initially anticipated, but I completed this PR to fix
`storybook:serve:dev` - it takes time to load, but it works the way it
used to with Storybook 8.
https://github.com/user-attachments/assets/7afc32c6-4bcf-4b37-b83b-8d00d28dda15
Upgraded from 8.6.15 to 9.1.17 in two steps:
- 8.6.15 -> 9.0.0
- 9.0.0 -> 9.1.17
I had to disable `storybook-addon-cookie` since it is not supported for
Storybook 9. However, I do intend to upgrade to Storybook 10 when this
is merged, so we can replace the aforementioned add-on with this fork
specifically created to support Storybook 10 and above:
https://www.npmjs.com/package/@storybook-community/storybook-addon-cookie.
Additionally, once we upgrade to Version 10 successfully, I will start
looking into integrating the official Vitest add-on.
- moved a few gql types to twenty shared to re-use in server
- added a new endpoint, onEventSubscription that expect a streamId to
create a connection
- two new endpoints to store queries in Redis
- updated the existing batch channel to directly use object record type
#16383
## Scope & Context
**Context**:
Added a **Bulk Update** feature to allow users to modify multiple
records simultaneously. This addresses the need for efficient data
management when dealing with large datasets.
**Scope**:
Enabling a "Bulk Update" flow that can be triggered for a filtered list
of records (e.g., current view). The feature guides the user through
selecting fields to modify, inputting new values, and executing the
update with real-time progress feedback.
## Current behavior
Currently, users must open and update records one by one.
To change the "Status" of 10 different opportunities, the user has to
navigate to each record detail page or use the inline cell editor 10
separate times. This is repetitive and time-consuming.
## Expected behavior
Users can trigger "Update Records" from the command menu or action bar.
1. **Choose Fields**: A step where users select which properties they
want to modify (e.g., check "Status" and "Assignee").
2. **Input Values**: Users provide the new values for the selected
fields.
3. **Execution**: Upon confirmation, the system updates all matching
records in the background.
4. **Feedback**: A progress indicator shows the number of processed
records, and a toast notification confirms completion.
*Key interactions:*
- Users can clear a field's value (set to null) by selecting the field
but leaving the input empty.
- The operation supports cancelling midway.
https://github.com/user-attachments/assets/c87366a3-246e-4615-9941-0bf63d70df86
## Technical inputs
**Core Functionality**:
- **Incremental Batch Processing**: Updates are performed in small
batches (using `useIncrementalUpdateManyRecords` to handle large
datasets without timing out or freezing the UI.
- **Global Feedback**: Integrated with the Snackbar system to notify
users of success or errors across the application.
<!-- CURSOR_SUMMARY -->
---
> [!NOTE]
> Introduces localization keys and generated messages to support the new
bulk update workflow.
>
> - New strings for command menu selection info: `{totalCount} selected`
> - Footer actions for bulk update: `Apply`, `Cancel` in
`UpdateMultipleRecordsFooter`
> - Toasts in `UpdateMultipleRecordsContainer`: `Successfully updated
{count} records`, `Failed to update records. Please try again.`
> - Action labels: `Update`, `Update records` in
`DefaultRecordActionsConfig` and command menu hook
> - Adds/updates entries across multiple locale `.po` files and
regenerates `af-ZA` messages
>
> <sup>Written by [Cursor
Bugbot](https://cursor.com/dashboard?tab=bugbot) for commit
b1dce4c599b85fa2947ca63b265fc750e80d9bb6. This will update automatically
on new commits. Configure
[here](https://cursor.com/dashboard?tab=bugbot).</sup>
<!-- /CURSOR_SUMMARY -->
---------
Co-authored-by: Samuel Arbibe <samuelarbibe@Samuels-MacBook-Pro.local>
Co-authored-by: Félix Malfait <felix@twenty.com>
<!-- CURSOR_SUMMARY -->
> [!NOTE]
> Implements last-modifier tracking and unified actor injection.
>
> - New `ActorFromAuthContextService` replaces createdBy-only logic;
pre-query hooks now inject both `createdBy` and `updatedBy` on create
and `updatedBy` on update
> - Add `updatedBy` standard field to core/custom objects (e.g.,
`person`, `company`, `task`, `note`, `attachment`, `dashboard`,
`workflow`, `workflowRun`) with IDs, metadata builders, and ORM entity
fields
> - Record CRUD: `create` now sets both `createdBy` and `updatedBy`;
`update` sets `updatedBy`; workflow actions use a shared workflow actor
builder; REST base handler uses the new actor service
> - Seeder data and snapshots updated to include `updatedBy`; GraphQL
result formatting adds defaults/handling for empty composite fields
(incl. actor)
> - Frontend `useUpdateOneRecord` upserts returned record into the local
store after mutation
>
> <sup>Written by [Cursor
Bugbot](https://cursor.com/dashboard?tab=bugbot) for commit
1f283778e9cb28a35bf84632a1a2997250bb3131. This will update automatically
on new commits. Configure
[here](https://cursor.com/dashboard?tab=bugbot).</sup>
<!-- /CURSOR_SUMMARY -->
---------
Co-authored-by: Félix Malfait <felix.malfait@gmail.com>
Co-authored-by: Félix Malfait <felix@twenty.com>
## Summary
This PR enforces the use of `@/` alias for imports instead of relative
parent imports (`../`).
## Changes
### ESLint Configuration
- Added `no-restricted-imports` pattern in `eslint.config.react.mjs` to
block `../*` imports with the message "Relative parent imports are not
allowed. Use @/ alias instead."
- Removed the non-working `import/no-relative-parent-imports` rule
(doesn't work properly in ESLint flat config)
### VS Code Settings
- Added `javascript.preferences.importModuleSpecifier: non-relative` to
`.vscode/settings.json` (TypeScript setting was already there)
### Code Fixes
- Fixed **941 relative parent imports** across **706 files** in
`packages/twenty-front`
- All `../` imports converted to use `@/` alias
## Why
- Consistent import style across the codebase
- Easier to move files without breaking imports
- Better IDE support for auto-imports
- Clearer understanding of where imports come from
- Added new `useUpdateManyRecords` hook for batch record updates with
optimistic cache updates
- Added `updateMessageFoldersSyncStatus` hook for managing message
folder sync state
- Redesigned Message Folders List with BreadCrumb and Animations
Fixes https://github.com/twentyhq/twenty/issues/16624
Original issue:
- while persisting a field (calling useUpdateOne), the response from the
backend is missing the taskTargets many to many (same for note). As we
optimistically update the cache, we lose the "Relations" in the UI
- I'm changing the behavior of useUpsertInRecordStore to accept
recordGqlFields to only update the fields we want in the record store
(this way, we are not losing the targets information in our case)
# Introduction
In this pull-request we introduce a service dedicated to the
twenty-standard app installation, we will later be able to re-use
existing logic to be more generic and allow any app installation.
For the moment sticking to this usage
https://github.com/twentyhq/core-team-issues/issues/1995
## Encountered issues
- We decided not to migrate deprecated fields ( also they will become
custom field for any existing workspace having them in the future )
- duplicate criteria
- wrong search index declaration
- forgotten isSearchable
- Attachement seed
- Restored standardId
## Note
For the moment we're still searching through standardId for code that
run on both existing and new workspaces.
For code running on new workspace exclusively we're searching using
universalIdentifier
We will standardize universalIdentifier usage later when we've migratred
all the existing workspaces
## Workspace creation
Will handle workspace creation the same way in another PR
Related https://github.com/twentyhq/twenty/pull/15065
## TODO
- [ ] Double all frontend hardcoded queries to not refer to deprecated
fields especially attachments
This PR improves the general UX and DX of boards, by modifying the query
effect to only use paged group by queries.
In this PR we implement two more things in the backend for group by
queries :
- Fixed ORDER BY in the PARTITION BY sub-query (this wasn't working
because it was applied in the main query, so it sorted randomly picked
records, which was a correct sort on an incorrect dataset returned by
the sub-query)
- Added offset paging in PARTITION BY
Miscellaneous, various bug fixes and improvements along the way :
- Throttled loading of cards to avoid React freeze
- Handling of drag & drop
- Handling of create / delete / update
- Reworked skeleton (the library slows down a lot with hundreds of
skeleton for a spinning effect that is hardly noticed)
- Fixed refetch of aggregate queries (I included the new group by
aggregates query we use in the existing refetch mechanism)
- Re-trigger queries on filters and sorts changes
- Unselect all record ids when deleting / restoring / detroying
- Fetch only groups that still have records to lighten the group by
query.
# What remains to be done
This is still a naïve fetch more implementation that will work for a few
fetch more rounds, but if you scroll and load say 200 cards per column
on a board, React will re-render all 200 cards of each column each time.
We would probably need to virtualize the board with paged queries as we
did for the table, this could be done after this PR but seems less
urgent.
What's nice is that this new query pattern is well designed for
virtualization also, drawing from our experience with table
virtualization, and adapted to a multi-column request pattern, like a
2:2 matrix of records, for our boards.
So the remaining work would be to design a UI solution for virtualizing
this matrix of records, which could be quite different from our table
virtualization mechanism.
Fixes#15508
Addresses the issue where the "Export View" functionality was not
exporting all records from a view. Users reported that only a subset of
records were exported, even when the view contained more. This led to
incomplete data exports.
The core of the problem was found within the `useLazyFetchAllRecords`
hook. The `fetchMoreRecordsLazy` function, which is responsible for
fetching subsequent pages of data during the export process, was
inadvertently using a default page limit (`DEFAULT_SEARCH_REQUEST_LIMIT`
of 60 records) instead of the intended `pageSize` (200 records). This
discrepancy caused the export process to prematurely stop fetching
records.
The PR modifies
`packages/twenty-front/src/modules/object-record/hooks/useLazyFetchAllRecords.ts`
to explicitly pass the `limit` parameter (which correctly reflects the
`pageSize`) to the `fetchMoreRecordsLazy` function. This ensures that
all subsequent data fetches during an export operation adhere to the
configured page size, allowing for the complete retrieval of all
records.
**Before**
- any user with workpace_members permission was able to remove a user
from their workspace. This triggered the deletion of workspaceMember +
of userWorkspace, but did not delete the user (even if they had no
workspace left) nor the roleTarget (acts as junction between role and
userWorkspace) which was left with a userWorkspaceId pointing to
nothing. This is because roleTarget points to userWorkspaceId but the
foreign key constraint was not implemented
- any user could delete their own account. This triggered the deletion
of all their workspaceMembers, but not of their userWorkspace nor their
user nor the roleTarget --> we have orphaned userWorkspace, not
technically but product wise - a userWorkspace without a workspaceMember
does not make sense
So the problems are
- we have some roleTargets pointing to non-existing userWorkspaceId
(which caused https://github.com/twentyhq/twenty/issues/14608 )
- we have userWorkspaces that should not exist and that have no
workspaceMember counterpart
- it is not possible for a user to leave a workspace by themselves, they
can only leave all workspaces at once, except if they are being removed
from the workspace by another user
**Now**
- if a user has multiple workspaces, they are given the possibility to
leave one workspace while remaining in the others (we show two buttons:
Leave workspace and Delete account buttons). if a user has just one
workspace, they only see Delete account
- when a user leaves a workspace, we delete their workspaceMember,
userWorkspace and roleTarget. If they don't belong to any other
workspace we also soft-delete their user
- soft-deleted users get hard deleted after 30 days thanks to a cron
- we have two commands to clean the orphans roleTarget and userWorkspace
(TODO: query db to see how many must be run)
**Next**
- once the commands have been run, we can implement and introduce the
foreign key constraint on roleTarget
Fixes https://github.com/twentyhq/twenty/issues/14608
Fixes https://github.com/twentyhq/twenty/issues/15369
It's the first time we are hitting this weird behavior on apollo
onCompleted / onError. Theoretically if the reference of the onCompleted
(resp onError) callback is changing, this will trigger a re-run of
onCompleted. But also theretically the reference here is a
recoilCallback so should never change.
As this is synchronous I have move this logic to a sync way
Fixes - https://github.com/twentyhq/twenty/issues/15263
- Replaced `useLoadSelectedRecordsInContextStore` with
`useLoadMergeRecords` in `useOpenMergeRecordsPageInCommandMenu` for
improved functionality.
- Updated `useMergePreview`, `useMergeRecordsActions`, and
`useMergeRecordsSettings` to utilize `mergeRecordsState` instead of the
deprecated context store hook.
- Cleaned up imports and ensured consistency across merge-related hooks.
https://github.com/user-attachments/assets/453539c9-7f2b-4e8c-bfa1-3ceebca07081
---------
Co-authored-by: Charles Bochet <charles@twenty.com>
This fixes#15156
Issue:
Restore and Destroy buttons not appearing in action menu for deleted
records until the record detail view is opened.
Cause:
The record index/table view queries only fetched fields that were
visible as table columns
The [deletedAt] field (along with [createdAt] and [updatedAt]) was not
included in these queries since it's not a visible column
The action menu logic checks [selectedRecord?.deletedAt] to determine if
a record is deleted and which actions to display
Without the [deletedAt] field in the record store, the action menu
couldn't detect deleted records
Opening the detailed view would fetch all fields (including
[deletedAt]), which is why the buttons would appear afterward
Solution
Modified [useRecordsFieldVisibleGqlFields] to always include the
standard fields ([createdAt], [updatedAt], [deletedAt]) in record index
queries, regardless of column visibility. This ensures the action menu
can immediately detect deleted records
---------
Co-authored-by: Félix Malfait <felix.malfait@gmail.com>
Co-authored-by: Félix Malfait <felix@twenty.com>
Avoid fetching full steps and trigger for versions that are not the
current version. Because those won't be used anyway. Better for
performances.
Only difficulty was for the `createDraft` mutation. I needed to return
the full created version so I can store it in cache and use it as new
`currentVersion`. Otherwise the current version is considered as
incomplete for a short time, since workflow is fetched separately from
the current version.
---------
Co-authored-by: Charles Bochet <charles@twenty.com>
This big PR implements table virtualization with an offset paging,
allowing a way more fluid UX.
It is a v1 that should be improved in the future with partial data
loading and optimization of the browser display performance of a row.
But with this PR we have the solid enough technical foundation, both
frontend and backend, to get to a smooth table UX.
Fixes and improvements after first successful round of development
(needed to have main clean) :
- [x] Delete should refresh virtualized portion only and reset all table
- [x] Fix add new : top and bottom
- [x] Table empty shouldn’t show when first loading
- [x] Fix d&d
- [x] Fix sorts
- [x] Fix drag when scrolling after a full virtual page (it throws an
error)
- [x] Si update mais qu’on a un sort ou filter, alors il faut trigger le
refresh
- [x] Reset scroll position between tables
- [x] Reset scroll shadows between tables
- [x] Setup d&n for virtual list :
https://github.com/hello-pangea/dnd/blob/main/docs/patterns/virtual-lists.md
- [x] Full table re-render when entering edit mode
- [x] Clean code and prepare for merge
Fixes https://github.com/twentyhq/core-team-issues/issues/1613 that
contains other bugs to be fixed before merge
---------
Co-authored-by: Charles Bochet <charles@twenty.com>
## Problem
With Twenty usage growing, we are facing challenges on server side to
respond to the demand. The current bottleneck we are facing is server
CPU.
While investigating CPU performances, we figured out that loading all
relation fields was the biggest issue.
### About graphql query response size
**Example:** on `People Index` Table page:
we query: `person.company` and we query **all non relation fields** on
company relation field.
`{ person { company { id, name, domainName, employees, address ... } }`
However we only need **company.id, company.name and company.domainName**
to be able to render the Company Chip in the company column (RELATION)
in the table.
`{ person { company { id, name, domainName } }`
We initially assumed that the querying all non relation fields was not
an issue because it was not adding additional load on database (which is
also partially true: if a field is containing a huge json this will add
load on postgres as data needs to be transfered)
This assumption is wrong on CPU side:
- when loading one to many relations (company.people), this quickly ads
up and we end up with response of 20kB quite quickly, even without
adding any custom field on person.
- even when loading a many to one relation, if the user is storing big
data in a given field (note.body for instance, or workflowRun.state),
this starts also being an issue.
- Worst case scenario are starting to happen: on a production workspace
with 20 active workflows. Loading workflow table while displaying
workflow runs commands (20 workflows x 60 workflowRuns x
workflowRun.state which is a big JSON) result in a response of... 125MB.
### Why is it an issue?
When yoga (graphql engine) parses a response to send it back in server
response, it's using `JSON.parse `(or stringify depending on the case).
Parsing data is CPU instensive (as you need to validate, transform) and
this is more or less `O(n)`.
This means returning 125MB is very intense on the CPU and will likely
use 100% of the CPU for ~1sec in our production servers.
NodeJs (nodeV8) is single threaded. It's able to process multiple
requests in parallel but to do that, it will cut them in "microTasks"
and process each microTasks (that can belong to different requests) one
after the other. Exactly like a single threaded CPU would allocate some
time to a process and then to the next one, etc.
This means that this big response request will actually block the other
request.
As a result, all requests are slow and our infrastructure is multi
tenant so some customers are impacting others.
This is even a bigger issue when our health checks start to fail and
containers are starting being considered as unhealthy and killed by the
orchestrator
## How to fix this
1. On Front end side, we should only query what we need. In this PR, I'm
forcing the relations to only query: `id`, `labelIdentifier`,
`imageIdentifier`
2. (later) as we are API first, we also need to do something to protect
the servers from this side too. To do that, Graphql APIs can associate a
cost to each graphql field (maybe 1 for a test fixed, 2 for a JSON
field, 10 for relations, etc...) and we can throttle based on that.
## Changes in this PR
It's mainly about refactor the tooling to generate `RecordGqlFields`
(`generateDepthOneGqlFields`). this tooling is used to generate the list
of fields we want to query. Most of the time we want to query a record
with one level of nesting.
1) Refactor to remove duplicate code => `generateDepthOne` become
`generateDepth` and can handle both `depth = 0 | 1`
2) Introduce `generateDepthRecordGqlFieldsFromFields.ts` which is the
base and can give you a list of gqlFields based on FieldMetadataItems
3) Introduce `generateDepthRecordGqlFieldsFromObject` which is a
shortcut for an object
4) Introduce `generateDepthRecordGqlFieldsFromRecords` which is the
intersection between `generateDepthRecordGqlFieldsFromObject` and a
given record (useful for cache tooling)
5) Replace all usages + introduce a hook
`useGenerateDepthRecordGqlFieldsFromObject` to ease usage
## Scope
- Refactor SingleRecordPicker to handle morphItem and match Multiple one
- fix query generation to take morph into account
Left:
- double check SingleRecordPicker in Workflows / Form
- Fix tests / stories
This PR follows the multiSelect PR merged previously. It will enable
morph relation Many to One to be handled from the table, using a
singleSelect picker
Main point : I decided to change the singleSelect API to take an array
of **objectMetadataName** instead of only one to deal with both our
usecases.
---------
Co-authored-by: Charles Bochet <charles@twenty.com>
Closes https://github.com/twentyhq/core-team-issues/issues/1560
If viewId is defined in a groupBy query, we want to apply all filters of
the view to the query.
This required to move a lot of code from twenty-front to twenty-shared
to convert the filters as stored in the db into graphql filters,
applying the right combinations between filters etc., which was
previously only done in the FE.
This PR does not handle any field filters, it will be done in a later pr
Following https://github.com/twentyhq/twenty/pull/14306 , working on the
deprecation of objectRecordsPermissions + renaming of objectPermissions
-> objectsPermissions
In this PR
- Removal of objectRecordsPermissions (was not used in the FE any
longer)
- Addition of objectsPermissions, same as objectPermission but renamed.
objectPermission is no longer used in the FE.
Next step
- Remove unused objectPermissions
Raphaël Bosi