This big PR implements table virtualization with an offset paging,
allowing a way more fluid UX.
It is a v1 that should be improved in the future with partial data
loading and optimization of the browser display performance of a row.
But with this PR we have the solid enough technical foundation, both
frontend and backend, to get to a smooth table UX.
Fixes and improvements after first successful round of development
(needed to have main clean) :
- [x] Delete should refresh virtualized portion only and reset all table
- [x] Fix add new : top and bottom
- [x] Table empty shouldn’t show when first loading
- [x] Fix d&d
- [x] Fix sorts
- [x] Fix drag when scrolling after a full virtual page (it throws an
error)
- [x] Si update mais qu’on a un sort ou filter, alors il faut trigger le
refresh
- [x] Reset scroll position between tables
- [x] Reset scroll shadows between tables
- [x] Setup d&n for virtual list :
https://github.com/hello-pangea/dnd/blob/main/docs/patterns/virtual-lists.md
- [x] Full table re-render when entering edit mode
- [x] Clean code and prepare for merge
Fixes https://github.com/twentyhq/core-team-issues/issues/1613 that
contains other bugs to be fixed before merge
---------
Co-authored-by: Charles Bochet <charles@twenty.com>
## Problem
With Twenty usage growing, we are facing challenges on server side to
respond to the demand. The current bottleneck we are facing is server
CPU.
While investigating CPU performances, we figured out that loading all
relation fields was the biggest issue.
### About graphql query response size
**Example:** on `People Index` Table page:
we query: `person.company` and we query **all non relation fields** on
company relation field.
`{ person { company { id, name, domainName, employees, address ... } }`
However we only need **company.id, company.name and company.domainName**
to be able to render the Company Chip in the company column (RELATION)
in the table.
`{ person { company { id, name, domainName } }`
We initially assumed that the querying all non relation fields was not
an issue because it was not adding additional load on database (which is
also partially true: if a field is containing a huge json this will add
load on postgres as data needs to be transfered)
This assumption is wrong on CPU side:
- when loading one to many relations (company.people), this quickly ads
up and we end up with response of 20kB quite quickly, even without
adding any custom field on person.
- even when loading a many to one relation, if the user is storing big
data in a given field (note.body for instance, or workflowRun.state),
this starts also being an issue.
- Worst case scenario are starting to happen: on a production workspace
with 20 active workflows. Loading workflow table while displaying
workflow runs commands (20 workflows x 60 workflowRuns x
workflowRun.state which is a big JSON) result in a response of... 125MB.
### Why is it an issue?
When yoga (graphql engine) parses a response to send it back in server
response, it's using `JSON.parse `(or stringify depending on the case).
Parsing data is CPU instensive (as you need to validate, transform) and
this is more or less `O(n)`.
This means returning 125MB is very intense on the CPU and will likely
use 100% of the CPU for ~1sec in our production servers.
NodeJs (nodeV8) is single threaded. It's able to process multiple
requests in parallel but to do that, it will cut them in "microTasks"
and process each microTasks (that can belong to different requests) one
after the other. Exactly like a single threaded CPU would allocate some
time to a process and then to the next one, etc.
This means that this big response request will actually block the other
request.
As a result, all requests are slow and our infrastructure is multi
tenant so some customers are impacting others.
This is even a bigger issue when our health checks start to fail and
containers are starting being considered as unhealthy and killed by the
orchestrator
## How to fix this
1. On Front end side, we should only query what we need. In this PR, I'm
forcing the relations to only query: `id`, `labelIdentifier`,
`imageIdentifier`
2. (later) as we are API first, we also need to do something to protect
the servers from this side too. To do that, Graphql APIs can associate a
cost to each graphql field (maybe 1 for a test fixed, 2 for a JSON
field, 10 for relations, etc...) and we can throttle based on that.
## Changes in this PR
It's mainly about refactor the tooling to generate `RecordGqlFields`
(`generateDepthOneGqlFields`). this tooling is used to generate the list
of fields we want to query. Most of the time we want to query a record
with one level of nesting.
1) Refactor to remove duplicate code => `generateDepthOne` become
`generateDepth` and can handle both `depth = 0 | 1`
2) Introduce `generateDepthRecordGqlFieldsFromFields.ts` which is the
base and can give you a list of gqlFields based on FieldMetadataItems
3) Introduce `generateDepthRecordGqlFieldsFromObject` which is a
shortcut for an object
4) Introduce `generateDepthRecordGqlFieldsFromRecords` which is the
intersection between `generateDepthRecordGqlFieldsFromObject` and a
given record (useful for cache tooling)
5) Replace all usages + introduce a hook
`useGenerateDepthRecordGqlFieldsFromObject` to ease usage
This PR follows the multiSelect PR merged previously. It will enable
morph relation Many to One to be handled from the table, using a
singleSelect picker
Main point : I decided to change the singleSelect API to take an array
of **objectMetadataName** instead of only one to deal with both our
usecases.
---------
Co-authored-by: Charles Bochet <charles@twenty.com>
Closes https://github.com/twentyhq/core-team-issues/issues/1560
If viewId is defined in a groupBy query, we want to apply all filters of
the view to the query.
This required to move a lot of code from twenty-front to twenty-shared
to convert the filters as stored in the db into graphql filters,
applying the right combinations between filters etc., which was
previously only done in the FE.
This PR does not handle any field filters, it will be done in a later pr
## Add Morph Relation Field Display Support for Tables (one to many &
many to one)
**Features:**
- Display morph relation fields (one-to-many, many-to-one) in table
cells
- Multi-field record retrieval for improved performance
- Enhanced table cell rendering with morph relation support
**Components Added:**
- `MorphRelationManyToOneFieldDisplay` - Shows many-to-one morph
relations
- `MorphRelationOneToManyFieldDisplay` - Shows one-to-many morph
relations
TODO :
handle notes and tasks case if we choose these kind of objects as
targets/source for a morph relation
---------
Co-authored-by: Charles Bochet <charles@twenty.com>
This PR introduces the usage of record fields in the manipulation of
table columns and board fields.
Since all of the actual system relies on states like tableColumns,
recordIndexFieldDefinitions and the likes, it was required to implement
temporary utils that modifies those states in parallel of the new
generic currentRecordFields, to keep a working product.
With this PR though, currentRecordFields becomes the single source of
truth that gets saved to DB, and the remaining work is just to make the
switch with this new state on all components that are plugged to
tableColumns and the like.
This will be done in another PR.
This PR prepares the refactor of record field definition and column
definition.
We need to separate the two concepts of FieldInput and FieldContext at
the cell level, and the new RecordField concept that will replace
FieldDefinition at the metadata and state level.
So we create a new ui sub-folder in the already existing record-field
folder, in order to separate those two concepts that are still very
close.
In this PR
- Introduction of readableFields and updatableFields in
objectMetadataItem selector to ease filtering from a developer
experience perspective ( + to help developers think to do it). In
discussion @lucasbordeau @charlesBochet
- Remove non-updatable field from CSV import process (@etiennejouan)
- QA fix / Non-readable fields should not show on show page
- QA fix / It should not be offered to create a kanban view on a
non-readable field
- QA fix / It should not be offered to create view groups on a
non-readable field
- QA fix / Rating field should have a readonly mode
---------
Co-authored-by: Charles Bochet <charles@twenty.com>
In our Graphql schema, ids are typed as UUID.
Our FE codegen is typing it as 'any', unless we say otherwise. Fixing it
and fixing the impacts in the codebase
Moves system-level operations (auth, billing, admin) to use the
/metadata endpoint instead of /graphql.
This cleans up the endpoint separation so /graphql is purely for core
objects (Company, People, etc.) and /metadata handles all system
operations.
Part of prep work for webhook/API key core migration.
Workflow views and versions are seed being opened by default into record
page. Issue is that:
- new views are set by default to side panel. Updated by copying the
current opensIn value to the new view
- users can still select side panel into their options. Disabling the
button.
<img width="650" alt="Capture d’écran 2025-06-19 à 16 15 34"
src="https://github.com/user-attachments/assets/0ddc3284-0fed-404f-9c1d-225c65549fd1"
/>
# What
Fully deprecate old relations because we have one bug tied to it and it
make the codebase complex
# How I've made this PR:
1. remove metadata datasource (we only keep 'core') => this was causing
extra complexity in the refactor + flaky reset
2. merge dev and demo datasets => as I needed to update the tests which
is very painful, I don't want to do it twice
3. remove all code tied to RELATION_METADATA /
relation-metadata.resolver, or anything tied to the old relation system
4. Remove ONE_TO_ONE and MANY_TO_MANY that are not supported
5. fix impacts on the different areas : see functional testing below
# Functional testing
## Functional testing from the front-end:
1. Database Reset ✅
2. Sign In ✅
3. Workspace sign-up ✅
5. Browsing table / kanban / show ✅
6. Assigning a record in a one to many / in a many to one ✅
7. Deleting a record involved in a relation ✅ => broken but not tied to
this PR
8. "Add new" from relation picker ✅ => broken but not tied to this PR
9. Creating a Task / Note, Updating a Task / Note relations, Deleting a
Task / Note (from table, show page, right drawer) ✅ => broken but not
tied to this PR
10. creating a relation from settings (custom / standard x oneToMany /
manyToOne) ✅
11. updating a relation from settings should not be possible ✅
12. deleting a relation from settings (custom / standard x oneToMany /
manyToOne) ✅
13. Make sure timeline activity still work (relation were involved
there), espacially with Task / Note => to be double checked ✅ => Cannot
convert undefined or null to object
14. Workspace deletion / User deletion ✅
15. CSV Import should keep working ✅
16. Permissions: I have tested without permissions V2 as it's still hard
to test v2 work and it's not in prod yet ✅
17. Workflows global test ✅
## From the API:
1. Review open-api documentation (REST) ✅
2. Make sure REST Api are still able to fetch relations ==> won't do, we
have a coupling Get/Update/Create there, this requires refactoring
3. Make sure REST Api is still able to update / remove relation => won't
do same
## Automated tests
1. lint + typescript ✅
2. front unit tests: ✅
3. server unit tests 2 ✅
4. front stories: ✅
5. server integration: ✅
6. chromatic check : expected 0
7. e2e check : expected no more that current failures
## Remove // Todos
1. All are captured by functional tests above, nothing additional to do
## (Un)related regressions
1. Table loading state is not working anymore, we see the empty state
before table content
2. Filtering by Creator Tim Ap return empty results
3. Not possible to add Tasks / Notes / Files from show page
# Result
## New seeds that can be easily extended
<img width="1920" alt="image"
src="https://github.com/user-attachments/assets/d290d130-2a5f-44e6-b419-7e42a89eec4b"
/>
## -5k lines of code
## No more 'metadata' dataSource (we only have 'core)
## No more relationMetadata (I haven't drop the table yet it's not
referenced in the code anymore)
## We are ready to fix the 6 months lag between current API results and
our mocked tests
## No more bug on relation creation / deletion
---------
Co-authored-by: Weiko <corentin@twenty.com>
Co-authored-by: Félix Malfait <felix@twenty.com>
This pull request introduces changes to improve handling of nullable
values in link-related data structures and simplifies field value
generation logic. Key updates include adjustments to type definitions,
utility functions, and component logic to support `null` values for
links, along with the removal of the `generateDefaultFieldValue`
function in favor of `generateEmptyFieldValue`.
There will be a few more follow-up Pull Requests.
---
Closes https://github.com/twentyhq/twenty/issues/11844
# Introduction
In this PR we've migrated `twenty-shared` from a `vite` app
[libary-mode](https://vite.dev/guide/build#library-mode) to a
[preconstruct](https://preconstruct.tools/) "atomic" application ( in
the future would like to introduce preconstruct to handle of all our
atomic dependencies such as `twenty-emails` `twenty-ui` etc it will be
integrated at the monorepo's root directly, would be to invasive in the
first, starting incremental via `twenty-shared`)
For more information regarding the motivations please refer to nor:
- https://github.com/twentyhq/core-team-issues/issues/587
-
https://github.com/twentyhq/core-team-issues/issues/281#issuecomment-2630949682
close https://github.com/twentyhq/core-team-issues/issues/589
close https://github.com/twentyhq/core-team-issues/issues/590
## How to test
In order to ease the review this PR will ship all the codegen at the
very end, the actual meaning full diff is `+2,411 −114`
In order to migrate existing dependent packages to `twenty-shared` multi
barrel new arch you need to run in local:
```sh
yarn tsx packages/twenty-shared/scripts/migrateFromSingleToMultiBarrelImport.ts && \
npx nx run-many -t lint --fix -p twenty-front twenty-ui twenty-server twenty-emails twenty-shared twenty-zapier
```
Note that `migrateFromSingleToMultiBarrelImport` is idempotent, it's atm
included in the PR but should not be merged. ( such as codegen will be
added before merging this script will be removed )
## Misc
- related opened issue preconstruct
https://github.com/preconstruct/preconstruct/issues/617
## Closed related PR
- https://github.com/twentyhq/twenty/pull/11028
- https://github.com/twentyhq/twenty/pull/10993
- https://github.com/twentyhq/twenty/pull/10960
## Upcoming enhancement: ( in others dedicated PRs )
- 1/ refactor generate barrel to export atomic module instead of `*`
- 2/ generate barrel own package with several files and tests
- 3/ Migration twenty-ui the same way
- 4/ Use `preconstruct` at monorepo global level
## Conclusion
As always any suggestions are welcomed !
## Introduction
Added coverage on the `useDeleteOneRecord` hooks, especially its
optimistic behavior feature.
Introduced a new testing tool `InMemoryTestingCacheInstance` that has
builtin very basic expectors in order to avoid future duplication when
covering others record hooks `update, create, destroy` etc etc
## Notes
Added few comments in this PR regarding some builtin functions I've
created around companies and people mocked object model and that I think
could be cool to spread and centralize within a dedicated "class
template"
Also put in light that unless I'm mistaken some tests are running on
`RecordNode` and not `RecordObject`
Took few directions on my own that as I always I would suggestion nor
remarks on them !
Let me know
## Misc
- Should we refactor `useDeleteOneRecord` tests to follow `eachTesting`
pattern ? => I feel like this is inappropriate as this hooks is already
high level, the only plus value would be less tests code despite
readability IMO
# Introduction
In this PR https://github.com/twentyhq/twenty/pull/10493 introduced a
regression on optimistic cache for record creation, by expecting a
`position` `fieldMetadataItem` on every `ObjectMetadataItem` which is a
wrong assertion
Some `Tasks` and `ApiKeys` do not have one
## Fix
Dynamically compute optimistic record input position depending on
current `ObjectMetadataItem`
## Refactor
Refactored a failing test following [jest
each](https://jestjs.io/docs/api#describeeachtablename-fn-timeout)
pattern to avoid error prone duplicated env tests. Created a "standard'
and applied it to an other test also using `it.each`.
## Introduction
This refactor results from this
https://github.com/twentyhq/twenty/pull/10493 review
Introduced a new abstraction to the extinsting
`generateDepthOneRecordGqlFields` that was accepting an optional record
in arg in order to map generated `recordGqlFields` to the keys in the
record
1/ Created a dedicated util method
`generateDepthOneRecordGqlFieldsFromRecord` to do so
2/ Updated each previous `generateDepthOneRecordGqlFields` passing a
record to call new `generateDepthOneRecordGqlFieldsFromRecord`
## Context
We are experiencing a lot of re-rendering / flash on MultiRecordSelect /
SingleRecordSelect / RelationPicker.
This PR is a first step to refactor this components
## Scope
Only move files to uniformize and prepare for the upcoming refactoring
## Vision
- SingleRecordPicker
- MultipleRecordPicker
- sharing RecordPicker tooling (RecordPickerComponentInstanceContext,
searchState)
Used in other areas:
- RelationPicker (which is actually only a subcomponent of
RelationToOneFieldInput)
- RelationFromManyFieldInput
- WorkflowRelationFieldInput
- etc.
+ remove all effects
+ migrate to the latest APIs
+ make a pass on re-renders to remove them completely (we likely need to
use a bit more familyStates here)
Closes https://github.com/twentyhq/core-team-issues/issues/410
- Added `openRecordIn` column in the `view` entity, which is set to
`SIDE_PANEL` by default
- Created a new option inside the view option dropdown to be able to set
`openRecordIn`
- Updated all record show page openings to reflect the setting behavior
- For `workflow`, `workflowVersion` and `workflowRun` (what I call
workflow objects), we want the default view `openRecordIn` to be set to
`RECORD_PAGE`. When seeding the views for the new workspaces, we set
`openRecordIn` to `RECORD_PAGE` for workflow objects. Since the workflow
objects views `openRecordIn` will be set to the default value
`SIDE_PANEL` for the existing workspaces when the sync metadata runs, I
created a script to run in the 0.43 update to update this value.
- Updated `closeCommandMenu` because of problems introduced by the
animate presence wrapper around the command menu. We now reset the
states at the end of the animation.
Note: We want to be able to open all workflow objects pages in the side
panel, but this requires some refactoring of the workflow module. For
now @Bonapara wanted to allow the possibility to change the
`openRecordIn` setting to `SIDE_PANEL` even for the workflows even if
it's buggy and not ready for the moment. Since this is an experimental
feature, it shouldn't cause too many problems.
# Introduction
While importing records encountering missing expected fields when
writting a fragment from apollo cache
## Updates
### 1/ `createdBy` Default value
When inserting in cache in create single or many we will now make
optimistic behavior on the createdBy value
### 2/ `createRecordInCache` dynamically create `recordGrqlFields`
When creating an entry in cache, we will now dynamically generate fields
to be written in the fragment instead of expecting all of them. As by
nature record could be partial
### 3/ Strictly typed `RecordGqlFields`
# Conclusion
closes#9927
# Introduction
At the moment when updating any record cache occurence, we will build a
fragment that will expect all of the object metadata item fields to be
provided.
Which result in the following traces: ( in the video companies aren't
fetch with companyId and other missing fields )
https://github.com/user-attachments/assets/56eab7c1-8f01-45ff-8f5d-78737b788b92
By definition as we're using graphql we might not request every record's
fields each time we wanna consume them.
In this way we will now dynamically compute or expect depending on the
CRUD operation specific fields to be written in the cache, and not all
of them
Tested all optimistic and failure management use cases
## Covering cache
Added coverage only for the `deleteOne` and `deleteMany` hooks, it cover
only the record record cache and not its relations hydratation ( for the
moment )
## Why not closing #9927
Unless I'm mistaken everything done here have fixed the same logs/traces
issue for updates and deletion but not creation.
Which means we still need to investigate the mass upload from import and
prefillRecord behavior
In a nutshell: went over each `updateRecordFromCache` calls, still need
to do all `createRecordInCache` calls
related to #9927
## Conlusion
Sorry for the big PR should have ejected into a specific one for the
`MinimalRecord` refactor
Will also continue covering others hooks later in my week as for the
`deleteOne`
As always any suggestions are welcomed !
# Introduction
When we create a new `view` from record table that has relation such as
opportunities.
Encountered invariant conditions:
## Unknown fiel `__typename`
`Should never occur, encountered unknown fields __typename in
objectMetadaItem viewGroup`,
### Fixed by ignoring unknown internal fields
## Provided both relation `view` and `viewId`
`Should never provide relation mutation through anything else than the
fieldId e.g companyId and not company, encountered: view`
### Fixed by sending only `viewId` to `createManyRecords` in
`usePersistViewGroupRecords.ts`
# Introduction
Avoid having multiple `isDefined` definition across our pacakges
Also avoid importing `isDefined` from `twenty-ui` which exposes a huge
barrel for a such little util function
## In a nutshell
Removed own `isDefined.ts` definition from `twenty-ui` `twenty-front`
and `twenty-server` to move it to `twenty-shared`.
Updated imports for each packages, and added explicit dependencies to
`twenty-shared` if not already in place
Related PR https://github.com/twentyhq/twenty/pull/9941
Lucas Bordeau