## Summary
- Renames `ObjectMetadataItem` to `EnrichedObjectMetadataItem` across
the entire frontend (~440 files) to clarify that this type includes
derived fields (`readableFields`, `updatableFields`, nested `fields[]`,
`indexMetadatas[]`) computed at read time from the metadata store
- Creates `splitObjectMetadataGqlResponse` that goes directly from a
GraphQL `ObjectMetadataItemsQuery` response to flat store items
(combining the old
`mapPaginatedObjectMetadataItemsToObjectMetadataItems` +
`splitObjectMetadataItemWithRelated` two-step flow into one call)
- Removes `ObjectMetadataItemWithRelated` type and all "WithRelated"
naming
- Renames `generatedMockObjectMetadataItems` to
`generateTestEnrichedObjectMetadataItemsMock` to make it clear this is
test-only enriched data
- Deletes `useLoadMockedObjectMetadataItems` hook (consolidated into
`useLoadMockedMinimalMetadata`)
- Ensures nothing destined for the metadata store computes
`readableFields`/`updatableFields` (preventing the localStorage bloat
from #18809)
## Type hierarchy (before → after)
**Before:**
```
ObjectMetadataItemsQuery → mapPaginated → ObjectMetadataItemWithRelated → enrich → ObjectMetadataItem
→ split → FlatObjectMetadataItem (store)
```
**After:**
```
ObjectMetadataItemsQuery → splitObjectMetadataGqlResponse → FlatObjectMetadataItem (store)
→ mapPaginated + enrich (tests only) → EnrichedObjectMetadataItem
```
## Test plan
- [x] `npx nx typecheck twenty-front` passes
- [x] `npx nx test twenty-front` passes (767 suites, 4505 tests)
- [x] `npx nx lint twenty-front` passes
- [ ] CI checks pass
Made with [Cursor](https://cursor.com)
- Remove feature flag
- Remove legacy methods in file-upload and file-service
- Migrate AI Chat to new file management
---------
Co-authored-by: Charles Bochet <charles@twenty.com>
## PR Description
- Uses `expr-eval` to enable front components (SDK plugins) to define
conditional availability as declarative expressions.
- Moves shared types and constants to `twenty-shared`
- Introduces a `conditionalAvailabilityExpression` field on
`CommandMenuItemEntity`, allowing command menu items to store an
`expr-eval` compatible expression string that is evaluated against a
CommandMenuContext to determine if the item should be shown.
- Creates an esbuild transform plugin
`conditional-availability-transform-plugin` in `twenty-sdk` that
converts TypeScript conditional availability expressions into
`expr-eval` compatible syntax at build time, so SDK developers can write
natural TS expressions that get transformed to evaluable strings.
- Removes deprecated `forceRegisteredActionsByKey` state and its usage.
- Creates `useCommandMenuContext` hook that builds the full
`CommandMenuContext` object from React state, which is then passed to
`useCommandMenuItemFrontComponentActions` for evaluating conditional
availability expressions.
- Migration command
- Check IS_FILES_FIELD_MIGRATED:false
- Check or create avatarFile field
- Fetch all people with avatarUrl
- Move (Copy/move) file in storage
- Create core.file record
- Update person record
- bonus : attachment migration : fullPath > file (same logic)
- BE logic
- Add avatarFile field on person
- FE logic
- Adapt logic to upload on/display avatarFile data
The whole imageIdentifier logic will be done later
## Problem
With Twenty usage growing, we are facing challenges on server side to
respond to the demand. The current bottleneck we are facing is server
CPU.
While investigating CPU performances, we figured out that loading all
relation fields was the biggest issue.
### About graphql query response size
**Example:** on `People Index` Table page:
we query: `person.company` and we query **all non relation fields** on
company relation field.
`{ person { company { id, name, domainName, employees, address ... } }`
However we only need **company.id, company.name and company.domainName**
to be able to render the Company Chip in the company column (RELATION)
in the table.
`{ person { company { id, name, domainName } }`
We initially assumed that the querying all non relation fields was not
an issue because it was not adding additional load on database (which is
also partially true: if a field is containing a huge json this will add
load on postgres as data needs to be transfered)
This assumption is wrong on CPU side:
- when loading one to many relations (company.people), this quickly ads
up and we end up with response of 20kB quite quickly, even without
adding any custom field on person.
- even when loading a many to one relation, if the user is storing big
data in a given field (note.body for instance, or workflowRun.state),
this starts also being an issue.
- Worst case scenario are starting to happen: on a production workspace
with 20 active workflows. Loading workflow table while displaying
workflow runs commands (20 workflows x 60 workflowRuns x
workflowRun.state which is a big JSON) result in a response of... 125MB.
### Why is it an issue?
When yoga (graphql engine) parses a response to send it back in server
response, it's using `JSON.parse `(or stringify depending on the case).
Parsing data is CPU instensive (as you need to validate, transform) and
this is more or less `O(n)`.
This means returning 125MB is very intense on the CPU and will likely
use 100% of the CPU for ~1sec in our production servers.
NodeJs (nodeV8) is single threaded. It's able to process multiple
requests in parallel but to do that, it will cut them in "microTasks"
and process each microTasks (that can belong to different requests) one
after the other. Exactly like a single threaded CPU would allocate some
time to a process and then to the next one, etc.
This means that this big response request will actually block the other
request.
As a result, all requests are slow and our infrastructure is multi
tenant so some customers are impacting others.
This is even a bigger issue when our health checks start to fail and
containers are starting being considered as unhealthy and killed by the
orchestrator
## How to fix this
1. On Front end side, we should only query what we need. In this PR, I'm
forcing the relations to only query: `id`, `labelIdentifier`,
`imageIdentifier`
2. (later) as we are API first, we also need to do something to protect
the servers from this side too. To do that, Graphql APIs can associate a
cost to each graphql field (maybe 1 for a test fixed, 2 for a JSON
field, 10 for relations, etc...) and we can throttle based on that.
## Changes in this PR
It's mainly about refactor the tooling to generate `RecordGqlFields`
(`generateDepthOneGqlFields`). this tooling is used to generate the list
of fields we want to query. Most of the time we want to query a record
with one level of nesting.
1) Refactor to remove duplicate code => `generateDepthOne` become
`generateDepth` and can handle both `depth = 0 | 1`
2) Introduce `generateDepthRecordGqlFieldsFromFields.ts` which is the
base and can give you a list of gqlFields based on FieldMetadataItems
3) Introduce `generateDepthRecordGqlFieldsFromObject` which is a
shortcut for an object
4) Introduce `generateDepthRecordGqlFieldsFromRecords` which is the
intersection between `generateDepthRecordGqlFieldsFromObject` and a
given record (useful for cache tooling)
5) Replace all usages + introduce a hook
`useGenerateDepthRecordGqlFieldsFromObject` to ease usage
Charles Bochet