## Recoil → Jotai progressive migration: infrastructure +
ChipFieldDisplay
### Benchmark
In the beginning, there was no hope:
<img width="1180" height="948" alt="image"
src="https://github.com/user-attachments/assets/f8635991-52e6-4958-8240-6ba7214132b2"
/>
Then the hope was reborn
<img width="2070" height="948" alt="image"
src="https://github.com/user-attachments/assets/be1182b9-1c8d-4fdc-ab4c-1484ad74449d"
/>
### Approach
We introduce a **V2 state management layer** backed by Jotai that
mirrors the existing Recoil API, enabling component-by-component
migration without a big-bang rewrite.
#### V2 API (Jotai-backed, Recoil-ergonomic)
- `createStateV2` / `createFamilyStateV2` — drop-in replacements for
`createState` / `createFamilyState`, returning wrapper types over Jotai
atoms
- `useRecoilValueV2`, `useRecoilStateV2`, `useFamilyRecoilValueV2`, etc.
— thin wrappers around Jotai's `useAtomValue` / `useAtom` / `useSetAtom`
- A shared `jotaiStore` (via `createStore()`) passed to a
`<JotaiProvider>` wrapping `<RecoilRoot>`, also accessible imperatively
for dual-writes
#### Dual-write bridge for progressive migration
For state shared between migrated and non-migrated components, we use
**dual-write**: writers update both the Recoil atom and the Jotai V2
atom (via `jotaiStore.set()`). This avoids sync components or extra
subscriptions.
Write sites updated: `useUpsertRecordsInStore`, `useSetRecordTableData`,
`ListenRecordUpdatesEffect`, `RecordShowEffect`,
`useLoadRecordIndexStates`, `useUpdateObjectViewOptions`.
#### First migration: ChipFieldDisplay render path
- `useChipFieldDisplay` → reads `recordStoreFamilyStateV2` via
`useFamilyRecoilValueV2` (was `useRecoilValue(recordStoreFamilyState)`)
- `RecordChip` → reads `recordIndexOpenRecordInStateV2` via
`useRecoilValueV2` (was `useRecoilValue(recordIndexOpenRecordInState)`)
- `Avatar` (twenty-ui) and event handlers (`useOpenRecordInCommandMenu`)
left on Recoil — not on the render path / in a different package
#### Pattern for migrating additional state
1. Create V2 atom: `createStateV2` or `createFamilyStateV2`
2. Add `jotaiStore.set(v2Atom, value)` at each write site
3. Switch readers to `useRecoilValueV2(v2Atom)`
4. Once all readers are migrated, remove the Recoil atom and dual-writes
#### Why not jotai-recoil-adapter?
Evaluated
[jotai-recoil-adapter](https://github.com/clockelliptic/jotai-recoil-adapter)
— not production-ready (21 open issues, no React 19, forces providerless
mode, missing types). We built a purpose-built thin layer instead.
## Reduce type leakage between GraphQL schemas
### Why
Twenty runs two separate GraphQL schemas: **core** and **metadata**.
NestJS's `@nestjs/graphql` uses a global `TypeMetadataStorage` that
accumulates all decorated types across all modules. When each schema is
built, every registered type leaks into both schemas regardless of which
module it belongs to.
This means the core schema's generated TypeScript
(`generated/graphql.ts`) contained ~2,700 lines of types that only
belong to the metadata schema (and vice versa). This creates confusion
about type ownership, inflates generated code, and makes it harder to
reason about which API surface each schema actually exposes.
### How
**1. Patch `@nestjs/graphql` to support schema-scoped type resolution**
- **(Already done)** Added a `resolverSchemaScope` option to
`GqlModuleOptions`, allowing each schema to declare a scope (e.g.
`'metadata'`)
- `ResolversExplorerService` now filters resolvers by a
`RESOLVER_SCHEMA_SCOPE` metadata key, so each schema only sees its own
resolvers
- `GraphQLSchemaFactory` now performs a **reachability walk**
(`computeReachableTypes`) starting from scoped resolver return types and
arguments, only including types that are transitively referenced —
handling unions, interfaces, and prototype chains
- Type definition storage and orphaned reference registry are cleared
between schema builds to prevent cross-contamination
**2. Register `ClientConfig` as orphaned type in metadata schema**
Since `ClientConfig` is needed in the metadata schema but not directly
returned by a resolver, it's explicitly declared via
`buildSchemaOptions.orphanedTypes`.
**3. Regenerate frontend types and fix imports**
- `generated/graphql.ts` shrank by ~2,700 lines (types moved to where
they belong)
- `generated-metadata/graphql.ts` gained types like `ClientConfig` that
were previously missing
- ~500 frontend files updated to import from the correct generated file
## Summary
- Remove `latestVersion` and `publishedVersions` columns from
`LogicFunction` entity
- Remove version parameter from logic function execution, build, and
source code retrieval
- Update all related services, DTOs, utilities, and tests
- Add database migration to drop the version columns
## Summary
Adds `app:build` command as a one-shot version of `app:dev` - builds
manifest, functions, and front components once then exits (no watching).
**Changes:**
- Added `watch` option to `FunctionsWatcher` and
`FrontComponentsWatcher` to support both watch and one-shot modes
- Created `AppBuildCommand` reusing the same build logic as `app:dev`
with `watch: false`
- Added integration tests for `app:build` on both `rich-app` and
`root-app`
- Updated manifest types: `handlerPath` → `sourceHandlerPath` +
`builtHandlerPath`, `componentPath` → `sourceComponentPath` +
`builtComponentPath`
- Updated test app `package.json` scripts to match `create-twenty-app`
template
## Context
- Add RLS entitlement to billing
- Check value in the backend (for RLS predicate entity
queries/mutations)
- Expose billingEntitlements to the API inside currentWorkspace to check
available features to the workspace and display the role components
accordingly
- Cleanup RLS when plan changes back to one without RLS.
This should cover almost everything, imho we don't need to check in the
ORM because => We can't create RLS without the correct PLAN and
switching back to a PLAN without RLS deletes existing RLS through
stripes webhooks
## Summary
Fixes#17101
When self-hosting TwentyCRM, users can now set workspace custom domains
without requiring CLOUDFLARE_API_KEY to be configured. This enables
manual DNS configuration for those not using Cloudflare.
## Changes
- Added isCloudflareConfigured method to DnsManagerService
- Modified all Cloudflare-dependent methods to gracefully handle missing
configuration
- Added getManualDnsRecords helper that provides DNS configuration
instructions when Cloudflare is not available
- isHostnameWorking returns true in manual mode, allowing the domain to
be saved and enabled
- Added comprehensive tests for non-Cloudflare scenarios
## Behavior
When CLOUDFLARE_API_KEY is set: Works exactly as before with automatic
Cloudflare provisioning
When CLOUDFLARE_API_KEY is NOT set:
- Custom domain can be saved to the database
- User receives manual DNS configuration instructions
- Domain is marked as working, user is responsible for external DNS and
TLS configuration
## Testing
Added tests covering non-Cloudflare scenarios. Full test suite requires
Docker which was not run locally.
<!-- CURSOR_SUMMARY -->
---
> [!NOTE]
> Introduces a Cloudflare integration feature flag and wires it through
server and front to gate the custom domain UI.
>
> - Server: adds `isCloudflareIntegrationEnabled` to `ClientConfig`,
computed from `CLOUDFLARE_API_KEY` and `CLOUDFLARE_ZONE_ID` in
`client-config.service`; updates GraphQL schema and unit tests.
> - Frontend: adds `isCloudflareIntegrationEnabledState`, extends
`ClientConfig` type, sets the flag in `useClientConfig`, and
conditionally renders `SettingsCustomDomain` in `SettingsDomain` when
enabled.
> - Updates mocked client config to include the new flag.
>
> <sup>Written by [Cursor
Bugbot](https://cursor.com/dashboard?tab=bugbot) for commit
0c76dde03b1ae940a9dae65e8673c3ba4cfec9dc. This will update automatically
on new commits. Configure
[here](https://cursor.com/dashboard?tab=bugbot).</sup>
<!-- /CURSOR_SUMMARY -->
---------
Signed-off-by: Vedant Madane <6527493+VedantMadane@users.noreply.github.com>
Co-authored-by: Félix Malfait <felix@twenty.com>
Co-authored-by: Félix Malfait <felix.malfait@gmail.com>
## Summary
Moves the custom ESLint rules from `tools/eslint-rules` to
`packages/twenty-eslint-rules` for better organization within the
monorepo packages structure.
## Changes
- Move `eslint-rules` from `tools/` to `packages/twenty-eslint-rules`
- Use `loadWorkspaceRules` from `@nx/eslint-plugin` to load custom rules
- Update all ESLint configs to use the `twenty/` rule prefix instead of
`@nx/workspace-`
- Update `project.json`, `jest.config.mjs` with new paths
- Update `package.json` workspaces and `nx.json` cache inputs
- Update Dockerfile reference
## Technical Details
The custom ESLint rules are now loaded using Nx's `loadWorkspaceRules`
utility which:
- Handles TypeScript transpilation automatically
- Allows loading workspace rules from any directory
- Provides a cleaner approach than the previous `@nx/workspace-`
convention
## Testing
- Verified all 17 custom ESLint rules load correctly from the new
location
- Verified linting works on dependent packages (twenty-front,
twenty-server, etc.)
## Context
Now that RLS predicates are applied, creating a record through the FE
(which is empty by default) is failing if your role has predicates and
your input does not respect them (which will always be true since, as
said above, input will be pretty much empty)
## Implementation
- Moved isMatching* filters to twenty-shared
- Implemented isMatchingRlsPredicates utils in the backend (ORM) to
check before insertion/update if the record is matching the current user
role Rls predicates, reusing the isMatching* filters utils moved to
twenty-shared
- Frontend now applies RLS predicates before creating a new record
(similarly to what we do with view filters)
Note:
It seems composite were not properly handled with view-filter insertion
logic, since I'm reusing the util for now, the issue remains for RLS and
will need to be addressed
## Summary
This PR introduces row-level security (RLS) permissions for roles in the
frontend, allowing fine-grained access control at the record level.
Users can now define permission rules that determine which specific
records a role can access based on dynamic conditions and filters.
## What's Changed
Implemented UI for configuring record-level permissions on object
permissions screens
Added support for defining permission predicates using filter conditions
(similar to advanced filters)
Introduced variable picker for dynamic permission rules (e.g., "me"
context for user-specific access)
Built predicate conversion layer to sync UI state with backend
permission structure
Extended GraphQL schema with mutations for upserting row-level
permission predicates
Fixed handling of orphaned RLS groups to prevent data inconsistencies
Added enterprise key validation for RLS features
The implementation enables scenarios like "users can only see their own
records" or "users can access records associated with their team."
<img width="687" height="702" alt="Screenshot 2026-01-09 at 23 07 02"
src="https://github.com/user-attachments/assets/33fe736e-6cbf-40bd-b2eb-c8a90c8d21bc"
/>
---------
Co-authored-by: Charles Bochet <charles@twenty.com>
Upgraded from 8.6.15 to 9.1.17 in two steps:
- 8.6.15 -> 9.0.0
- 9.0.0 -> 9.1.17
I had to disable `storybook-addon-cookie` since it is not supported for
Storybook 9. However, I do intend to upgrade to Storybook 10 when this
is merged, so we can replace the aforementioned add-on with this fork
specifically created to support Storybook 10 and above:
https://www.npmjs.com/package/@storybook-community/storybook-addon-cookie.
Additionally, once we upgrade to Version 10 successfully, I will start
looking into integrating the official Vitest add-on.
## Context
The feature has not been maintained for more than a year and was never
officially launched.
This PR removes its code due to the upcoming refactoring of the
sync-metadata that will break its logic if not handled properly and it
would be too costly for the time being.
TODO:
- remove isRemote from object/field metadata
# Introduction
In this pull-request we're refactoring the page layout widget
configuration entity to be containing its discriminated key simplifying
underlying code and maintainability
- Made the configuration and title non nullable
- Introduced a generic predicate for the `widgetConfigurationType`
- Upgraded command to remove `graphType` and insert new
`configurationType` to existing entries
- Migrated frontend to new type system
---------
Co-authored-by: bosiraphael <raphael.bosi@gmail.com>
Co-authored-by: Raphaël Bosi <71827178+bosiraphael@users.noreply.github.com>
## Summary
This PR enforces the use of `@/` alias for imports instead of relative
parent imports (`../`).
## Changes
### ESLint Configuration
- Added `no-restricted-imports` pattern in `eslint.config.react.mjs` to
block `../*` imports with the message "Relative parent imports are not
allowed. Use @/ alias instead."
- Removed the non-working `import/no-relative-parent-imports` rule
(doesn't work properly in ESLint flat config)
### VS Code Settings
- Added `javascript.preferences.importModuleSpecifier: non-relative` to
`.vscode/settings.json` (TypeScript setting was already there)
### Code Fixes
- Fixed **941 relative parent imports** across **706 files** in
`packages/twenty-front`
- All `../` imports converted to use `@/` alias
## Why
- Consistent import style across the codebase
- Easier to move files without breaking imports
- Better IDE support for auto-imports
- Clearer understanding of where imports come from
Fixes https://github.com/twentyhq/twenty/issues/16624
Original issue:
- while persisting a field (calling useUpdateOne), the response from the
backend is missing the taskTargets many to many (same for note). As we
optimistically update the cache, we lose the "Relations" in the UI
- I'm changing the behavior of useUpsertInRecordStore to accept
recordGqlFields to only update the fields we want in the record store
(this way, we are not losing the targets information in our case)
Fixes https://github.com/twentyhq/core-team-issues/issues/1382
Current issue : all step output schemas are computed and stored on
backend side. Which means that, when the database schema is updated -
like a field creation - steps needs to be deleted an recreated. Which is
invisible to users.
Solution : schema generation is moved on frontend side
1. Coming on the page the first time, the schema is populated for all
steps except a few ones that are handled differently (Code, Webhook,
http node, Agent)
2. A separated state allow to determine if a step needs a recomputation.
3. The user only needs a refresh to see the whole schema re-computed
Follow-up:
- check if remaining backend steps could be moved to runtime
computation. But Code will still require storage.
- Clean backend service that is not used anymore
Should be merged once https://github.com/twentyhq/twenty/pull/16206 has
been released + command run to prod
In this PR
- Remove usage of viewGroup.fieldMetadataId, both in BE and FE states.
- But we still need to properly populate it until we fully remove
viewGroup.fieldMetadataId from db and ORM entity (upcoming 3rd PR out of
3). fieldMetadataId was removed from CoreViewGroup type and
CreateViewGroupInput and is determined BE-side based on the associated
view's mainGroupByFieldMetadataId. **I expect this means a downtime on
viewGroup creation, until both FE and BE are deployed and cache is
flushed.** This seems acceptable to me as it only regards viewGroup
creation.
- this information is replaced by view.mainGroupByFieldMetadataID
- Handle view group creation, update and deletion in the BE as a
side-effect of a view creation, update or deletion. Optimistic effects
are still used
- Add validation at view creation or update regarding
mainGroupByFieldMetadata
Left to do in 3rd PR
- Remove viewGroup.fieldMetadataId from db and ORM entity
- Restore feature allowing to update an existing grouped view's group by
field (already OK on BE side but need to rebuild FE optimistic)
In this PR (1/3)
- introduce view.mainGroupByFieldMetadataId as the new reference
determining which fieldMetadataId is used in a grouped view, in order to
deprecate viewGroup.fieldMetadataId which creates inconsistencies.
view.mainGroupByFieldMetadataId is now filled at every view creation,
though not in use yet.
- Introduce a command to backfill view.mainGroupByFieldMetadataId for
existing views + delete all viewGroup.fieldMetadataId with a
fieldMetadataId that is not view.mainGroupByFieldMetadataId. (It should
concern 37 active workspaces)
- Temporarily disable the option to change a grouped view's
fieldMetadataId as for now it creates inconsistencies. This feature can
be reintroduced when we have done the full migration.
In a next PR
- (2/3) use view.mainGroupByFieldMetadataId instead of
viewGroup.fieldMetadataId. In FE we may keep viewGroup.fieldMetadataId
as a state (TBD). View groups will now be created / deleted as a side
effect of view's mainGroupByFieldMetadataId update.
- (3/3) remove viewGroup.fieldMetadataId
---------
Co-authored-by: Charles Bochet <charles@twenty.com>
## Overview
This PR replaces the dynamic agent handoff system with a more
predictable planning-based router that decides upfront how to handle
multi-agent coordination.
## Major Changes
### 🔄 Architecture Shift: Handoffs → Planning
**Removed:**
- `AgentHandoffEntity` and handoff tracking system
- `AgentHandoffService` and `AgentHandoffExecutorService`
- Dynamic agent-to-agent transfers during execution
- Handoff tool generation and description templates
**Added:**
- `AiRouterService` with two strategies: `simple` (single agent) and
`planned` (multi-agent)
- `AgentPlanExecutorService` for executing multi-step plans
- Plan validation (cycle detection, dependency resolution)
- `UnifiedRouterResult` type with discriminated union
### 🤖 New Standard Agents
Added two new specialized agents:
- **Researcher Agent**: Web search, fact-finding, competitive
intelligence
- **Code Agent**: TypeScript function generation for serverless
workflows
### 🏗️ Router Refactoring (Latest)
Split router responsibilities into focused services:
- `AiRouterStrategyDeciderService`: Decides simple vs planned strategy
- `AiRouterPlanGeneratorService`: Generates and validates execution
plans
- `AiRouterService`: Coordinates between services (reduced from 426→275
lines)
### ⚙️ Configuration Improvements
- Added `outputStrategy` to agent definitions (`direct` vs `synthesize`)
- Removed hardcoded special cases for workflow-builder
- Added `plannerModel` field to workspace entity
- Increased `MAX_STEPS` from 10 to 25 for complex workflows
### 📝 Agent Prompt Refinements
Significantly simplified prompts for better clarity:
- Workflow Builder: 51→36 lines
- Helper: 49→28 lines
- Data Manipulator: Enhanced with sorting guidance
### 🔍 Enhanced Debugging
- Plan reasoning and step count in data message parts
- Router debug info with token usage tracking
- Better logging throughout execution pipeline
## Benefits
1. **Simpler Mental Model**: Router decides upfront vs dynamic transfers
2. **Better Predictability**: Users see the plan before execution
3. **Cleaner Architecture**: SRP with focused services
4. **Configuration Over Code**: Agent behavior via config, not hardcoded
logic
5. **Plan Validation**: Catches invalid dependencies and cycles
## Migration Notes
- Database migration removes `agentHandoff` table
- Adds `plannerModel` column to workspace table
- No API breaking changes (agent endpoints unchanged)
## Testing
- Integration tests updated to remove handoff dependencies
- Agent tool test utilities simplified
- Plan validation covered by new logic
## Next Steps (Future PRs)
- Parallel execution of independent plan steps
- Dynamic re-planning based on results
- Plan caching for common routing patterns
- Error recovery strategies in plan executor
# Introduction
Two things:
- Enforcing non nullable workspace custom application Id for any
workspace
- Fixing front non editable data models following
https://github.com/twentyhq/twenty/pull/15911 that associate any custom
entities to an applicationId. The front was putting everything as
readonly when under an app ( we will have to handle the twenty standard
application in the future too )
## Fallback
### Migration
The non nullable migration will fail when released, that's why it's
being swallowed and re-run in an upgrade command post workspace custom
application creation for those that miss one. Allowing the migration to
pass in the end
The typeorm migration still need to exists for any new workspaces
### GetCurrentUser
In order to dynamically display isReadOnly in data model settings we're
fetching the workspaceCustomApplicationId through the `getCurrentUser`
If not fallback this endpoint would throw until we're handling existing
workspaces that do not have a custom workspace application
The fallback should be removed post release
## Overview
This PR strengthens our permission system by introducing more granular
role-based access control across the platform.
## Changes
### New Permissions Added
- **Applications** - Control who can install and manage applications
- **Layouts** - Control who can customize page layouts and UI structure
- **AI** - Control access to AI features and agents
- **Upload File** - Separate permission for file uploads
- **Download File** - Separate permission for file downloads (frontend
visibility)
### Security Enhancements
- Implemented whitelist-based validation for workspace field updates
- Added explicit permission guards to core entity resolvers
- Enhanced ESLint rule to enforce permission checks on all mutations
- Created `CustomPermissionGuard` and `NoPermissionGuard` for better
code documentation
### Affected Components
- Core entity resolvers: webhooks, files, domains, applications,
layouts, postgres credentials
- Workspace update mutations now use whitelist validation
- Settings UI updated with new permission controls
### Developer Experience
- ESLint now catches missing permission guards during development
- Explicit guard markers make permission requirements clear in code
review
- Comprehensive test coverage for new permission logic
## Testing
- ✅ All TypeScript type checks pass
- ✅ ESLint validation passes
- ✅ New permission guards properly enforced
- ✅ Frontend UI displays new permissions correctly
## Migration Notes
Existing workspaces will need to assign the new permissions to roles as
needed. By default, all new permissions are set to `false` for non-admin
roles.
Here is what the PR does:
- Surface password state in validatePasswordResetToken, returning
hasPassword so the client can tell whether a user is setting or changing
their password.
- Consume that flag throughout the front end (mock data, stories,
GraphQL types) and update the Reset/Set Password modal to swap the
heading/button label and success toast accordingly.
- After a successful password set/reset, immediately update the
logged-in user’s hasPassword flag so the Settings screen reflects the
new state without a reload.
Modal has two states now - reset password modal uses change password
state since it made intuitive sense.
<p align="center">
<img width="404" height="397" alt="image"
src="https://github.com/user-attachments/assets/c54cc581-1248-4395-833d-0202758e1947"
/>
</p>
<p align="center">
<img width="403" height="393" alt="image"
src="https://github.com/user-attachments/assets/d8a39a95-27e6-4037-86f2-1f74176002ba"
/>
</p>
---------
Co-authored-by: Félix Malfait <felix.malfait@gmail.com>
Closes [Core Issue
#1772](https://github.com/twentyhq/core-team-issues/issues/1772).
<!-- CURSOR_SUMMARY -->
---
> [!NOTE]
> Introduces SSO bypass with a new permission flag and workspace-level
provider toggles, enabling permitted users to log in via
Google/Microsoft/Password when SSO-only, with backend enforcement and
frontend UI/hooks/queries.
>
> - **Backend**:
> - **Permission & Enforcement**: Add `PermissionFlagType.SSO_BYPASS`;
update `AuthService` to allow login via non-SSO providers when workspace
bypass is enabled and user has `SSO_BYPASS`.
> - **Workspace Model**: Add `isGoogleAuthBypassEnabled`,
`isMicrosoftAuthBypassEnabled`, `isPasswordAuthBypassEnabled`
(migration, entity, update input, service validation).
> - **Public API**: Extend `PublicWorkspaceDataOutput` with
`authBypassProviders`; resolver computes it; permissions defaults
include `SSO_BYPASS`.
> - **Frontend**:
> - **GraphQL/State**: Generate new types/fields; add
`authBypassProviders` to `GetPublicWorkspaceDataByDomain`; new states
`workspaceAuthBypassProvidersState`, `workspaceBypassModeState`.
> - **Auth UI/Logic**: Add `useWorkspaceBypass`; update sign-in form and
footer to offer "Bypass SSO" and use merged providers when enabled;
remove auto-redirect when single SSO.
> - **Settings**: Add Security section to toggle bypass methods per
provider; conditionally show Change Password via `useCanChangePassword`.
> - **Tests/Mocks**: Update mocks and tests to include bypass
flags/providers.
>
> <sup>Written by [Cursor
Bugbot](https://cursor.com/dashboard?tab=bugbot) for commit
8c393b2bad387fb6e8b8f40027f8637dd6e85723. This will update automatically
on new commits. Configure
[here](https://cursor.com/dashboard?tab=bugbot).</sup>
<!-- /CURSOR_SUMMARY -->
---------
Co-authored-by: Félix Malfait <felix.malfait@gmail.com>
Now that all existing and new workspaces have the following syncStage:
- CALENDAR_EVENT_LIST_FETCH_PENDING
- MESSAGE_LIST_FETCH_PENDING
We can fully deprecate the old FULL_CALENDAR_EVENT_LIST_FETCH_PENDING
and PARTIAL_CALENDAR_EVENT_LIST_FETCH_PENDING (full vs partial is now
directly inferred from the presence of a cursor)
Adds intelligent routing system that automatically selects the best
agent for user queries based on conversation context.
### Changes:
- Added `routerModel` column to workspace table for configurable router
LLM selection
- Implemented `RouterService` with conversation history analysis and
agent matching logic
- Created router settings UI in AI Settings page with model dropdown
- Removed agent-specific thread associations - threads are now
agent-agnostic
- Added real-time routing status notification in chat UI with shimmer
effect
- Removed automatic default assistant agent creation
- Renamed GraphQL operations from agent-specific to generic (e.g.,
`agentChatThreads` → `chatThreads`)
---------
Co-authored-by: Félix Malfait <felix.malfait@gmail.com>
Co-authored-by: Félix Malfait <felix@twenty.com>
This PR fixes two bugs :
- Data loading not working between 60 and 120 records on an object
- Table not refreshing when creating a record in an empty table.
Fixes https://github.com/twentyhq/twenty/issues/15196
Closes#14726
### Added
- `trashRetentionDays` field to workspace entity (default: 14 days)
- Automated trash cleanup using BullMQ jobs
- Daily cron (00:10 UTC) that enqueues cleanup jobs for all active
workspaces
- Per-workspace limit: 100k records deleted per day
- Calendar-based retention: records deleted on day X are cleaned up X+14
days later (at midnight UTC boundaries)
### Architecture
- **Cron (WorkspaceTrashCleanupCronJob):** Runs daily, enqueues jobs in
parallel for all workspaces
- **Job (WorkspaceTrashCleanupJob):** Processes individual workspace
cleanup
- **Service (WorkspaceTrashCleanupService):** Discovers tables with
`deletedAt`, deletes old records with quota enforcement
- **Command:** `npx nx run twenty-server:command
cron:workspace:cleanup-trash` to register the cron
### Testing
- Unit tests for service with 100% coverage of public API
- Tested quota enforcement, error handling, and edge cases
---------
Co-authored-by: Félix Malfait <felix.malfait@gmail.com>
Avoid fetching full steps and trigger for versions that are not the
current version. Because those won't be used anyway. Better for
performances.
Only difficulty was for the `createDraft` mutation. I needed to return
the full created version so I can store it in cache and use it as new
`currentVersion`. Otherwise the current version is considered as
incomplete for a short time, since workflow is fetched separately from
the current version.
---------
Co-authored-by: Charles Bochet <charles@twenty.com>
## Problem
With Twenty usage growing, we are facing challenges on server side to
respond to the demand. The current bottleneck we are facing is server
CPU.
While investigating CPU performances, we figured out that loading all
relation fields was the biggest issue.
### About graphql query response size
**Example:** on `People Index` Table page:
we query: `person.company` and we query **all non relation fields** on
company relation field.
`{ person { company { id, name, domainName, employees, address ... } }`
However we only need **company.id, company.name and company.domainName**
to be able to render the Company Chip in the company column (RELATION)
in the table.
`{ person { company { id, name, domainName } }`
We initially assumed that the querying all non relation fields was not
an issue because it was not adding additional load on database (which is
also partially true: if a field is containing a huge json this will add
load on postgres as data needs to be transfered)
This assumption is wrong on CPU side:
- when loading one to many relations (company.people), this quickly ads
up and we end up with response of 20kB quite quickly, even without
adding any custom field on person.
- even when loading a many to one relation, if the user is storing big
data in a given field (note.body for instance, or workflowRun.state),
this starts also being an issue.
- Worst case scenario are starting to happen: on a production workspace
with 20 active workflows. Loading workflow table while displaying
workflow runs commands (20 workflows x 60 workflowRuns x
workflowRun.state which is a big JSON) result in a response of... 125MB.
### Why is it an issue?
When yoga (graphql engine) parses a response to send it back in server
response, it's using `JSON.parse `(or stringify depending on the case).
Parsing data is CPU instensive (as you need to validate, transform) and
this is more or less `O(n)`.
This means returning 125MB is very intense on the CPU and will likely
use 100% of the CPU for ~1sec in our production servers.
NodeJs (nodeV8) is single threaded. It's able to process multiple
requests in parallel but to do that, it will cut them in "microTasks"
and process each microTasks (that can belong to different requests) one
after the other. Exactly like a single threaded CPU would allocate some
time to a process and then to the next one, etc.
This means that this big response request will actually block the other
request.
As a result, all requests are slow and our infrastructure is multi
tenant so some customers are impacting others.
This is even a bigger issue when our health checks start to fail and
containers are starting being considered as unhealthy and killed by the
orchestrator
## How to fix this
1. On Front end side, we should only query what we need. In this PR, I'm
forcing the relations to only query: `id`, `labelIdentifier`,
`imageIdentifier`
2. (later) as we are API first, we also need to do something to protect
the servers from this side too. To do that, Graphql APIs can associate a
cost to each graphql field (maybe 1 for a test fixed, 2 for a JSON
field, 10 for relations, etc...) and we can throttle based on that.
## Changes in this PR
It's mainly about refactor the tooling to generate `RecordGqlFields`
(`generateDepthOneGqlFields`). this tooling is used to generate the list
of fields we want to query. Most of the time we want to query a record
with one level of nesting.
1) Refactor to remove duplicate code => `generateDepthOne` become
`generateDepth` and can handle both `depth = 0 | 1`
2) Introduce `generateDepthRecordGqlFieldsFromFields.ts` which is the
base and can give you a list of gqlFields based on FieldMetadataItems
3) Introduce `generateDepthRecordGqlFieldsFromObject` which is a
shortcut for an object
4) Introduce `generateDepthRecordGqlFieldsFromRecords` which is the
intersection between `generateDepthRecordGqlFieldsFromObject` and a
given record (useful for cache tooling)
5) Replace all usages + introduce a hook
`useGenerateDepthRecordGqlFieldsFromObject` to ease usage
Storybook had issues. This PR intends to solve them
Main things to care :
I had to change the way the provider was given to the Calendar view. Not
sure how it may impact the final component, so if someone with some
knowledge about it could take a look that would be great
TODO:
Only one remaining test to fix : prefetch loading. Couldn't find out how
to solve this issue. When you open the story you see that the companies
are not fetched.
## Context
We are experiencing bad performance on Twenty. One of the root cause
hypothesis is that computing `currentUser.currentWorkspace.views` is CPU
consuming. Without views, the GetCurrentUser response is ~1000 lines.
With its ~10000 lines.
As graphql is going through all fields recursively this can be quite
heavy on CPU. We had a similar issues on ObjectMetadataItems 2 years ago
and came with storing the response in redis.
Note: I thought there was also a cache in RAM but this is not the case,
so to invalidate the cache we can just empty redis.
## How
- Extract getting all views from GetCurrentUser and update frontend to
perform both queries
- Add views to cached graphql operations
- invalidate the cache manually on view or related core entities update
/ create / delete / destroy
## Tests
I have tested a lot on v1
Closes https://github.com/twentyhq/core-team-issues/issues/1560
If viewId is defined in a groupBy query, we want to apply all filters of
the view to the query.
This required to move a lot of code from twenty-front to twenty-shared
to convert the filters as stored in the db into graphql filters,
applying the right combinations between filters etc., which was
previously only done in the FE.
This PR does not handle any field filters, it will be done in a later pr
This PR fixes https://github.com/twentyhq/twenty/issues/13022
It was due to a mouse leave event that worked for table without groups
but not with table with groups, the fix is to handle this event higher
up in the table.
This PR also fixes a bug that happened when hiding a column, the page
crashed because of an indexed array returning undefined that TS didn't
catch, linked to
https://github.com/twentyhq/core-team-issues/issues/1205.
Charles Bochet