Validations :
- relations count (in common api)
- oneToMany relation nested count (in common)
- requested fields count (in gql)
- root resolver count (in gql)
- root resolver duplicates (in gql)
- specific complexity for metadata / nesting count (in gql)
Fixes a regression on seeding introduced by
https://github.com/twentyhq/twenty/pull/16296.
In the seeding process an async function was used without awaiting. This
caused all the seeded configs to be empty.
Note: The need to await is because `transformRichTextV2Value` is async
due to `const { ServerBlockNoteEditor } = await
import('@blocknote/server-util');`. @etiennejouan do you know why is it
done this way? Is it to reduce bundle size? Would it be possible to
create another util which is not async and which imports
`@blocknote/server-util` at the top?
## Context
Call to incrementMetadataVersion is only done in sync-metadata (to be
deprecated) and migration runner v2. They both already invalidate the
cache so we were doing it twice which is quite heavy with all the
object/field/view/etc maps in it.
pipeline.exec() is a a blocking operation which means if we want to
write a large chunk of data (or many maps in our case) it will block
write operations and impact performances.
We prefer to simply call set multiple times in a promise.all, this is an
acceptable tradeoff
In this PR, we are adding a new command to reset a message channel.
We are also refactoring a bit cursor reset as we had multiple
implementations at different places in the code base
---------
Co-authored-by: Charles Bochet <charles@twenty.com>
fixes https://github.com/twentyhq/twenty/issues/16270
It happens with api user only.
Integration tests were ok because we expect a returned empty string
(check in test) even if null value is expected in db (not test). How can
we improve this ?
This is a temporary fix that needs to be revisited.
It seems that with the new enqueue system we are enqueuing jobs multiple
times due to race condition. We likely want to only consider job that
have been created more than x secs ago
## Problem
buildEntityMetadatas in GlobalWorkspaceOrmManager is computationally
expensive and was running on every executeInWorkspaceContext call. This
method uses
TypeORM's EntitySchemaTransformer and EntityMetadataBuilder to build
metadata for all workspace entities (30-50+ objects with many fields
each).
The resulting EntityMetadata[] is not serialisable which means it cannot
be cached in Redis because they contain:
- Circular references
- Functions/methods
- References to the DataSource instance
## Solution
Extended the workspace cache system to support local-only caching, then
created a cache provider for entityMetadatas.
## Implementation details
Updated @WorkspaceCache decorator (workspace-cache.decorator.ts)
- Added localOnly?: boolean option to skip Redis storage for
non-serializable data
Created WorkspaceEntityMetadatasCacheService
- Computes entity metadatas from DB to avoid race condition, this is
acceptable
Simplified GlobalWorkspaceOrmManager
- Now fetches entityMetadatas from cache instead of rebuilding on every
call
Updated Workspace migration runner - the only entry point where metadata
can change
- Now invalidate the new 'entityMetadata' local cache when
shouldIncrementMetadataGraphqlSchemaVersion is true (== field/object
mutations)
# Introduction
On production facing a migration error with
UpdateRoleTargetsUniqueConstraint1764329720503
```ts
Migration "UpdateRoleTargetsUniqueConstraint1764329720503" failed, error: could not create unique index "IDX_ROLE_TARGETS_UNIQUE_AGENT"
query: ROLLBACK
Error during migration run:
QueryFailedError: could not create unique index "IDX_ROLE_TARGETS_UNIQUE_AGENT"
at PostgresQueryRunner.query (/Users/paulrastoin/ws/twenty/node_modules/typeorm/driver/postgres/PostgresQueryRunner.js:219:19)
at process.processTicksAndRejections (node:internal/process/task_queues:105:5)
at async UpdateRoleTargetsUniqueConstraint1764329720503.up (/Users/paulrastoin/ws/twenty/packages/twenty-server/dist/database/typeorm/core/migrations/common/1764329720503-update-role-targets-unique-constraint.js:15:9)
at async MigrationExecutor.executePendingMigrations (/Users/paulrastoin/ws/twenty/node_modules/typeorm/migration/MigrationExecutor.js:225:17)
at async DataSource.runMigrations (/Users/paulrastoin/ws/twenty/node_modules/typeorm/data-source/DataSource.js:265:35)
at async Object.handler (/Users/paulrastoin/ws/twenty/node_modules/typeorm/commands/MigrationRunCommand.js:68:13) {
query: 'ALTER TABLE "core"."roleTargets" ADD CONSTRAINT "IDX_ROLE_TARGETS_UNIQUE_AGENT" UNIQUE ("workspaceId", "agentId")',
parameters: undefined,
driverError: error: could not create unique index "IDX_ROLE_TARGETS_UNIQUE_AGENT"
at /Users/paulrastoin/ws/twenty/node_modules/pg/lib/client.js:526:17
at process.processTicksAndRejections (node:internal/process/task_queues:105:5)
at async PostgresQueryRunner.query (/Users/paulrastoin/ws/twenty/node_modules/typeorm/driver/postgres/PostgresQueryRunner.js:184:25)
at async UpdateRoleTargetsUniqueConstraint1764329720503.up (/Users/paulrastoin/ws/twenty/packages/twenty-server/dist/database/typeorm/core/migrations/common/1764329720503-update-role-targets-unique-constraint.js:15:9)
at async MigrationExecutor.executePendingMigrations (/Users/paulrastoin/ws/twenty/node_modules/typeorm/migration/MigrationExecutor.js:225:17)
at async DataSource.runMigrations (/Users/paulrastoin/ws/twenty/node_modules/typeorm/data-source/DataSource.js:265:35)
at async Object.handler (/Users/paulrastoin/ws/twenty/node_modules/typeorm/commands/MigrationRunCommand.js:68:13) {
length: 314,
severity: 'ERROR',
code: '23505',
```
As several agent has duplicated role target in database
```sql
SELECT constraint_name, COUNT(*) AS duplicate_groups, SUM(duplicate_count - 1) AS rows_to_delete
FROM (
SELECT 'IDX_ROLE_TARGET_UNIQUE_API_KEY' AS constraint_name, COUNT(*) AS duplicate_count
FROM "core"."roleTargets" rt
WHERE rt."apiKeyId" IS NOT NULL
GROUP BY rt."workspaceId", rt."apiKeyId"
HAVING COUNT(*) > 1
UNION ALL
SELECT 'IDX_ROLE_TARGET_UNIQUE_AGENT', COUNT(*)
FROM "core"."roleTargets" rt
WHERE rt."agentId" IS NOT NULL
GROUP BY rt."workspaceId", rt."agentId"
HAVING COUNT(*) > 1
UNION ALL
SELECT 'IDX_ROLE_TARGET_UNIQUE_USER_WORKSPACE', COUNT(*)
FROM "core"."roleTargets" rt
WHERE rt."userWorkspaceId" IS NOT NULL
GROUP BY rt."workspaceId", rt."userWorkspaceId"
HAVING COUNT(*) > 1
) AS all_duplicates
GROUP BY constraint_name;
```
with constraint name, duplicated_groups, row_to_delete
`IDX_ROLE_TARGET_UNIQUE_AGENT 2507 7229`
Please note that only active or suspended workspaces contains duplicated
role target
## Fix
Introduced an upgrade command that will only keep the latest inserted
role target
Swallowing migration error on typeorm atomic migration ( still required
for self host new instances etc )
```ts
[Nest] 91886 - 12/03/2025, 2:56:28 PM LOG [DeduplicateRoleTargetsCommand] Running command on workspace SOME_WORKSPACE_ID 2587/2587
flatFieldMetadataMaps,flatIndexMaps,flatObjectMetadataMaps out of 298
query: SELECT version();
[Nest] 91886 - 12/03/2025, 2:56:29 PM LOG [DeduplicateRoleTargetsCommand] Command completed!
```
## Test
Tested through an extract in local, tested both the upgrade command and
the swallowed migration
test
## Description
When switching a view's visibility from WORKSPACE to UNLISTED, the code
in `view-v2.service.ts` correctly sets `createdByUserWorkspaceId` to the
current user (to prevent the view from disappearing). However, this
property was not included in `FLAT_VIEW_EDITABLE_PROPERTIES`, which
meant:
1. The comparison logic didn't detect the change
2. No update action was created for the property
3. The value was never persisted to the database
4. The view disappeared because it became UNLISTED with no owner
This fix adds `createdByUserWorkspaceId` to the editable properties so
the change is properly detected and persisted.
Might fix https://github.com/twentyhq/twenty/issues/15955
# Introduction
Closes https://github.com/twentyhq/core-team-issues/issues/1980
In this PR we migrate the agent from v1 to v2.
## New FlatRoleTargetByAgentIdMaps
Derivated the `flatRoleTargetMaps` to be building a
`flatRoleTargetByAgentIdMaps` to ease retrieving a roleId to associate
to an agent
## Coverage
Added strong coverage on both failing and successful CRU agents
operations
---------
Co-authored-by: Weiko <corentin@twenty.com>
- if >5000 workflows per hour, new ones should failed
- if >100 workflow per min, new ones should be set as not started.
Except manual trigger
- when enqueued, we check if there a not started workflows that may be
queued. If yes, we call the associated job
---------
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
In this PR (1/3)
- introduce view.mainGroupByFieldMetadataId as the new reference
determining which fieldMetadataId is used in a grouped view, in order to
deprecate viewGroup.fieldMetadataId which creates inconsistencies.
view.mainGroupByFieldMetadataId is now filled at every view creation,
though not in use yet.
- Introduce a command to backfill view.mainGroupByFieldMetadataId for
existing views + delete all viewGroup.fieldMetadataId with a
fieldMetadataId that is not view.mainGroupByFieldMetadataId. (It should
concern 37 active workspaces)
- Temporarily disable the option to change a grouped view's
fieldMetadataId as for now it creates inconsistencies. This feature can
be reintroduced when we have done the full migration.
In a next PR
- (2/3) use view.mainGroupByFieldMetadataId instead of
viewGroup.fieldMetadataId. In FE we may keep viewGroup.fieldMetadataId
as a state (TBD). View groups will now be created / deleted as a side
effect of view's mainGroupByFieldMetadataId update.
- (3/3) remove viewGroup.fieldMetadataId
---------
Co-authored-by: Charles Bochet <charles@twenty.com>
Because of [this code I forget to
clean](https://github.com/twentyhq/twenty/pull/16217),
- __workspace created between 1.12.0 and 1.12.2 (from friday 28 nov PM >
monday 1 dec PM)__ have standard objects with empty string default value
set on related TEXT type column (but default value null on field
metadata) (ex: jobTitle on person)
- __custom objects created between 1.12.0 and 1.12.2 (from friday 28 nov
PM > monday 1 dec PM)__ have "name" TEXT field with empty string default
value set + NOT NULL constraint on column
Command cleans data and updates table structure
This PR improves the general UX and DX of boards, by modifying the query
effect to only use paged group by queries.
In this PR we implement two more things in the backend for group by
queries :
- Fixed ORDER BY in the PARTITION BY sub-query (this wasn't working
because it was applied in the main query, so it sorted randomly picked
records, which was a correct sort on an incorrect dataset returned by
the sub-query)
- Added offset paging in PARTITION BY
Miscellaneous, various bug fixes and improvements along the way :
- Throttled loading of cards to avoid React freeze
- Handling of drag & drop
- Handling of create / delete / update
- Reworked skeleton (the library slows down a lot with hundreds of
skeleton for a spinning effect that is hardly noticed)
- Fixed refetch of aggregate queries (I included the new group by
aggregates query we use in the existing refetch mechanism)
- Re-trigger queries on filters and sorts changes
- Unselect all record ids when deleting / restoring / detroying
- Fetch only groups that still have records to lighten the group by
query.
# What remains to be done
This is still a naïve fetch more implementation that will work for a few
fetch more rounds, but if you scroll and load say 200 cards per column
on a board, React will re-render all 200 cards of each column each time.
We would probably need to virtualize the board with paged queries as we
did for the table, this could be done after this PR but seems less
urgent.
What's nice is that this new query pattern is well designed for
virtualization also, drawing from our experience with table
virtualization, and adapted to a multi-column request pattern, like a
2:2 matrix of records, for our boards.
So the remaining work would be to design a UI solution for virtualizing
this matrix of records, which could be quite different from our table
virtualization mechanism.
## Context
We've recently introduced a new workspace cache service which now acts
as a cache access and local storage for all workspace related data,
deprecating the individual specific services.
- Better performance through multiple caching/fetching strategies
- Consistent data access patterns across the codebase
- Reduced redis queries through MGET/MSET/PIPELINE with multiple cache
keys
Closes [1918](https://github.com/twentyhq/core-team-issues/issues/1918).
- For the first point in the issue, we just show the deactivated entries
along with the deactivated text.
---
- For the second point, we show a banner and control the
enabled/disabled state of save button depending on whether we're
allowing the user to create table with the typed name.
- For example, we do not want to allow the user to create a table with
reserved name, so we disable the save button without showing a banner.
- Similarly, we do not want the user to create a table with a name that
already exists in the database. In this case, we show a banner and we
also disable the save button.
- Finally, we do not want to allow the user to create a table where
singular and plural name are the same. Therefore, we disable the save
button for names like `works`.
---
- For the third point, if we add the delete button, it logically means
that we allow the user to delete a custom object/field even it has not
been deactivated yet, so did that.
- Upon deleting the object/field, if we wait for the metadata to refetch
before we navigate, this is what we see because the path does not exist
any longer after deletion and we're waiting for refetch on the path
until we navigate away.
https://github.com/user-attachments/assets/dbe0569c-db88-4285-851f-22551b1ca81e
- To avoid this page from appearing, I replaced awaiting refetch to not
awaiting refetch and redirecting while the refetch happens in the
background.
- Therefore, when we delete something, there is a slight delay for when
it is actually cleared out from the list, but the Not Found view does
not appear on the screen.
https://github.com/user-attachments/assets/47f49579-ce51-4d6a-b857-72046247bb4b
- I tried optimistically removing the object/field from the metadata,
but it leads to some issues (crashes the app) and I have not been able
to find a solution for it yet.
- Therefore, instead of getting stuck at perfection and blocking myself,
I stopped getting into the issue further and created this PR by ensuring
that the desired functionality works.
<!-- CURSOR_SUMMARY -->
---
> [!NOTE]
> Display deactivated objects/fields by default, add delete actions with
confirmation, and unify metadata name computation (auto-suffix reserved
keywords) across front/back with conflict checks in object creation.
>
> - **Frontend (Settings/Data Model)**:
> - **Visibility/UX**: Show `Deactivated` labels for objects/fields;
filters default to include inactive (`showDeactivated`/`showInactive`
true); replace field action dropdown with chevron link.
> - **Delete flows**: Add delete buttons for custom objects/fields with
confirmation modals and background refetch to avoid Not Found flashes.
> - **Creation/Edit validation**: Add name conflict detection banner in
`SettingsDataModelObjectAboutForm` and disable Save on conflicts;
simplify `metadataLabelSchema` to use computed name; form fields
validate on change and sync API names.
> - **Shared (twenty-shared/metadata)**:
> - Add `computeMetadataNameFromLabel` util (slugify+camelCase) and
`RESERVED_METADATA_NAME_KEYWORDS`; auto-append `Custom` to reserved
names; export constants/utilities.
> - **Backend**:
> - Migrate to shared `computeMetadataNameFromLabel`; update validators
to use shared reserved keywords with new messages; allow deletion of
active custom fields/objects (keep standard guards); adjust
services/decorators accordingly.
> - **Tests/Stories**:
> - Update unit/integration snapshots for new reserved-name messages and
behaviors; add missing i18n/router decorators in stories.
>
> <sup>Written by [Cursor
Bugbot](https://cursor.com/dashboard?tab=bugbot) for commit
5b126155606f6dbc8f7f91e2192cffb7bd2ebd2c. This will update automatically
on new commits. Configure
[here](https://cursor.com/dashboard?tab=bugbot).</sup>
<!-- /CURSOR_SUMMARY -->
---------
Co-authored-by: Félix Malfait <felix.malfait@gmail.com>
Co-authored-by: Félix Malfait <felix@twenty.com>
# Introduction
close https://github.com/twentyhq/core-team-issues/issues/1930
close https://github.com/twentyhq/core-team-issues/issues/1929
Migrating role and roleTarget entities to the v2 core engine, allowing
v2 caching leverage and allow migrating agent to v2 that needs role
target in prior
After agent we should be able to pass twenty standard app totally though
workspace migration
## Role target assignation
Please note that role target have 3 creation entrypoints:
- Agent
- User workspace
- ApiKey
Refactored all 3 of them to pass through a new role-target.service.ts
that consumes the v2 under the hood.
---------
Co-authored-by: Weiko <corentin@twenty.com>
martmull