Commit Graph

184 Commits

Author SHA1 Message Date
Paul Rastoin 859e718cf2 Identify view group (#17220)
# Introduction
Related to https://github.com/twentyhq/core-team-issues/issues/1989

1/ Migration, applicationId and universalIdentifier are required on
entity ( save point migration + upgrade command fallback pattern )
2/ Backfill using previous standard ids

## Test
tested prod extract
2026-01-19 12:56:52 +00:00
Paul Rastoin 6070dbf16a Identify agent (#17221)
# Introduction
Related to https://github.com/twentyhq/core-team-issues/issues/1989

1/ Migration, applicationId and universalIdentifier are required on
entity ( save point migration + upgrade command fallback pattern )
2/ Backfill using previous standard ids

## Test
tested prod extract
2026-01-18 16:31:25 +00:00
Paul Rastoin a91bac9dd9 Identify view filter (#17197)
# Introduction
Related to https://github.com/twentyhq/core-team-issues/issues/1989

1/ Migration, applicationId and universalIdentifier are required on
entity ( save point migration + upgrade command fallback pattern )
2/ Backfill using previous standard ids

## Test
tested prod extract
2026-01-18 11:08:21 +00:00
Paul Rastoin 9750bcba81 Improve view field identification command perfs and reliability (#17168)
# Introduction
Related to https://github.com/twentyhq/core-team-issues/issues/1989
Same continuity than what we've done in
https://github.com/twentyhq/twenty/pull/17161 comparing expect to
existing instead of existing versus expected

Plus various fixes
Tried both view and view field identification on prod extract, ( clean
soft deleted workspace and applied the migration WIP )
2026-01-16 09:35:41 +00:00
Paul Rastoin d8f0115fef Improve view identification command perfs and reliability (#17161)
# Introduction
Instead of comparing existing to expected comparing expected to existing
Also improved logs and fallback use cases

# Test
Tested on a prod extract
2026-01-15 12:27:07 +00:00
Paul Rastoin 2c24180b44 Identify standard view fields and views (#17118)
# Introduction
Related https://github.com/twentyhq/core-team-issues/issues/1989

1/ Migration, applicationId and universalIdentifier are required on
entity ( save point migration + upgrade command fallback pattern )
2/ Backfill using previous `standardId` or `isCustom`

## Test
Both tested on prod extract
Some view field are set as non custom is prod whereas they should for
several manually handle-able workspace amount
2026-01-14 14:15:07 +00:00
Paul Rastoin df108ea040 Identify standard objects (#17091)
# Introduction
Followup https://github.com/twentyhq/twenty/pull/16981

1/ Migration, applicationId and universalIdentifier are required on
entity ( save point migration + upgrade command fallback pattern )
2/ Backfill using previous standard ids
2026-01-12 18:06:13 +00:00
Paul Rastoin a6f371a42a Identify standard field do deploy until IS_WORKSPACE_CREATION_V2_ENABLED is enabled in prod (#16981)
# Introduction

fixes https://github.com/twentyhq/twenty/issues/16905

Do not merge until `IS_WORKSPACE_CREATION_V2_ENABLED` has been activated
by default, and so sync metadata has been deprecated by doing so. As the
sync metadata will attempt to insert `null` `applicationId` and
`universalIdentifier` values while creating a workspace

In this PR we're introducing a new `SyncableEntityRequired` which
enforces the non nullable `applicationId` and `universalIdentifier` on
extending entity

In this PR we also migrate the field metadata entity to extend the
required

## Identification upgrade command
This command will search for workspace field metadata entities that
aren't associated to an applicationId, dispatch them to either the
workspace-custom `applicationId` or the twenty-standard `applicationId`.
For the standard entities it will also set their universal identifier
based on the `STANDARD_OBJECTS` const hashmap

## Typeorm migration
As the non nullable `applicationId` and `universalIdentifier`migration
won't pass in the first we've been using the save point and upgrade
command migration fallback pattern

## Tests
Tested the command on a prod extract locally

Both `twenty-eng` and `twenty-for-twenty` have unexpected standard
objects
Please note that we will deprecate the `isCustom` and `standardId` col
later in the future

### Twenty-eng
```ts
[Nest] 98971  - 01/01/2026, 3:18:00 PM     LOG [IdentifyStandardEntitiesCommand] Successfully validated 600/600 field metadata update(s) for workspace 9870323e-22c3-4d14-9b7f-5bdc84f7d6ee (309 custom, 291 standard)
[Nest] 98971  - 01/01/2026, 3:18:00 PM    WARN [IdentifyStandardEntitiesCommand] Found 35 warning(s) while processing field metadata for workspace 9870323e-22c3-4d14-9b7f-5bdc84f7d6ee. These fields will become custom.
```

### Twenty for twenty

### Just created workspace
2026-01-12 09:59:01 +00:00
nitin 46e5420d20 dashboard workspace standard seeds (#16962)
# Introduction

In this pull request we're introducing new standard page layout, tabs
and widget ( 1 page layout, 1 tab and 8 widgets ) and also a new
opportunity field

Also now prefilling new records, 6 opportunities and a dashboard.

## Standard declaration
### New workspace creation
Relies on existing standard declaration builder

### Backfill command
We've been hacking through the standard builder in order to extract only
the standard page layout entities, updated their entity dependencies to
match the workspace ids so the validation passes


## Remark
- Refactored the `PageLayoutWidget` configuration type to be dynamically
typed through a generic discriminated union

---------

Co-authored-by: prastoin <paul@twenty.com>
2026-01-12 09:34:15 +00:00
Etienne d7b7e9def1 Cascade delete Task targets when tasks deleted - logic + migration command (#17019)
Co-authored-by: prastoin <paul@twenty.com>
2026-01-09 14:10:40 +00:00
Paul Rastoin 942d2fef83 Remove sync-metadata and IS_WORKSPACE_CREATION_V2_ENABLED feature flag (#16997)
# Introduction
Followup of
https://github.com/twentyhq/twenty/pull/17001#pullrequestreview-3638508738
close https://github.com/twentyhq/core-team-issues/issues/1910

We've completely decom the `sync-metadata` in production. We're now then
removing its implementation in favor of the v2.

## TODO:
- [x] Remove sync-metadata implem and commands
- [x] Remove workspace decorators 
- [x] Type each deprecated field to deprecated on their workspaceEntity
- [x] Remove the `workspace-sync-metadata` folder entirely
- [x] remove workspace migration
- [x] workspace migration removal migration
- [x] remove the `v2` references from workspace manager file names
- [x] remove the `v2` references from workspace manager modules
- [ ] Double check impact on translation file path updates


## Note
- Removed the gate logic
- Remains some service v2 naming, serverless needs to be migrated on v2
fully
- Removed workspaceMigration service app health consumption, making it
always returning up ( no more down ) cc @FelixMalfait ( quite obsolete
health check now, will require complete refactor once we introduce inter
app dependency etc )
2026-01-08 15:45:12 +00:00
Paul Rastoin 96f7f1cb0e Fix 1-15 upgrade commands bundle (#16957) 2026-01-06 14:13:40 +00:00
Paul Rastoin 98a9ae2a0e Migrate view filter group to v2 (#16876)
## Introduction
On the side hanlded PR with a // agents on another repo
Made several iterations to fix behavior and direction
Find below auto-generated PR description

closes https://github.com/twentyhq/core-team-issues/issues/2037

Created generic tooling for entity circular dep checking, will be useful
for permissions validation too @Weiko


## Migrate `viewFilterGroup` entity to v2 flat architecture

### Summary
Migrates the `viewFilterGroup` entity from v1 to the v2 flat entity
architecture, following the established patterns for other v2 entities
like `viewFilter`, `view`, and `viewField`.

### Changes

**Types & Constants**
- Added `FlatViewFilterGroup` and `FlatViewFilterGroupMaps` types
- Added editable properties constant for `viewFilterGroup`
- Registered `viewFilterGroup` in `ALL_METADATA_NAME`,
`ALL_METADATA_RELATION_PROPERTIES`,
`ALL_METADATA_MANY_TO_ONE_RELATIONS`, and related constants

**Cache Service**
- Created `WorkspaceFlatViewFilterGroupMapCacheService` with proper
relation loading for `viewFilters` and `childViewFilterGroups`
- Updated `WorkspaceFlatViewMapCacheService` to load `viewFilterGroups`
relation

**Builder & Validator**
- Created `WorkspaceMigrationV2ViewFilterGroupActionsBuilderService`
- Created `FlatViewFilterGroupValidatorService` with creation, update,
and deletion validation
- Integrated validation into the orchestrator service (runs before
`viewFilter` validation)

**Action Handlers**
- Created create, update, and delete action handlers for
`viewFilterGroup`

**Service Migration**
- Rewrote `ViewFilterGroupService` to use v2 migration pattern with
`WorkspaceMigrationValidateBuildAndRunService`
- Created utility functions for transforming DTOs to flat entities

**Database Migration**
- Added migration to make `parentViewFilterGroupId` foreign key
deferrable (handles self-referential parent/child insertions)

**ViewFilter Integration**
- Added `viewFilterGroupId` validation in
`FlatViewFilterValidatorService`
- Updated `viewFilter` many-to-one relations to include
`viewFilterGroup`

**Tests**
- Added integration tests for successful creation, update, deletion, and
destruction
- Added failing test cases for non-existent entities and invalid
references
- Added failing test for `viewFilter` creation with non-existent
`viewFilterGroupId`

### Breaking Changes
None - existing API contracts are preserved.
2026-01-02 14:16:01 +01:00
Paul Rastoin eeb91d9a96 [DO_NOT_RELEASE_MAIN_UNTIL_MERGED] Prevent migration failure due to workspace orphan metadata rows (#16863)
# Introduction
The `AddWorkspaceForeignKeys1767002571103` migration would fail when
released in production right now, as `foreignKey` be applicable as
there's a lof of orphan entries in database

As a workaround in order not to block any patch release we're
fallbacking the migration using save point and an upgrade command that
will attempt to apply the `foreignKey` on every workspace upgrade until
it succeed

We should keep in mind that any new fresh self installation will have
the foreignKey double checked that it would not implies regression on
workspace deletion using the integration tests

## Cleaning upgrade command
We won't implement the cleaning command in this PR yet either will I as
discussed with @Weiko someone else might be taking the subject starting
next week

<!-- CURSOR_SUMMARY -->
---

> [!NOTE]
> Strengthens workspace data integrity and makes the FK migration
resilient.
> 
> - Adds `upgrade:1-16:add-workspace-foreign-keys-migration` command to
apply `workspaceId` FKs once per run; wires into
`V1_16_UpgradeVersionCommandModule` and 1.16 upgrade sequence
> - Refactors migration `1767002571103` to use
`addWorkspaceForeignKeysQueries` util and wrap in a savepoint,
swallowing errors to avoid blocking releases
> - Extracts FK DDL into
`utils/1767002571103-addWorkspaceForeignKeys.util` for reuse by command
and migration
> - Removes duplicate `workspaceId` columns from entities (e.g.,
`cronTrigger`, `databaseEventTrigger`, `indexMetadata`,
`objectMetadata`, `roleTarget`, `role`, `serverlessFunction`) relying on
`SyncableEntity`; keeps indexes/relations
> - Marks legacy delete paths as deprecated; temporarily extends
`WorkspaceManagerService.delete` to also delete `serverlessFunction` by
`workspaceId`
> - Updates wiring to inject `ServerlessFunctionEntity` repository in
`workspace-manager` module/service and corresponding unit test
> - Extends integration tests and adds GraphQL helpers to create
serverless functions and triggers; verifies cascade deletion of related
metadata on workspace removal
> 
> <sup>Written by [Cursor
Bugbot](https://cursor.com/dashboard?tab=bugbot) for commit
6805bf5d1b32828b4bb1e9f130bfe6e478f66aee. This will update automatically
on new commits. Configure
[here](https://cursor.com/dashboard?tab=bugbot).</sup>
<!-- /CURSOR_SUMMARY -->
2025-12-31 11:52:21 +00:00
Abdul Rahman 720348c583 Add upgrade commands to backfill updatedBy field and view fields (#16845)
---
prastoin edit
## Introduction
As we're deprecating the sync metadata we cannot rely on it anymore in
order to create the new standard updatedBy field
In this PR we introduce a command that will backfill the dynamic field
on both workspace standard and custom objects, for standard object it
will create a standard fields ( twenty-standard app scoped ) and for
custom one it will create a custom field ( worksapce-custom-app scoped )

## Testing method
Checkout `1.14.0` `yarn` and `npx nx database:reset twenty-server`
checkout PR and `yarn` `npx nx build twenty-server` and `yarn
database:migrate:prod` finally running the command

#### Before
<img width="2244" height="1464" alt="image"
src="https://github.com/user-attachments/assets/2fb866cd-13b8-4152-99b8-1fcc813b1d46"
/>

#### After
<img width="2244" height="1464" alt="image"
src="https://github.com/user-attachments/assets/b836ac06-c7b8-4add-bee1-bc1963418f29"
/>


#### Logs
```ts
[Nest] 20678  - 12/30/2025, 1:47:35 PM     LOG [BackfillUpdatedByFieldCommand] Found 12 objects that need updatedBy field
[EntityBuilder fieldMetadata] matrix computation: 3.994ms
[EntityBuilder fieldMetadata] creation validation: 1.822ms
[EntityBuilder fieldMetadata] deletion validation: 0.016ms
[EntityBuilder fieldMetadata] update validation: 0.009ms
[EntityBuilder fieldMetadata] entity processing: 6.098ms
[EntityBuilder fieldMetadata] validateAndBuild: 10.217ms
[EntityBuilder index] matrix computation: 0.857ms
[EntityBuilder index] creation validation: 0.003ms
[EntityBuilder index] deletion validation: 0.017ms
[EntityBuilder index] update validation: 0.008ms
[EntityBuilder index] entity processing: 4.721ms
[EntityBuilder index] validateAndBuild: 5.632ms
[Runner] Initial cache retrieval: 0.07ms
[BaseWorkspaceMigrationRunnerActionHandlerService] create_field executeForMetadata: 7.69ms
[BaseWorkspaceMigrationRunnerActionHandlerService] create_field executeForWorkspaceSchema: 10.531ms
[BaseWorkspaceMigrationRunnerActionHandlerService] create_field executeForMetadata: 5.354ms
[BaseWorkspaceMigrationRunnerActionHandlerService] create_field executeForWorkspaceSchema: 6.134ms
[BaseWorkspaceMigrationRunnerActionHandlerService] create_field executeForMetadata: 2.507ms
[BaseWorkspaceMigrationRunnerActionHandlerService] create_field executeForWorkspaceSchema: 3.745ms
[BaseWorkspaceMigrationRunnerActionHandlerService] create_field executeForMetadata: 1.113ms
[BaseWorkspaceMigrationRunnerActionHandlerService] create_field executeForWorkspaceSchema: 1.585ms
[BaseWorkspaceMigrationRunnerActionHandlerService] create_field executeForMetadata: 1.223ms
[BaseWorkspaceMigrationRunnerActionHandlerService] create_field executeForWorkspaceSchema: 2.221ms
[BaseWorkspaceMigrationRunnerActionHandlerService] create_field executeForMetadata: 5.375ms
[BaseWorkspaceMigrationRunnerActionHandlerService] create_field executeForWorkspaceSchema: 5.943ms
[BaseWorkspaceMigrationRunnerActionHandlerService] create_field executeForMetadata: 1.363ms
[BaseWorkspaceMigrationRunnerActionHandlerService] create_field executeForWorkspaceSchema: 1.997ms
[BaseWorkspaceMigrationRunnerActionHandlerService] create_field executeForMetadata: 2.03ms
[BaseWorkspaceMigrationRunnerActionHandlerService] create_field executeForWorkspaceSchema: 2.982ms
[BaseWorkspaceMigrationRunnerActionHandlerService] create_field executeForMetadata: 0.88ms
[BaseWorkspaceMigrationRunnerActionHandlerService] create_field executeForWorkspaceSchema: 2.977ms
[BaseWorkspaceMigrationRunnerActionHandlerService] create_field executeForMetadata: 9.638ms
[BaseWorkspaceMigrationRunnerActionHandlerService] create_field executeForWorkspaceSchema: 11.297ms
[BaseWorkspaceMigrationRunnerActionHandlerService] create_field executeForMetadata: 1.581ms
[BaseWorkspaceMigrationRunnerActionHandlerService] create_field executeForWorkspaceSchema: 2.248ms
[BaseWorkspaceMigrationRunnerActionHandlerService] create_field executeForMetadata: 0.847ms
[BaseWorkspaceMigrationRunnerActionHandlerService] create_field executeForWorkspaceSchema: 1.732ms
[Runner] Transaction execution: 82.555ms
[Runner] Cache invalidation flatFieldMetadataMaps,flatIndexMaps,flatObjectMetadataMaps: 89.247ms
[Runner] Total execution: 171.963ms
[Nest] 20678  - 12/30/2025, 1:47:35 PM     LOG [BackfillUpdatedByFieldCommand] Successfully backfilled updatedBy field for 12 objects in workspace 20202020-1c25-4d02-bf25-6aeccf7ea419
[Nest] 20678  - 12/30/2025, 1:47:35 PM     LOG [BackfillUpdatedByFieldCommand] Running command on workspace 3b8e6458-5fc1-4e63-8563-008ccddaa6db 2/2
[Nest] 20678  - 12/30/2025, 1:47:35 PM     LOG [BackfillUpdatedByFieldCommand] Starting backfill of updatedBy field for workspace 3b8e6458-5fc1-4e63-8563-008ccddaa6db
[Nest] 20678  - 12/30/2025, 1:47:35 PM     LOG [BackfillUpdatedByFieldCommand] Found 10 objects that need updatedBy field
[EntityBuilder fieldMetadata] matrix computation: 3.224ms
[EntityBuilder fieldMetadata] creation validation: 0.969ms
[EntityBuilder fieldMetadata] deletion validation: 0.014ms
[EntityBuilder fieldMetadata] update validation: 0.002ms
[EntityBuilder fieldMetadata] entity processing: 5.149ms
[EntityBuilder fieldMetadata] validateAndBuild: 8.472ms
[EntityBuilder index] matrix computation: 0.785ms
[EntityBuilder index] creation validation: 0.002ms
[EntityBuilder index] deletion validation: 0.014ms
[EntityBuilder index] update validation: 0.002ms
[EntityBuilder index] entity processing: 3.818ms
[EntityBuilder index] validateAndBuild: 4.637ms
[Runner] Initial cache retrieval: 0.05ms
[BaseWorkspaceMigrationRunnerActionHandlerService] create_field executeForMetadata: 2.964ms
[BaseWorkspaceMigrationRunnerActionHandlerService] create_field executeForWorkspaceSchema: 3.902ms
[BaseWorkspaceMigrationRunnerActionHandlerService] create_field executeForMetadata: 1.502ms
[BaseWorkspaceMigrationRunnerActionHandlerService] create_field executeForWorkspaceSchema: 2.039ms
[BaseWorkspaceMigrationRunnerActionHandlerService] create_field executeForMetadata: 2.673ms
[BaseWorkspaceMigrationRunnerActionHandlerService] create_field executeForWorkspaceSchema: 3.403ms
[BaseWorkspaceMigrationRunnerActionHandlerService] create_field executeForMetadata: 1.143ms
[BaseWorkspaceMigrationRunnerActionHandlerService] create_field executeForWorkspaceSchema: 2.123ms
[BaseWorkspaceMigrationRunnerActionHandlerService] create_field executeForMetadata: 2.033ms
[BaseWorkspaceMigrationRunnerActionHandlerService] create_field executeForWorkspaceSchema: 2.954ms
[BaseWorkspaceMigrationRunnerActionHandlerService] create_field executeForMetadata: 1.826ms
[BaseWorkspaceMigrationRunnerActionHandlerService] create_field executeForWorkspaceSchema: 2.903ms
[BaseWorkspaceMigrationRunnerActionHandlerService] create_field executeForMetadata: 1.495ms
[BaseWorkspaceMigrationRunnerActionHandlerService] create_field executeForWorkspaceSchema: 2.106ms
[BaseWorkspaceMigrationRunnerActionHandlerService] create_field executeForMetadata: 14.014ms
[BaseWorkspaceMigrationRunnerActionHandlerService] create_field executeForWorkspaceSchema: 14.373ms
[BaseWorkspaceMigrationRunnerActionHandlerService] create_field executeForMetadata: 1.669ms
[BaseWorkspaceMigrationRunnerActionHandlerService] create_field executeForWorkspaceSchema: 2.48ms
[BaseWorkspaceMigrationRunnerActionHandlerService] create_field executeForMetadata: 0.62ms
[BaseWorkspaceMigrationRunnerActionHandlerService] create_field executeForWorkspaceSchema: 1.048ms
[Runner] Transaction execution: 73.222ms
[Runner] Cache invalidation flatFieldMetadataMaps,flatIndexMaps,flatObjectMetadataMaps: 37.703ms
[Runner] Total execution: 111.071ms
```
---

---------

Co-authored-by: prastoin <paul@twenty.com>
2025-12-30 14:05:26 +01:00
Paul Rastoin 4135a6473a [BREAKING_CHANGE_DASHBOARDS][DASHBOARD_CACHE_FLUSH_REQUIRED] Refactor page layout widget configuration type (#16671)
# Introduction
In this pull-request we're refactoring the page layout widget
configuration entity to be containing its discriminated key simplifying
underlying code and maintainability
- Made the configuration and title non nullable
- Introduced a generic predicate for the `widgetConfigurationType`
- Upgraded command to remove `graphType` and insert new
`configurationType` to existing entries
- Migrated frontend to new type system

---------

Co-authored-by: bosiraphael <raphael.bosi@gmail.com>
Co-authored-by: Raphaël Bosi <71827178+bosiraphael@users.noreply.github.com>
2025-12-29 15:46:59 +01:00
Marie bb18f78f1b [Fix] Fix NaN position in notes resulting in not being able to create notes (#16818)
Following [discord
thread](https://discord.com/channels/1130383047699738754/1453910755387899996/1453910755387899996),
reproductible on twenty-eng


https://github.com/user-attachments/assets/ae7f363d-87e1-44fa-8fe2-ee78412d62a7

Records positions are computed at record creation, depending on the
position arg from the request, being equal to `last`, `first`, or not
present.
When being equal to last, as it is done when creating a note from the
product, the position is calculated using `.maximum()` function which
uses postgres' MAX function. If there is a `NaN` value among the list,
the MAX will return `NaN` too. So if for some reason there is a NaN
somewhere in the position column, all subsequent records being created
with last position argument will be created with NaN value. Until [this
PR](https://github.com/twentyhq/twenty/pull/16630), where we introduced
a validation on position at record creation which throws when NaN is
trying to be introduced as a position, this was going silent. (fyi
@etiennejouan , not on you at all but for info)

Looking into twenty-eng workspace, I found note records with NaN
position dating back to august 2025, making it hard to understand and
debug why they were introduced with NaN position. So I did not find the
real root cause, but I suggest to
- update record-position.service to fix the issue for subsequent records
that go through this service (which is what is currently broken)
- run a command to fix the existing records with NaN position for Notes,
as it is where the issue happened for both the user reporting the issue
on discord, and us on twenty-eng. So hopefully the problem was limited
to Notes
2025-12-29 13:55:20 +01:00
Paul Rastoin 8acfa60412 Fix standard agent and roles deletion command (#16686)
# Introduction
Caught red handed, introduced a failing command in
https://github.com/twentyhq/twenty/pull/16499 that was failing even in
system build which is should not
2025-12-18 19:14:24 +01:00
Paul Rastoin 47a8a15598 Metadata modules for PageLayoutTab PageLayoutWidget (#16662)
# Introduction
Creating dedicated folders and module for both `page-layout-tab` and
`page-layout-widget`

The addition diff with deletion is due to the module being added
2025-12-18 11:15:29 +00:00
martmull 3ac72683da Update migration command (#16578)
Avoid pg_ table request
2025-12-16 13:35:03 +00:00
Paul Rastoin 75bba5a8ee Standard Agent, Role, Role target (#16499)
# Introduction
In this pullrequest have been migrated to the flat standard entities:
Role, Agent and RoleTargets.

## What happens
- Removed createStandardMorph tool util in favor of dynamic typing of
`createMorphOrRelationStandardField`
- Implemented a command to remove standard agents and their role that
has been removed in https://github.com/twentyhq/twenty/pull/16513 also
added a default role target to data manipulator role to the only
remaining agent
- Implemented an agent deleteMany service handler
2025-12-16 14:17:31 +01:00
Marie a0b963ef86 Remove viewGroup.fieldMetadataId (#16571)
Final step of
https://github.com/orgs/twentyhq/projects/1/views/8?pane=issue&itemId=142348748&issue=twentyhq%7Ccore-team-issues%7C1965

Removing viewGroup.fieldMetadataId.
It's already not used in FE anymore
2025-12-15 17:58:45 +01:00
martmull e289f3056e 1895 extensibility v1 application tokens 3 (#16504)
- moves applicationRoleId to application entity
- add new `APPLICATION` FieldActorSource and `APPLICATION`
JwtTokenTypeEnum value
- create a new token with applicationId when executing a function
- when applicationId is in token, check for application.defaultRole
permissions
-use twenty-shared types in `twenty-sdk/application`
- create a new import from generate called "Twenty" that you can use
directly without having to set TWENTY_API_KEY AND TWENTY_API_URL (keep
metadata or core parameter only)
- provide to serverless unique one time BEARER TOKEN to run it

Result
<img width="977" height="566" alt="image"
src="https://github.com/user-attachments/assets/e78428a0-5b13-4975-aa13-58ee3b32450c"
/>

<img width="910" height="596" alt="image"
src="https://github.com/user-attachments/assets/6ec72bf5-7655-4093-a45e-ad269595a324"
/>

<img width="741" height="568" alt="image"
src="https://github.com/user-attachments/assets/7683944c-fd79-4417-8fb2-8e4815cc112f"
/>
2025-12-15 17:44:23 +01:00
Etienne 3e55e772d4 Update null command (#16567)
Remove version condition to re-run the command on upgraded workspaces on
which it fails
2025-12-15 13:55:32 +00:00
Paul Rastoin 8c79353bfb Fix rename index collides with existing v2 index (#16560)
## Introduction
When migrating a v1 index name to v2 it might collide with an existing
v2 index
In this case we remove both the v1 metadata and pg index

In case of a metadata and pg_index desync this might occur too late in
the process that's why we have two fallback
One computing the v1 deletion from the metadata and another one in the
catch block of the v1 to migration transaction commit
2025-12-15 11:06:09 +01:00
Paul Rastoin d189e3f57e Fix rename command error code edge case (#16556) 2025-12-15 08:36:35 +00:00
Paul Rastoin f0af947331 [RenameIndexCommand] Fix orphaned index edge case (#16539)
We've been facing orphaned metadata index that would result in alter
error, now if we encounter one we just delete it
2025-12-12 16:44:00 +00:00
Paul Rastoin 59663ec34e fix(server): non blocking CleanEmptyStringNullInTextFieldsCommand error (#16529)
# Introduction
Getting query timeout on prod upgrade for workspace that has huge
timeline activity table
2025-12-12 16:02:23 +01:00
Weiko 083a038f8a Deprecate workspace datasoure (#16507) 2025-12-11 18:27:49 +01:00
Paul Rastoin f06a5ccacd Remove sync metadata from upgrade (#16491)
# Introduction
Related https://github.com/twentyhq/core-team-issues/issues/1910
From now on the upgrade won't integrate any sync metadata as it's going
to be deprecated very soon
Any updates to about to removes workspace-entity or standard flat entity
will require a dedicated upgrade command, what we've already started
doing during the 1.13 sprint, until we have totally migrated the v2 to
be workspace agnostic
2025-12-11 14:50:56 +01:00
Guillim b83dc3aaff TimelineActivity migration to morph (#15652)
# TimelineActivity migration to morph


- Creates `timelineActivities2` relations on Company, Dashboard, Note,
Opportunity, Person, Task, Workflow, WorkflowRun, and WorkflowVersion
entities with proper metadata and cascade delete behavior.

It was required to create standard fields as well since the
mapObjectMetadataByUniqueIdentifier needs it. otherwise the fields won't
be considered

- Feature Flag `IS_TIMELINE_ACTIVITY_MIGRATED` necessary to have the two
states in parallel. It is used as a stamp once the migration has been
run

- Migration is done using the coreDataSource. Why ?
even though is unsafe to use, the first implementation of the migration
took forever on each workspace. See [this
commit](https://github.com/twentyhq/twenty/pull/15652/commits/477011e8d7d4c580f79ba7ec4a8fb002a3ec86b2)

The plan for this complex migration is as follows :

![plan](https://github.com/user-attachments/assets/51c63ea6-fb0d-40b4-b99d-3e0b35a204e6)
Note: we will need to rename fields in the release 1.12 (there is no
easy way to do all this in one release)
2025-12-11 14:28:26 +01:00
Weiko 9bd8f94b3a Refactor global datasource part 3 (#16447)
## Context
Following https://github.com/twentyhq/twenty/pull/16399
Now using the new global orm manager everywhere and returning a
GlobalDatasource/WorkspaceDatasource based on a feature flag.
This means we now need to wrap all our ORM calls within
executeInWorkspaceContext callback (at least for now) so the global
datasource can dynamically hydrate its context via the new store (the
global datasource does not store anything related to workspaces as it is
now a unique singleton). If feature flag is off it still uses local data
stored in the workspace datasource.
2025-12-10 17:17:33 +01:00
Paul Rastoin 4996f3dd28 Finalize twenty standard app as workspace migration object and fields (#16353)
# Introduction
Related to https://github.com/twentyhq/core-team-issues/issues/1995
In this PR we're fixing the remaining object/fields validation errors
resulting from standard objects and fields now passing a validation that
wasn't when using the sync metadata

## Key Changes
- **Field naming**: Renamed `iCalUID` to `iCalUid` for consistent
camelCase convention across calendar events
- **Enum standardization**: Uppercased enum values for message channels
(email→EMAIL), message participants (from→FROM, to→TO, cc→CC, bcc→BCC),
and message direction (incoming→INCOMING, outgoing→OUTGOING)
- **Label simplification**: Removed example values from workspace member
number format labels for cleaner UI
- **Migration infrastructure**: Added `isSystemBuild` flag throughout
field metadata service pipeline to allow system-level updates of
standard fields that bypass normal restrictions

## Migrating the existing data
We've created an upgrade command that will identify using the existing
object and field standard id field that needs to be updated, even though
the sync metadata still in usage could have fix them ( and the goal is
to deprecate it by the end of the sprint )
We will call the updateOneField for each of them, we're passing by the
field service in order to battle test what are going to be the temporary
way to handle standard migrations when we will start deprecating the
sync metadata but haven't still refactored the v2 workspace migration to
be workspace agnostic

## Twenty eng migration
Tested the whole migration + upgrade on twenty eng
Here are generated workspace migration
Records are handled natively gracefully too

### ICalUid
```json
{
  "status": "success",
  "workspaceMigration": {
    "relatedFlatEntityMapsKeys": [
      "flatFieldMetadataMaps",
      "flatIndexMaps",
      "flatViewFilterMaps",
      "flatViewGroupMaps",
      "flatViewMaps",
      "flatViewFieldMaps",
      "flatObjectMetadataMaps"
    ],
    "actions": [
      {
        "type": "update_field",
        "fieldMetadataId": "",
        "objectMetadataId": "",
        "updates": [
          {
            "from": "iCalUID",
            "to": "iCalUid",
            "property": "name"
          }
        ]
      }
    ],
    "workspaceId": ""
  }
}
```

### Incoming Outgoing
None as already caps in database somehow
```json
{
  "status": "success",
  "workspaceMigration": {
    "relatedFlatEntityMapsKeys": [
      "flatFieldMetadataMaps",
      "flatIndexMaps",
      "flatViewFilterMaps",
      "flatViewGroupMaps",
      "flatViewMaps",
      "flatViewFieldMaps",
      "flatObjectMetadataMaps"
    ],
    "actions": [],
    "workspaceId": ""
  }
}
```

### EMAIL
```json
{
  "status": "success",
  "workspaceMigration": {
    "relatedFlatEntityMapsKeys": [
      "flatFieldMetadataMaps",
      "flatIndexMaps",
      "flatViewFilterMaps",
      "flatViewGroupMaps",
      "flatViewMaps",
      "flatViewFieldMaps",
      "flatObjectMetadataMaps"
    ],
    "actions": [
      {
        "type": "update_field",
        "fieldMetadataId": "",
        "objectMetadataId": "",
        "updates": [
          {
            "from": "'email'",
            "to": "'EMAIL'",
            "property": "defaultValue"
          },
          {
            "from": [
              {
                "color": "green",
                "id": "",
                "label": "Email",
                "position": 0,
                "value": "email"
              },
              {
                "color": "blue",
                "id": "",
                "label": "SMS",
                "position": 1,
                "value": "sms"
              }
            ],
            "to": [
              {
                "color": "green",
                "id": "",
                "label": "Email",
                "position": 0,
                "value": "EMAIL"
              },
              {
                "color": "blue",
                "id": "",
                "label": "SMS",
                "position": 1,
                "value": "SMS"
              }
            ],
            "property": "options"
          }
        ]
      }
    ],
    "workspaceId": "e"
  }
}
```

### MessageParticipantRole
```json
{
  "status": "success",
  "workspaceMigration": {
    "relatedFlatEntityMapsKeys": [
      "flatFieldMetadataMaps",
      "flatIndexMaps",
      "flatViewFilterMaps",
      "flatViewGroupMaps",
      "flatViewMaps",
      "flatViewFieldMaps",
      "flatObjectMetadataMaps"
    ],
    "actions": [
      {
        "type": "update_field",
        "fieldMetadataId": "",
        "objectMetadataId": "",
        "updates": [
          {
            "from": "'from'",
            "to": "'FROM'",
            "property": "defaultValue"
          },
          {
            "from": [
              {
                "color": "green",
                "id": "",
                "label": "From",
                "position": 0,
                "value": "from"
              },
              {
                "color": "blue",
                "id": "",
                "label": "To",
                "position": 1,
                "value": "to"
              },
              {
                "color": "orange",
                "id": "",
                "label": "Cc",
                "position": 2,
                "value": "cc"
              },
              {
                "color": "red",
                "id": "",
                "label": "Bcc",
                "position": 3,
                "value": "bcc"
              }
            ],
            "to": [
              {
                "color": "green",
                "id": "",
                "label": "From",
                "position": 0,
                "value": "FROM"
              },
              {
                "color": "blue",
                "id": "",
                "label": "To",
                "position": 1,
                "value": "TO"
              },
              {
                "color": "orange",
                "id": "",
                "label": "Cc",
                "position": 2,
                "value": "CC"
              },
              {
                "color": "red",
                "id": "",
                "label": "Bcc",
                "position": 3,
                "value": "BCC"
              }
            ],
            "property": "options"
          }
        ]
      }
    ],
    "workspaceId": ""
  }
}
```

### Workspace member number format labels
```json
{
  "status": "success",
  "workspaceMigration": {
    "relatedFlatEntityMapsKeys": [
      "flatFieldMetadataMaps",
      "flatIndexMaps",
      "flatViewFilterMaps",
      "flatViewGroupMaps",
      "flatViewMaps",
      "flatViewFieldMaps",
      "flatObjectMetadataMaps"
    ],
    "actions": [
      {
        "type": "update_field",
        "fieldMetadataId": "",
        "objectMetadataId": "",
        "updates": [
          {
            "from": [
              {
                "color": "turquoise",
                "id": "",
                "label": "System",
                "position": 0,
                "value": "SYSTEM"
              },
              {
                "color": "blue",
                "id": "",
                "label": "Commas and dot (1,234.56)",
                "position": 1,
                "value": "COMMAS_AND_DOT"
              },
              {
                "color": "green",
                "id": "",
                "label": "Spaces and comma (1 234,56)",
                "position": 2,
                "value": "SPACES_AND_COMMA"
              },
              {
                "color": "orange",
                "id": "",
                "label": "Dots and comma (1.234,56)",
                "position": 3,
                "value": "DOTS_AND_COMMA"
              },
              {
                "color": "purple",
                "id": "",
                "label": "Apostrophe and dot (1'234.56)",
                "position": 4,
                "value": "APOSTROPHE_AND_DOT"
              }
            ],
            "to": [
              {
                "color": "turquoise",
                "id": "",
                "label": "System",
                "position": 0,
                "value": "SYSTEM"
              },
              {
                "color": "blue",
                "id": "",
                "label": "Commas and dot",
                "position": 1,
                "value": "COMMAS_AND_DOT"
              },
              {
                "color": "green",
                "id": "",
                "label": "Spaces and comma",
                "position": 2,
                "value": "SPACES_AND_COMMA"
              },
              {
                "color": "orange",
                "id": "",
                "label": "Dots and comma",
                "position": 3,
                "value": "DOTS_AND_COMMA"
              },
              {
                "color": "purple",
                "id": "",
                "label": "Apostrophe and dot",
                "position": 4,
                "value": "APOSTROPHE_AND_DOT"
              }
            ],
            "property": "options"
          }
        ]
      }
    ],
    "workspaceId": ""
  }
}
```
2025-12-09 14:09:48 +01:00
Etienne be91d870f9 Fix - revert standard sync logic on index + fix command (#16421) 2025-12-09 12:54:01 +00:00
Etienne 07cfaa78ef Fix unique standard field (#16371)
Fixes https://github.com/twentyhq/twenty/issues/15925

- update field metadata update logic
- uniformize the way index are named
- command to migrate v1-named unique index
- add integration testing

---------

Co-authored-by: prastoin <paul@twenty.com>
2025-12-08 18:05:28 +00:00
Raphaël Bosi 5fb7e76005 Migrate page layout to v2 (#16364) 2025-12-08 14:11:24 +00:00
Paul Rastoin 3e57aa14d3 Refactor page layout tab and widgets migration (#16367)
# Introduction
Making columns nullable instead of required + metadata on the fly
migration in typeorm migration that could affect other breaking change
migrations to be run
2025-12-06 10:23:24 +01:00
Paul Rastoin 28cdb02fbb Twenty standard application Objects and fields as allFlatEntityMaps ID non-agnostic (#16298)
# Introduction

Related to https://github.com/twentyhq/core-team-issues/issues/1995
This PR introduces the basis of the `twentyStandard` application as code
on demand, it's highly tied to `ids` where it will becomes workspace
agnostic following the builder and runner `universalIdentifier` refactor
later.

The goal here to allow computing the `allFlatEntityMaps` `to` of the
`twentyStandard` application on a empty workspace ( workspace creation
). Allowing installing the twenty standard app through a workspace
migration instead of passing by the sync metadata

Nothing done will be run in production for the moment if it's not the
small validation refactor we've introduced

Please note that everything introduced here will be replaced at some
point by a twenty app instance when the twenty sdk is mature enough to
handle of the edge cases we need here

## How we've proceeded
We've been iterating over every workspace entity both objects and their
fields, and transpiled them to flatEntity.
Being sure we migrate the defaultValue, settings and so on accordingly.
We've also compute all the ids in prior of the whole entities
computation so we don't face any hoisting issue.

## Current state
At the moment only handling all of the 29 standard objects and their
fields
Settings a unique universalIdentifier for all of them

Will come views, agent role targets and so on later

## `workspace:compute-twenty-standard-migration` command
This command allow generating a workspace migration that will result in
installing the twenty standard app in an empty workspace
It's temporary and aims to allow debugging for the moment we might not
keep it in the future as it is right now
It contains debug writeFileSync which is expected no worries greptile

## `LabelFieldMetadataIdentifierId`
Small refactor allowing defining the label identifier field metadata id
of a uuid field metadata type for system object, as some of our standard
object don't have a name field and don't aim to
Also please note that we might remove this build options later in the
sake of the currently installed universal identifier application that we
could compare with the deterministic twenty standard one

## `runFlatFieldMetadataValidators`
Deprecated this pattern which was redundant and not v2 friendly pattern

## Current errors that will address in upcoming PR
Current standard objects and fields metadata does not pass the
validation that we have in place, as historically the sync metadata
would directly consume the repositories and would just ignore the
validation. This is about to change.
Will handle the below errors in dedicated PRs as they will required
upgrade commands in order to migrate the data, or will handle that from
the sync metadata instead still to be determined but nothing critical
here

- camel case field metadata name
- options label invalid format

```json
{
  "status": "fail",
  "report": {
    "fieldMetadata": [
      {
        "status": "fail",
        "errors": [
          {
            "code": "INVALID_FIELD_INPUT",
            "message": "Name should be in camelCase",
            "userFriendlyMessage": {
              "id": "P+jdmX",
              "message": "Name should be in camelCase"
            },
            "value": "iCalUID"
          }
        ],
        "flatEntityMinimalInformation": {
          "id": "68dd83cd-92c8-4233-bb28-47939bab6124",
          "name": "iCalUID",
          "objectMetadataId": "11c16ab6-9176-439e-a2db-a12c5a58a524"
        },
        "type": "create_field"
      },
      {
        "status": "fail",
        "errors": [
          {
            "code": "INVALID_FIELD_INPUT",
            "message": "Value must be in UPPER_CASE and follow snake_case \"email\"",
            "userFriendlyMessage": {
              "id": "UBPzFQ",
              "message": "Value must be in UPPER_CASE and follow snake_case \"{sanitizedValue}\"",
              "values": {
                "sanitizedValue": "email"
              }
            },
            "value": "email"
          },
          {
            "code": "INVALID_FIELD_INPUT",
            "message": "Value must be in UPPER_CASE and follow snake_case \"sms\"",
            "userFriendlyMessage": {
              "id": "UBPzFQ",
              "message": "Value must be in UPPER_CASE and follow snake_case \"{sanitizedValue}\"",
              "values": {
                "sanitizedValue": "sms"
              }
            },
            "value": "sms"
          }
        ],
        "flatEntityMinimalInformation": {
          "id": "e3caaf2a-e07d-4146-8dfc-9eef904e82c9",
          "name": "type",
          "objectMetadataId": "4b777de5-4c7b-4af4-9b92-655c0f87512b"
        },
        "type": "create_field"
      },
      {
        "status": "fail",
        "errors": [
          {
            "code": "INVALID_FIELD_INPUT",
            "message": "Value must be in UPPER_CASE and follow snake_case \"incoming\"",
            "userFriendlyMessage": {
              "id": "UBPzFQ",
              "message": "Value must be in UPPER_CASE and follow snake_case \"{sanitizedValue}\"",
              "values": {
                "sanitizedValue": "incoming"
              }
            },
            "value": "incoming"
          },
          {
            "code": "INVALID_FIELD_INPUT",
            "message": "Value must be in UPPER_CASE and follow snake_case \"outgoing\"",
            "userFriendlyMessage": {
              "id": "UBPzFQ",
              "message": "Value must be in UPPER_CASE and follow snake_case \"{sanitizedValue}\"",
              "values": {
                "sanitizedValue": "outgoing"
              }
            },
            "value": "outgoing"
          }
        ],
        "flatEntityMinimalInformation": {
          "id": "d96233a4-93be-45ea-9548-3b50f3c700cf",
          "name": "direction",
          "objectMetadataId": "480a648a-d2e5-482a-992f-ef053e1b4bb0"
        },
        "type": "create_field"
      },
      {
        "status": "fail",
        "errors": [
          {
            "code": "INVALID_FIELD_INPUT",
            "message": "Value must be in UPPER_CASE and follow snake_case \"from\"",
            "userFriendlyMessage": {
              "id": "UBPzFQ",
              "message": "Value must be in UPPER_CASE and follow snake_case \"{sanitizedValue}\"",
              "values": {
                "sanitizedValue": "from"
              }
            },
            "value": "from"
          },
          {
            "code": "INVALID_FIELD_INPUT",
            "message": "Value must be in UPPER_CASE and follow snake_case \"to\"",
            "userFriendlyMessage": {
              "id": "UBPzFQ",
              "message": "Value must be in UPPER_CASE and follow snake_case \"{sanitizedValue}\"",
              "values": {
                "sanitizedValue": "to"
              }
            },
            "value": "to"
          },
          {
            "code": "INVALID_FIELD_INPUT",
            "message": "Value must be in UPPER_CASE and follow snake_case \"cc\"",
            "userFriendlyMessage": {
              "id": "UBPzFQ",
              "message": "Value must be in UPPER_CASE and follow snake_case \"{sanitizedValue}\"",
              "values": {
                "sanitizedValue": "cc"
              }
            },
            "value": "cc"
          },
          {
            "code": "INVALID_FIELD_INPUT",
            "message": "Value must be in UPPER_CASE and follow snake_case \"bcc\"",
            "userFriendlyMessage": {
              "id": "UBPzFQ",
              "message": "Value must be in UPPER_CASE and follow snake_case \"{sanitizedValue}\"",
              "values": {
                "sanitizedValue": "bcc"
              }
            },
            "value": "bcc"
          }
        ],
        "flatEntityMinimalInformation": {
          "id": "961c598e-67c3-452d-8bb2-b92c0bc64404",
          "name": "role",
          "objectMetadataId": "8af8a13c-ff97-4cd3-b70d-52a7dc2924b4"
        },
        "type": "create_field"
      },
      {
        "status": "fail",
        "errors": [
          {
            "code": "INVALID_FIELD_INPUT",
            "message": "Label must not contain a comma",
            "userFriendlyMessage": {
              "id": "k731jp",
              "message": "Label must not contain a comma"
            },
            "value": "Commas and dot (1,234.56)"
          },
          {
            "code": "INVALID_FIELD_INPUT",
            "message": "Label must not contain a comma",
            "userFriendlyMessage": {
              "id": "k731jp",
              "message": "Label must not contain a comma"
            },
            "value": "Spaces and comma (1 234,56)"
          },
          {
            "code": "INVALID_FIELD_INPUT",
            "message": "Label must not contain a comma",
            "userFriendlyMessage": {
              "id": "k731jp",
              "message": "Label must not contain a comma"
            },
            "value": "Dots and comma (1.234,56)"
          }
        ],
        "flatEntityMinimalInformation": {
          "id": "7fa20caf-2597-42e3-84e5-15a91b125b9b",
          "name": "numberFormat",
          "objectMetadataId": "a6974302-9e72-461c-aa09-9390f4ff16fc"
        },
        "type": "create_field"
      }
    ],
    "objectMetadata": [],
    "view": [],
    "viewField": [],
    "viewGroup": [],
    "index": [],
    "serverlessFunction": [],
    "cronTrigger": [],
    "databaseEventTrigger": [],
    "routeTrigger": [],
    "viewFilter": [],
    "role": [],
    "roleTarget": [],
    "agent": []
  }
}
```
2025-12-04 17:39:12 +01:00
Paul Rastoin 3e75e39650 Upgrade command remove duplicated role target (#16281)
# Introduction
On production facing a migration error with
UpdateRoleTargetsUniqueConstraint1764329720503
```ts
Migration "UpdateRoleTargetsUniqueConstraint1764329720503" failed, error: could not create unique index "IDX_ROLE_TARGETS_UNIQUE_AGENT"
query: ROLLBACK
Error during migration run:
QueryFailedError: could not create unique index "IDX_ROLE_TARGETS_UNIQUE_AGENT"
    at PostgresQueryRunner.query (/Users/paulrastoin/ws/twenty/node_modules/typeorm/driver/postgres/PostgresQueryRunner.js:219:19)
    at process.processTicksAndRejections (node:internal/process/task_queues:105:5)
    at async UpdateRoleTargetsUniqueConstraint1764329720503.up (/Users/paulrastoin/ws/twenty/packages/twenty-server/dist/database/typeorm/core/migrations/common/1764329720503-update-role-targets-unique-constraint.js:15:9)
    at async MigrationExecutor.executePendingMigrations (/Users/paulrastoin/ws/twenty/node_modules/typeorm/migration/MigrationExecutor.js:225:17)
    at async DataSource.runMigrations (/Users/paulrastoin/ws/twenty/node_modules/typeorm/data-source/DataSource.js:265:35)
    at async Object.handler (/Users/paulrastoin/ws/twenty/node_modules/typeorm/commands/MigrationRunCommand.js:68:13) {
  query: 'ALTER TABLE "core"."roleTargets" ADD CONSTRAINT "IDX_ROLE_TARGETS_UNIQUE_AGENT" UNIQUE ("workspaceId", "agentId")',
  parameters: undefined,
  driverError: error: could not create unique index "IDX_ROLE_TARGETS_UNIQUE_AGENT"
      at /Users/paulrastoin/ws/twenty/node_modules/pg/lib/client.js:526:17
      at process.processTicksAndRejections (node:internal/process/task_queues:105:5)
      at async PostgresQueryRunner.query (/Users/paulrastoin/ws/twenty/node_modules/typeorm/driver/postgres/PostgresQueryRunner.js:184:25)
      at async UpdateRoleTargetsUniqueConstraint1764329720503.up (/Users/paulrastoin/ws/twenty/packages/twenty-server/dist/database/typeorm/core/migrations/common/1764329720503-update-role-targets-unique-constraint.js:15:9)
      at async MigrationExecutor.executePendingMigrations (/Users/paulrastoin/ws/twenty/node_modules/typeorm/migration/MigrationExecutor.js:225:17)
      at async DataSource.runMigrations (/Users/paulrastoin/ws/twenty/node_modules/typeorm/data-source/DataSource.js:265:35)
      at async Object.handler (/Users/paulrastoin/ws/twenty/node_modules/typeorm/commands/MigrationRunCommand.js:68:13) {
    length: 314,
    severity: 'ERROR',
    code: '23505',
```

As several agent has duplicated role target in database
```sql
SELECT constraint_name, COUNT(*) AS duplicate_groups, SUM(duplicate_count - 1) AS rows_to_delete
FROM (
    SELECT 'IDX_ROLE_TARGET_UNIQUE_API_KEY' AS constraint_name, COUNT(*) AS duplicate_count
    FROM "core"."roleTargets" rt
    WHERE rt."apiKeyId" IS NOT NULL
    GROUP BY rt."workspaceId", rt."apiKeyId"
    HAVING COUNT(*) > 1
    
    UNION ALL
    
    SELECT 'IDX_ROLE_TARGET_UNIQUE_AGENT', COUNT(*)
    FROM "core"."roleTargets" rt
    WHERE rt."agentId" IS NOT NULL
    GROUP BY rt."workspaceId", rt."agentId"
    HAVING COUNT(*) > 1
    
    UNION ALL
    
    SELECT 'IDX_ROLE_TARGET_UNIQUE_USER_WORKSPACE', COUNT(*)
    FROM "core"."roleTargets" rt
    WHERE rt."userWorkspaceId" IS NOT NULL
    GROUP BY rt."workspaceId", rt."userWorkspaceId"
    HAVING COUNT(*) > 1
) AS all_duplicates
GROUP BY constraint_name;
```

with constraint name, duplicated_groups, row_to_delete
`IDX_ROLE_TARGET_UNIQUE_AGENT	 2507	7229`

Please note that only active or suspended workspaces contains duplicated
role target

## Fix
Introduced an upgrade command that will only keep the latest inserted
role target
Swallowing migration error on typeorm atomic migration ( still required
for self host new instances etc )

```ts
[Nest] 91886  - 12/03/2025, 2:56:28 PM     LOG [DeduplicateRoleTargetsCommand] Running command on workspace SOME_WORKSPACE_ID 2587/2587
flatFieldMetadataMaps,flatIndexMaps,flatObjectMetadataMaps out of 298
query: SELECT version();
[Nest] 91886  - 12/03/2025, 2:56:29 PM     LOG [DeduplicateRoleTargetsCommand] Command completed!
```

## Test
Tested through an extract in local, tested both the upgrade command and
the swallowed migration
test
2025-12-03 16:17:46 +01:00
Marie 77409b6eb2 [Requires "warm" cache flush (no immediate downtime before flush)] Migrate viewGroup.fieldMetadataId -> view.mainGroupByFieldMetadataId (1/3) (#16206)
In this PR (1/3)
- introduce view.mainGroupByFieldMetadataId as the new reference
determining which fieldMetadataId is used in a grouped view, in order to
deprecate viewGroup.fieldMetadataId which creates inconsistencies.
view.mainGroupByFieldMetadataId is now filled at every view creation,
though not in use yet.
- Introduce a command to backfill view.mainGroupByFieldMetadataId for
existing views + delete all viewGroup.fieldMetadataId with a
fieldMetadataId that is not view.mainGroupByFieldMetadataId. (It should
concern 37 active workspaces)
- Temporarily disable the option to change a grouped view's
fieldMetadataId as for now it creates inconsistencies. This feature can
be reintroduced when we have done the full migration.

In a next PR
- (2/3) use view.mainGroupByFieldMetadataId instead of
viewGroup.fieldMetadataId. In FE we may keep viewGroup.fieldMetadataId
as a state (TBD). View groups will now be created / deleted as a side
effect of view's mainGroupByFieldMetadataId update.
- (3/3) remove viewGroup.fieldMetadataId

---------

Co-authored-by: Charles Bochet <charles@twenty.com>
2025-12-02 16:31:07 +01:00
Weiko 13e283fc3a Rename roleTargets -> roleTarget (#16247) 2025-12-02 14:39:54 +01:00
Etienne 00e970bdf4 Null - Second command - Cleaning remaining empty values (#16241)
Because of [this code I forget to
clean](https://github.com/twentyhq/twenty/pull/16217),
- __workspace created between 1.12.0 and 1.12.2 (from friday 28 nov PM >
monday 1 dec PM)__ have standard objects with empty string default value
set on related TEXT type column (but default value null on field
metadata) (ex: jobTitle on person)
- __custom objects created between 1.12.0 and 1.12.2 (from friday 28 nov
PM > monday 1 dec PM)__ have "name" TEXT field with empty string default
value set + NOT NULL constraint on column

Command cleans data and updates table structure
2025-12-02 13:39:39 +01:00
Etienne 68c429a54a Null equivalence - remove feature flag (#16222) 2025-12-01 19:06:24 +01:00
Paul Rastoin 9f62188ba6 Field and object metadata naming does not refer to v2 (#16187)
Related https://github.com/twentyhq/core-team-issues/issues/1911
2025-11-28 18:06:44 +01:00
Charles Bochet eb362c6d5f Update workspace entities to make all TEXT nullable (#16144)
Follow up on #15926

---------

Co-authored-by: Etienne <45695613+etiennejouan@users.noreply.github.com>
Co-authored-by: guillim <guigloo@msn.com>
2025-11-28 10:50:38 +01:00
Etienne ca5bd76c6a Null equivalence - migration command (#16018)
Awaiting https://github.com/twentyhq/twenty/pull/15926 approval, before
un-drafting it

---------

Co-authored-by: prastoin <paul@twenty.com>
2025-11-27 13:18:56 +00:00
Paul Rastoin 74eab77539 Refactor upgrade devx to allow configuring workspaces status to pass over (#16066)
# Introduction
We need to be able to create custom workspace application on all
workspaces, even pending and ongoing etc
Right now the upgrade devx only allows and expect active or suspended
workspace to be passed to runOnWorkspace.

## WorkspacesMigrationRunner
Created an intermediate class `WorkspacesMigrationRunner` that expect an
array `WorkspaceStatus` to be fetched for the current command to be run
on
The `ActiveOrSuspendedCommandRunner` statically passes both `SUSPENDED`
and `ACTIVE`, whereas the create workspace custom application passed all
the enum values

## DataSource
Workspace that are not fully init don't have a `workspace_schema` so
they don't have `dataSource`
Made a not very elegant check to see if current workspace we're about to
create dataSource on has one historically
Which means that dataSource is now optional, it had only one impact on
an existing command and the desired devx will become consuming existing
services that do not expect dataSource ( or at least yet )
2025-11-26 12:53:33 +01:00
Charles Bochet b1c03b533f Fix upgrade command messaging (#16067) 2025-11-25 18:14:51 +01:00
Charles Bochet 8455ecc3e8 Add import scheduled status to messaging sync (#16058)
We have introduced to new syncStage statuses:
`messageChannel.MESSAGES_IMPORT_SCHEDULED` and
`calendarChannel.CALENDAR_EVENTS_IMPORT_SCHEDULED`

We need to make sure all existing workspaces have it
2025-11-25 15:27:58 +01:00
Félix Malfait e7ebf51e50 Replace agent handoff system with planning-based router (#16003)
## Overview

This PR replaces the dynamic agent handoff system with a more
predictable planning-based router that decides upfront how to handle
multi-agent coordination.

## Major Changes

### 🔄 Architecture Shift: Handoffs → Planning

**Removed:**
- `AgentHandoffEntity` and handoff tracking system
- `AgentHandoffService` and `AgentHandoffExecutorService`
- Dynamic agent-to-agent transfers during execution
- Handoff tool generation and description templates

**Added:**
- `AiRouterService` with two strategies: `simple` (single agent) and
`planned` (multi-agent)
- `AgentPlanExecutorService` for executing multi-step plans
- Plan validation (cycle detection, dependency resolution)
- `UnifiedRouterResult` type with discriminated union

### 🤖 New Standard Agents

Added two new specialized agents:
- **Researcher Agent**: Web search, fact-finding, competitive
intelligence
- **Code Agent**: TypeScript function generation for serverless
workflows

### 🏗️ Router Refactoring (Latest)

Split router responsibilities into focused services:
- `AiRouterStrategyDeciderService`: Decides simple vs planned strategy
- `AiRouterPlanGeneratorService`: Generates and validates execution
plans
- `AiRouterService`: Coordinates between services (reduced from 426→275
lines)

### ⚙️ Configuration Improvements

- Added `outputStrategy` to agent definitions (`direct` vs `synthesize`)
- Removed hardcoded special cases for workflow-builder
- Added `plannerModel` field to workspace entity
- Increased `MAX_STEPS` from 10 to 25 for complex workflows

### 📝 Agent Prompt Refinements

Significantly simplified prompts for better clarity:
- Workflow Builder: 51→36 lines
- Helper: 49→28 lines
- Data Manipulator: Enhanced with sorting guidance

### 🔍 Enhanced Debugging

- Plan reasoning and step count in data message parts
- Router debug info with token usage tracking
- Better logging throughout execution pipeline

## Benefits

1. **Simpler Mental Model**: Router decides upfront vs dynamic transfers
2. **Better Predictability**: Users see the plan before execution
3. **Cleaner Architecture**: SRP with focused services
4. **Configuration Over Code**: Agent behavior via config, not hardcoded
logic
5. **Plan Validation**: Catches invalid dependencies and cycles

## Migration Notes

- Database migration removes `agentHandoff` table
- Adds `plannerModel` column to workspace table
- No API breaking changes (agent endpoints unchanged)

## Testing

- Integration tests updated to remove handoff dependencies
- Agent tool test utilities simplified
- Plan validation covered by new logic

## Next Steps (Future PRs)

- Parallel execution of independent plan steps
- Dynamic re-planning based on results
- Plan caching for common routing patterns
- Error recovery strategies in plan executor
2025-11-25 12:10:14 +01:00