Commit Graph

3069 Commits

Author SHA1 Message Date
Paul Rastoin 5016c25daa Remove viewGroup v1 implem (#16178)
# Introduction
Removing v1 implementation of view groups and both view group and view
field relicas front fetchers

Related https://github.com/twentyhq/core-team-issues/issues/1911
2025-11-28 16:36:28 +01:00
Charles Bochet 9387680020 Rollback standard id removal on relation object creation (#16177) 2025-11-28 15:11:45 +01:00
Weiko 470888a23a Fix missing metadata version in legacy datasource (#16173) 2025-11-28 11:54:54 +00:00
Raphaël Bosi 9bc58a4ef9 Release line chart and pie chart (#16166)
- Remove the feature flag for these two charts.
- Reorder the charts
- Hide gauge chart
2025-11-28 10:17:03 +00:00
Charles Bochet eb362c6d5f Update workspace entities to make all TEXT nullable (#16144)
Follow up on #15926

---------

Co-authored-by: Etienne <45695613+etiennejouan@users.noreply.github.com>
Co-authored-by: guillim <guigloo@msn.com>
2025-11-28 10:50:38 +01:00
Weiko 9620a4b0ba Optimize EntityMetadata caching in GlobalWorkspaceDataSource (#16146)
## Context
EntityMetadata was being rebuilt from scratch on every
findMetadata()/getMetadata() call (~20 times per request). This involved
running EntitySchemaTransformer.transform() and
EntityMetadataBuilder.build() repeatedly, causing unnecessary CPU
overhead.

## Implementation
Cache entityMetadatas in ORMWorkspaceContext: Build EntityMetadata once
during workspace context initialization instead of on every metadata
lookup
Remove redundant entitySchemas caching: Since flatMetadata is already
cached, the additional Redis cache for entitySchemaOptions was
unnecessary overhead
Remove WorkspaceEntitiesStorage: Replaced with direct lookup from
FlatObjectMetadataMap
Simplify getObjectMetadataFromEntityTarget: Now only accepts string
targets, using flat metadata maps directly

Also:
Removed unused injections in some services
2025-11-28 10:09:52 +01:00
Marie fd9ea2f5ee [groupBy] Fix order by nested date field (#16135)
Fixes https://github.com/twentyhq/core-team-issues/issues/1935
2025-11-28 09:02:06 +00:00
Abdullah. 04b01170ed Introduce a workspace member page. (#16031)
- Refactored workspace member details into a focused Infos-only page.
- Aligned the flow with SettingsProfile, including controlled name
inputs, debounced saves, and stable instance IDs.
- Added a dedicated member-picture upload flow. Introduced the
MemberPictureUploader, connected to the
uploadWorkspaceMemberProfilePicture mutation.
- Backend now includes a workspace-member resolver/module for
profile-picture uploads. The endpoint is permission-guarded, streams
files through FileUploadService, and returns the signed file without
modifying the member entity.

<!-- CURSOR_SUMMARY -->
---

> [!NOTE]
> Adds a workspace member detail page with picture/name management,
integrates a new avatar upload mutation, and updates list routing;
replaces the old profile picture uploader across profile and onboarding.
> 
> - **Frontend**
>   - **Settings Members**:
> - Add `pages/settings/members/SettingsWorkspaceMember` with
`MemberInfosTab`, `MemberNameFields`, and `MemberEmailField` for
viewing/editing member info.
> - Update routes in `SettingsRoutes` and add
`SettingsPath.WorkspaceMemberPage`.
> - Update `SettingsWorkspaceMembers` to navigate to member detail on
row click and simplify row actions (remove dropdown menu).
>   - **Avatar Upload**:
> - Introduce `WorkspaceMemberPictureUploader` using
`uploadWorkspaceMemberProfilePicture` mutation.
> - Replace `ProfilePictureUploader` in `SettingsProfile` and
`onboarding/CreateProfile`.
>   - **GraphQL (client)**:
> - Add `uploadWorkspaceMemberProfilePicture` mutation types/hooks in
`generated(-metadata)/graphql.ts`.
> - **Backend**
> - Add `UserWorkspaceResolver` with
`uploadWorkspaceMemberProfilePicture` mutation guarded by
`WorkspaceAuthGuard` and `SettingsPermissionGuard` (WORKSPACE_MEMBERS),
using `FileUploadService`.
> - Register resolver and `PermissionsModule` in `UserWorkspaceModule`.
> 
> <sup>Written by [Cursor
Bugbot](https://cursor.com/dashboard?tab=bugbot) for commit
359652f8c94d093a69469969874131e525d3dc6f. This will update automatically
on new commits. Configure
[here](https://cursor.com/dashboard?tab=bugbot).</sup>
<!-- /CURSOR_SUMMARY -->

---------

Co-authored-by: Félix Malfait <felix@twenty.com>
2025-11-28 08:23:51 +00:00
Paul Rastoin e53e0d266d Remove view filter v1 implem (#16154)
# Introduction
Removing view filter v1 implem 

Related https://github.com/twentyhq/core-team-issues/issues/1911
2025-11-28 00:17:17 +01:00
Abdul Rahman a343bc1aee feat: workflow agent node permissions tab (#16092) 2025-11-28 02:57:33 +05:30
Baptiste Devessier eaac569812 Fix variable usage in Search Record workflow action (#16147)
Closes https://github.com/twentyhq/twenty/issues/16141

---------

Co-authored-by: prastoin <paul@twenty.com>
2025-11-27 18:56:56 +01:00
Abdullah. 3b8db734a5 fix: glob CLI command injection via -c/--cmd executes matches with shell:true (#16139)
Resolves [Dependabot Alert
318](https://github.com/twentyhq/twenty/security/dependabot/318),
[Dependabot Alert
321](https://github.com/twentyhq/twenty/security/dependabot/321) and
[Dependabot Alert
322](https://github.com/twentyhq/twenty/security/dependabot/322).

<!-- CURSOR_SUMMARY -->
---

> [!NOTE]
> Bumps glob to 10.5.0 across packages and adds @types/node and
twenty-sdk to rollup-engine dependencies.
> 
> - **Dependencies**:
>   - Upgrade `glob` to `10.5.0` across multiple `yarn.lock` files.
>   - In `packages/twenty-apps/community/rollup-engine`:
>     - Add `@types/node@^24.7.2` (adds `undici-types`).
>     - Add `twenty-sdk@0.0.3`.
> 
> <sup>Written by [Cursor
Bugbot](https://cursor.com/dashboard?tab=bugbot) for commit
0aee78e0fa90918320aa35bf38d89c3a4c801666. This will update automatically
on new commits. Configure
[here](https://cursor.com/dashboard?tab=bugbot).</sup>
<!-- /CURSOR_SUMMARY -->

Co-authored-by: Félix Malfait <felix.malfait@gmail.com>
2025-11-27 16:36:36 +01:00
Etienne 3590bf1e83 Null equivalence - fix on dashboard entity (#16136) 2025-11-27 14:18:43 +00:00
Weiko da626f70b7 optimize buildFieldMapsFromFlatObjectMetadata usages (#16132)
This newly introduced util can be a bit expensive especially when done
recursively.
This PR improves that


File | Pattern Fixed | Impact
-- | -- | --
format-result.util.ts | Recursive array/object processing | N array
items → 1 call
format-data.util.ts | Recursive array processing | N array items → 1
call
process-nested-relations-v2.helper.ts | Duplicate call in call chain | 2
calls → 1 call
common-result-getters.service.ts | Per-record processing in array | N
records → 1 call
compute-relation-connect-query-configs.util.ts | Nested loop (entities ×
connect fields) | N×M calls → 1 call
2025-11-27 15:01:51 +01:00
Etienne 57ae12ff7c Null equivalence - update filter (#16123) 2025-11-27 13:31:39 +00:00
github-actions[bot] 76ed82b598 i18n - translations (#16130)
Created by Github action

---------

Co-authored-by: github-actions <github-actions@twenty.com>
2025-11-27 14:23:42 +01:00
Etienne ca5bd76c6a Null equivalence - migration command (#16018)
Awaiting https://github.com/twentyhq/twenty/pull/15926 approval, before
un-drafting it

---------

Co-authored-by: prastoin <paul@twenty.com>
2025-11-27 13:18:56 +00:00
Weiko 1607aebcc6 Deprecate object metadata maps in favor of flat entities (#16080)
## Context
Deprecating the old objectMetadataMap type in favour of split flat
entities to match with our new caching.
In the long run, trying to achieve:
- Better performance through caching
- Consistent data access patterns across the codebase
- Reduced database queries

Now that everything is based on flat entities, which are cached, we can
finish the refactoring of workspace context cache which should already
improve performances.
Then the last step will be to consume that new cache in the new global
datasource to get rid of the many workspace datasources stored in the
server
2025-11-27 13:43:34 +01:00
github-actions[bot] 44f4203d58 i18n - translations (#16125)
Created by Github action

---------

Co-authored-by: github-actions <github-actions@twenty.com>
2025-11-27 13:01:57 +01:00
nitin d7f8c2d338 Center metric layer on pie chart + cleanup (#16105)
https://github.com/user-attachments/assets/bb48bebe-d0be-4006-8f63-e686c70c0011

---------

Co-authored-by: bosiraphael <raphael.bosi@gmail.com>
2025-11-27 11:15:48 +00:00
Guillim 1a45576990 Morph-add-new-object-destination (#16027)
Add new object target to an existing morph relation (backend only)

Fixes https://github.com/twentyhq/core-team-issues/issues/1898

---------

Co-authored-by: prastoin <paul@twenty.com>
2025-11-27 11:10:13 +00:00
Marie 9c9a01d55a [groupBy][Requires cache flush] Add WEEK date granularity (#16099)
Closes https://github.com/twentyhq/core-team-issues/issues/1921

https://github.com/user-attachments/assets/bf400ec1-ce25-4d9e-b875-774168452514
2025-11-27 10:22:57 +00:00
Félix Malfait 4f20fd35c5 feat: Add Agent Evaluation System and Refactor AI Modules (#16111)
## Summary

This PR introduces a comprehensive agent evaluation system and refactors
the AI module structure for better organization.

## Key Changes

### 🎯 Agent Evaluation System
- Added **Agent Turn Evaluation** entities, DTOs, and database schema
- New GraphQL mutations: `evaluateAgentTurn` and `runEvaluationInput`
- Added `evaluationInputs` field to Agent entity for storing test inputs
- New `AgentTurnGraderService` for automatic turn evaluation
- Added evaluation UI with new **Evals** and **Logs** tabs in agent
detail pages

### 🏗️ Entity & Module Refactoring
- Renamed `AgentChatMessage` → `AgentMessage` for clarity
- Consolidated chat entities: `AgentMessage`, `AgentTurn`, and
`AgentChatThread`
- Reorganized AI modules under `ai/` subdirectory structure
- Updated imports across codebase to reflect new module paths

### 🤖 New Agents & Roles
- Added **Dashboard Builder Agent** for dashboard creation and
management
- Added **Dashboard Manager Role** with appropriate permissions
- Updated role permissions to be more granular (users vs agents vs API
keys)

### 🔐 Permission System Updates
- Added `HTTP_REQUEST_TOOL` permission flag
- Updated Workflow Manager role permissions (restricted tool access)
- Enhanced permission flag types to differentiate between user/agent/API
key contexts
- Added `isRelevantForAgents`, `isRelevantForApiKeys`,
`isRelevantForUsers` to permission flags

### 📨 Message Role Enhancement
- Added `system` role to `AgentMessageRole` enum (alongside
user/assistant)
- Updated message handling to support system prompts

### 🎨 UI/UX Improvements
- New tabs in agent detail: **Evals** and **Logs**
- Added turn detail page: `/ai/agents/:agentId/turns/:turnId`
- Fixed text overflow in `SettingsListItemCardContent`
- Updated role applicability labels ("Assignable to Workspace Members")

### 🛠️ Technical Improvements
- Fixed Zod schema validation for UUID and Date fields (use string
validators)
- Updated `ToolRegistryService` to properly register HTTP tool with
permission flag
- Enhanced error handling in agent execution services
- Updated database migrations for new entity schema

## Database Migrations
- `1764210000000-add-system-role-to-agent-message.ts`
- `1764220000000-add-evaluation-inputs-to-agent.ts`
- `1764200000000-add-agent-turn-evaluation.ts`
- `1764100000000-refactor-agent-chat-entities.ts`

## Testing
- [ ] Agent evaluation flow tested
- [ ] Dashboard Builder agent tested
- [ ] Permission system validated
- [ ] UI tabs and navigation tested
- [ ] Database migrations run successfully

## Breaking Changes
⚠️ **Entity Rename**: `AgentChatMessage` renamed to `AgentMessage` -
GraphQL queries need updating

## Related Issues
<!-- Link any related issues here -->

## Screenshots
<!-- Add screenshots if applicable -->
2025-11-27 08:25:40 +01:00
Charles Bochet b1d1bcb712 Fix messaging import (#16112) 2025-11-26 23:13:12 +01:00
github-actions[bot] ee6ac6bb4c i18n - docs translations (#16109)
Created by Github action

---------

Co-authored-by: github-actions <github-actions@twenty.com>
2025-11-26 19:22:05 +01:00
github-actions[bot] 3c658b209d i18n - translations (#16106)
Created by Github action

Co-authored-by: github-actions <github-actions@twenty.com>
2025-11-26 18:46:23 +01:00
Etienne 70f48ba445 Security - Add complexity max on gql queries (#16069)
closes https://github.com/twentyhq/private-issues/issues/348
closes https://github.com/twentyhq/private-issues/issues/352
closes https://github.com/twentyhq/private-issues/issues/353
closes https://github.com/twentyhq/private-issues/issues/354
2025-11-26 18:36:21 +01:00
Charles Bochet 5202e2b2db Refactor error messages messaging (#16094)
We should try catch locally gmail errors when:
- fetching message list
- fetching messages
- refreshing aliases
- fetching folders
2025-11-26 18:03:44 +01:00
Raphaël Bosi fc4dbb80de [DASHBOARDS] Create display legend setting + animations (#16089)
## PR Description
- Create display legend setting
- Create pagination inside the legend
- Animate the legend on enter/exit
- Animate the transition between pages

## Video QA


https://github.com/user-attachments/assets/a68a4ea3-195f-4399-88f7-f83bc1a39cc4
2025-11-26 17:03:23 +00:00
Guillim 7f441d741b Timelineactivites-views-seeder (#16103)
seeder for Timelineactivites views

Fixes https://github.com/twentyhq/core-team-issues/issues/1904


<img width="850" height="348" alt="CleanShot 2025-11-26 at 17 30 32"
src="https://github.com/user-attachments/assets/41a311c1-0b60-4201-885f-d0e8400dd6d5"
/>
2025-11-26 18:00:06 +01:00
neo773 4120c191f8 fix scheduling after folder actions are processed (#16097) 2025-11-26 17:09:06 +01:00
Paul Rastoin 996ccd8353 Non composite and non morph or relation field update fix (#16091) 2025-11-26 11:54:04 +00:00
Paul Rastoin 74eab77539 Refactor upgrade devx to allow configuring workspaces status to pass over (#16066)
# Introduction
We need to be able to create custom workspace application on all
workspaces, even pending and ongoing etc
Right now the upgrade devx only allows and expect active or suspended
workspace to be passed to runOnWorkspace.

## WorkspacesMigrationRunner
Created an intermediate class `WorkspacesMigrationRunner` that expect an
array `WorkspaceStatus` to be fetched for the current command to be run
on
The `ActiveOrSuspendedCommandRunner` statically passes both `SUSPENDED`
and `ACTIVE`, whereas the create workspace custom application passed all
the enum values

## DataSource
Workspace that are not fully init don't have a `workspace_schema` so
they don't have `dataSource`
Made a not very elegant check to see if current workspace we're about to
create dataSource on has one historically
Which means that dataSource is now optional, it had only one impact on
an existing command and the desired devx will become consuming existing
services that do not expect dataSource ( or at least yet )
2025-11-26 12:53:33 +01:00
Charles Bochet 4d0c469157 Fix message import scheduled (#16071)
Co-authored-by: neo773 <neo773@protonmail.com>
2025-11-26 08:54:53 +01:00
Marie 6c1c78ea3d [fix] User have no firstName nor lastName (#16057)
Closes https://github.com/twentyhq/twenty/issues/15692, fixes
https://github.com/twentyhq/twenty/issues/14678

The issue was:
- When a user is signing up, they are asked for their first name and
last name with which we fulfill their workspaceMember entity for the
workspace they are signing up to (which is the one they are creating if
they are not joining an existing workspace). user.firstName and
user.lastName remain empty
- When we create a record, we are using user.firstName and user.lastName
to fulfill the text field "createdByName", which intends to save the
name of the creator at the moment of the creation. This field is then
use 1) when filtering on createdBy (as a trick to be able to filter by
this, since we don't have filters on nested fields, ie we dont have
filters on person.createdByWorkspaceMember) 2) to display the user
creator when a user has left a workspace - otherwise we rely on the
dynamic createdByWorkspaceMember.firstName and lastName
- So filtering on createdBy was not working as createdByName is empty.
We did not see that in dev mode because we have seeded users with names

The fix
- When we create a record, we use workspaceMember.firstName and
workspaceMember.lastName as we should.
- When an existing user joins a workspace, they are asked to give their
name again so that we set their workspaceMember name
- This will not fix the history of the records (they will still have
createdByName empty so the filter will not work properly), but a command
to fix it would be very long as it would iterate over all the records of
each active workspace

The long-term view is to get rid of user and to store the name in
userWorkspace
2025-11-25 17:23:43 +00:00
Charles Bochet b1c03b533f Fix upgrade command messaging (#16067) 2025-11-25 18:14:51 +01:00
neo773 f3416d435c move folder cron to message-list-fetch (#16062) 2025-11-25 18:06:38 +01:00
neo773 1740a2217a Update channel sync service to immediately enqueue jobs (#16064) 2025-11-25 18:05:32 +01:00
Etienne 71724de7dd Security - disable gql introspection for non-auth user (#16047)
closes https://github.com/twentyhq/private-issues/issues/351
closes https://github.com/twentyhq/private-issues/issues/350

Before, introspection query works without token. After, fails.

```

query IntrospectionQuery {
  __schema {
    queryType {
      name
    }
    mutationType {
      name
    }
    subscriptionType {
      name
    }
    types {
      ...FullType
    }
    directives {
      name
      description
      locations
      args {
        ...InputValue
      }
    }
  }
}

fragment FullType on __Type {
  kind
  name
  description
  fields(includeDeprecated: true) {
    name
    description
    args {
      ...InputValue
    }
    type {
      ...TypeRef
    }
    isDeprecated
    deprecationReason
  }
  inputFields {
    ...InputValue
  }
  interfaces {
    ...TypeRef
  }
  enumValues(includeDeprecated: true) {
    name
    description
    isDeprecated
    deprecationReason
  }
  possibleTypes {
    ...TypeRef
  }
}

fragment InputValue on __InputValue {
  name
  description
  type {
    ...TypeRef
  }
  defaultValue
}

fragment TypeRef on __Type {
  kind
  name
  ofType {
    kind
    name
    ofType {
      kind
      name
      ofType {
        kind
        name
        ofType {
          kind
          name
          ofType {
            kind
            name
            ofType {
              kind
              name
              ofType {
                kind
                name
              }
            }
          }
        }
      }
    }
  }
}
```
2025-11-25 15:39:29 +01:00
Charles Bochet 8455ecc3e8 Add import scheduled status to messaging sync (#16058)
We have introduced to new syncStage statuses:
`messageChannel.MESSAGES_IMPORT_SCHEDULED` and
`calendarChannel.CALENDAR_EVENTS_IMPORT_SCHEDULED`

We need to make sure all existing workspaces have it
2025-11-25 15:27:58 +01:00
Etienne 2028a8f9be Null equivalence - Fix (#16050)
@charlesBochet 
In workflow codebase, NULL (instead of empty object) is expected on
workflow version object step field, when a new workflow is created for
example.
(packages/twenty-front/src/modules/workflow/workflow-diagram/utils/generateWorkflowDiagram.ts
- 42)
This case is not isolated and it creates many issues.

We decided to format NULL value to equivalent (empty string for text
field, empty object for raw_json) but it seems it complicates the dev x.

To unlock @Devessier I prefer revert the logic, the time we discuss how
to solve this cases.
2025-11-25 14:07:32 +01:00
github-actions[bot] 26169a2136 i18n - translations (#16052)
Created by Github action

---------

Co-authored-by: github-actions <github-actions@twenty.com>
2025-11-25 12:46:03 +01:00
Félix Malfait e7ebf51e50 Replace agent handoff system with planning-based router (#16003)
## Overview

This PR replaces the dynamic agent handoff system with a more
predictable planning-based router that decides upfront how to handle
multi-agent coordination.

## Major Changes

### 🔄 Architecture Shift: Handoffs → Planning

**Removed:**
- `AgentHandoffEntity` and handoff tracking system
- `AgentHandoffService` and `AgentHandoffExecutorService`
- Dynamic agent-to-agent transfers during execution
- Handoff tool generation and description templates

**Added:**
- `AiRouterService` with two strategies: `simple` (single agent) and
`planned` (multi-agent)
- `AgentPlanExecutorService` for executing multi-step plans
- Plan validation (cycle detection, dependency resolution)
- `UnifiedRouterResult` type with discriminated union

### 🤖 New Standard Agents

Added two new specialized agents:
- **Researcher Agent**: Web search, fact-finding, competitive
intelligence
- **Code Agent**: TypeScript function generation for serverless
workflows

### 🏗️ Router Refactoring (Latest)

Split router responsibilities into focused services:
- `AiRouterStrategyDeciderService`: Decides simple vs planned strategy
- `AiRouterPlanGeneratorService`: Generates and validates execution
plans
- `AiRouterService`: Coordinates between services (reduced from 426→275
lines)

### ⚙️ Configuration Improvements

- Added `outputStrategy` to agent definitions (`direct` vs `synthesize`)
- Removed hardcoded special cases for workflow-builder
- Added `plannerModel` field to workspace entity
- Increased `MAX_STEPS` from 10 to 25 for complex workflows

### 📝 Agent Prompt Refinements

Significantly simplified prompts for better clarity:
- Workflow Builder: 51→36 lines
- Helper: 49→28 lines
- Data Manipulator: Enhanced with sorting guidance

### 🔍 Enhanced Debugging

- Plan reasoning and step count in data message parts
- Router debug info with token usage tracking
- Better logging throughout execution pipeline

## Benefits

1. **Simpler Mental Model**: Router decides upfront vs dynamic transfers
2. **Better Predictability**: Users see the plan before execution
3. **Cleaner Architecture**: SRP with focused services
4. **Configuration Over Code**: Agent behavior via config, not hardcoded
logic
5. **Plan Validation**: Catches invalid dependencies and cycles

## Migration Notes

- Database migration removes `agentHandoff` table
- Adds `plannerModel` column to workspace table
- No API breaking changes (agent endpoints unchanged)

## Testing

- Integration tests updated to remove handoff dependencies
- Agent tool test utilities simplified
- Plan validation covered by new logic

## Next Steps (Future PRs)

- Parallel execution of independent plan steps
- Dynamic re-planning based on results
- Plan caching for common routing patterns
- Error recovery strategies in plan executor
2025-11-25 12:10:14 +01:00
github-actions[bot] a47b7dfed0 i18n - translations (#16041)
Created by Github action

---------

Co-authored-by: github-actions <github-actions@twenty.com>
2025-11-25 10:36:15 +01:00
Marie 46d1ea6505 [groupBy] groupBy relation fields (#15951)
Example query
Here person has 
- a N - 1 relationship with company
- a N - 1 morph relationship with pet or company

<img width="862" height="374" alt="image"
src="https://github.com/user-attachments/assets/59bc9b82-c943-43de-ad82-d3393b76904b"
/>
<img width="415" height="629" alt="image"
src="https://github.com/user-attachments/assets/9a3176bc-99cd-4983-8611-68ca3a2cf527"
/>

truncated response
<img width="299" height="447" alt="image"
src="https://github.com/user-attachments/assets/45af0322-9e66-4eae-8353-6c0dda487bbe"
/>

We don't allow grouping by relations of relations.

Left to do
- rest api
- tests on permissions
2025-11-25 09:03:16 +00:00
neo773 316aec3c40 fix: prevent NUMERIC field type creation via API (#16038)
Block users from creating NUMERIC, POSITION, and TS_VECTOR fields via
the API as these are system-only types. Users should use NUMBER instead
of NUMERIC.

/closes #16023
2025-11-25 09:51:48 +01:00
Weiko 04562b11fb Migrate metadata cache (#16030)
## Context
Deprecating legacy ObjectMetadata from cache in favor of flat entities.
Introducing utils to build byName/byNameSingular/byNamePlural in
isolated cases

## Next
- I had to introduce a util to build from flat to legacy
objectMetadataMaps, we should instead use flat maps directly when needed
(datasource, schema generation, etc)
- Deprecate metadata version in the cache
- Use the new cache strategy for flat entities with permissions and
feature flags and inject in the global datasource context
2025-11-24 19:50:47 +01:00
Etienne 208c0857ee common api - null equivalence (#15926)
closes https://github.com/twentyhq/core-team-issues/issues/1629

To do before requesting review : 
- filter update

Migration to come in an other PR


Strat : 
1/  Null transformation

- [x] Transform NULL equivalent value to NULL in field validation in
common api - pre-query - with feature flag
- [ ] Same logic in ORM (Not done, complex to handle feature flag here)
- [x] Transform NULL value to equivalent in data formatting in ORM -
post-query

2/ Migration (in other PR) for fieldMetadata not nullable with default
defaultValue (empty string, ...)

- [ ] Remove NOT NULL db constraint
- [ ] Update record value to NULL
- [ ] Update field metadata : isNullable:true
- [ ] Update uniqueIndex whereClause (also for standard uniqueIndex) 
- [ ] Activate feature flag

3/ Update metadata creation

- [x] No more default default value
- [x] Update standard field nullability
- [x] Remove index default whereClause for standard field

4/ Update filter

- [x] When filtering on NULL or empty string, be sure all records are
returned (the one with NULL + the one with "")

5/ Test

- [ ] Strat. to do
2025-11-24 18:57:47 +01:00
Paul Rastoin 439adc6ac0 Restore transaction on WorkspaceCustomApplicationIdNonNullable1763977334519 failure (#16032)
# Introduction
When a transaction query fails it gets aborted, even if we catch the js
exception typeorm ack it and fails
By adding a save point we isolate the issue and restore the transaction
🥷

## Through database:migrate:prod
Tested but lost logs

## Upgrade
```ts

➜  twenty-server git:(fix-migration-runner) ✗ npx nx command twenty-server upgrade

   ✔  3/3 dependent project tasks succeeded [3 read from cache]

   Hint: you can run the command with --verbose to see the full dependent project outputs

———————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————


> nx run twenty-server:build  [existing outputs match the cache, left as is]

> rimraf dist

> nest build --path ./tsconfig.build.json

>  SWC  Running...
Successfully compiled: 3747 files with swc (65.82ms)

> nx run twenty-server:command upgrade

query: SELECT * FROM current_schema()
query: SELECT version();
[Nest] 82844  - 11/24/2025, 6:05:11 PM     LOG [NestFactory] Starting Nest application...
[Nest] 82844  - 11/24/2025, 6:05:11 PM     LOG [InstanceLoader] CommandRootModule dependencies initialized
[Nest] 82844  - 11/24/2025, 6:05:11 PM     LOG [InstanceLoader] CommandModule dependencies initialized
// ...
[Nest] 82844  - 11/24/2025, 6:05:11 PM     LOG [InstanceLoader] WorkflowApiModule dependencies initialized
[Nest] 82844  - 11/24/2025, 6:05:11 PM     LOG [InstanceLoader] AuthModule dependencies initialized
[Nest] 82844  - 11/24/2025, 6:05:11 PM     LOG [PgPoolSharedService] Pool sharing will use max 10 connections per pool with 600000ms idle timeout and allowExitOnIdle=true
[Nest] 82844  - 11/24/2025, 6:05:11 PM     LOG [PgPoolSharedService] pg.Pool patched successfully by this service instance.
[Nest] 82844  - 11/24/2025, 6:05:11 PM     LOG [PgPoolSharedService] Pg pool sharing initialized - pools will be shared across tenants
[Nest] 82844  - 11/24/2025, 6:05:11 PM   DEBUG [PgPoolSharedService] No active pg pools to log stats for
[Nest] 82844  - 11/24/2025, 6:05:11 PM   DEBUG [PgPoolSharedService] Pool statistics logging enabled (30s interval)
[Nest] 82844  - 11/24/2025, 6:05:11 PM     LOG [PgPoolSharedService] Created new shared pg Pool for key "localhost|5432|postgres||no-ssl" with 10 max connections and 600000 ms idle timeout. Total pools: 1
[Nest] 82844  - 11/24/2025, 6:05:11 PM   DEBUG [PgPoolSharedService] Pool[localhost|5432|postgres||no-ssl]: New connection established
[Nest] 82844  - 11/24/2025, 6:05:11 PM   DEBUG [PgPoolSharedService] Pool[localhost|5432|postgres||no-ssl]: Client acquired from pool
[Nest] 82844  - 11/24/2025, 6:05:11 PM   DEBUG [PgPoolSharedService] Pool[localhost|5432|postgres||no-ssl]: Client acquired from pool
[Nest] 82844  - 11/24/2025, 6:05:11 PM   DEBUG [PgPoolSharedService] Pool[localhost|5432|postgres||no-ssl]: Client acquired from pool
[Nest] 82844  - 11/24/2025, 6:05:11 PM     LOG [DatabaseConfigDriver] [INIT] Loading initial config variables from database
[Nest] 82844  - 11/24/2025, 6:05:11 PM     LOG [DatabaseConfigDriver] [INIT] Config variables loaded: 0 values found in DB, 59 falling to env vars/defaults
[Nest] 82844  - 11/24/2025, 6:05:11 PM     LOG [UpgradeCommand] Initialized upgrade context with:
   - currentVersion (migrating to): 1.12.0
   - fromWorkspaceVersion: 1.11.0
   - 3 commands
[Nest] 82844  - 11/24/2025, 6:05:11 PM     LOG [UpgradeCommand] Running global database migrations
[Nest] 82844  - 11/24/2025, 6:05:11 PM     LOG [UpgradeCommand] Running core datasource migrations...
[Nest] 82844  - 11/24/2025, 6:05:12 PM     LOG [UpgradeCommand] query: SELECT * FROM current_schema()
query: CREATE EXTENSION IF NOT EXISTS "uuid-ossp"
query: SELECT version();
query: SELECT * FROM "information_schema"."tables" WHERE "table_schema" = 'core' AND "table_name" = '_typeorm_migrations'
query: SELECT * FROM "core"."_typeorm_migrations" "_typeorm_migrations" ORDER BY "id" DESC
46 migrations are already loaded in the database.
47 migrations were found in the source code.
AddCanBeUninstalledColumnToApplication1763731277403 is the last executed migration. It was executed on Fri Nov 21 2025 14:21:17 GMT+0100 (Central European Standard Time).
1 migrations are new migrations must be executed.
query: START TRANSACTION
query: SAVEPOINT sp_workspace_custom_application_id_non_nullable
query: ALTER TABLE "core"."workspace" DROP CONSTRAINT "FK_3b1acb13a5dac9956d1a4b32755"
query: ALTER TABLE "core"."workspace" ALTER COLUMN "workspaceCustomApplicationId" SET NOT NULL
query failed: ALTER TABLE "core"."workspace" ALTER COLUMN "workspaceCustomApplicationId" SET NOT NULL
error: error: column "workspaceCustomApplicationId" of relation "workspace" contains null values
query: ROLLBACK TO SAVEPOINT sp_workspace_custom_application_id_non_nullable
query: RELEASE SAVEPOINT sp_workspace_custom_application_id_non_nullable
query: INSERT INTO "core"."_typeorm_migrations"("timestamp", "name") VALUES ($1, $2) -- PARAMETERS: [1763977334519,"WorkspaceCustomApplicationIdNonNullable1763977334519"]
Migration WorkspaceCustomApplicationIdNonNullable1763977334519 has been  executed successfully.
query: COMMIT

[Nest] 82844  - 11/24/2025, 6:05:12 PM     LOG [UpgradeCommand] Database migrations completed successfully
[Nest] 82844  - 11/24/2025, 6:05:12 PM     LOG [UpgradeCommand] Running command on workspace 20202020-1c25-4d02-bf25-6aeccf7ea419 1/2
Computing new Datasource for cacheKey: 20202020-1c25-4d02-bf25-6aeccf7ea419-10 out of 0
[Nest] 82844  - 11/24/2025, 6:05:12 PM   DEBUG [PgPoolSharedService] Reusing existing pg Pool for key "localhost|5432|postgres||no-ssl"
[Nest] 82844  - 11/24/2025, 6:05:12 PM   DEBUG [PgPoolSharedService] Pool[localhost|5432|postgres||no-ssl]: Client acquired from pool
[Nest] 82844  - 11/24/2025, 6:05:12 PM   DEBUG [PgPoolSharedService] Pool[localhost|5432|postgres||no-ssl]: Client acquired from pool
query: SELECT * FROM current_schema()
[Nest] 82844  - 11/24/2025, 6:05:12 PM   DEBUG [PgPoolSharedService] Pool[localhost|5432|postgres||no-ssl]: Client acquired from pool
query: SELECT version();
[Nest] 82844  - 11/24/2025, 6:05:12 PM     LOG [UpgradeCommand] Upgrading workspace 20202020-1c25-4d02-bf25-6aeccf7ea419 from=1.11.0 to=1.12.0 1/2
[Nest] 82844  - 11/24/2025, 6:05:12 PM     LOG [CreateWorkspaceCustomApplicationCommand] Checking standard applications for workspace 20202020-1c25-4d02-bf25-6aeccf7ea419
[Nest] 82844  - 11/24/2025, 6:05:12 PM     LOG [CreateWorkspaceCustomApplicationCommand] 20202020-1c25-4d02-bf25-6aeccf7ea419 skipping custom workspace application creation as already exists
query failed: ALTER TABLE "core"."workspace" ALTER COLUMN "workspaceCustomApplicationId" SET NOT NULL
error: error: column "workspaceCustomApplicationId" of relation "workspace" contains null values
[Nest] 82844  - 11/24/2025, 6:05:12 PM     LOG [WorkspaceCustomApplicationIdNonNullableCommand] Rollbacking WorkspaceCustomApplicationIdNonNullableCommand: column "workspaceCustomApplicationId" of relation "workspace" contains null values
[Nest] 82844  - 11/24/2025, 6:05:12 PM     LOG [SyncWorkspaceMetadataCommand] Running workspace sync for workspace: 20202020-1c25-4d02-bf25-6aeccf7ea419 (0 out of 2)
[Nest] 82844  - 11/24/2025, 6:05:12 PM     LOG [WorkspaceSyncMetadataService] Syncing standard objects and fields metadata
[Nest] 82844  - 11/24/2025, 6:05:12 PM     LOG [WorkspaceSyncMetadataService] Syncing standard objects and fields metadata
[Nest] 82844  - 11/24/2025, 6:05:12 PM     LOG [WorkspaceSyncObjectMetadataService] Comparing standard objects and fields metadata
[Nest] 82844  - 11/24/2025, 6:05:12 PM     LOG [WorkspaceSyncObjectMetadataService] Updating workspace metadata
[Nest] 82844  - 11/24/2025, 6:05:12 PM     LOG [WorkspaceSyncObjectMetadataService] Generating migrations
[Nest] 82844  - 11/24/2025, 6:05:12 PM     LOG [WorkspaceSyncObjectMetadataService] Saving migrations
[Nest] 82844  - 11/24/2025, 6:05:12 PM     LOG [WorkspaceSyncMetadataService] Workspace object migrations took 50.206083000000035ms
[Nest] 82844  - 11/24/2025, 6:05:12 PM     LOG [WorkspaceSyncFieldMetadataService] Updating workspace metadata
[Nest] 82844  - 11/24/2025, 6:05:12 PM     LOG [WorkspaceSyncFieldMetadataService] Generating migrations
[Nest] 82844  - 11/24/2025, 6:05:12 PM     LOG [WorkspaceSyncFieldMetadataService] Saving migrations
[Nest] 82844  - 11/24/2025, 6:05:12 PM     LOG [WorkspaceSyncMetadataService] Workspace field migrations took 79.43987500000003ms
[Nest] 82844  - 11/24/2025, 6:05:12 PM     LOG [WorkspaceSyncFieldMetadataRelationService] Updating workspace metadata
[Nest] 82844  - 11/24/2025, 6:05:12 PM     LOG [WorkspaceSyncFieldMetadataRelationService] Generating migrations
[Nest] 82844  - 11/24/2025, 6:05:12 PM     LOG [WorkspaceSyncFieldMetadataRelationService] Saving migrations
[Nest] 82844  - 11/24/2025, 6:05:12 PM     LOG [WorkspaceSyncMetadataService] Workspace relation migrations took 91.85295899999983ms
[Nest] 82844  - 11/24/2025, 6:05:12 PM     LOG [WorkspaceSyncIndexMetadataService] Syncing index metadata
[Nest] 82844  - 11/24/2025, 6:05:12 PM     LOG [WorkspaceSyncMetadataService] Workspace index migrations took 153.78104199999962ms
[Nest] 82844  - 11/24/2025, 6:05:13 PM     LOG [WorkspaceSyncMetadataService] Workspace object metadata identifiers took 130.9623330000004ms
[Nest] 82844  - 11/24/2025, 6:05:13 PM     LOG [WorkspaceSyncRoleService] Syncing standard role metadata
[Nest] 82844  - 11/24/2025, 6:05:13 PM     LOG [WorkspaceSyncMetadataService] Workspace role migrations took 2.154790999999932ms
[Nest] 82844  - 11/24/2025, 6:05:13 PM     LOG [WorkspaceSyncAgentService] Syncing standard agent.
[Nest] 82844  - 11/24/2025, 6:05:13 PM     LOG [WorkspaceSyncMetadataService] Workspace agent migrations took 2.3623329999991256ms
[Nest] 82844  - 11/24/2025, 6:05:13 PM     LOG [WorkspaceSyncMetadataService] Workspace migrations save took 5.95837500000016ms
[Nest] 82844  - 11/24/2025, 6:05:13 PM     LOG [WorkspaceSyncMetadataService] Executing pending migrations
[Nest] 82844  - 11/24/2025, 6:05:13 PM     LOG [WorkspaceSyncMetadataService] Execute migrations took 66.94295799999963ms
[Nest] 82844  - 11/24/2025, 6:05:13 PM     LOG [SyncWorkspaceMetadataCommand] Finished synchronizing workspace.
[Nest] 82844  - 11/24/2025, 6:05:13 PM     LOG [SetStandardApplicationNotUninstallableCommand] Checking workspace applications for workspace 20202020-1c25-4d02-bf25-6aeccf7ea419
[Nest] 82844  - 11/24/2025, 6:05:13 PM     LOG [SetStandardApplicationNotUninstallableCommand] Successfully updated workspace application
[Nest] 82844  - 11/24/2025, 6:05:13 PM     LOG [UpgradeCommand] Upgrade for workspace 20202020-1c25-4d02-bf25-6aeccf7ea419 completed.
PromiseMemoizer Event: A WorkspaceDataSource for workspace 20202020-1c25-4d02-bf25-6aeccf7ea419 is being cleared. Actual pool closure managed by PgPoolSharedService. Not calling dataSource.destroy().
[Nest] 82844  - 11/24/2025, 6:05:13 PM     LOG [UpgradeCommand] Running command on workspace 3b8e6458-5fc1-4e63-8563-008ccddaa6db 2/2
Computing new Datasource for cacheKey: 3b8e6458-5fc1-4e63-8563-008ccddaa6db-6 out of 0
[Nest] 82844  - 11/24/2025, 6:05:13 PM   DEBUG [PgPoolSharedService] Reusing existing pg Pool for key "localhost|5432|postgres||no-ssl"
[Nest] 82844  - 11/24/2025, 6:05:13 PM   DEBUG [PgPoolSharedService] Pool[localhost|5432|postgres||no-ssl]: Client acquired from pool
[Nest] 82844  - 11/24/2025, 6:05:13 PM   DEBUG [PgPoolSharedService] Pool[localhost|5432|postgres||no-ssl]: Client acquired from pool
query: SELECT * FROM current_schema()
[Nest] 82844  - 11/24/2025, 6:05:13 PM   DEBUG [PgPoolSharedService] Pool[localhost|5432|postgres||no-ssl]: Client acquired from pool
query: SELECT version();
[Nest] 82844  - 11/24/2025, 6:05:13 PM     LOG [UpgradeCommand] Upgrading workspace 3b8e6458-5fc1-4e63-8563-008ccddaa6db from=1.11.0 to=1.12.0 2/2
[Nest] 82844  - 11/24/2025, 6:05:13 PM     LOG [CreateWorkspaceCustomApplicationCommand] Checking standard applications for workspace 3b8e6458-5fc1-4e63-8563-008ccddaa6db
[Nest] 82844  - 11/24/2025, 6:05:13 PM     LOG [CreateWorkspaceCustomApplicationCommand] Successfully create workspace custom application
[Nest] 82844  - 11/24/2025, 6:05:13 PM     LOG [WorkspaceCustomApplicationIdNonNullableCommand] Successfully run WorkspaceCustomApplicationIdNonNullableCommand
[Nest] 82844  - 11/24/2025, 6:05:13 PM     LOG [SyncWorkspaceMetadataCommand] Running workspace sync for workspace: 3b8e6458-5fc1-4e63-8563-008ccddaa6db (1 out of 2)
[Nest] 82844  - 11/24/2025, 6:05:13 PM     LOG [WorkspaceSyncMetadataService] Syncing standard objects and fields metadata
[Nest] 82844  - 11/24/2025, 6:05:13 PM     LOG [WorkspaceSyncMetadataService] Syncing standard objects and fields metadata
[Nest] 82844  - 11/24/2025, 6:05:13 PM     LOG [WorkspaceSyncObjectMetadataService] Comparing standard objects and fields metadata
[Nest] 82844  - 11/24/2025, 6:05:13 PM     LOG [WorkspaceSyncObjectMetadataService] Updating workspace metadata
[Nest] 82844  - 11/24/2025, 6:05:13 PM     LOG [WorkspaceSyncObjectMetadataService] Generating migrations
[Nest] 82844  - 11/24/2025, 6:05:13 PM     LOG [WorkspaceSyncObjectMetadataService] Saving migrations
[Nest] 82844  - 11/24/2025, 6:05:13 PM     LOG [WorkspaceSyncMetadataService] Workspace object migrations took 19.26008400000046ms
[Nest] 82844  - 11/24/2025, 6:05:13 PM     LOG [WorkspaceSyncFieldMetadataService] Updating workspace metadata
[Nest] 82844  - 11/24/2025, 6:05:13 PM     LOG [WorkspaceSyncFieldMetadataService] Generating migrations
[Nest] 82844  - 11/24/2025, 6:05:13 PM     LOG [WorkspaceSyncFieldMetadataService] Saving migrations
[Nest] 82844  - 11/24/2025, 6:05:13 PM     LOG [WorkspaceSyncMetadataService] Workspace field migrations took 31.371917000000394ms
[Nest] 82844  - 11/24/2025, 6:05:13 PM     LOG [WorkspaceSyncFieldMetadataRelationService] Updating workspace metadata
[Nest] 82844  - 11/24/2025, 6:05:13 PM     LOG [WorkspaceSyncFieldMetadataRelationService] Generating migrations
[Nest] 82844  - 11/24/2025, 6:05:13 PM     LOG [WorkspaceSyncFieldMetadataRelationService] Saving migrations
[Nest] 82844  - 11/24/2025, 6:05:13 PM     LOG [WorkspaceSyncMetadataService] Workspace relation migrations took 36.84983300000022ms
[Nest] 82844  - 11/24/2025, 6:05:13 PM     LOG [WorkspaceSyncIndexMetadataService] Syncing index metadata
[Nest] 82844  - 11/24/2025, 6:05:13 PM     LOG [WorkspaceSyncMetadataService] Workspace index migrations took 43.85666599999968ms
[Nest] 82844  - 11/24/2025, 6:05:13 PM     LOG [WorkspaceSyncMetadataService] Workspace object metadata identifiers took 74.27345799999966ms
[Nest] 82844  - 11/24/2025, 6:05:13 PM     LOG [WorkspaceSyncRoleService] Syncing standard role metadata
[Nest] 82844  - 11/24/2025, 6:05:13 PM     LOG [WorkspaceSyncMetadataService] Workspace role migrations took 1.3081249999995634ms
[Nest] 82844  - 11/24/2025, 6:05:13 PM     LOG [WorkspaceSyncAgentService] Syncing standard agent.
[Nest] 82844  - 11/24/2025, 6:05:13 PM     LOG [WorkspaceSyncMetadataService] Workspace agent migrations took 0.7992909999993572ms
[Nest] 82844  - 11/24/2025, 6:05:13 PM     LOG [WorkspaceSyncMetadataService] Workspace migrations save took 2.1899590000002718ms
[Nest] 82844  - 11/24/2025, 6:05:13 PM     LOG [WorkspaceSyncMetadataService] Executing pending migrations
[Nest] 82844  - 11/24/2025, 6:05:13 PM     LOG [WorkspaceSyncMetadataService] Execute migrations took 33.817165999999816ms
[Nest] 82844  - 11/24/2025, 6:05:13 PM     LOG [SyncWorkspaceMetadataCommand] Finished synchronizing workspace.
[Nest] 82844  - 11/24/2025, 6:05:13 PM     LOG [SetStandardApplicationNotUninstallableCommand] Checking workspace applications for workspace 3b8e6458-5fc1-4e63-8563-008ccddaa6db
[Nest] 82844  - 11/24/2025, 6:05:13 PM     LOG [SetStandardApplicationNotUninstallableCommand] Successfully updated workspace application
[Nest] 82844  - 11/24/2025, 6:05:13 PM     LOG [UpgradeCommand] Upgrade for workspace 3b8e6458-5fc1-4e63-8563-008ccddaa6db completed.
PromiseMemoizer Event: A WorkspaceDataSource for workspace 3b8e6458-5fc1-4e63-8563-008ccddaa6db is being cleared. Actual pool closure managed by PgPoolSharedService. Not calling dataSource.destroy().
[Nest] 82844  - 11/24/2025, 6:05:13 PM     LOG [UpgradeCommand] Command completed!
[Nest] 82844  - 11/24/2025, 6:05:13 PM     LOG [PgPoolSharedService] pg Pool for key "localhost|5432|postgres||no-ssl" has been closed. Remaining pools: 0
[Nest] 82844  - 11/24/2025, 6:05:13 PM   DEBUG [PgPoolSharedService] Pool[localhost|5432|postgres||no-ssl]: Connection removed from pool
[Nest] 82844  - 11/24/2025, 6:05:13 PM     LOG [PgPoolSharedService] onApplicationShutdown called in PgPoolSharedService
[Nest] 82844  - 11/24/2025, 6:05:13 PM     LOG [PgPoolSharedModule] Shutting down PgPoolSharedModule
[Nest] 82844  - 11/24/2025, 6:05:13 PM     LOG [PgPoolSharedService] onApplicationShutdown called in PgPoolSharedService

———————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————

 NX   Successfully ran target command for project twenty-server and 4 tasks it depends on (7s)

      With additional flags:
        upgrade
```
2025-11-24 17:20:07 +00:00
Charles Bochet d526b07078 Add ability to discard Information Banner (#16019)
Fixes https://github.com/twentyhq/twenty/issues/14028

<img width="1917" height="965" alt="image"
src="https://github.com/user-attachments/assets/eeb5eab7-2c50-48be-8218-af6052b30777"
/>
2025-11-24 17:34:10 +01:00