## Context
Introduces a middleware that automatically sets the workspace auth
context in AsyncLocalStorage for HTTP requests, making it available
throughout the request lifecycle without explicit parameter passing.
The motivation behind this change is to reduce boilerplate and simplify
the developer experience when working with workspace data in HTTP
request handlers.
The Problem (Before)
Every HTTP request handler that needed to access workspace data had to:
- Extract auth-related info from decorators (@AuthWorkspace(),
@AuthUserWorkspaceId(), etc.) in controller/resolver and pass down to
services
- Build or pass the authContext explicitly (sometimes with type
assertion which was flaky)
Then call executeInWorkspaceContext(authContext, async () => { ... })
## Changes
- Add WorkspaceAuthContextMiddleware that extracts auth context from the
request and stores it in AsyncLocalStorage
- Register middleware for GraphQL, metadata, and REST routes (runs after
hydration middlewares)
- Simplify executeInWorkspaceContext signature: fn is now the first
parameter, authContext is optional second
- If authContext is not provided, it's automatically retrieved from the
storage (set by middleware)
- Update all callers (~120 files) to use the new parameter order
- Fixes a bug in search where system auth context was used, bypassing
RLS feature.
# Introduction
In this PR we're migrating the serverless function service that was
using the SF repo directly to the v2 build and runner.
The whole serverless engine now deals with flat entities only
## Resolvers
Refactored the resolvers ( serverlessFunction, route, database and cron
trigger) :
- return types to `dto`
- Standardized the flat to dto transpilation within the resolvers
- Find and findMany passing by the cached data
## Services
Refactored the services ( serverlessFunction, route, database and cron
trigger) :
- return type to be `flat`
- always calling v2 and computing cache
## New additional caches
- application variables ( cf
https://github.com/twentyhq/core-team-issues/issues/2116 )
- serverless function layer
## What to test:
- CRUD ( database trigger ✅ , route trigger, cron trigger, serverless
function through workflows ✅ )
- Duplicating a workflow with a serverless function code node ✅
## Concerns
We need to implement the cron that will hard delete soft deleted s3
serverless functions, not in this PR though ( cf
https://github.com/twentyhq/twenty/pull/17285#discussion_r2709168570 and
https://github.com/twentyhq/core-team-issues/issues/2118 )
# Introduction
Followup of
https://github.com/twentyhq/twenty/pull/17001#pullrequestreview-3638508738
close https://github.com/twentyhq/core-team-issues/issues/1910
We've completely decom the `sync-metadata` in production. We're now then
removing its implementation in favor of the v2.
## TODO:
- [x] Remove sync-metadata implem and commands
- [x] Remove workspace decorators
- [x] Type each deprecated field to deprecated on their workspaceEntity
- [x] Remove the `workspace-sync-metadata` folder entirely
- [x] remove workspace migration
- [x] workspace migration removal migration
- [x] remove the `v2` references from workspace manager file names
- [x] remove the `v2` references from workspace manager modules
- [ ] Double check impact on translation file path updates
## Note
- Removed the gate logic
- Remains some service v2 naming, serverless needs to be migrated on v2
fully
- Removed workspaceMigration service app health consumption, making it
always returning up ( no more down ) cc @FelixMalfait ( quite obsolete
health check now, will require complete refactor once we introduce inter
app dependency etc )
Two challenges with error messages
- always provide a useful/meaningful error message for the end user
instead of the generic one. eg: show "Wrong password" and not "An error
occured"
- avoid technical details unless error regards a technical feature. eg:
show "An error occured" and not "Invalid post-hook payload."; but do
show "Invalid issuer URL." as it occurs while configuring SSO
What this PR does
- Make userFriendlyMessage mandatory for widely used
GraphqlQueryRunnerException and CommonQueryRunnerException, so that
developers are forced to ask themselves what the error message should
be, and as it contains very wide error codes (eg: "Bad request") which
should not be mapped to just one default message
- Keep userFriendlyMessage optional for service-specific exceptions (eg:
workflowStepExecutorException), but convert the error code to
userFriendlyMessage mapper to a switch case function with a typecheck
ensuring that all codes are mapped to a message. These default messages
are still overridable where they are thrown.
<!-- CURSOR_SUMMARY -->
> [!NOTE]
> Implements last-modifier tracking and unified actor injection.
>
> - New `ActorFromAuthContextService` replaces createdBy-only logic;
pre-query hooks now inject both `createdBy` and `updatedBy` on create
and `updatedBy` on update
> - Add `updatedBy` standard field to core/custom objects (e.g.,
`person`, `company`, `task`, `note`, `attachment`, `dashboard`,
`workflow`, `workflowRun`) with IDs, metadata builders, and ORM entity
fields
> - Record CRUD: `create` now sets both `createdBy` and `updatedBy`;
`update` sets `updatedBy`; workflow actions use a shared workflow actor
builder; REST base handler uses the new actor service
> - Seeder data and snapshots updated to include `updatedBy`; GraphQL
result formatting adds defaults/handling for empty composite fields
(incl. actor)
> - Frontend `useUpdateOneRecord` upserts returned record into the local
store after mutation
>
> <sup>Written by [Cursor
Bugbot](https://cursor.com/dashboard?tab=bugbot) for commit
1f283778e9cb28a35bf84632a1a2997250bb3131. This will update automatically
on new commits. Configure
[here](https://cursor.com/dashboard?tab=bugbot).</sup>
<!-- /CURSOR_SUMMARY -->
---------
Co-authored-by: Félix Malfait <felix.malfait@gmail.com>
Co-authored-by: Félix Malfait <felix@twenty.com>
Solves #16015
- Added a check in `useTimelineActivities.ts` to see if the system
object page we're viewing has timelineActivity being tracked before
querying to get the activity history.
- Removed @WorkspaceIsObjectUIReadOnly decorator from system objects and
added @WorkspaceIsFieldUIReadOnly to standard and system fields to allow
custom field edits as requested in the issue.
- Did not add calendarEvents or other system objects to timelineActivity
just yet since keeping track of timeline activity for every one of them
felt counter-intuitive and bloated. In order to determine which objects
need timelineActivity, I think we need to fix the broken views of system
objects first, such as the one in the attached screenshots - a good
number of them are broken. The check I added to
`useTimelineActivities.ts` hook displays timeline activity as empty for
the time - error would not be shown as suggested by Thomas.
<img width="1062" height="858" alt="image"
src="https://github.com/user-attachments/assets/e877e0fe-b665-46e3-b785-e84f2af7f833"
/>
<br />
<img width="1061" height="858" alt="image"
src="https://github.com/user-attachments/assets/0eba8c1c-444a-4b13-beda-64b95cf39077"
/>
- Editing custom fields on calendar events (and other objects without
position fields) crashed with TypeError: Cannot convert undefined or
null to object in sortCachedObjectEdges. This happened because some
cached queries had empty orderBy arrays ([]), and the optimistic effect
tried to sort with them. Objects without position fields returned
empty orderBy arrays when there were no sorts, while objects with
position fields automatically got [{ position: 'AscNullsFirst' }].
The backend always adds { id: 'AscNullsFirst' } as a fallback, but
the frontend didn't match this. So, I added { id: 'AscNullsFirst' } to
the Frontend as default orderBy when there are no sorts and no position
field.
<!-- CURSOR_SUMMARY -->
---
> [!NOTE]
> Apply per-field UI read-only to system/standard fields and skip
timeline activity fetching when the target object isn’t related; refine
record-table cell open/navigation behavior.
>
> - Server: Replace object-level UI read-only with per-field
`isUIReadOnly` across many standard objects (e.g., `calendarEvent`,
`workspaceMember`, messaging, calendar, favorites, attachments,
workflows, etc.), and mirror this via `@WorkspaceIsFieldUIReadOnly` on
workspace entities.
> - Frontend: In `useTimelineActivities.ts`, check object metadata for a
relation to `timelineActivity` and `skip` the query when absent,
preventing errors on system object pages.
> - Frontend: Simplify/adjust record-table cell logic—remove unused
args, allow navigation from first column when non-empty, block editing
for read-only records (while still allowing navigation), and update
button handlers/hover styles accordingly.
>
> <sup>Written by [Cursor
Bugbot](https://cursor.com/dashboard?tab=bugbot) for commit
dac1262d70d97ce84b39730454dd3579c494994a. This will update automatically
on new commits. Configure
[here](https://cursor.com/dashboard?tab=bugbot).</sup>
<!-- /CURSOR_SUMMARY -->
---------
Co-authored-by: Félix Malfait <felix@twenty.com>
## Summary
This PR enforces that all custom exceptions must provide a
`userFriendlyMessage`, ensuring end users always see readable error
messages.
## Changes
### Core Changes
- **`CustomException` simplified**: Removed the `ForceFriendlyMessage`
generic parameter - `userFriendlyMessage` is now always required
- **Type safety**: The constructor now requires `{ userFriendlyMessage:
MessageDescriptor }` (no longer optional)
### Updated Files
- **74+ exception classes** updated to provide default user-friendly
messages using Lingui `msg` macro
- Each exception class has a sensible fallback message (e.g., `msg\`An
authentication error occurred.\``)
- Exception classes that had code-specific message maps retain their
behavior
## Benefits
- **Compile-time enforcement**: Forgetting to add a user-friendly
message now causes a TypeScript error
- **Better UX**: End users always see a localized, human-readable error
message
- **Simpler API**: No more boolean generic parameter to think about
## Testing
- `npx nx run twenty-server:typecheck` passes
- `npx nx run twenty-server:lint` passes
<!-- CURSOR_SUMMARY -->
---
> [!NOTE]
> Enforces `userFriendlyMessage` on `CustomException` and updates all
exception classes to supply localized default messages, with
filters/tests adjusted accordingly.
>
> - **Core**:
> - Enforce required `userFriendlyMessage` in `CustomException` (remove
optional generic; constructor now requires `{ userFriendlyMessage:
MessageDescriptor }`).
> - **Exceptions**:
> - Update ~70+ exception classes to set default localized messages via
Lingui `msg` maps and pass them in constructors (e.g., `AuthException`,
`ObjectMetadataException`, `FieldMetadataException`, etc.).
> - Add fallback messages where needed (e.g., `INTERNAL_SERVER_ERROR` or
domain-specific defaults).
> - **HTTP/GraphQL Filters**:
> - Ensure fallbacks create `UnknownException` with `msg` for
user-friendly text in REST/GraphQL exception filters.
> - **Tests**:
> - Adjust unit tests to pass `userFriendlyMessage` to exceptions.
> - Update Jest snapshots to include `extensions.userFriendlyMessage` or
message objects where applicable.
>
> <sup>Written by [Cursor
Bugbot](https://cursor.com/dashboard?tab=bugbot) for commit
221004fdfc0d97b7d152a258b347bf571e70f10e. This will update automatically
on new commits. Configure
[here](https://cursor.com/dashboard?tab=bugbot).</sup>
<!-- /CURSOR_SUMMARY -->
# TimelineActivity migration to morph
- Creates `timelineActivities2` relations on Company, Dashboard, Note,
Opportunity, Person, Task, Workflow, WorkflowRun, and WorkflowVersion
entities with proper metadata and cascade delete behavior.
It was required to create standard fields as well since the
mapObjectMetadataByUniqueIdentifier needs it. otherwise the fields won't
be considered
- Feature Flag `IS_TIMELINE_ACTIVITY_MIGRATED` necessary to have the two
states in parallel. It is used as a stamp once the migration has been
run
- Migration is done using the coreDataSource. Why ?
even though is unsafe to use, the first implementation of the migration
took forever on each workspace. See [this
commit](https://github.com/twentyhq/twenty/pull/15652/commits/477011e8d7d4c580f79ba7ec4a8fb002a3ec86b2)
The plan for this complex migration is as follows :
Note: we will need to rename fields in the release 1.12 (there is no
easy way to do all this in one release)
## Context
Following https://github.com/twentyhq/twenty/pull/16399
Now using the new global orm manager everywhere and returning a
GlobalDatasource/WorkspaceDatasource based on a feature flag.
This means we now need to wrap all our ORM calls within
executeInWorkspaceContext callback (at least for now) so the global
datasource can dynamically hydrate its context via the new store (the
global datasource does not store anything related to workspaces as it is
now a unique singleton). If feature flag is off it still uses local data
stored in the workspace datasource.
## Context
Deprecating TwentyORMManager in favor of TwentyORMGlobalManager
(temporarily, as this will simplify the ultimate goal to later replace
all usages with the new TwentyORMGlobalManagerV2 which will have a
similar signature)
This means this PR had to refactor a bit of code to pass down the
workspaceId when not available directly as it is now a requirement,
meaning we also deprecated scopedWorkspaceContextFactory to have a less
obscure way to fetch the workspaceId and have something more
declarative.
Step 3 will be to update TwentyORMGlobalManager to use a featureFlag
toggling and use the new GlobalWorkspaceOrmManager internally using the
new cache service
Step 4 will be to remove the feature flag and pg_pool patch
## Context
Deprecating the old objectMetadataMap type in favour of split flat
entities to match with our new caching.
In the long run, trying to achieve:
- Better performance through caching
- Consistent data access patterns across the codebase
- Reduced database queries
Now that everything is based on flat entities, which are cached, we can
finish the refactoring of workspace context cache which should already
improve performances.
Then the last step will be to consume that new cache in the new global
datasource to get rid of the many workspace datasources stored in the
server
@charlesBochet
In workflow codebase, NULL (instead of empty object) is expected on
workflow version object step field, when a new workflow is created for
example.
(packages/twenty-front/src/modules/workflow/workflow-diagram/utils/generateWorkflowDiagram.ts
- 42)
This case is not isolated and it creates many issues.
We decided to format NULL value to equivalent (empty string for text
field, empty object for raw_json) but it seems it complicates the dev x.
To unlock @Devessier I prefer revert the logic, the time we discuss how
to solve this cases.
## Overview
This PR replaces the dynamic agent handoff system with a more
predictable planning-based router that decides upfront how to handle
multi-agent coordination.
## Major Changes
### 🔄 Architecture Shift: Handoffs → Planning
**Removed:**
- `AgentHandoffEntity` and handoff tracking system
- `AgentHandoffService` and `AgentHandoffExecutorService`
- Dynamic agent-to-agent transfers during execution
- Handoff tool generation and description templates
**Added:**
- `AiRouterService` with two strategies: `simple` (single agent) and
`planned` (multi-agent)
- `AgentPlanExecutorService` for executing multi-step plans
- Plan validation (cycle detection, dependency resolution)
- `UnifiedRouterResult` type with discriminated union
### 🤖 New Standard Agents
Added two new specialized agents:
- **Researcher Agent**: Web search, fact-finding, competitive
intelligence
- **Code Agent**: TypeScript function generation for serverless
workflows
### 🏗️ Router Refactoring (Latest)
Split router responsibilities into focused services:
- `AiRouterStrategyDeciderService`: Decides simple vs planned strategy
- `AiRouterPlanGeneratorService`: Generates and validates execution
plans
- `AiRouterService`: Coordinates between services (reduced from 426→275
lines)
### ⚙️ Configuration Improvements
- Added `outputStrategy` to agent definitions (`direct` vs `synthesize`)
- Removed hardcoded special cases for workflow-builder
- Added `plannerModel` field to workspace entity
- Increased `MAX_STEPS` from 10 to 25 for complex workflows
### 📝 Agent Prompt Refinements
Significantly simplified prompts for better clarity:
- Workflow Builder: 51→36 lines
- Helper: 49→28 lines
- Data Manipulator: Enhanced with sorting guidance
### 🔍 Enhanced Debugging
- Plan reasoning and step count in data message parts
- Router debug info with token usage tracking
- Better logging throughout execution pipeline
## Benefits
1. **Simpler Mental Model**: Router decides upfront vs dynamic transfers
2. **Better Predictability**: Users see the plan before execution
3. **Cleaner Architecture**: SRP with focused services
4. **Configuration Over Code**: Agent behavior via config, not hardcoded
logic
5. **Plan Validation**: Catches invalid dependencies and cycles
## Migration Notes
- Database migration removes `agentHandoff` table
- Adds `plannerModel` column to workspace table
- No API breaking changes (agent endpoints unchanged)
## Testing
- Integration tests updated to remove handoff dependencies
- Agent tool test utilities simplified
- Plan validation covered by new logic
## Next Steps (Future PRs)
- Parallel execution of independent plan steps
- Dynamic re-planning based on results
- Plan caching for common routing patterns
- Error recovery strategies in plan executor
## Context
Deprecating legacy ObjectMetadata from cache in favor of flat entities.
Introducing utils to build byName/byNameSingular/byNamePlural in
isolated cases
## Next
- I had to introduce a util to build from flat to legacy
objectMetadataMaps, we should instead use flat maps directly when needed
(datasource, schema generation, etc)
- Deprecate metadata version in the cache
- Use the new cache strategy for flat entities with permissions and
feature flags and inject in the global datasource context
## Summary
This PR adds configurable response format support for AI agents,
allowing them to return either plain text or structured JSON data based
on a defined schema.
## Key Features
### 1. Agent Response Format Configuration
- Added `AgentResponseFormat` type supporting:
- `text`: Returns plain text responses (default)
- `json`: Returns structured JSON based on defined schema
- New `AgentResponseSchema` type moved to `twenty-shared/ai` for sharing
between frontend/backend
### 2. Settings UI
- New `SettingsAgentResponseFormat` component for configuring response
format
- Visual schema builder for defining JSON output structure
- Real-time validation and preview
- Integrated into agent settings tab
### 3. Workflow Integration
- AI Agent workflow action automatically uses agent's configured
response format
- Output schema dynamically generated from agent's response format
- Workflow variable picker shows structured fields for JSON responses
- Backward compatible with existing text-only agents
### 4. Backend Implementation
- Added `convertAgentSchemaToZod` utility to validate JSON responses
- Agent executor service handles both text and JSON generation
- Automatic agent creation/cloning when adding AI agent steps to
workflows
- Unique agent naming with conflict resolution
### 5. Database Migration
- Migration `1763622159656-update-agent-response-format.ts`
- Sets default `responseFormat` to `{"type":"text"}` for existing agents
- Updated all standard agents with proper response format
## Changes by Module
### Frontend (`twenty-front`)
- 🆕 `AgentResponseFormat` type
- 🆕 `SettingsAgentResponseFormat` component
- ✏️ Updated `WorkflowEditActionAiAgent` to support response format
configuration
- 🗑️ Removed deprecated `useAiAgentOutputSchema` hook and
`AiAgentOutputSchema` type
### Backend (`twenty-server`)
- 🆕 `AgentResponseFormat` type in agent entity
- 🆕 `convertAgentSchemaToZod` utility for schema validation
- ✏️ Updated `AiAgentExecutorService` to handle both text and JSON
generation
- ✏️ Updated `WorkflowSchemaWorkspaceService` to generate output schema
from agent config
- ✏️ Enhanced `WorkflowVersionStepOperationsWorkspaceService` with agent
creation/cloning
- 🆕 Agent naming constants for conflict resolution
### Shared (`twenty-shared`)
- 🆕 `AgentResponseSchema` type
- 🆕 `ModelConfiguration` type moved to shared package
- Updated exports in `ai/index.ts`
## Code Quality
- Removed useless comments following code style guidelines
- All linter checks passed
- Type-safe implementation with proper TypeScript types
## Testing
- ✅ Database migration tested
- ✅ Agent creation/cloning in workflows verified
- ✅ Response format switching (text ↔ JSON) validated
- ✅ Backward compatibility with existing agents confirmed
## Migration Notes
- Existing agents will have `responseFormat: {type: 'text'}` set
automatically
- No breaking changes - all existing functionality preserved
- Agents can be updated to use JSON format through settings UI
## Summary
This PR replaces the Active/Inactive accordion sections with a modern
filter dropdown button and enables full access to system objects in
advanced mode.
## Changes
### UI Improvements
- ✅ Replaced accordion sections with a filter button dropdown (matching
the design pattern from the Group filter)
- ✅ Added 'Deactivated' toggle filter (hidden by default, uses
IconArchive)
- ✅ Added 'System objects' toggle filter (only visible in advanced mode,
uses IconSettings)
- ✅ Fixed search input width to properly fill available space
- ✅ Proper button sizing and alignment
### System Objects Support
- ✅ Made system objects visible when 'System objects' filter is toggled
on
- ✅ System objects are now fully clickable and accessible
- ✅ Updated object detail page to support system objects
- ✅ Updated field creation/edit pages to support system objects
- ✅ System objects can now have custom fields added
### Architecture
- ✅ Implemented scalable filter architecture using a single filtered
list
- ✅ Easy to add more filters in the future (e.g., show remote objects)
- ✅ All filters work independently and can be combined
## Testing
- [x] Tested deactivated objects toggle
- [x] Tested system objects toggle (only shows in advanced mode)
- [x] Tested clicking on system objects
- [x] Tested adding custom fields to system objects
- [x] No linter errors
## Screenshots
See attached screenshots in the conversation for the new filter UI.
<!-- CURSOR_SUMMARY -->
---
> [!NOTE]
> Adds a filter dropdown to the Settings Objects table (incl.
deactivated/system toggles), enables system objects across
settings/field flows, seeds core views (workspace members, messages,
threads, calendar events), and adds actions for Workspace Members.
>
> - **Settings UI**
> - **Objects Table**: Replaces Active/Inactive sections with a single
filterable list and dropdown (`Deactivated`, `System objects` in
advanced mode); updates props to `objectMetadataItems` and removes
accordion sections.
> - **Search/UX**: Search input fills available space; inactive rows
show activation/delete menu; active rows remain navigable.
> - **Pages Updated**: `SettingsObjects`,
`SettingsApplicationDetailContentTab` switch to new table API; object
detail and new-field flows use `findObjectMetadataItemByNamePlural`
(works with system objects).
> - **Action Menu**
> - **Workspace Members**: Adds `WORKSPACE_MEMBERS_ACTIONS_CONFIG` with
"Manage members in settings" action; wired into `getActionConfig` for
`WorkspaceMember`.
> - **Server (Core Views Seed)**
> - Adds default views: `workspaceMembersAllView`, `messagesAllView`,
`messageThreadsAllView`, `calendarEventsAllView`; included in
`prefillCoreViews`.
> - Marks workflow entities as system (`WorkflowRun`,
`WorkflowVersion`).
>
> <sup>Written by [Cursor
Bugbot](https://cursor.com/dashboard?tab=bugbot) for commit
6a2856df85104665bd986cc63f2d54f19bf668d0. This will update automatically
on new commits. Configure
[here](https://cursor.com/dashboard?tab=bugbot).</sup>
<!-- /CURSOR_SUMMARY -->
# Introduction
related to https://github.com/twentyhq/core-team-issues/issues/1833
In this PR we're starting the sync-metadata and standardIds deprecation
by introducing `twenty-standard` application that will regroup every
standard object such as company and opportunities. But also the
`custom-workspace-application` which is an app created at the same time
as a workspace and that will regroup everything configure within the
workspace ( custom objects fields etc )
## What's done
On both new workspace and seeded workspace creation:
- Creating a custom workspace app
- Creating a twenty standard app
- Refactored the seed core schema and workspace creation to be run
within a transaction in order to handle circular dependency foreignkey
requirements ( which is deferred for app toward workspace )
- Updated workspace entity to have a custom workspace relation (
nullable for the moment until we implem an upgrade command to handle
retro comp )
- Integration testing on user, workspace creation deletion and expected
default apps creation
- ~~Soft deleted user on `deleteUser`~~ Done by marie and rebased on it
## What's next
- Update seeder to propagate the `twenty-standard` workspace
`applicationId` to every standard synchronized entities ( cheap and fast
iteration through the about to be deprecated sync-metadata as an easy
way to synchronize standards metadata entities ).
- Update seeder to propagate the `custom-workspace-application`
workspace `applicationId` to anything custom ( `pets` and `rockets` )
- Prepend `custom-workspace-application` `applicationId` to every
metadata API operations ( create a specific cache etc )
- Upgrade command on all existing workspace to create a custom app and
associate its applicationId to any existing custom entities
- Make `universalIdentifier` and `applicationId` required for any
syncable entity
As title
- adds decorators in twenty-sdk
- update twenty-cli load-manifest to it gets @FieldMetadata infos +
testing
- update twenty-server so it CRUD fields properly, using
universalIdentifier
- Fix UI so we can update managed objects records
- move FieldMetadata items from twenty-server to twenty-shared
- on draft creation, do not fetch the full version. Avoid the fetch of
steps and trigger
- on activation, we were performing 8 queries/mutations synchronously +
2 additional for automated triggers. I refacto the call it it gets
reduced to 4 queries/mutations + 2 additional for automated triggers
## Improvements
- Add logs to all gql operations and rest calls to help debug CPU issues
on the backend. These are temporary and should be removed
- Remove nested relations from workflowVersions load (used to add manual
triggers in the side bar). On some workspaces this call result in a
response of 4MB which is heavy on CPU
- Investigated Redis Usage ==> made a few improvements, we are should
still migrate to the new cache service once available
- investigated db calls in messaging / calendar fetch list + workflow
enqueue run cron jobs. Everything seems to be properly batched
- replace save by insert
- if the insert output is needed, cast the generatedMaps
- remove transactions from trigger services. Doing it manually would be
complex and not reliable
Improve workflow enqueue cron : instead of relying on the cache to know
how many workflows we can enqueue, query the DB. Then set the cache and
process the not started workflows.
Also adding a second cron that will look for workflows enqueued one hour
ago or more and put these back in the not started status. This will
allow the first cron to start these again.
## Context
Add an eventEmitter instance to twenty datasources so we can emit DB
events.
Add input and output formatting to twenty orm (formatData, formatResult)
Those 2 elements simplified existing logic when we interact with the
ORM, input will be formatted by the ORM so we can directly use
field-like structure instead of column-like. The output will be
formatted, for builder queries it will be in `result.generatedMaps`
where `result.raw` preserves the previous column-like structure.
Important change: We now have an authContext that we can pass when we
get a repository, this will be used for the different events emitted in
the ORM. We also removed the caching for repositories as it was not
scaling well and not necessary imho
Note: An upcoming PR should handle the onDelete: cascade behavior where
we send DESTROY events in cascade when there is an onDelete: CASCADE on
the FK.
---------
Co-authored-by: Charles Bochet <charles@twenty.com>
This PR does not produce any functional changes for our users. It
prepares the branches for workflows by:
- decommissioning `output` and `context` fields or `workflowRun` records
and use newly created `state` field from front-end and back-end
- use `stepStatus` computed by `back-end` in `front-end`
- add utils and types in `twenty-shared/workflow` (not completed, a
follow-up is scheduled
https://github.com/twentyhq/core-team-issues/issues/1211)
- add concurrency to `workflowQueue` message queue to avoid weird branch
execution when using forms in workflow branches
- add a WithLock decorator for better dev experience of
`CacheLockService.withLock` usage
Here is an example of such a workflow running (front branch display is
not yet done that's why it looks ugly) ->
https://discord.com/channels/1130383047699738754/1258024460238192691/1392897615171158098
In this PR
- introduction of fieldPermission entity
- addition of upsertFieldPermission in role resolver
- computing of permissions taking fieldPermission into account. In order
to limit what is stored in Redis we only store fields restrictions. For
instance for objectMetadata with id XXX with a restriction on field with
id YYY we store:
`"XXX":{"canRead":true,"canUpdate":false,"canSoftDelete":false,"canDestroy":false,"restrictedFields":{"YYY":{"canRead":false,"canUpdate":null}}}`
---------
Co-authored-by: Charles Bochet <charlesBochet@users.noreply.github.com>
This PR is purely technical, it does produces any functional change to
the user
- add Lock mecanism to run steps concurrently
- update `workflow-executor.workspace-service.ts` to handle multi branch
workflow execution
- stop passing `context` through steps, it causes race condition issue
- refactor a little bit
- simplify `workflow-run.workspace-service.ts` to prepare `output` and
`context` removal
- move workflowRun status computing from `run-workflow.job.ts` to
`workflow-executor.workspace-service.ts`
## NOTA BENE
When a code step depends of 2 parents like in this config (see image
below)
If the form is submitted before the "Code - 2s" step succeed, the branch
merge "Form" step is launched twice.
- once because form is submission Succeed resumes the workflow in an
asynchronous job
- the second time is when the asynchronous job is launched when "Code -
2s" is succeeded
- the merge "Form" step makes the workflow waiting for response to
trigger the resume in another job
- during that time, the first resume job is launched, running the merge
"Form" step again
This issue only occurs with branch workflows. It will be solved by
checking if the currentStepToExecute is already in a SUCCESS state or
not
<img width="505" alt="image"
src="https://github.com/user-attachments/assets/b73839a1-16fe-45e1-a0d9-3efa26ab4f8b"
/>
Currently, when a server query or mutation from the front-end fails, the
error message defined server-side is displayed in a snackbar in the
front-end.
These error messages usually contain technical details that don't belong
to the user interface, such as "ObjectMetadataCollection not found" or
"invalid ENUM value for ...".
**BE**
In addition to the original error message that is still needed (for the
request response, debugging, sentry monitoring etc.), we add a
`displayedErrorMessage` that will be used in the snackbars. It's only
relevant to add it for the messages that will reach the FE (ie. not in
jobs or in rest api for instance) and if it can help the user sort out /
fix things (ie. we do add displayedErrorMessage for "Cannot create
multiple draft versions for the same workflow" or "Cannot delete
[field], please update the label identifier field first", but not
"Object metadata does not exist"), even if in practice in the FE users
should not be able to perform an action that will not work (ie should
not be able to save creation of multiple draft versions of the same
workflows).
**FE**
To ease the usage we replaced enqueueSnackBar with enqueueErrorSnackBar
and enqueueSuccessSnackBar with an api that only requires to pass on the
error.
If no displayedErrorMessage is specified then the default error message
is `An error occured.`
This PR does not produce any functional change
First step of the workflow branch feature
- add gather `workflowRun.output` and `workflowRun.context` into one
column `workflowRun.runContext`
- add a command to fill `runContext` from `output` and `context` in
existing records
- maintain `runContext` up to date during workflow runs
Fixes: #12722
The problem is that there is no TS_VECTOR field in workflow objects.
Thus, I have added this field to three objects: workflow,
workflowVersions, and workflowRuns.
---------
Co-authored-by: Thomas Trompette <thomas.trompette@sfr.fr>
- new status `ENQUEUED` added. With a command to backfill
- counter in cache per workspace, managed by a new service
[workflow-run-queue.workspace-service.ts](https://github.com/twentyhq/twenty/compare/tt-improve-workflow-run-queueing?expand=1#diff-1e2de2a48cd482a3bd7e8dedf1150a19d0b200afbd9282181a24ecddddb56927)
- cron added that will run every minute to look for not started
workflows
Here is the new flow:
- When executing a workflow, we check if the queue is not full. If not,
run is created as `ENQUEUED` and the run workflow job is triggered as
usual. If full, create the run as NOT_STARTED and do not trigger the job
- Cron will look for NOT_STARTED workflows and queue some if there is
some place again
- Only MANUAL and Form submit skip the queue limit
## Done
Update manually (without status update job) workflow and
workflowVersions statuses when workflow is deleted
## Not Done
Status optimistic rendering on workflow index deleted page. This page is
already buggy, this will be fix by
https://discord.com/channels/1130383047699738754/1384177035244732487
Charles Bochet