In this PR:
- add more logs to troubleshoot issues in production
- use messageChannelSyncStatus in messageSync service instead of making
a manual update
First PR to implement application tokens
- add new application role in twenty-server
- move duplicated constants and types to twenty-shared
- will add role configuration utils into twenty-sdk in another PR
## Summary
Consolidates the AI tool provider architecture by creating a single
`ToolProviderService` as the entry point for all tool generation. This
removes multiple intermediate services and simplifies the codebase.
## Changes
### New Architecture
- **`ToolProviderService`**: Single service for all tool generation
with:
- `getTools(spec)` - Get tools by category with permissions
- `getToolByType(type)` - Get specific tool for workflow execution
- **`ToolCategory` enum**: Declarative specification of tool types:
- `DATABASE_CRUD` - Record CRUD operations
- `ACTION` - HTTP requests, email sending, article search
- `WORKFLOW` - Workflow management tools
- `METADATA` - Object/field metadata tools
- `NATIVE_MODEL` - Model-specific tools (e.g., web search)
- **`ToolSpecification` type**: Clean API for requesting tools with
permissions
### Removed
- `AiToolsModule` - No longer needed
- `ToolService` - Logic inlined into ToolProviderService
- `ToolAdapterService` - Logic inlined into ToolProviderService
- `ToolRegistryService` - Logic inlined into ToolProviderService
### Updated
- All consumers (agents, chat, MCP, workflows) now use
`ToolProviderService`
- Test files updated accordingly
## Stats
- **547 insertions, 1146 deletions** (net ~600 lines removed)
- 4 services deleted
- 1 module deleted
## Testing
- [x] Typecheck passes
- [x] Lint passes
Mainly clarifying naming (scheduleMessageImport was actually tagging as
PENDING)
+ tagging as SCHEDULED in case of direct import (we skip PENDING state)
In this PR:
- change messaging / calendar stale duration check to 30minutes (cron is
running every 1h, duration check was 1h, so evaluation was flaky)
- when temporary error (throttling), preserve syncStageStartedAt as this
is necessary to assess exponential throttling
In this PR, we are adding a new command to reset a message channel.
We are also refactoring a bit cursor reset as we had multiple
implementations at different places in the code base
---------
Co-authored-by: Charles Bochet <charles@twenty.com>
This is a temporary fix that needs to be revisited.
It seems that with the new enqueue system we are enqueuing jobs multiple
times due to race condition. We likely want to only consider job that
have been created more than x secs ago
- if >5000 workflows per hour, new ones should failed
- if >100 workflow per min, new ones should be set as not started.
Except manual trigger
- when enqueued, we check if there a not started workflows that may be
queued. If yes, we call the associated job
---------
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
## Context
EntityMetadata was being rebuilt from scratch on every
findMetadata()/getMetadata() call (~20 times per request). This involved
running EntitySchemaTransformer.transform() and
EntityMetadataBuilder.build() repeatedly, causing unnecessary CPU
overhead.
## Implementation
Cache entityMetadatas in ORMWorkspaceContext: Build EntityMetadata once
during workspace context initialization instead of on every metadata
lookup
Remove redundant entitySchemas caching: Since flatMetadata is already
cached, the additional Redis cache for entitySchemaOptions was
unnecessary overhead
Remove WorkspaceEntitiesStorage: Replaced with direct lookup from
FlatObjectMetadataMap
Simplify getObjectMetadataFromEntityTarget: Now only accepts string
targets, using flat metadata maps directly
Also:
Removed unused injections in some services
## Context
Deprecating the old objectMetadataMap type in favour of split flat
entities to match with our new caching.
In the long run, trying to achieve:
- Better performance through caching
- Consistent data access patterns across the codebase
- Reduced database queries
Now that everything is based on flat entities, which are cached, we can
finish the refactoring of workspace context cache which should already
improve performances.
Then the last step will be to consume that new cache in the new global
datasource to get rid of the many workspace datasources stored in the
server
## Summary
This PR introduces a comprehensive agent evaluation system and refactors
the AI module structure for better organization.
## Key Changes
### 🎯 Agent Evaluation System
- Added **Agent Turn Evaluation** entities, DTOs, and database schema
- New GraphQL mutations: `evaluateAgentTurn` and `runEvaluationInput`
- Added `evaluationInputs` field to Agent entity for storing test inputs
- New `AgentTurnGraderService` for automatic turn evaluation
- Added evaluation UI with new **Evals** and **Logs** tabs in agent
detail pages
### 🏗️ Entity & Module Refactoring
- Renamed `AgentChatMessage` → `AgentMessage` for clarity
- Consolidated chat entities: `AgentMessage`, `AgentTurn`, and
`AgentChatThread`
- Reorganized AI modules under `ai/` subdirectory structure
- Updated imports across codebase to reflect new module paths
### 🤖 New Agents & Roles
- Added **Dashboard Builder Agent** for dashboard creation and
management
- Added **Dashboard Manager Role** with appropriate permissions
- Updated role permissions to be more granular (users vs agents vs API
keys)
### 🔐 Permission System Updates
- Added `HTTP_REQUEST_TOOL` permission flag
- Updated Workflow Manager role permissions (restricted tool access)
- Enhanced permission flag types to differentiate between user/agent/API
key contexts
- Added `isRelevantForAgents`, `isRelevantForApiKeys`,
`isRelevantForUsers` to permission flags
### 📨 Message Role Enhancement
- Added `system` role to `AgentMessageRole` enum (alongside
user/assistant)
- Updated message handling to support system prompts
### 🎨 UI/UX Improvements
- New tabs in agent detail: **Evals** and **Logs**
- Added turn detail page: `/ai/agents/:agentId/turns/:turnId`
- Fixed text overflow in `SettingsListItemCardContent`
- Updated role applicability labels ("Assignable to Workspace Members")
### 🛠️ Technical Improvements
- Fixed Zod schema validation for UUID and Date fields (use string
validators)
- Updated `ToolRegistryService` to properly register HTTP tool with
permission flag
- Enhanced error handling in agent execution services
- Updated database migrations for new entity schema
## Database Migrations
- `1764210000000-add-system-role-to-agent-message.ts`
- `1764220000000-add-evaluation-inputs-to-agent.ts`
- `1764200000000-add-agent-turn-evaluation.ts`
- `1764100000000-refactor-agent-chat-entities.ts`
## Testing
- [ ] Agent evaluation flow tested
- [ ] Dashboard Builder agent tested
- [ ] Permission system validated
- [ ] UI tabs and navigation tested
- [ ] Database migrations run successfully
## Breaking Changes
⚠️ **Entity Rename**: `AgentChatMessage` renamed to `AgentMessage` -
GraphQL queries need updating
## Related Issues
<!-- Link any related issues here -->
## Screenshots
<!-- Add screenshots if applicable -->
# Introduction
We need to be able to create custom workspace application on all
workspaces, even pending and ongoing etc
Right now the upgrade devx only allows and expect active or suspended
workspace to be passed to runOnWorkspace.
## WorkspacesMigrationRunner
Created an intermediate class `WorkspacesMigrationRunner` that expect an
array `WorkspaceStatus` to be fetched for the current command to be run
on
The `ActiveOrSuspendedCommandRunner` statically passes both `SUSPENDED`
and `ACTIVE`, whereas the create workspace custom application passed all
the enum values
## DataSource
Workspace that are not fully init don't have a `workspace_schema` so
they don't have `dataSource`
Made a not very elegant check to see if current workspace we're about to
create dataSource on has one historically
Which means that dataSource is now optional, it had only one impact on
an existing command and the desired devx will become consuming existing
services that do not expect dataSource ( or at least yet )
Closes https://github.com/twentyhq/twenty/issues/15692, fixes
https://github.com/twentyhq/twenty/issues/14678
The issue was:
- When a user is signing up, they are asked for their first name and
last name with which we fulfill their workspaceMember entity for the
workspace they are signing up to (which is the one they are creating if
they are not joining an existing workspace). user.firstName and
user.lastName remain empty
- When we create a record, we are using user.firstName and user.lastName
to fulfill the text field "createdByName", which intends to save the
name of the creator at the moment of the creation. This field is then
use 1) when filtering on createdBy (as a trick to be able to filter by
this, since we don't have filters on nested fields, ie we dont have
filters on person.createdByWorkspaceMember) 2) to display the user
creator when a user has left a workspace - otherwise we rely on the
dynamic createdByWorkspaceMember.firstName and lastName
- So filtering on createdBy was not working as createdByName is empty.
We did not see that in dev mode because we have seeded users with names
The fix
- When we create a record, we use workspaceMember.firstName and
workspaceMember.lastName as we should.
- When an existing user joins a workspace, they are asked to give their
name again so that we set their workspaceMember name
- This will not fix the history of the records (they will still have
createdByName empty so the filter will not work properly), but a command
to fix it would be very long as it would iterate over all the records of
each active workspace
The long-term view is to get rid of user and to store the name in
userWorkspace
We have introduced to new syncStage statuses:
`messageChannel.MESSAGES_IMPORT_SCHEDULED` and
`calendarChannel.CALENDAR_EVENTS_IMPORT_SCHEDULED`
We need to make sure all existing workspaces have it
@charlesBochet
In workflow codebase, NULL (instead of empty object) is expected on
workflow version object step field, when a new workflow is created for
example.
(packages/twenty-front/src/modules/workflow/workflow-diagram/utils/generateWorkflowDiagram.ts
- 42)
This case is not isolated and it creates many issues.
We decided to format NULL value to equivalent (empty string for text
field, empty object for raw_json) but it seems it complicates the dev x.
To unlock @Devessier I prefer revert the logic, the time we discuss how
to solve this cases.
## Overview
This PR replaces the dynamic agent handoff system with a more
predictable planning-based router that decides upfront how to handle
multi-agent coordination.
## Major Changes
### 🔄 Architecture Shift: Handoffs → Planning
**Removed:**
- `AgentHandoffEntity` and handoff tracking system
- `AgentHandoffService` and `AgentHandoffExecutorService`
- Dynamic agent-to-agent transfers during execution
- Handoff tool generation and description templates
**Added:**
- `AiRouterService` with two strategies: `simple` (single agent) and
`planned` (multi-agent)
- `AgentPlanExecutorService` for executing multi-step plans
- Plan validation (cycle detection, dependency resolution)
- `UnifiedRouterResult` type with discriminated union
### 🤖 New Standard Agents
Added two new specialized agents:
- **Researcher Agent**: Web search, fact-finding, competitive
intelligence
- **Code Agent**: TypeScript function generation for serverless
workflows
### 🏗️ Router Refactoring (Latest)
Split router responsibilities into focused services:
- `AiRouterStrategyDeciderService`: Decides simple vs planned strategy
- `AiRouterPlanGeneratorService`: Generates and validates execution
plans
- `AiRouterService`: Coordinates between services (reduced from 426→275
lines)
### ⚙️ Configuration Improvements
- Added `outputStrategy` to agent definitions (`direct` vs `synthesize`)
- Removed hardcoded special cases for workflow-builder
- Added `plannerModel` field to workspace entity
- Increased `MAX_STEPS` from 10 to 25 for complex workflows
### 📝 Agent Prompt Refinements
Significantly simplified prompts for better clarity:
- Workflow Builder: 51→36 lines
- Helper: 49→28 lines
- Data Manipulator: Enhanced with sorting guidance
### 🔍 Enhanced Debugging
- Plan reasoning and step count in data message parts
- Router debug info with token usage tracking
- Better logging throughout execution pipeline
## Benefits
1. **Simpler Mental Model**: Router decides upfront vs dynamic transfers
2. **Better Predictability**: Users see the plan before execution
3. **Cleaner Architecture**: SRP with focused services
4. **Configuration Over Code**: Agent behavior via config, not hardcoded
logic
5. **Plan Validation**: Catches invalid dependencies and cycles
## Migration Notes
- Database migration removes `agentHandoff` table
- Adds `plannerModel` column to workspace table
- No API breaking changes (agent endpoints unchanged)
## Testing
- Integration tests updated to remove handoff dependencies
- Agent tool test utilities simplified
- Plan validation covered by new logic
## Next Steps (Future PRs)
- Parallel execution of independent plan steps
- Dynamic re-planning based on results
- Plan caching for common routing patterns
- Error recovery strategies in plan executor
## Context
Deprecating legacy ObjectMetadata from cache in favor of flat entities.
Introducing utils to build byName/byNameSingular/byNamePlural in
isolated cases
## Next
- I had to introduce a util to build from flat to legacy
objectMetadataMaps, we should instead use flat maps directly when needed
(datasource, schema generation, etc)
- Deprecate metadata version in the cache
- Use the new cache strategy for flat entities with permissions and
feature flags and inject in the global datasource context
closes https://github.com/twentyhq/core-team-issues/issues/1629
To do before requesting review :
- filter update
Migration to come in an other PR
Strat :
1/ Null transformation
- [x] Transform NULL equivalent value to NULL in field validation in
common api - pre-query - with feature flag
- [ ] Same logic in ORM (Not done, complex to handle feature flag here)
- [x] Transform NULL value to equivalent in data formatting in ORM -
post-query
2/ Migration (in other PR) for fieldMetadata not nullable with default
defaultValue (empty string, ...)
- [ ] Remove NOT NULL db constraint
- [ ] Update record value to NULL
- [ ] Update field metadata : isNullable:true
- [ ] Update uniqueIndex whereClause (also for standard uniqueIndex)
- [ ] Activate feature flag
3/ Update metadata creation
- [x] No more default default value
- [x] Update standard field nullability
- [x] Remove index default whereClause for standard field
4/ Update filter
- [x] When filtering on NULL or empty string, be sure all records are
returned (the one with NULL + the one with "")
5/ Test
- [ ] Strat. to do
Charles Bochet