Files
twenty/packages/twenty-server/test/integration/metadata/suites/agent/utils/agent-tool-test-utils.ts
T
Félix Malfait e7ebf51e50 Replace agent handoff system with planning-based router (#16003)
## Overview

This PR replaces the dynamic agent handoff system with a more
predictable planning-based router that decides upfront how to handle
multi-agent coordination.

## Major Changes

### 🔄 Architecture Shift: Handoffs → Planning

**Removed:**
- `AgentHandoffEntity` and handoff tracking system
- `AgentHandoffService` and `AgentHandoffExecutorService`
- Dynamic agent-to-agent transfers during execution
- Handoff tool generation and description templates

**Added:**
- `AiRouterService` with two strategies: `simple` (single agent) and
`planned` (multi-agent)
- `AgentPlanExecutorService` for executing multi-step plans
- Plan validation (cycle detection, dependency resolution)
- `UnifiedRouterResult` type with discriminated union

### 🤖 New Standard Agents

Added two new specialized agents:
- **Researcher Agent**: Web search, fact-finding, competitive
intelligence
- **Code Agent**: TypeScript function generation for serverless
workflows

### 🏗️ Router Refactoring (Latest)

Split router responsibilities into focused services:
- `AiRouterStrategyDeciderService`: Decides simple vs planned strategy
- `AiRouterPlanGeneratorService`: Generates and validates execution
plans
- `AiRouterService`: Coordinates between services (reduced from 426→275
lines)

### ⚙️ Configuration Improvements

- Added `outputStrategy` to agent definitions (`direct` vs `synthesize`)
- Removed hardcoded special cases for workflow-builder
- Added `plannerModel` field to workspace entity
- Increased `MAX_STEPS` from 10 to 25 for complex workflows

### 📝 Agent Prompt Refinements

Significantly simplified prompts for better clarity:
- Workflow Builder: 51→36 lines
- Helper: 49→28 lines
- Data Manipulator: Enhanced with sorting guidance

### 🔍 Enhanced Debugging

- Plan reasoning and step count in data message parts
- Router debug info with token usage tracking
- Better logging throughout execution pipeline

## Benefits

1. **Simpler Mental Model**: Router decides upfront vs dynamic transfers
2. **Better Predictability**: Users see the plan before execution
3. **Cleaner Architecture**: SRP with focused services
4. **Configuration Over Code**: Agent behavior via config, not hardcoded
logic
5. **Plan Validation**: Catches invalid dependencies and cycles

## Migration Notes

- Database migration removes `agentHandoff` table
- Adds `plannerModel` column to workspace table
- No API breaking changes (agent endpoints unchanged)

## Testing

- Integration tests updated to remove handoff dependencies
- Agent tool test utilities simplified
- Plan validation covered by new logic

## Next Steps (Future PRs)

- Parallel execution of independent plan steps
- Dynamic re-planning based on results
- Plan caching for common routing patterns
- Error recovery strategies in plan executor
2025-11-25 12:10:14 +01:00

332 lines
11 KiB
TypeScript

import { Test, type TestingModule } from '@nestjs/testing';
import { getRepositoryToken } from '@nestjs/typeorm';
import { type Repository } from 'typeorm';
import { type ApplicationEntity } from 'src/engine/core-modules/application/application.entity';
import { CreateRecordService } from 'src/engine/core-modules/record-crud/services/create-record.service';
import { DeleteRecordService } from 'src/engine/core-modules/record-crud/services/delete-record.service';
import { FindRecordsService } from 'src/engine/core-modules/record-crud/services/find-records.service';
import { UpdateRecordService } from 'src/engine/core-modules/record-crud/services/update-record.service';
import { RecordInputTransformerService } from 'src/engine/core-modules/record-transformer/services/record-input-transformer.service';
import { ToolRegistryService } from 'src/engine/core-modules/tool/services/tool-registry.service';
import { SearchArticlesTool } from 'src/engine/core-modules/tool/tools/search-articles-tool/search-articles-tool';
import { SendEmailTool } from 'src/engine/core-modules/tool/tools/send-email-tool/send-email-tool';
import { TwentyConfigService } from 'src/engine/core-modules/twenty-config/twenty-config.service';
import { AgentService } from 'src/engine/metadata-modules/ai-agent/agent.service';
import { AgentEntity } from 'src/engine/metadata-modules/ai-agent/entities/agent.entity';
import { AgentToolGeneratorService } from 'src/engine/metadata-modules/ai-agent/services/agent-tool-generator.service';
import { ToolAdapterService } from 'src/engine/metadata-modules/ai-tools/services/tool-adapter.service';
import { ToolService } from 'src/engine/metadata-modules/ai-tools/services/tool.service';
import { WorkspaceManyOrAllFlatEntityMapsCacheService } from 'src/engine/metadata-modules/flat-entity/services/workspace-many-or-all-flat-entity-maps-cache.service';
import { ObjectMetadataServiceV2 } from 'src/engine/metadata-modules/object-metadata/object-metadata-v2.service';
import { type ObjectMetadataEntity } from 'src/engine/metadata-modules/object-metadata/object-metadata.entity';
import { PermissionsService } from 'src/engine/metadata-modules/permissions/permissions.service';
import { RoleEntity } from 'src/engine/metadata-modules/role/role.entity';
import { WorkspacePermissionsCacheService } from 'src/engine/metadata-modules/workspace-permissions-cache/workspace-permissions-cache.service';
import { ScopedWorkspaceContextFactory } from 'src/engine/twenty-orm/factories/scoped-workspace-context.factory';
import { TwentyORMGlobalManager } from 'src/engine/twenty-orm/twenty-orm-global.manager';
import { MessagingSendMessageService } from 'src/modules/messaging/message-import-manager/services/messaging-send-message.service';
import { WorkflowToolWorkspaceService } from 'src/modules/workflow/workflow-tools/services/workflow-tool.workspace-service';
import { getMockObjectMetadataEntity } from 'src/utils/__test__/get-object-metadata-entity.mock';
export interface AgentToolTestContext {
module: TestingModule;
agentToolService: AgentToolGeneratorService;
agentService: AgentService;
objectMetadataService: ObjectMetadataServiceV2;
roleRepository: Repository<RoleEntity>;
workspacePermissionsCacheService: WorkspacePermissionsCacheService;
twentyORMGlobalManager: TwentyORMGlobalManager;
testAgent: AgentEntity & { roleId: string | null };
testRole: RoleEntity;
testObjectMetadata: ObjectMetadataEntity;
testWorkspaceId: string;
testAgentId: string;
testRoleId: string;
}
export const createAgentToolTestModule =
async (): Promise<AgentToolTestContext> => {
const testWorkspaceId = 'test-workspace-id';
const testAgentId = 'test-agent-id';
const testRoleId = 'test-role-id';
const module = await Test.createTestingModule({
providers: [
AgentToolGeneratorService,
{
provide: AgentService,
useValue: {
findOneAgent: jest.fn(),
},
},
{
provide: getRepositoryToken(RoleEntity),
useValue: {
findOne: jest.fn(),
find: jest.fn(),
},
},
{
provide: getRepositoryToken(AgentEntity),
useValue: {
findOne: jest.fn(),
find: jest.fn(),
},
},
{
provide: ObjectMetadataServiceV2,
useValue: {
findManyWithinWorkspace: jest.fn(),
findOneWithinWorkspace: jest.fn(),
},
},
{
provide: TwentyORMGlobalManager,
useValue: {
getRepositoryForWorkspace: jest.fn(),
},
},
{
provide: WorkspacePermissionsCacheService,
useValue: {
getRolesPermissionsFromCache: jest.fn(),
},
},
{
provide: ToolService,
useClass: ToolService,
},
{
provide: CreateRecordService,
useValue: {
execute: jest.fn(),
},
},
{
provide: UpdateRecordService,
useValue: {
execute: jest.fn(),
},
},
{
provide: DeleteRecordService,
useValue: {
execute: jest.fn(),
},
},
{
provide: FindRecordsService,
useValue: {
execute: jest.fn().mockResolvedValue({
success: true,
message: 'Records found successfully',
result: [],
}),
},
},
{
provide: RecordInputTransformerService,
useValue: {
process: jest.fn(async ({ recordInput }) => recordInput),
},
},
{
provide: WorkspaceManyOrAllFlatEntityMapsCacheService,
useValue: {
getOrRecomputeManyOrAllFlatEntityMaps: jest.fn(),
},
},
{
provide: ToolAdapterService,
useClass: ToolAdapterService,
},
{
provide: ToolRegistryService,
useClass: ToolRegistryService,
},
{
provide: SendEmailTool,
useValue: {
description: 'mock',
inputSchema: {},
execute: jest.fn(),
},
},
{
provide: SearchArticlesTool,
useValue: {
description: 'Search for articles and documentation',
inputSchema: {},
execute: jest.fn(),
},
},
{
provide: ScopedWorkspaceContextFactory,
useValue: {
create: jest.fn(() => ({ workspaceId: 'test-workspace-id' })),
},
},
{
provide: MessagingSendMessageService,
useValue: { sendMessage: jest.fn() },
},
{
provide: PermissionsService,
useValue: {
hasToolPermission: jest.fn(),
checkRolePermissions: jest.fn().mockReturnValue(true),
checkRolesPermissions: jest.fn().mockResolvedValue(true),
},
},
{
provide: WorkflowToolWorkspaceService,
useValue: {
generateWorkflowTools: jest.fn().mockResolvedValue({}),
},
},
{
provide: TwentyConfigService,
useValue: {
get: jest.fn(),
},
},
],
}).compile();
const agentToolService = module.get<AgentToolGeneratorService>(
AgentToolGeneratorService,
);
const agentService = module.get<AgentService>(AgentService);
const objectMetadataService = module.get<ObjectMetadataServiceV2>(
ObjectMetadataServiceV2,
);
const roleRepository = module.get<Repository<RoleEntity>>(
getRepositoryToken(RoleEntity),
);
const workspacePermissionsCacheService =
module.get<WorkspacePermissionsCacheService>(
WorkspacePermissionsCacheService,
);
const twentyORMGlobalManager = module.get<TwentyORMGlobalManager>(
TwentyORMGlobalManager,
);
const testAgent: AgentEntity & { roleId: string | null } = {
id: testAgentId,
name: 'test-agent',
label: 'Test Agent',
icon: 'IconTest',
isCustom: false,
applicationId: null,
application: {} as ApplicationEntity,
standardId: null,
deletedAt: null,
universalIdentifier: testAgentId,
description: 'Test agent for integration tests',
prompt: 'You are a test agent',
modelId: 'gpt-4o',
responseFormat: { type: 'text' },
workspaceId: testWorkspaceId,
workspace: {} as any,
roleId: testRoleId,
createdAt: new Date(),
updatedAt: new Date(),
modelConfiguration: {},
};
const testRole: RoleEntity = {
id: testRoleId,
label: 'Test Role',
description: 'Test role for integration tests',
canUpdateAllSettings: false,
canReadAllObjectRecords: true,
canUpdateAllObjectRecords: true,
canSoftDeleteAllObjectRecords: true,
canDestroyAllObjectRecords: false,
workspaceId: testWorkspaceId,
createdAt: new Date(),
updatedAt: new Date(),
isEditable: true,
} as RoleEntity;
const testObjectMetadata = getMockObjectMetadataEntity({
id: 'test-object-id',
standardId: null,
dataSourceId: 'test-data-source-id',
nameSingular: 'testObject',
namePlural: 'testObjects',
labelSingular: 'Test Object',
labelPlural: 'Test Objects',
description: 'Test object for integration tests',
icon: 'IconTest',
targetTableName: 'test_objects',
isActive: true,
isSystem: false,
isCustom: false,
isRemote: false,
isAuditLogged: true,
isSearchable: false,
shortcut: '',
isLabelSyncedWithName: false,
workspaceId: testWorkspaceId,
createdAt: new Date(),
updatedAt: new Date(),
fields: [],
indexMetadatas: [],
targetRelationFields: [],
dataSource: {} as any,
objectPermissions: [],
fieldPermissions: [],
});
// Ensure ToolService input transformation has access to minimal metadata maps
const workspaceManyOrAllFlatEntityMapsCacheService =
module.get<WorkspaceManyOrAllFlatEntityMapsCacheService>(
WorkspaceManyOrAllFlatEntityMapsCacheService,
);
// Return a barebones flat object metadata map where fields are unknown (so transformer is a no-op)
const getMapsMock =
workspaceManyOrAllFlatEntityMapsCacheService.getOrRecomputeManyOrAllFlatEntityMaps as jest.Mock;
getMapsMock.mockResolvedValue({
flatObjectMetadataMaps: {
byId: {
[testObjectMetadata.id]: {
...testObjectMetadata,
fieldMetadataIds: [],
indexMetadataIds: [],
viewIds: [],
universalIdentifier: testObjectMetadata.id,
applicationId: null,
} as any,
},
idByUniversalIdentifier: {},
universalIdentifiersByApplicationId: {},
},
flatFieldMetadataMaps: {
byId: {},
idByUniversalIdentifier: {},
universalIdentifiersByApplicationId: {},
},
} as any);
return {
module,
agentToolService,
agentService,
objectMetadataService,
roleRepository,
workspacePermissionsCacheService,
twentyORMGlobalManager,
testAgent,
testRole,
testObjectMetadata,
testWorkspaceId,
testAgentId,
testRoleId,
};
};