Félix Malfait
4f20fd35c5
## Summary This PR introduces a comprehensive agent evaluation system and refactors the AI module structure for better organization. ## Key Changes ### 🎯 Agent Evaluation System - Added **Agent Turn Evaluation** entities, DTOs, and database schema - New GraphQL mutations: `evaluateAgentTurn` and `runEvaluationInput` - Added `evaluationInputs` field to Agent entity for storing test inputs - New `AgentTurnGraderService` for automatic turn evaluation - Added evaluation UI with new **Evals** and **Logs** tabs in agent detail pages ### 🏗️ Entity & Module Refactoring - Renamed `AgentChatMessage` → `AgentMessage` for clarity - Consolidated chat entities: `AgentMessage`, `AgentTurn`, and `AgentChatThread` - Reorganized AI modules under `ai/` subdirectory structure - Updated imports across codebase to reflect new module paths ### 🤖 New Agents & Roles - Added **Dashboard Builder Agent** for dashboard creation and management - Added **Dashboard Manager Role** with appropriate permissions - Updated role permissions to be more granular (users vs agents vs API keys) ### 🔐 Permission System Updates - Added `HTTP_REQUEST_TOOL` permission flag - Updated Workflow Manager role permissions (restricted tool access) - Enhanced permission flag types to differentiate between user/agent/API key contexts - Added `isRelevantForAgents`, `isRelevantForApiKeys`, `isRelevantForUsers` to permission flags ### 📨 Message Role Enhancement - Added `system` role to `AgentMessageRole` enum (alongside user/assistant) - Updated message handling to support system prompts ### 🎨 UI/UX Improvements - New tabs in agent detail: **Evals** and **Logs** - Added turn detail page: `/ai/agents/:agentId/turns/:turnId` - Fixed text overflow in `SettingsListItemCardContent` - Updated role applicability labels ("Assignable to Workspace Members") ### 🛠️ Technical Improvements - Fixed Zod schema validation for UUID and Date fields (use string validators) - Updated `ToolRegistryService` to properly register HTTP tool with permission flag - Enhanced error handling in agent execution services - Updated database migrations for new entity schema ## Database Migrations - `1764210000000-add-system-role-to-agent-message.ts` - `1764220000000-add-evaluation-inputs-to-agent.ts` - `1764200000000-add-agent-turn-evaluation.ts` - `1764100000000-refactor-agent-chat-entities.ts` ## Testing - [ ] Agent evaluation flow tested - [ ] Dashboard Builder agent tested - [ ] Permission system validated - [ ] UI tabs and navigation tested - [ ] Database migrations run successfully ## Breaking Changes ⚠️ **Entity Rename**: `AgentChatMessage` renamed to `AgentMessage` - GraphQL queries need updating ## Related Issues <!-- Link any related issues here --> ## Screenshots <!-- Add screenshots if applicable -->
40 lines
2.0 KiB
TypeScript
40 lines
2.0 KiB
TypeScript
import { Module } from '@nestjs/common';
|
|
import { TypeOrmModule } from '@nestjs/typeorm';
|
|
|
|
import { ApiKeyModule } from 'src/engine/core-modules/api-key/api-key.module';
|
|
import { ApplicationModule } from 'src/engine/core-modules/application/application.module';
|
|
import { FileModule } from 'src/engine/core-modules/file/file.module';
|
|
import { UserWorkspaceEntity } from 'src/engine/core-modules/user-workspace/user-workspace.entity';
|
|
import { UserWorkspaceModule } from 'src/engine/core-modules/user-workspace/user-workspace.module';
|
|
import { WorkspaceEntity } from 'src/engine/core-modules/workspace/workspace.entity';
|
|
import { AiAgentRoleModule } from 'src/engine/metadata-modules/ai/ai-agent-role/ai-agent-role.module';
|
|
import { ObjectPermissionModule } from 'src/engine/metadata-modules/object-permission/object-permission.module';
|
|
import { PermissionFlagModule } from 'src/engine/metadata-modules/permission-flag/permission-flag.module';
|
|
import { PermissionsModule } from 'src/engine/metadata-modules/permissions/permissions.module';
|
|
import { RoleTargetsEntity } from 'src/engine/metadata-modules/role/role-targets.entity';
|
|
import { RoleEntity } from 'src/engine/metadata-modules/role/role.entity';
|
|
import { RoleResolver } from 'src/engine/metadata-modules/role/role.resolver';
|
|
import { RoleService } from 'src/engine/metadata-modules/role/role.service';
|
|
import { UserRoleModule } from 'src/engine/metadata-modules/user-role/user-role.module';
|
|
import { WorkspacePermissionsCacheModule } from 'src/engine/metadata-modules/workspace-permissions-cache/workspace-permissions-cache.module';
|
|
|
|
@Module({
|
|
imports: [
|
|
TypeOrmModule.forFeature([RoleEntity, RoleTargetsEntity]),
|
|
TypeOrmModule.forFeature([UserWorkspaceEntity, WorkspaceEntity]),
|
|
UserRoleModule,
|
|
AiAgentRoleModule,
|
|
ApiKeyModule,
|
|
PermissionsModule,
|
|
UserWorkspaceModule,
|
|
ObjectPermissionModule,
|
|
PermissionFlagModule,
|
|
WorkspacePermissionsCacheModule,
|
|
FileModule,
|
|
ApplicationModule,
|
|
],
|
|
providers: [RoleService, RoleResolver],
|
|
exports: [RoleService],
|
|
})
|
|
export class RoleModule {}
|