feat: add configurable response format for AI agents (text/JSON) (#15953)

## Summary
This PR adds configurable response format support for AI agents,
allowing them to return either plain text or structured JSON data based
on a defined schema.

## Key Features

### 1. Agent Response Format Configuration
- Added `AgentResponseFormat` type supporting:
  - `text`: Returns plain text responses (default)
  - `json`: Returns structured JSON based on defined schema
- New `AgentResponseSchema` type moved to `twenty-shared/ai` for sharing
between frontend/backend

### 2. Settings UI
- New `SettingsAgentResponseFormat` component for configuring response
format
- Visual schema builder for defining JSON output structure
- Real-time validation and preview
- Integrated into agent settings tab

### 3. Workflow Integration
- AI Agent workflow action automatically uses agent's configured
response format
- Output schema dynamically generated from agent's response format
- Workflow variable picker shows structured fields for JSON responses
- Backward compatible with existing text-only agents

### 4. Backend Implementation
- Added `convertAgentSchemaToZod` utility to validate JSON responses
- Agent executor service handles both text and JSON generation
- Automatic agent creation/cloning when adding AI agent steps to
workflows
- Unique agent naming with conflict resolution

### 5. Database Migration
- Migration `1763622159656-update-agent-response-format.ts` 
- Sets default `responseFormat` to `{"type":"text"}` for existing agents
- Updated all standard agents with proper response format

## Changes by Module

### Frontend (`twenty-front`)
- 🆕 `AgentResponseFormat` type
- 🆕 `SettingsAgentResponseFormat` component
- ✏️ Updated `WorkflowEditActionAiAgent` to support response format
configuration
- 🗑️ Removed deprecated `useAiAgentOutputSchema` hook and
`AiAgentOutputSchema` type

### Backend (`twenty-server`)
- 🆕 `AgentResponseFormat` type in agent entity
- 🆕 `convertAgentSchemaToZod` utility for schema validation
- ✏️ Updated `AiAgentExecutorService` to handle both text and JSON
generation
- ✏️ Updated `WorkflowSchemaWorkspaceService` to generate output schema
from agent config
- ✏️ Enhanced `WorkflowVersionStepOperationsWorkspaceService` with agent
creation/cloning
- 🆕 Agent naming constants for conflict resolution

### Shared (`twenty-shared`)
- 🆕 `AgentResponseSchema` type
- 🆕 `ModelConfiguration` type moved to shared package
- Updated exports in `ai/index.ts`

## Code Quality
- Removed useless comments following code style guidelines
- All linter checks passed
- Type-safe implementation with proper TypeScript types

## Testing
-  Database migration tested
-  Agent creation/cloning in workflows verified
-  Response format switching (text ↔ JSON) validated
-  Backward compatibility with existing agents confirmed

## Migration Notes
- Existing agents will have `responseFormat: {type: 'text'}` set
automatically
- No breaking changes - all existing functionality preserved
- Agents can be updated to use JSON format through settings UI
This commit is contained in:
Félix Malfait
2025-11-20 18:32:44 +01:00
committed by GitHub
parent 5476879f77
commit a281f2a773
49 changed files with 1207 additions and 641 deletions
@@ -1,66 +0,0 @@
import { type OutputSchemaField } from '@/ai/constants/OutputFieldTypeOptions';
import { type WorkflowAiAgentAction } from '@/workflow/types/Workflow';
import { type AiAgentOutputSchema } from '@/workflow/workflow-variables/types/AiAgentOutputSchema';
import { useState } from 'react';
import { isDefined } from 'twenty-shared/utils';
import { useDebouncedCallback } from 'use-debounce';
import { v4 } from 'uuid';
export const useAiAgentOutputSchema = (
outputSchema?: AiAgentOutputSchema,
onActionUpdate?: (action: WorkflowAiAgentAction) => void,
action?: WorkflowAiAgentAction,
readonly?: boolean,
) => {
const [outputFields, setOutputFields] = useState<OutputSchemaField[]>(
Object.entries(outputSchema || {}).map(([name, field]) => ({
id: v4(),
name,
type: field.type,
})),
);
const debouncedSave = useDebouncedCallback(
async (fields: OutputSchemaField[]) => {
if (readonly === true) {
return;
}
const newOutputSchema = fields.reduce<AiAgentOutputSchema>(
(schema, field) => {
if (isDefined(field.name)) {
schema[field.name] = {
isLeaf: true,
type: field.type,
value: null,
label: field.name,
};
}
return schema;
},
{},
);
if (isDefined(onActionUpdate) && isDefined(action)) {
onActionUpdate({
...action,
settings: {
...action.settings,
outputSchema: newOutputSchema,
},
});
}
},
500,
);
const handleOutputSchemaChange = async (fields: OutputSchemaField[]) => {
setOutputFields(fields);
await debouncedSave(fields);
};
return {
handleOutputSchemaChange,
outputFields,
};
};