Commit Graph

20 Commits

Author SHA1 Message Date
Félix Malfait 6f251a6f8e Add @mention support in AI Chat input (#17943)
## Summary
- Add `@mention` support to the AI Chat text input by replacing the
plain textarea with a minimal Tiptap editor and building a shared
`mention` module with reusable Tiptap extensions (`MentionTag`,
`MentionSuggestion`), search hook (`useMentionSearch`), and suggestion
menu — all shared with the existing BlockNote-based Notes mentions to
avoid code duplication
- Mentions are serialized as
`[[record:objectName:recordId:displayName]]` markdown (the format
already understood by the backend and rendered in chat messages), and
displayed using the existing `RecordLink` chip component for visual
consistency
- Fix images in chat messages overflowing their container by
constraining to `max-width: 100%`
- Fix web_search tool display showing literal `{query}` instead of the
actual query (ICU single-quote escaping issue in Lingui `t` tagged
templates)

## Test plan
- [ ] Open AI Chat, type `@` and verify the suggestion menu appears with
searchable records
- [ ] Select a mention from the dropdown (via click or keyboard
Enter/ArrowUp/Down) and verify the record chip renders inline
- [ ] Send a message containing a mention and verify it appears
correctly in the conversation as a clickable `RecordLink`
- [ ] Verify Enter sends the message when the suggestion menu is closed,
and selects a mention when the menu is open
- [ ] Verify images in AI chat responses are constrained to the
container width
- [ ] Verify the web_search tool step shows the actual search query
(e.g. "Searched the web for Salesforce") instead of `{query}`
- [ ] Verify Notes @mentions still work as before


Made with [Cursor](https://cursor.com)

---------

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-02-14 14:37:33 +01:00
Félix Malfait 21c51ec251 Improve AI agent chat, tool display, and workflow agent management (#17876)
## Summary

- **Fix token renewal endpoint**: Use `/metadata` instead of `/graphql`
for token renewal in agent chat, fixing auth issues
- **Improve tool display**: Add `load_skills` support, show formatted
tool names (underscores → spaces) with finish/loading states, display
tool icons during loading, and support custom loading messages from tool
input
- **Refactor workflow agent management**: Replace direct
`AgentRepository` access with `AgentService` for create/delete/find
operations in workflow steps, improving encapsulation and consistency
- **Simplify Apollo client usage**: Remove explicit Apollo client
override in `useGetToolIndex`, add `AgentChatProvider` to
`AppRouterProviders`
- **Fix load-skill tool**: Change parameter type from `string` to `json`
for proper schema parsing
- **Update agent-chat-streaming**: Use `AgentService` for agent
resolution and tool registration instead of direct repository queries

## Test plan

- [ ] Verify AI agent chat works end-to-end (send message, receive
response)
- [ ] Verify tool steps display correctly with icons and proper messages
during loading and after completion
- [ ] Verify workflow AI agent step creation and deletion works
correctly
- [ ] Verify workflow version cloning preserves agent configuration
- [ ] Verify token renewal works when tokens expire during agent chat


Made with [Cursor](https://cursor.com)

---------

Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: cubic-dev-ai[bot] <191113872+cubic-dev-ai[bot]@users.noreply.github.com>
2026-02-13 09:27:38 +00:00
Charles Bochet b456f79167 Reduce leak between gql schema (#17878)
## Reduce type leakage between GraphQL schemas

### Why

Twenty runs two separate GraphQL schemas: **core** and **metadata**.
NestJS's `@nestjs/graphql` uses a global `TypeMetadataStorage` that
accumulates all decorated types across all modules. When each schema is
built, every registered type leaks into both schemas regardless of which
module it belongs to.

This means the core schema's generated TypeScript
(`generated/graphql.ts`) contained ~2,700 lines of types that only
belong to the metadata schema (and vice versa). This creates confusion
about type ownership, inflates generated code, and makes it harder to
reason about which API surface each schema actually exposes.

### How

**1. Patch `@nestjs/graphql` to support schema-scoped type resolution**

- **(Already done)** Added a `resolverSchemaScope` option to
`GqlModuleOptions`, allowing each schema to declare a scope (e.g.
`'metadata'`)
- `ResolversExplorerService` now filters resolvers by a
`RESOLVER_SCHEMA_SCOPE` metadata key, so each schema only sees its own
resolvers
- `GraphQLSchemaFactory` now performs a **reachability walk**
(`computeReachableTypes`) starting from scoped resolver return types and
arguments, only including types that are transitively referenced —
handling unions, interfaces, and prototype chains
- Type definition storage and orphaned reference registry are cleared
between schema builds to prevent cross-contamination

**2. Register `ClientConfig` as orphaned type in metadata schema**

Since `ClientConfig` is needed in the metadata schema but not directly
returned by a resolver, it's explicitly declared via
`buildSchemaOptions.orphanedTypes`.

**3. Regenerate frontend types and fix imports**

- `generated/graphql.ts` shrank by ~2,700 lines (types moved to where
they belong)
- `generated-metadata/graphql.ts` gained types like `ClientConfig` that
were previously missing
- ~500 frontend files updated to import from the correct generated file
2026-02-12 10:58:52 +01:00
Félix Malfait 3216b634a3 feat: improve AI chat - system prompt, tool output, context window display (#17769)
⚠️ **AI-generated PR — not ready for review** ⚠️

cc @FelixMalfait

---

## Changes

### System prompt improvements
- Explicit skill-before-tools workflow to prevent the model from calling
tools without loading the matching skill first
- Data efficiency guidance (default small limits, use filters)
- Pluralized `load_skill` → `load_skills` for consistency with
`load_tools`

### Token usage reduction
- Output serialization layer: strips null/undefined/empty values from
tool results
- Lowered default `find_*` limit from 100 → 10, max from 1000 → 100

### System object tool generation
- System objects (calendar events, messages, etc.) now generate AI tools
- Only workflow-related and favorite-related objects are excluded

### Context window display fix
- **Bug**: UI compared cumulative tokens (sum of all turns) against
single-request context window → showed 100% after a few turns
- **Fix**: Track `conversationSize` (last step's `inputTokens`) which
represents the actual conversation history size sent to the model
- New `conversationSize` column on thread entity with migration

### Workspace AI instructions
- Support for custom workspace-level AI instructions

---------

Co-authored-by: claude[bot] <41898282+claude[bot]@users.noreply.github.com>
2026-02-09 14:26:02 +01:00
Félix Malfait b46e9d2e64 feat: add AI chat error handling for billing and API key errors (#16797)
## Summary

This PR adds user-friendly error handling for AI chat features,
specifically for **billing credits exhausted** and **API key not
configured** errors.

## Changes

### Backend
- Added `BILLING_CREDITS_EXHAUSTED` exception code with 402 status
- Added `API_KEY_NOT_CONFIGURED` exception code with 503 status
- Added billing check before AI chat streaming in
`agent-chat.controller.ts`
- Added error code to HTTP exception response body for frontend error
type detection
- Created `AgentRestApiExceptionFilter` for agent-specific errors

### Frontend
- Created `AIChatBanner` - reusable banner component for error/warning
messages
- Created `AIChatCreditsExhaustedMessage` - shows upgrade prompts based
on user permissions
- Created `AIChatApiKeyNotConfiguredMessage` - shows configuration
guidance with docs link
- Created `AIChatErrorRenderer` - encapsulates error type switching
logic (fixes nested ternary)
- Created `AIChatStandaloneError` - displays errors when there are no
messages
- Split `aiChatErrorUtils.ts` into separate files (1 export per file):
  - `AIChatErrorCode.ts`
  - `extractErrorCode.ts`
  - `isAIChatErrorOfType.ts`
  - `isBillingCreditsExhaustedError.ts`
  - `isApiKeyNotConfiguredError.ts`
- Added comprehensive test coverage (27 tests)

### Other
- Updated trial period banner messaging

## Testing
- All lint checks pass
- All 27 new tests pass
- TypeScript typecheck passes
2025-12-24 15:30:28 +01:00
Félix Malfait b27a97f2c5 feat: enforce @/ alias for imports and fix all relative parent imports (#16787)
## Summary
This PR enforces the use of `@/` alias for imports instead of relative
parent imports (`../`).

## Changes

### ESLint Configuration
- Added `no-restricted-imports` pattern in `eslint.config.react.mjs` to
block `../*` imports with the message "Relative parent imports are not
allowed. Use @/ alias instead."
- Removed the non-working `import/no-relative-parent-imports` rule
(doesn't work properly in ESLint flat config)

### VS Code Settings
- Added `javascript.preferences.importModuleSpecifier: non-relative` to
`.vscode/settings.json` (TypeScript setting was already there)

### Code Fixes
- Fixed **941 relative parent imports** across **706 files** in
`packages/twenty-front`
- All `../` imports converted to use `@/` alias

## Why
- Consistent import style across the codebase
- Easier to move files without breaking imports
- Better IDE support for auto-imports
- Clearer understanding of where imports come from
2025-12-23 22:57:51 +01:00
Félix Malfait 4f91b48470 feat(ai): add context usage display to AI chat (BREAKING: deploy server first) (#16518)
## Summary

- Add a context usage indicator to the AI chat interface inspired by
Vercel's AI SDK Context component
- Display token consumption, context window utilization percentage, and
estimated cost in credits
- Show a circular progress ring with percentage, revealing detailed
breakdown on hover

## Changes

### Backend
- Stream usage metadata (tokens, model config) via `messageMetadata`
callback in `agent-chat-streaming.service.ts`
- Return model config from `chat-execution.service.ts`
- Add usage and model types to `ExtendedUIMessage` metadata

### Frontend
- New `ContextUsageProgressRing` component - circular SVG progress
indicator
- New `AIChatContextUsageButton` component with hover card showing:
  - Progress bar with used/total tokens
  - Input/output token counts with credit costs
  - Total credits consumed
- Track cumulative usage in Recoil state (`agentChatUsageState`)
- Reset usage when creating new chat thread
- Integrate button into `AIChatTab`

## Test plan

- [ ] Open AI chat and send a message
- [ ] Verify the context usage button appears with percentage
- [ ] Hover over the button to see detailed breakdown
- [ ] Verify credits are calculated correctly
- [ ] Create a new chat thread and verify usage resets to 0
2025-12-12 13:42:00 +01:00
Félix Malfait 4f20fd35c5 feat: Add Agent Evaluation System and Refactor AI Modules (#16111)
## Summary

This PR introduces a comprehensive agent evaluation system and refactors
the AI module structure for better organization.

## Key Changes

### 🎯 Agent Evaluation System
- Added **Agent Turn Evaluation** entities, DTOs, and database schema
- New GraphQL mutations: `evaluateAgentTurn` and `runEvaluationInput`
- Added `evaluationInputs` field to Agent entity for storing test inputs
- New `AgentTurnGraderService` for automatic turn evaluation
- Added evaluation UI with new **Evals** and **Logs** tabs in agent
detail pages

### 🏗️ Entity & Module Refactoring
- Renamed `AgentChatMessage` → `AgentMessage` for clarity
- Consolidated chat entities: `AgentMessage`, `AgentTurn`, and
`AgentChatThread`
- Reorganized AI modules under `ai/` subdirectory structure
- Updated imports across codebase to reflect new module paths

### 🤖 New Agents & Roles
- Added **Dashboard Builder Agent** for dashboard creation and
management
- Added **Dashboard Manager Role** with appropriate permissions
- Updated role permissions to be more granular (users vs agents vs API
keys)

### 🔐 Permission System Updates
- Added `HTTP_REQUEST_TOOL` permission flag
- Updated Workflow Manager role permissions (restricted tool access)
- Enhanced permission flag types to differentiate between user/agent/API
key contexts
- Added `isRelevantForAgents`, `isRelevantForApiKeys`,
`isRelevantForUsers` to permission flags

### 📨 Message Role Enhancement
- Added `system` role to `AgentMessageRole` enum (alongside
user/assistant)
- Updated message handling to support system prompts

### 🎨 UI/UX Improvements
- New tabs in agent detail: **Evals** and **Logs**
- Added turn detail page: `/ai/agents/:agentId/turns/:turnId`
- Fixed text overflow in `SettingsListItemCardContent`
- Updated role applicability labels ("Assignable to Workspace Members")

### 🛠️ Technical Improvements
- Fixed Zod schema validation for UUID and Date fields (use string
validators)
- Updated `ToolRegistryService` to properly register HTTP tool with
permission flag
- Enhanced error handling in agent execution services
- Updated database migrations for new entity schema

## Database Migrations
- `1764210000000-add-system-role-to-agent-message.ts`
- `1764220000000-add-evaluation-inputs-to-agent.ts`
- `1764200000000-add-agent-turn-evaluation.ts`
- `1764100000000-refactor-agent-chat-entities.ts`

## Testing
- [ ] Agent evaluation flow tested
- [ ] Dashboard Builder agent tested
- [ ] Permission system validated
- [ ] UI tabs and navigation tested
- [ ] Database migrations run successfully

## Breaking Changes
⚠️ **Entity Rename**: `AgentChatMessage` renamed to `AgentMessage` -
GraphQL queries need updating

## Related Issues
<!-- Link any related issues here -->

## Screenshots
<!-- Add screenshots if applicable -->
2025-11-27 08:25:40 +01:00
Félix Malfait a281f2a773 feat: add configurable response format for AI agents (text/JSON) (#15953)
## Summary
This PR adds configurable response format support for AI agents,
allowing them to return either plain text or structured JSON data based
on a defined schema.

## Key Features

### 1. Agent Response Format Configuration
- Added `AgentResponseFormat` type supporting:
  - `text`: Returns plain text responses (default)
  - `json`: Returns structured JSON based on defined schema
- New `AgentResponseSchema` type moved to `twenty-shared/ai` for sharing
between frontend/backend

### 2. Settings UI
- New `SettingsAgentResponseFormat` component for configuring response
format
- Visual schema builder for defining JSON output structure
- Real-time validation and preview
- Integrated into agent settings tab

### 3. Workflow Integration
- AI Agent workflow action automatically uses agent's configured
response format
- Output schema dynamically generated from agent's response format
- Workflow variable picker shows structured fields for JSON responses
- Backward compatible with existing text-only agents

### 4. Backend Implementation
- Added `convertAgentSchemaToZod` utility to validate JSON responses
- Agent executor service handles both text and JSON generation
- Automatic agent creation/cloning when adding AI agent steps to
workflows
- Unique agent naming with conflict resolution

### 5. Database Migration
- Migration `1763622159656-update-agent-response-format.ts` 
- Sets default `responseFormat` to `{"type":"text"}` for existing agents
- Updated all standard agents with proper response format

## Changes by Module

### Frontend (`twenty-front`)
- 🆕 `AgentResponseFormat` type
- 🆕 `SettingsAgentResponseFormat` component
- ✏️ Updated `WorkflowEditActionAiAgent` to support response format
configuration
- 🗑️ Removed deprecated `useAiAgentOutputSchema` hook and
`AiAgentOutputSchema` type

### Backend (`twenty-server`)
- 🆕 `AgentResponseFormat` type in agent entity
- 🆕 `convertAgentSchemaToZod` utility for schema validation
- ✏️ Updated `AiAgentExecutorService` to handle both text and JSON
generation
- ✏️ Updated `WorkflowSchemaWorkspaceService` to generate output schema
from agent config
- ✏️ Enhanced `WorkflowVersionStepOperationsWorkspaceService` with agent
creation/cloning
- 🆕 Agent naming constants for conflict resolution

### Shared (`twenty-shared`)
- 🆕 `AgentResponseSchema` type
- 🆕 `ModelConfiguration` type moved to shared package
- Updated exports in `ai/index.ts`

## Code Quality
- Removed useless comments following code style guidelines
- All linter checks passed
- Type-safe implementation with proper TypeScript types

## Testing
-  Database migration tested
-  Agent creation/cloning in workflows verified
-  Response format switching (text ↔ JSON) validated
-  Backward compatibility with existing agents confirmed

## Migration Notes
- Existing agents will have `responseFormat: {type: 'text'}` set
automatically
- No breaking changes - all existing functionality preserved
- Agents can be updated to use JSON format through settings UI
2025-11-20 18:32:44 +01:00
Abdul Rahman 06d8c8c76a Fix find tool filters by mapping many-to-one relations to fieldId (#15716)
**Root Cause**  
Many-to-one relation filters were being exposed under the relation name
(e.g., `company`) instead of the corresponding foreign-key attribute
(e.g., `companyId`).

**Change**  
- Detect many-to-one relation metadata and remap those filter keys to
`<fieldName>Id`.
- Removed some unused code unrelated to this fix.
2025-11-08 14:32:06 +01:00
Abdul Rahman e518f03031 Fix: AI Agent tool errors and relation field handling (#15668)
### Problems Fixed

1. **Tool execution errors broke conversations**
- Failed tool executions showed "Processing..." indefinitely instead of
error messages
- Tool errors with `input: null` caused subsequent messages to fail with
`Missing required parameter: 'input[X].arguments'`

2. **Relation fields not saved in AI Agent**
- AI Agent couldn't save relation fields (e.g., `companyId`) when
creating/upserting records
   - Join column names weren't recognized during field validation

### Solutions

**Tool Error Handling:**
- Display error messages in UI with expandable error details
- Ensure tool parts always have valid `input` field (`input:
part.toolInput ?? {}`)
- Refactored `ToolStepRenderer` to accept complete `toolPart` object

**Relation Field Support:**
- Updated field validation in `create-record.service.ts` and
`upsert-record.service.ts`
- Check both `fieldIdByName` and `fieldIdByJoinColumnName` mappings

### Changes
- `packages/twenty-front/src/modules/ai/`
  - `ToolStepRenderer.tsx` - Error state handling
  - `AIChatAssistantMessageRenderer.tsx` - Pass complete toolPart
  - `mapDBPartToUIMessagePart.ts` - Prevent null tool input
- `packages/twenty-server/src/engine/core-modules/record-crud/services/`
  - `create-record.service.ts` - Add join column validation
  - `upsert-record.service.ts` - Add join column validation
2025-11-06 12:28:00 +01:00
Abdul Rahman 32558673c6 feat: Implement AI Router for Dynamic Agent Selection (#15227)
Adds intelligent routing system that automatically selects the best
agent for user queries based on conversation context.

### Changes:
- Added `routerModel` column to workspace table for configurable router
LLM selection
- Implemented `RouterService` with conversation history analysis and
agent matching logic
- Created router settings UI in AI Settings page with model dropdown
- Removed agent-specific thread associations - threads are now
agent-agnostic
- Added real-time routing status notification in chat UI with shimmer
effect
- Removed automatic default assistant agent creation
- Renamed GraphQL operations from agent-specific to generic (e.g.,
`agentChatThreads` → `chatThreads`)

---------

Co-authored-by: Félix Malfait <felix.malfait@gmail.com>
Co-authored-by: Félix Malfait <felix@twenty.com>
2025-10-22 15:02:41 +02:00
Abdul Rahman 20403664e3 Feat: native model capabilities (#14787) 2025-10-01 18:37:21 +02:00
Abdul Rahman 2685f4a5b9 Restructure agent chat messages with parts-based architecture (#14749)
Co-authored-by: Félix Malfait <felix@twenty.com>
2025-09-29 13:31:55 +02:00
Abdul Rahman 216d72b5d7 AI SDK v5 migration (#14549)
Co-authored-by: Félix Malfait <felix@twenty.com>
2025-09-22 22:13:43 +02:00
Abdul Rahman 8d9617131c Feat: Add reasoning summary and tool details display for AI responses (#14414)
Closes [#1405](https://github.com/twentyhq/core-team-issues/issues/1405)

### Implemented Features
- Reasoning Summary: Collapsible display of AI thinking process with
real-time streaming
- Tool Execution: Detailed tool call rendering with expandable results
and loading states
- Error Handling: Clean error stream event display with proper
formatting


### Demo



https://github.com/user-attachments/assets/76ad29b7-d831-4bc8-a115-ad25f7ace6e2

<img width="504" height="819" alt="Screenshot 2025-09-12 at 3 23 39 AM"
src="https://github.com/user-attachments/assets/82dc177d-f009-4e8a-8901-ee1a49201984"
/>

---------

Co-authored-by: Félix Malfait <felix@twenty.com>
2025-09-17 11:30:24 +02:00
Félix Malfait 464a480043 Continue ESLINT9 Migration (#13795)
Might already fix #13793
2025-08-10 23:25:58 +02:00
Antoine Moreaux 153739b9c3 feat(ai): add current context to ai chat (#13315)
## TODO

- [ ] add dropdown to use records from outside the context
- [x] add loader for files chip
- [x] add roleId where it's necessary
- [x] Split AvatarChip in two components. One with the icon that will
call the second with leftComponent.
- [ ] Fix tests
- [x] Fix UI regression on Search
2025-07-22 17:27:19 +02:00
Abdul Rahman 0c73c9df50 Agent chat file drag and drop (#13226)
- Added drag and drop support
- The AIChatTab component has been refactored into smaller, more focused
components to improve readability, maintainability, and scalability of
the codebase.
- Introduced a custom useAIChatFileUpload hook to encapsulate and manage
file upload logic, promoting code reuse and separation of concerns.

### Demo

https://github.com/user-attachments/assets/c4b2a67a-2736-48ae-9ba8-8e124e4b6069

---------

Co-authored-by: Raphaël Bosi <71827178+bosiraphael@users.noreply.github.com>
Co-authored-by: neo773 <62795688+neo773@users.noreply.github.com>
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
Co-authored-by: Félix Malfait <felix.malfait@gmail.com>
Co-authored-by: Félix Malfait <felix@twenty.com>
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: github-actions <github-actions@twenty.com>
Co-authored-by: MD Readul Islam <99027968+readul-islam@users.noreply.github.com>
Co-authored-by: readul-islam <developer.readul@gamil.com>
Co-authored-by: Thomas des Francs <tdesfrancs@gmail.com>
Co-authored-by: Guillim <guillim@users.noreply.github.com>
Co-authored-by: Lucas Bordeau <bordeau.lucas@gmail.com>
Co-authored-by: Jean-Baptiste Ronssin <65334819+jbronssin@users.noreply.github.com>
Co-authored-by: kahkashan shaik <93042682+kahkashanshaik@users.noreply.github.com>
Co-authored-by: martmull <martmull@hotmail.fr>
Co-authored-by: Paul Rastoin <45004772+prastoin@users.noreply.github.com>
Co-authored-by: bosiraphael <raphael.bosi@gmail.com>
Co-authored-by: Naifer <161821705+omarNaifer12@users.noreply.github.com>
Co-authored-by: Marie Stoppa <marie.stoppa@essec.edu>
2025-07-16 13:39:54 +02:00
Abdul Rahman 8edf59a521 Feat: Agent chat multi thread support (#13216)
Co-authored-by: Raphaël Bosi <71827178+bosiraphael@users.noreply.github.com>
Co-authored-by: neo773 <62795688+neo773@users.noreply.github.com>
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
Co-authored-by: Félix Malfait <felix.malfait@gmail.com>
Co-authored-by: Félix Malfait <felix@twenty.com>
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: github-actions <github-actions@twenty.com>
Co-authored-by: MD Readul Islam <99027968+readul-islam@users.noreply.github.com>
Co-authored-by: readul-islam <developer.readul@gamil.com>
Co-authored-by: Thomas des Francs <tdesfrancs@gmail.com>
Co-authored-by: Guillim <guillim@users.noreply.github.com>
Co-authored-by: Lucas Bordeau <bordeau.lucas@gmail.com>
Co-authored-by: Jean-Baptiste Ronssin <65334819+jbronssin@users.noreply.github.com>
Co-authored-by: kahkashan shaik <93042682+kahkashanshaik@users.noreply.github.com>
Co-authored-by: martmull <martmull@hotmail.fr>
Co-authored-by: Paul Rastoin <45004772+prastoin@users.noreply.github.com>
Co-authored-by: bosiraphael <raphael.bosi@gmail.com>
Co-authored-by: Naifer <161821705+omarNaifer12@users.noreply.github.com>
Co-authored-by: Marie Stoppa <marie.stoppa@essec.edu>
2025-07-16 09:26:40 +02:00