## Summary
- The exception class under `ai-agent/` was serving every AI surface
(agent, chat, role, models, generate-text), so `Agent` was a misnomer.
Promoted to the `ai/` namespace; renamed `AgentException` →
`AiException`, `AgentExceptionCode` → `AiExceptionCode`, and related
interceptor / filter / handler / file names accordingly.
- Split the single `AGENT_NOT_FOUND` code into entity-specific codes.
Chat-thread lookups no longer reuse the agent identifier.
- **Fixes Sentry 500s on `GetChatMessages` / `chatThread`.** Every
"Thread not found" and "Queued message not found" throw site in ai-chat
was previously wired to `AGENT_EXECUTION_FAILED`, which maps to
`InternalServerError` (HTTP 500). They now use `THREAD_NOT_FOUND` /
`MESSAGE_NOT_FOUND`, both of which map to `NotFoundError` (HTTP 404) in
the GraphQL and REST handlers.
The underlying cause of *why* clients are asking for threads that no
longer resolve for them — per-user chat-thread create events being
broadcast workspace-wide — is addressed separately in a follow-up PR.
### Code map
- Added: `ai/ai.exception.ts`,
`ai/utils/ai-graphql-api-exception-handler.util.ts` (+ spec with new
THREAD/MESSAGE cases),
`ai/interceptors/ai-graphql-api-exception.interceptor.ts`,
`ai/filters/ai-api-exception.filter.ts`
- Deleted: `ai/ai-agent/agent.exception.ts`,
`ai/ai-agent/utils/agent-graphql-api-exception-handler.util.ts` (+
spec),
`ai/ai-agent/interceptors/agent-graphql-api-exception.interceptor.ts`,
`ai/ai-agent/filters/agent-api-exception.filter.ts`
- Updated: 21 call sites across ai-agent, ai-agent-execution,
ai-agent-role, ai-chat, ai-generate-text, ai-models, role, and
workspace-migration validators.
## Test plan
- [x] `npx nx typecheck twenty-server`
- [x] `npx jest ai-graphql-api-exception-handler` (3/3 including new
THREAD_NOT_FOUND and MESSAGE_NOT_FOUND cases)
- [x] `npx jest agent-role.service` (9/9)
- [x] `npx oxlint --type-aware` on all changed files (0 warnings/errors)
- [x] `npx prettier --check` on all changed files
- [ ] CI
Automated daily sync of `ai-providers.json` from
[models.dev](https://models.dev).
This PR updates pricing, context windows, and model availability based
on the latest data.
New models meeting inclusion criteria (tool calling, pricing data,
context limits) are added automatically.
Deprecated models are detected based on cost-efficiency within the same
model family.
**Please review before merging** — verify no critical models were
incorrectly deprecated.
Co-authored-by: FelixMalfait <6399865+FelixMalfait@users.noreply.github.com>
Automated daily sync of `ai-providers.json` from
[models.dev](https://models.dev).
This PR updates pricing, context windows, and model availability based
on the latest data.
New models meeting inclusion criteria (tool calling, pricing data,
context limits) are added automatically.
Deprecated models are detected based on cost-efficiency within the same
model family.
**Please review before merging** — verify no critical models were
incorrectly deprecated.
Co-authored-by: FelixMalfait <6399865+FelixMalfait@users.noreply.github.com>
Automated daily sync of `ai-providers.json` from
[models.dev](https://models.dev).
This PR updates pricing, context windows, and model availability based
on the latest data.
New models meeting inclusion criteria (tool calling, pricing data,
context limits) are added automatically.
Deprecated models are detected based on cost-efficiency within the same
model family.
**Please review before merging** — verify no critical models were
incorrectly deprecated.
Co-authored-by: FelixMalfait <6399865+FelixMalfait@users.noreply.github.com>
Automated daily sync of `ai-providers.json` from
[models.dev](https://models.dev).
This PR updates pricing, context windows, and model availability based
on the latest data.
New models meeting inclusion criteria (tool calling, pricing data,
context limits) are added automatically.
Deprecated models are detected based on cost-efficiency within the same
model family.
**Please review before merging** — verify no critical models were
incorrectly deprecated.
Co-authored-by: FelixMalfait <6399865+FelixMalfait@users.noreply.github.com>
## Summary
- `buildBedrockProvider` ignored `authType: "role"` and only honored
static `accessKeyId`/`secretAccessKey` pairs, so deployments relying on
IRSA (e.g. EKS pods with `AWS_WEB_IDENTITY_TOKEN_FILE` + `AWS_ROLE_ARN`)
had no way to authenticate — `@ai-sdk/amazon-bedrock` does not walk the
AWS default credential chain on its own.
- When `authType === 'role'`, wire `fromNodeProviderChain()` from
`@aws-sdk/credential-providers` (already a server dependency, used by
the S3 and logic-function drivers) into `createAmazonBedrock` so IRSA
and other ambient AWS credentials resolve correctly.
- The static-credentials path is unchanged for `authType !== 'role'`.
## Test plan
- [ ] Deploy a server pod with IRSA + an `ai-models-config` entry using
`"authType": "role"` and verify Bedrock calls succeed (no `Could not
load credentials from any providers` error).
- [ ] Verify static-credentials providers (`accessKeyId` +
`secretAccessKey`) still work as before.
- [ ] `npx nx lint:diff-with-main twenty-server` passes.
- [ ] `npx nx typecheck twenty-server` passes.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
---------
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
## Summary
Fixes#19422.
Self-hosted users hitting "No AI models are available" after configuring
API keys via the admin panel were victims of a stale
`AiModelRegistryService` cache. Only the four `addAiProvider` /
`removeAiProvider` / `addModelToProvider` / `removeModelFromProvider`
mutations called `refreshRegistry()` — setting an API key through
`set/update/deleteDatabaseConfigVariable` left the registry pointing at
the pre-mutation provider state.
Rather than patch each mutation site (and re-introduce the same class of
bug on the next one), the registry now invalidates lazily based on the
LLM config-group hash, mirroring the pattern `WebSearchDriverFactory`
already uses via `DriverFactoryBase`. Any mutation to an LLM-tagged
config variable is picked up automatically on the next read — callers
never have to remember to refresh.
- Extracted `getConfigGroupHash` into a shared util reused by both
`DriverFactoryBase` and `AiModelRegistryService` (and switched the hash
to `JSON.stringify` so object-typed config vars like `AI_PROVIDERS`
actually contribute meaningfully).
- `AiModelRegistryService` gates all internal `Map` access behind
private getters that call `ensureFresh()`, so future read paths can't
accidentally observe stale state. Build path uses underscored backing
fields directly to avoid recursing.
- Dropped the now-redundant `refreshRegistry()` public method and its
four call sites in `admin-panel.resolver.ts`.
## Test plan
- [x] `nx typecheck twenty-server` passes
- [x] Existing admin-panel + ai-models specs pass (79 tests)
- [ ] Manual: on a self-hosted instance, set `OPENAI_API_KEY` (or
another provider key) via the admin panel `setDatabaseConfigVariable`
mutation and confirm AI features work without a server restart
- [ ] Manual: existing flows (`addAiProvider`, `addModelToProvider`,
etc.) still pick up new providers on the next read
🤖 Generated with [Claude Code](https://claude.com/claude-code)
---------
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Automated daily sync of `ai-providers.json` from
[models.dev](https://models.dev).
This PR updates pricing, context windows, and model availability based
on the latest data.
New models meeting inclusion criteria (tool calling, pricing data,
context limits) are added automatically.
Deprecated models are detected based on cost-efficiency within the same
model family.
**Please review before merging** — verify no critical models were
incorrectly deprecated.
Co-authored-by: FelixMalfait <6399865+FelixMalfait@users.noreply.github.com>
Automated daily sync of `ai-providers.json` from
[models.dev](https://models.dev).
This PR updates pricing, context windows, and model availability based
on the latest data.
New models meeting inclusion criteria (tool calling, pricing data,
context limits) are added automatically.
Deprecated models are detected based on cost-efficiency within the same
model family.
**Please review before merging** — verify no critical models were
incorrectly deprecated.
Co-authored-by: FelixMalfait <6399865+FelixMalfait@users.noreply.github.com>
Automated daily sync of `ai-providers.json` from
[models.dev](https://models.dev).
This PR updates pricing, context windows, and model availability based
on the latest data.
New models meeting inclusion criteria (tool calling, pricing data,
context limits) are added automatically.
Deprecated models are detected based on cost-efficiency within the same
model family.
**Please review before merging** — verify no critical models were
incorrectly deprecated.
Co-authored-by: FelixMalfait <6399865+FelixMalfait@users.noreply.github.com>
Automated daily sync of `ai-providers.json` from
[models.dev](https://models.dev).
This PR updates pricing, context windows, and model availability based
on the latest data.
New models meeting inclusion criteria (tool calling, pricing data,
context limits) are added automatically.
Deprecated models are detected based on cost-efficiency within the same
model family.
**Please review before merging** — verify no critical models were
incorrectly deprecated.
Co-authored-by: FelixMalfait <6399865+FelixMalfait@users.noreply.github.com>
Automated daily sync of `ai-providers.json` from
[models.dev](https://models.dev).
This PR updates pricing, context windows, and model availability based
on the latest data.
New models meeting inclusion criteria (tool calling, pricing data,
context limits) are added automatically.
Deprecated models are detected based on cost-efficiency within the same
model family.
**Please review before merging** — verify no critical models were
incorrectly deprecated.
Co-authored-by: FelixMalfait <6399865+FelixMalfait@users.noreply.github.com>
Automated daily sync of `ai-providers.json` from
[models.dev](https://models.dev).
This PR updates pricing, context windows, and model availability based
on the latest data.
New models meeting inclusion criteria (tool calling, pricing data,
context limits) are added automatically.
Deprecated models are detected based on cost-efficiency within the same
model family.
**Please review before merging** — verify no critical models were
incorrectly deprecated.
Co-authored-by: FelixMalfait <6399865+FelixMalfait@users.noreply.github.com>
## Summary
- Adds an `authType` field (`'api_key' | 'access_key' | 'iam_role'`) to
AI provider config
- Providers like Amazon Bedrock that authenticate via IAM role (instance
profile) can now be registered without explicit API keys or access keys
- Backend: `isProviderConfigured()` checks `apiKey || accessKeyId ||
authType`
- Frontend admin panel: shows green "Configured" badge and "IAM role"
description for providers with `authType: "iam_role"`
- Provider detail page shows "IAM role (instance profile)" in the
credentials row
## Companion PR
- twentyhq/twenty-infra#528 — patches `authType: "iam_role"` into
dev/staging Bedrock catalogs
## Changed files
- **Backend**: new `AiProviderAuthType` type, `isProviderConfigured`
util, updated registry + resolver
- **Frontend**: new `AiProviderAuthType` type, updated provider list
card + detail page
## Test plan
- [ ] Deploy with a Bedrock catalog that includes `"authType":
"iam_role"` — verify Bedrock shows "Configured" in admin AI panel
- [ ] Verify OpenAI/Anthropic with `apiKey` still show "Configured"
- [ ] Verify a provider with no credentials and no `authType` still
shows "No credentials"
Made with [Cursor](https://cursor.com)
Automated daily sync of `ai-providers.json` from
[models.dev](https://models.dev).
This PR updates pricing, context windows, and model availability based
on the latest data.
New models meeting inclusion criteria (tool calling, pricing data,
context limits) are added automatically.
Deprecated models are detected based on cost-efficiency within the same
model family.
**Please review before merging** — verify no critical models were
incorrectly deprecated.
Co-authored-by: FelixMalfait <6399865+FelixMalfait@users.noreply.github.com>
## Summary
- File serving endpoints (`GET file/:fileFolder/:id` and `GET
public-assets/...`) were piping S3/local file streams directly to the
response without any HTTP headers, allowing a stored XSS attack via
uploaded HTML files rendered inline on the CRM origin.
- Adds `Content-Type`, `Content-Disposition`, and
`X-Content-Type-Options: nosniff` headers to all file serving responses.
Only known-safe MIME types (images, PDF, plain text, audio, video) are
served inline; everything else (HTML, SVG, XML, etc.) forces
`Content-Disposition: attachment` to trigger download instead of
rendering.
- New `setFileResponseHeaders` utility with an explicit allowlist of
inline-safe MIME types.
## Test plan
- [x] Unit tests pass (9 tests including 2 new ones: header assertions
and attachment-disposition for HTML)
- [x] Lint clean (`lint:diff-with-main`)
- [x] Typecheck clean (`nx typecheck twenty-server`)
- [ ] Manual: upload an HTML file via `uploadWorkflowFile`, access the
returned URL — should download instead of rendering
- [ ] Manual: upload a PNG image, access the URL — should render inline
with correct `Content-Type: image/png`
- [ ] Manual: verify `X-Content-Type-Options: nosniff` header is present
on all file responses
Made with [Cursor](https://cursor.com)
---------
Co-authored-by: claude[bot] <41898282+claude[bot]@users.noreply.github.com>
Co-authored-by: Etienne <etiennejouan@users.noreply.github.com>
Co-authored-by: Etienne <45695613+etiennejouan@users.noreply.github.com>
Automated daily sync of `ai-providers.json` from
[models.dev](https://models.dev).
This PR updates pricing, context windows, and model availability based
on the latest data.
New models meeting inclusion criteria (tool calling, pricing data,
context limits) are added automatically.
Deprecated models are detected based on cost-efficiency within the same
model family.
**Please review before merging** — verify no critical models were
incorrectly deprecated.
Co-authored-by: FelixMalfait <6399865+FelixMalfait@users.noreply.github.com>
## Summary
- Replaces per-provider TypeScript constant files
(`openai-models.const.ts`, `anthropic-models.const.ts`, etc.) with a
single `ai-providers.json` catalog as the source of truth
- Adds runtime model discovery via AI SDK for self-hosted providers,
with `models.dev` enrichment for pricing/capabilities
- Introduces composite model IDs (`provider/modelId`) for canonical,
conflict-free identification
- Simplifies provider configuration: API keys are injected from
environment variables (e.g., `OPENAI_API_KEY`)
- Adds admin panel UI for provider management (add/remove/test), model
discovery, recommended model configuration, and default fast/smart model
selection per workspace
- Removes deprecated config variables (`AI_DISABLED_MODEL_IDS`,
`AUTO_ENABLE_NEW_AI_MODELS`, etc.)
- Adds database migration for composite model ID format
## Test plan
- [ ] Server typecheck passes
- [ ] Frontend typecheck passes
- [ ] Server unit tests pass
- [ ] Frontend unit tests pass
- [ ] CI pipeline green
- [ ] Admin panel AI tab loads correctly
- [ ] Provider discovery works for configured providers
- [ ] Model recommendation toggles persist
- [ ] Default fast/smart model selection works
Made with [Cursor](https://cursor.com)
## Summary
- Adds `gemini-3.1-flash-lite-preview` to the Google AI models registry
- Ultra-low-cost Gemini model ($0.25/M input, $1.50/M output) — half the
price of Gemini 3 Flash
- 1M context window, 64K max output, supports dynamic thinking
- No service code changes needed — the existing `AiModelRegistryService`
auto-discovers models from constants
## Changes
- `ai-models-types.const.ts`: Added `gemini-3.1-flash-lite-preview` to
the `ModelId` type union
- `google-models.const.ts`: Added model configuration with pricing,
context window, and capabilities
## Test plan
- [ ] `npx nx typecheck twenty-server` passes
- [ ] `npx nx lint twenty-server` passes
- [ ] With `GOOGLE_API_KEY` set, model appears in available models list
- [ ] Existing Gemini models unaffected
---------
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
## Summary
- **Admin-level filtering**: New AI tab in admin panel with server-wide
model availability controls (whitelist/blacklist via
`AI_AUTO_ENABLE_NEW_MODELS`, `AI_DISABLED_MODEL_IDS`,
`AI_ENABLED_MODEL_IDS` config variables). Dedicated
`setAdminAiModelEnabled` mutation replaces frontend config-variable
manipulation. Filter dropdown to show/hide unconfigured and deprecated
models.
- **Workspace-level filtering**: Per-workspace controls with "Use best
models only" mode (curated list backed by `isRecommended` flag), or
custom whitelist/blacklist. Separate Smart/Fast model selectors with
"Best (...)" virtual options.
- **Security enforcement**: Both layers enforced at every backend
execution point — workspace update, agent create/update, chat execution.
Model ID validated against known models before config mutation. All
admin endpoints protected by `AdminPanelGuard`.
## Changes
### Backend (`twenty-server`)
- New config variables for admin-level model filtering
- `AiModelRegistryService`: `getAllModelsWithStatus()`,
`setModelAdminEnabled()` with model ID validation,
`isModelAdminAllowed()`
- `AdminPanelResolver`: `getAdminAiModels` query,
`setAdminAiModelEnabled` mutation
- `WorkspaceEntity`: new fields (`autoEnableNewAiModels`,
`disabledAiModelIds`, `enabledAiModelIds`, `useRecommendedModels`)
- `WorkspaceService`: model validation on `smartModel`/`fastModel`
updates
- `AgentResolver`: model availability checks on create/update
- `isModelAllowedByWorkspace` centralized utility
- `isRecommended` flag on model definitions
- Two TypeORM migrations
### Frontend (`twenty-front`)
- New `SettingsAdminAI` component with search, filter dropdown
(unconfigured/deprecated), and model toggle cards
- AI tab added to admin panel navigation
- `useWorkspaceAiModelAvailability` hook for workspace-level filtering
- `SettingsAIModelsTab` redesigned: merged sections, "Use best models
only" toggle, conditional available models list
- `getModelIcon`/`getModelProviderLabel` shared utilities with GraphQL
enum casing normalization
- Updated generated GraphQL types and mock data
## Test plan
- [ ] Toggle models on/off in admin panel AI tab and verify they
appear/disappear in workspace settings
- [ ] Enable "Use best models only" in workspace settings and verify
only recommended models are selectable
- [ ] Disable recommended mode and verify whitelist/blacklist toggles
work correctly
- [ ] Verify deprecated models hidden by default, shown greyed out when
filter enabled
- [ ] Verify unconfigured models hidden by default, shown disabled when
filter enabled
- [ ] Try setting a disabled model as Smart/Fast model — should be
rejected
- [ ] Try creating an agent with a disabled model — should be rejected
- [ ] Verify admin panel AI tab requires admin access
Made with [Cursor](https://cursor.com)
---------
Co-authored-by: Cursor <cursoragent@cursor.com>
## Summary
- **Model pricing overhaul**: All model constants updated with accurate
pricing in dollars per 1M tokens, including cached input rates, cache
creation rates, and tiered >200k context pricing
- **New providers**: Added Google (Gemini 3.x), Mistral, and AWS Bedrock
as inference providers. Bedrock serves Claude Opus 4.6 and Sonnet 4.6
via AWS, with proper credential handling following the existing S3/SES
pattern
- **InferenceProvider/ModelFamily split**: Refactored `ModelProvider`
into two orthogonal enums — `InferenceProvider` (who serves the model:
auth, SDK, metadata format) and `ModelFamily` (who created it: token
counting semantics). This eliminates growing `||` chains for token
normalization checks like `excludesCachedTokens`
- **Billing improvements**: Reasoning tokens charged at output rate,
cache token discounts applied accurately, real errors thrown to Sentry
on billing failures
## Test plan
- [x] All existing unit tests updated and passing (23 tests across 3
test files)
- [x] Lint passes for both twenty-server and twenty-front
- [ ] CI checks pass
Made with [Cursor](https://cursor.com)
---------
Co-authored-by: Cursor <cursoragent@cursor.com>
⚠️ **AI-generated PR — not ready for review** ⚠️
cc @FelixMalfait
---
## Changes
### System prompt improvements
- Explicit skill-before-tools workflow to prevent the model from calling
tools without loading the matching skill first
- Data efficiency guidance (default small limits, use filters)
- Pluralized `load_skill` → `load_skills` for consistency with
`load_tools`
### Token usage reduction
- Output serialization layer: strips null/undefined/empty values from
tool results
- Lowered default `find_*` limit from 100 → 10, max from 1000 → 100
### System object tool generation
- System objects (calendar events, messages, etc.) now generate AI tools
- Only workflow-related and favorite-related objects are excluded
### Context window display fix
- **Bug**: UI compared cumulative tokens (sum of all turns) against
single-request context window → showed 100% after a few turns
- **Fix**: Track `conversationSize` (last step's `inputTokens`) which
represents the actual conversation history size sent to the model
- New `conversationSize` column on thread entity with migration
### Workspace AI instructions
- Support for custom workspace-level AI instructions
---------
Co-authored-by: claude[bot] <41898282+claude[bot]@users.noreply.github.com>
## Summary
This PR introduces a Skills system for AI agents, inspired by the [Agent
Skills specification](https://agentskills.io/specification).
## Changes
### Backend
- **SkillEntity**: New database entity with migration for storing skills
- **V2 Sync Mechanism**: Implemented FlatSkill, builders, validators,
and action handlers following the v2 flat entity pattern
- **Standard Skills**: Pre-defined skills (workflow-building,
data-manipulation, dashboard-building, metadata-building, research,
code-interpreter, xlsx, pdf, docx, pptx)
- **GraphQL API**: CRUD operations for skills with proper guards and
permissions
- **Workspace Cache**: Integrated skills into the workspace cache system
### Frontend
- **Skills Table**: Searchable table in AI settings showing all skills
- **Skill Form**: Create/edit page with Label (primary), Description,
and Content (markdown editor)
- **API Name**: Following existing patterns, name is derived from label
with advanced settings toggle for custom API names
- **Standard vs Custom**: Standard skills are read-only, custom skills
can be edited/deleted
## Key Design Decisions
- Skills are stored in the database (Salesforce-like approach) rather
than files
- Name is derived from Label by default (isLabelSyncedWithName pattern)
- Skills reference functions/files via @ mentions in markdown content
rather than explicit relations
- Standard skills are synced from code, custom skills are created via UI
## Screenshots
Skills table and form UI follow existing settings patterns.
## Testing
- [x] Lint passes
- [x] Typecheck passes
- [ ] CI tests
## Summary
This PR adds user-friendly error handling for AI chat features,
specifically for **billing credits exhausted** and **API key not
configured** errors.
## Changes
### Backend
- Added `BILLING_CREDITS_EXHAUSTED` exception code with 402 status
- Added `API_KEY_NOT_CONFIGURED` exception code with 503 status
- Added billing check before AI chat streaming in
`agent-chat.controller.ts`
- Added error code to HTTP exception response body for frontend error
type detection
- Created `AgentRestApiExceptionFilter` for agent-specific errors
### Frontend
- Created `AIChatBanner` - reusable banner component for error/warning
messages
- Created `AIChatCreditsExhaustedMessage` - shows upgrade prompts based
on user permissions
- Created `AIChatApiKeyNotConfiguredMessage` - shows configuration
guidance with docs link
- Created `AIChatErrorRenderer` - encapsulates error type switching
logic (fixes nested ternary)
- Created `AIChatStandaloneError` - displays errors when there are no
messages
- Split `aiChatErrorUtils.ts` into separate files (1 export per file):
- `AIChatErrorCode.ts`
- `extractErrorCode.ts`
- `isAIChatErrorOfType.ts`
- `isBillingCreditsExhaustedError.ts`
- `isApiKeyNotConfiguredError.ts`
- Added comprehensive test coverage (27 tests)
### Other
- Updated trial period banner messaging
## Testing
- All lint checks pass
- All 27 new tests pass
- TypeScript typecheck passes
## Summary
- Add latest AI models from OpenAI (GPT-4.1, o3, o4-mini), Anthropic
(Claude 4.5 Opus/Sonnet/Haiku), and xAI (Grok 4.1)
- Mark deprecated models (GPT-4o, GPT-4o-mini, GPT-4-turbo, Claude Opus
4, Claude Sonnet 4) with a `deprecated` flag
- Split AI models into separate files per provider for better
maintainability
- Support comma-separated default model lists for automatic fallback
across providers (works out of the box for self-hosters regardless of
which provider they configure)
- Filter deprecated models from dropdown selection while keeping them
functional for existing agents
## Changes
### New Models Added
| Provider | Models |
|----------|--------|
| OpenAI | gpt-4.1, gpt-4.1-mini, o3, o4-mini |
| Anthropic | claude-opus-4-5, claude-sonnet-4-5, claude-haiku-4-5 |
| xAI | grok-4-1-fast-reasoning |
### Deprecated Models
- gpt-4o, gpt-4o-mini, gpt-4-turbo (OpenAI)
- claude-opus-4-20250514, claude-sonnet-4-20250514 (Anthropic)
### Config Changes
Default model configs now support comma-separated fallback lists:
-
`DEFAULT_AI_SPEED_MODEL_ID=gpt-4.1-mini,claude-haiku-4-5-20251001,grok-3-mini`
-
`DEFAULT_AI_PERFORMANCE_MODEL_ID=gpt-4.1,claude-sonnet-4-5-20250929,grok-4`
## Test plan
- [x] Unit tests pass
- [x] Typecheck passes
- [x] Lint passes
- [ ] Verify deprecated models don't appear in model dropdowns
- [ ] Verify agents with deprecated models still work correctly
- [ ] Verify default model fallback works when only one provider is
configured
## Summary
Consolidates the AI tool provider architecture by creating a single
`ToolProviderService` as the entry point for all tool generation. This
removes multiple intermediate services and simplifies the codebase.
## Changes
### New Architecture
- **`ToolProviderService`**: Single service for all tool generation
with:
- `getTools(spec)` - Get tools by category with permissions
- `getToolByType(type)` - Get specific tool for workflow execution
- **`ToolCategory` enum**: Declarative specification of tool types:
- `DATABASE_CRUD` - Record CRUD operations
- `ACTION` - HTTP requests, email sending, article search
- `WORKFLOW` - Workflow management tools
- `METADATA` - Object/field metadata tools
- `NATIVE_MODEL` - Model-specific tools (e.g., web search)
- **`ToolSpecification` type**: Clean API for requesting tools with
permissions
### Removed
- `AiToolsModule` - No longer needed
- `ToolService` - Logic inlined into ToolProviderService
- `ToolAdapterService` - Logic inlined into ToolProviderService
- `ToolRegistryService` - Logic inlined into ToolProviderService
### Updated
- All consumers (agents, chat, MCP, workflows) now use
`ToolProviderService`
- Test files updated accordingly
## Stats
- **547 insertions, 1146 deletions** (net ~600 lines removed)
- 4 services deleted
- 1 module deleted
## Testing
- [x] Typecheck passes
- [x] Lint passes
## Summary
This PR introduces a comprehensive agent evaluation system and refactors
the AI module structure for better organization.
## Key Changes
### 🎯 Agent Evaluation System
- Added **Agent Turn Evaluation** entities, DTOs, and database schema
- New GraphQL mutations: `evaluateAgentTurn` and `runEvaluationInput`
- Added `evaluationInputs` field to Agent entity for storing test inputs
- New `AgentTurnGraderService` for automatic turn evaluation
- Added evaluation UI with new **Evals** and **Logs** tabs in agent
detail pages
### 🏗️ Entity & Module Refactoring
- Renamed `AgentChatMessage` → `AgentMessage` for clarity
- Consolidated chat entities: `AgentMessage`, `AgentTurn`, and
`AgentChatThread`
- Reorganized AI modules under `ai/` subdirectory structure
- Updated imports across codebase to reflect new module paths
### 🤖 New Agents & Roles
- Added **Dashboard Builder Agent** for dashboard creation and
management
- Added **Dashboard Manager Role** with appropriate permissions
- Updated role permissions to be more granular (users vs agents vs API
keys)
### 🔐 Permission System Updates
- Added `HTTP_REQUEST_TOOL` permission flag
- Updated Workflow Manager role permissions (restricted tool access)
- Enhanced permission flag types to differentiate between user/agent/API
key contexts
- Added `isRelevantForAgents`, `isRelevantForApiKeys`,
`isRelevantForUsers` to permission flags
### 📨 Message Role Enhancement
- Added `system` role to `AgentMessageRole` enum (alongside
user/assistant)
- Updated message handling to support system prompts
### 🎨 UI/UX Improvements
- New tabs in agent detail: **Evals** and **Logs**
- Added turn detail page: `/ai/agents/:agentId/turns/:turnId`
- Fixed text overflow in `SettingsListItemCardContent`
- Updated role applicability labels ("Assignable to Workspace Members")
### 🛠️ Technical Improvements
- Fixed Zod schema validation for UUID and Date fields (use string
validators)
- Updated `ToolRegistryService` to properly register HTTP tool with
permission flag
- Enhanced error handling in agent execution services
- Updated database migrations for new entity schema
## Database Migrations
- `1764210000000-add-system-role-to-agent-message.ts`
- `1764220000000-add-evaluation-inputs-to-agent.ts`
- `1764200000000-add-agent-turn-evaluation.ts`
- `1764100000000-refactor-agent-chat-entities.ts`
## Testing
- [ ] Agent evaluation flow tested
- [ ] Dashboard Builder agent tested
- [ ] Permission system validated
- [ ] UI tabs and navigation tested
- [ ] Database migrations run successfully
## Breaking Changes
⚠️ **Entity Rename**: `AgentChatMessage` renamed to `AgentMessage` -
GraphQL queries need updating
## Related Issues
<!-- Link any related issues here -->
## Screenshots
<!-- Add screenshots if applicable -->
Félix Malfait