Commit Graph

32 Commits

Author SHA1 Message Date
Félix Malfait bb3d556799 Refined demo workspace creation skill (rebased, review fixes) (#19274)
## Summary

Rebased version of #19051 with all review comments addressed. Clean
branch on latest main, lint/typecheck/tests passing.

### Changes from original PR
- AI can now create, update, and delete **view filters**
(`ViewFilterToolsFactory`) and **view sorts** (`ViewSortToolsFactory`)
- `create_view` now accepts `calendarFieldName`, `calendarLayout`, and
`fieldNames` to configure views at creation time
- Three new standard skills: `view-building`, `view-filters-and-sorts`,
`custom-objects-cleanup`
- `workspace-demo-seeding` skill reworked to keep standard objects and
enrich them with custom fields
- Cache invalidation for nav menu items when object `isActive` changes
- Dashboard tool descriptions improved (RECORD_TABLE widget workflow)

### Review comments addressed (all 10 from #19051)
1. **Sentry + Cubic**: Calendar field DATE/DATE_TIME validation — added
`resolveCalendarFieldMetadataId` using `isFieldMetadataDateKind`
2. **Cubic**: "navigate tool" → "navigate_app tool" in skill metadata
(all 7 occurrences)
3. **Copilot**: KANBAN views now require `mainGroupByFieldName` — throws
clear error if missing
4. **Copilot**: CALENDAR views now require both `calendarFieldName` and
`calendarLayout` — validated before DB call
5. **Copilot**: Mock field fixtures include `type` property (DATE_TIME,
TEXT, SELECT)
6. **Copilot**: `ViewFilterValue` type assertion instead of unsafe `as
string` casts (3 locations)
7. **FelixMalfait**: Removed
`NavigationMenuItemObjectDeactivationListener` — replaced with cache
invalidation
8. **FelixMalfait**: Consolidated `ViewFilterToolProvider` and
`ViewSortToolProvider` into single `ViewToolProvider`
9. Removed `VIEW_FILTER` and `VIEW_SORT` from `ToolCategory` enum
(merged into `VIEW`)
10. Removed stale `existingFeatureFlagsMap` param incompatible with
current main

## Test plan
- [x] `npx nx lint:diff-with-main twenty-server` — passes
- [x] `npx nx typecheck twenty-server` — passes
- [x] `view-tools.factory.spec.ts` — all 20 tests pass (including 3 new
validation tests)

Supersedes #19051

https://claude.ai/code/session_01QPV74NU6vzmJb32e4i899E

---------

Co-authored-by: Claude <noreply@anthropic.com>
2026-04-03 09:19:46 +02:00
nitin 3c067f072c [AI] Improve tools tab (#19221) 2026-04-02 13:36:33 +00:00
nitin 223943550c [AI] Unify code-interpreter streaming rendering and fix assistant width jitter (#19235)
closes
https://discord.com/channels/1130383047699738754/1480991390782455838

- Use data-code-execution as the streaming source of truth and hide
duplicate code-interpreter tool parts (including tool-execute_tool
wrappers).
- Ensure wrapped execute_tool code-interpreter outputs still render
correctly after refetch.
- Gate code-interpreter server behavior by enablement state and keep
assistant messages full-width to avoid streaming vs completed width
shifts.

Co-authored-by: Félix Malfait <felix.malfait@gmail.com>
2026-04-02 10:58:14 +00:00
Charles Bochet c407341912 feat: optimize hot database queries with multi-layer caching (#19068)
## Summary

Introduces multi-layer caching for the 5 most frequent database queries
identified in production (Sentry data), targeting the JWT authentication
hot path and cron job logic.

### Problem
Our database is under heavy load from uncached queries on the auth hot
path:
- `WorkspaceEntity` lookups: **638 queries/min**
- `ApiKeyEntity` lookups: **491 queries/min**
- `UserEntity` lookups: **147 queries/min**
- `UserWorkspaceEntity` lookups: **143 queries/min**
- `LogicFunctionEntity` lookups: **1800 queries/min** (cron job)

### Solution

**1. New `CoreEntityCacheService`** for non-workspace-scoped entities
(Workspace, User, UserWorkspace):
- Mirrors `WorkspaceCacheService` architecture (in-process Map + Redis
with hash validation)
- Provider pattern with `@CoreEntityCache` decorator
- Keyed by entity primary key (not workspaceId)
- 100ms local TTL, Redis-backed hash validation for cross-instance
consistency
- Three providers: `WorkspaceEntityCacheProviderService`,
`UserEntityCacheProviderService`,
`UserWorkspaceEntityCacheProviderService`

**2. New `apiKeyMap` WorkspaceCache** for workspace-scoped API key
lookups:
- `WorkspaceApiKeyMapCacheService` loads all API keys for a workspace
into a map by ID
- Leverages existing `WorkspaceCacheService` infrastructure
- Cache invalidation on API key create/update/revoke

**3. `CronTriggerCronJob` refactored** to use existing
`flatLogicFunctionMaps` workspace cache:
- Eliminates per-workspace `LogicFunctionEntity` repository queries
(~1800/min)
- Filters cached data in-memory instead

**4. `JwtAuthStrategy` refactored** to use caches for all entity
lookups:
- Workspace, User, UserWorkspace → `CoreEntityCacheService`
- ApiKey → `WorkspaceCacheService` (`apiKeyMap`)
- Impersonation queries kept as direct DB queries (rare path, requires
relations)

**5. Cache invalidation** wired into mutation paths:
- `WorkspaceService` → invalidates `workspaceEntity` on
save/update/delete
- `ApiKeyService` → invalidates `apiKeyMap` on create/update/revoke

### Architecture

```
Request → JwtAuthStrategy
  ├── Workspace lookup → CoreEntityCacheService (in-process → Redis → DB)
  ├── User lookup → CoreEntityCacheService (in-process → Redis → DB)
  ├── UserWorkspace lookup → CoreEntityCacheService (in-process → Redis → DB)
  └── ApiKey lookup → WorkspaceCacheService (in-process → Redis → DB)

CronTriggerCronJob
  └── LogicFunction lookup → WorkspaceCacheService (flatLogicFunctionMaps)
```

### Expected Impact
| Query | Before | After |
|-------|--------|-------|
| WorkspaceEntity | 638/min | ~0 (cached) |
| ApiKeyEntity | 491/min | ~0 (cached) |
| UserEntity | 147/min | ~0 (cached) |
| UserWorkspaceEntity | 143/min | ~0 (cached) |
| LogicFunctionEntity | 1800/min | ~0 (cached) |

### Not included (ongoing separately)
- DataSourceEntity query optimization (IS_DATASOURCE_MIGRATED migration)
- ObjectMetadataEntity query optimization (already partially cached)
2026-03-28 22:53:34 +01:00
Lucas Bordeau 2a82df7073 AI tools to create a demo workspace (#18236)
This PR adds the necessary tool to create a demo workspace with :
relevant custom objects and fields, mock data and a real dashboard with
graph widgets.

It is still a bit under-optimized and slow but it works.

This PR also adds an AI tool that allows to see what happens in real
time, it navigates the app and waits when necessary.

---------

Co-authored-by: Félix Malfait <felix.malfait@gmail.com>
2026-03-05 11:39:31 +01:00
Abdullah. 9107f5bbc7 feat: upgrade ai package to version six and the corresponding @ai-sdk/* packages to compatible versions (#18172)
Used the migration guide to carry out this upgrade:
https://ai-sdk.dev/docs/migration-guides/migration-guide-6-0

I have not been able to test locally due to credits.

<img width="220" height="450" alt="image"
src="https://github.com/user-attachments/assets/050b34b9-3239-4010-8c47-b43d44571994"
/>

---------

Co-authored-by: Félix Malfait <felix.malfait@gmail.com>
2026-02-25 16:49:26 +01:00
Félix Malfait 66da296799 Unify MCP into single endpoint with lazy tool discovery (#18113)
## Summary

- Consolidates the MCP server from two endpoints (`/mcp` +
`/mcp/metadata`) into a **single `POST /mcp`** endpoint exposing five
high-level tools: `get_tool_catalog`, `learn_tools`, `execute_tool`,
`load_skills`, and `search_help_center`
- Fixes **DATABASE_CRUD tools not accessible via API key auth** by
removing an unnecessary `userId`/`userWorkspaceId` guard in
`DatabaseToolProvider` and threading `ApiKeyWorkspaceAuthContext`
through the tool context chain
- Improves tool descriptions with **STEP 1/2/3 workflow guidance** so AI
clients follow the correct discovery flow (catalog → learn → execute)
instead of guessing tool names
- Simplifies the **frontend AI settings** by removing the schema picker
dropdown (no more "Core Schema" vs "Metadata Schema" choice)

## Changes

### Backend
- **Deleted**: `mcp-metadata.controller.ts`, `mcp-metadata.service.ts`
(merged into core)
- **New**: `get-tool-catalog.tool.ts` — browsable, categorized tool
discovery
- **Fixed**: `DatabaseToolProvider.generateDescriptors` — removed guard
that blocked API key access to CRUD tools
- **Fixed**: `ToolContext` type + `ToolRegistryService` — `authContext`
now flows through so API key CRUD execution works end-to-end
- **Updated**: `McpProtocolService` — builds
`ApiKeyWorkspaceAuthContext` for API key requests
- **Updated**: All MCP tool descriptions with explicit step numbering

### Frontend
- **Simplified**: `SettingsAIMCP.tsx` — removed schema selector
dropdown, shows single MCP config

## Test plan
- [x] All 19 MCP unit tests pass (controller + protocol service)
- [x] Server and frontend lint clean
- [x] Server and frontend typecheck pass
- [x] Live-tested on localhost:3000 with API key: `get_tool_catalog`
returns 236 tools including 196 DATABASE_CRUD tools
- [x] Live-tested `execute_tool` with `find_companies` via API key —
returns real data
- [x] Tested MCP connection from Cursor IDE via project-level
`.cursor/mcp.json`

Made with [Cursor](https://cursor.com)

---------

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-02-20 13:09:54 +01:00
martmull da064d5e88 Support define is tool logic function (#17926)
- supports isTool and timeout settings in defineLogicFunction in apps
and in setting tabs definition
- compute for all toolInputSchema for logic funciton, in settings and in
code steps

<img width="991" height="802" alt="image"
src="https://github.com/user-attachments/assets/05dc1221-cac9-45a3-87b0-3b13161446fd"
/>
2026-02-16 10:43:29 +01:00
neo773 84afbb4d2c feat: add draft email workflow action (#17793)
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Co-authored-by: cubic-dev-ai[bot] <191113872+cubic-dev-ai[bot]@users.noreply.github.com>
2026-02-13 18:20:20 +01:00
Félix Malfait 21c51ec251 Improve AI agent chat, tool display, and workflow agent management (#17876)
## Summary

- **Fix token renewal endpoint**: Use `/metadata` instead of `/graphql`
for token renewal in agent chat, fixing auth issues
- **Improve tool display**: Add `load_skills` support, show formatted
tool names (underscores → spaces) with finish/loading states, display
tool icons during loading, and support custom loading messages from tool
input
- **Refactor workflow agent management**: Replace direct
`AgentRepository` access with `AgentService` for create/delete/find
operations in workflow steps, improving encapsulation and consistency
- **Simplify Apollo client usage**: Remove explicit Apollo client
override in `useGetToolIndex`, add `AgentChatProvider` to
`AppRouterProviders`
- **Fix load-skill tool**: Change parameter type from `string` to `json`
for proper schema parsing
- **Update agent-chat-streaming**: Use `AgentService` for agent
resolution and tool registration instead of direct repository queries

## Test plan

- [ ] Verify AI agent chat works end-to-end (send message, receive
response)
- [ ] Verify tool steps display correctly with icons and proper messages
during loading and after completion
- [ ] Verify workflow AI agent step creation and deletion works
correctly
- [ ] Verify workflow version cloning preserves agent configuration
- [ ] Verify token renewal works when tokens expire during agent chat


Made with [Cursor](https://cursor.com)

---------

Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: cubic-dev-ai[bot] <191113872+cubic-dev-ai[bot]@users.noreply.github.com>
2026-02-13 09:27:38 +00:00
Charles Bochet 9e21e55db4 Prevent leak between /metadata and /graphql GQL schemas (#17845)
## Fix resolver schema leaking between `/metadata` and `/graphql`
endpoints

### Summary
- Patch `@nestjs/graphql` to support a `resolverSchemaScope` option that
filters resolvers at both schema generation and runtime, preventing
cross-endpoint leaking
- Introduce `@CoreResolver()` and `@MetadataResolver()` decorators to
explicitly scope each resolver to its endpoint
- Move most resolvers (auth, billing, workspace, user, etc.) to the
metadata schema where the frontend expects them; only workflow and
timeline calendar/messaging resolvers remain on `/graphql`
- Fix frontend `SSEQuerySubscribeEffect` to use the default (metadata)
Apollo client instead of the core client

### Problem
NestJS GraphQL's module-based resolver discovery traverses transitive
imports, causing resolvers from `/metadata` modules to leak into the
`/graphql` schema and vice versa. This made the schemas unpredictable
and tightly coupled to module import order.

### Approach
- Added `resolverSchemaScope` to `GqlModuleOptions` via a patch on
`@nestjs/graphql`, filtering in both `filterResolvers()` (runtime
binding) and `getAllCtors()` (schema generation)
- Each resolver is explicitly decorated with `@CoreResolver()` or
`@MetadataResolver()`
- Organized decorator, constant, and type files under `graphql-config/`
following project conventions


Core GQL Schema: (see: no more fields!)
<img width="827" height="894" alt="image"
src="https://github.com/user-attachments/assets/668f3f0f-485e-43f0-92be-4345aeccacb6"
/>

Metadata GQL Schema (see no more getTimelineCalendarEventsFromCompany)
<img width="827" height="894" alt="image"
src="https://github.com/user-attachments/assets/443913db-e5fe-4161-b0e7-4a971cc80a71"
/>
2026-02-11 10:05:24 +00:00
Félix Malfait 3216b634a3 feat: improve AI chat - system prompt, tool output, context window display (#17769)
⚠️ **AI-generated PR — not ready for review** ⚠️

cc @FelixMalfait

---

## Changes

### System prompt improvements
- Explicit skill-before-tools workflow to prevent the model from calling
tools without loading the matching skill first
- Data efficiency guidance (default small limits, use filters)
- Pluralized `load_skill` → `load_skills` for consistency with
`load_tools`

### Token usage reduction
- Output serialization layer: strips null/undefined/empty values from
tool results
- Lowered default `find_*` limit from 100 → 10, max from 1000 → 100

### System object tool generation
- System objects (calendar events, messages, etc.) now generate AI tools
- Only workflow-related and favorite-related objects are excluded

### Context window display fix
- **Bug**: UI compared cumulative tokens (sum of all turns) against
single-request context window → showed 100% after a few turns
- **Fix**: Track `conversationSize` (last step's `inputTokens`) which
represents the actual conversation history size sent to the model
- New `conversationSize` column on thread entity with migration

### Workspace AI instructions
- Support for custom workspace-level AI instructions

---------

Co-authored-by: claude[bot] <41898282+claude[bot]@users.noreply.github.com>
2026-02-09 14:26:02 +01:00
martmull 9162685b2e Reorganize logic function files (#17766)
reorganize according to

<img width="1243" height="725" alt="Pasted Graphic"
src="https://github.com/user-attachments/assets/ba65dd10-8eec-4b13-ad49-9726edd3b79c"
/>

Not working yet
2026-02-09 12:36:39 +01:00
Paul Rastoin 476bdf764c Refactor flat entity maps to be universal oriented (#17665)
# Introduction
In preparation of the workspace agnostic builder, we're migrating
`FlatEntityMaps` to be universal identifier oriented and based
As in the builder context there're won't be any ids at all
Please also note that the FlatEntity is a UniversalFlatEntity superset

From
```ts
import { type SyncableFlatEntity } from 'src/engine/metadata-modules/flat-entity/types/flat-entity-from.type';

export type FlatEntityMaps<T extends SyncableFlatEntity> = {
  byId: Partial<Record<string, T>>;
  idByUniversalIdentifier: Partial<Record<string, string>>;
  universalIdentifiersByApplicationId: Partial<Record<string, string[]>>;
};
```

To
```ts
export type FlatEntityMaps<
  T extends SyncableFlatEntity | UniversalSyncableFlatEntity,
> = {
  byUniversalIdentifier: Partial<Record<string, T>>;
  universalIdentifierById: Partial<Record<string, string>>;
  universalIdentifiersByApplicationId: Partial<Record<string, string[]>>; // this might make more sense to be migrated to universalIdentifiersByApplicationUniversalIdentifier but it's the main topic of this PR
};
```

## Low level maps tools
Had to refactor find | create | delete | replace | find-many | get-sub
tools ( through mutations and or throw equivalent )
2026-02-03 16:16:02 +00:00
Charles Bochet 46d28509b9 Keep simplifying logic functions (#17595)
## Summary

Refactors the `LogicFunctionService` API by consolidating v1 and v2
services:

In metadata-module (presentation layer module)
- **Renamed methods**: `deleteOneLogicFunction` → `destroyOne`,
`updateOneLogicFunction` → `updateOne`, `createOneLogicFunction` →
`createOne`
- **Added duplicate methods**: `duplicateLogicFunction`,
`createLogicFunctionFromExistingLogicFunctionById`
- **Removed soft delete/restore** functionality - only hard delete
(`destroyOne`) is supported

In core-module (lower level module)
- **Moved execution methods** to `LogicFunctionExecutorService` which is
lower level: `executeOneLogicFunction`, `getAvailablePackages`,
`getLogicFunctionSourceCode`
2026-01-30 20:30:52 +01:00
Charles Bochet 6eebf6f23a Remove versions from logicFunction (#17540)
## Summary

- Remove `latestVersion` and `publishedVersions` columns from
`LogicFunction` entity
- Remove version parameter from logic function execution, build, and
source code retrieval
- Update all related services, DTOs, utilities, and tests
- Add database migration to drop the version columns
2026-01-30 14:30:49 +01:00
Weiko fc908e9d87 Refactor WorkspaceAuthContext to use discriminated union types (#17491)
## Context
The previous WorkspaceAuthContext was a single interface with many
optional fields, making it unclear which fields are available in
different authentication scenarios. This made the code harder to reason
about and required runtime checks scattered throughout the codebase.

## Changes
- Introduced a discriminated union type for WorkspaceAuthContext with
four specific variants:
-> UserWorkspaceAuthContext - for authenticated users
-> ApiKeyWorkspaceAuthContext - for API key authentication
-> ApplicationWorkspaceAuthContext - for application-based auth
-> SystemWorkspaceAuthContext - for system/internal operations
- Added type guard functions (isUserAuthContext, isApiKeyAuthContext,
etc.) for safe type narrowing
- Added builder utilities (buildUserAuthContext, buildApiKeyAuthContext,
etc.) to construct each context variant with proper type safety
- Refactored WorkspaceAuthContextMiddleware to use the new builders
instead of constructing a loosely-typed object
- Moved the type definition from twenty-orm/interfaces/ to
core-modules/auth/types/ for better organization
- Updated all consumers across query runners, tool providers, and
modules to use the new type location


## Notes
- I had to query User and WorkspaceMember in some parts of tool module
that were expecting userWorkspaceId but not the rest of
UserWorkspaceAuthContext (that should be required with the new proper
type otherwise it would break a lot of logic and mostly permissions with
the newly added RLS -> This is what we expect from
UserWorkspaceAuthContext and how it's done in the "normal" path in HTTP
middleware)
- WorkspaceMember is in the cache already but ideally we should move
User (And Workspace?) in the cache as well to avoid querying the DB
after each request (this is also valid for HTTP middleware when we
hydrate the Request object btw)
2026-01-28 17:50:42 +00:00
Charles Bochet da6f1bbef3 Rename serverlessFunction to logicFunction (#17494)
## Summary

Rename "Serverless Function" to "Logic Function" across the codebase for
clearer naming.

### Environment Variable Changes

| Old | New |
|-----|-----|
| `SERVERLESS_TYPE` | `LOGIC_FUNCTION_TYPE` |
| `SERVERLESS_LAMBDA_REGION` | `LOGIC_FUNCTION_LAMBDA_REGION` |
| `SERVERLESS_LAMBDA_ROLE` | `LOGIC_FUNCTION_LAMBDA_ROLE` |
| `SERVERLESS_LAMBDA_SUBHOSTING_URL` |
`LOGIC_FUNCTION_LAMBDA_SUBHOSTING_URL` |
| `SERVERLESS_LAMBDA_ACCESS_KEY_ID` |
`LOGIC_FUNCTION_LAMBDA_ACCESS_KEY_ID` |
| `SERVERLESS_LAMBDA_SECRET_ACCESS_KEY` |
`LOGIC_FUNCTION_LAMBDA_SECRET_ACCESS_KEY` |

### Breaking Changes

- Environment variables must be updated in production deployments
- Database migration renames `serverlessFunction` → `logicFunction`
tables
2026-01-28 01:42:19 +01:00
Paul Rastoin 942d2fef83 Remove sync-metadata and IS_WORKSPACE_CREATION_V2_ENABLED feature flag (#16997)
# Introduction
Followup of
https://github.com/twentyhq/twenty/pull/17001#pullrequestreview-3638508738
close https://github.com/twentyhq/core-team-issues/issues/1910

We've completely decom the `sync-metadata` in production. We're now then
removing its implementation in favor of the v2.

## TODO:
- [x] Remove sync-metadata implem and commands
- [x] Remove workspace decorators 
- [x] Type each deprecated field to deprecated on their workspaceEntity
- [x] Remove the `workspace-sync-metadata` folder entirely
- [x] remove workspace migration
- [x] workspace migration removal migration
- [x] remove the `v2` references from workspace manager file names
- [x] remove the `v2` references from workspace manager modules
- [ ] Double check impact on translation file path updates


## Note
- Removed the gate logic
- Remains some service v2 naming, serverless needs to be migrated on v2
fully
- Removed workspaceMigration service app health consumption, making it
always returning up ( no more down ) cc @FelixMalfait ( quite obsolete
health check now, will require complete refactor once we introduce inter
app dependency etc )
2026-01-08 15:45:12 +00:00
Félix Malfait 0173e40a20 feat: Serverless Functions as AI Tools (#16919)
## Summary

This PR enables serverless functions to be exposed as AI tools, allowing
them to be used by AI agents.

### Changes

- Added new `SERVERLESS_FUNCTION` tool category
- Added `toolDescription`, `toolInputSchema`, and `toolOutputSchema`
fields to serverless functions
- Created database migration for the new schema columns
- Added tool index query and resolver for fetching available tools
- Added Settings AI page tabs (Skills, Tools, Settings) with new tools
table
- Added utility to convert tool schema to JSON schema format
- Updated frontend to display tools in the settings page

### Implementation Details

- Serverless functions can now define tool metadata (description,
input/output schemas)
- These functions are automatically registered in the tool registry
- The tool index endpoint allows querying available tools with their
schemas
- Settings page now has a dedicated Tools tab showing all available
tools
2026-01-05 11:13:06 +01:00
Félix Malfait 21ff42074d feat: implement skills system for AI agents (#16865)
## Summary
This PR introduces a Skills system for AI agents, inspired by the [Agent
Skills specification](https://agentskills.io/specification).

## Changes

### Backend
- **SkillEntity**: New database entity with migration for storing skills
- **V2 Sync Mechanism**: Implemented FlatSkill, builders, validators,
and action handlers following the v2 flat entity pattern
- **Standard Skills**: Pre-defined skills (workflow-building,
data-manipulation, dashboard-building, metadata-building, research,
code-interpreter, xlsx, pdf, docx, pptx)
- **GraphQL API**: CRUD operations for skills with proper guards and
permissions
- **Workspace Cache**: Integrated skills into the workspace cache system

### Frontend  
- **Skills Table**: Searchable table in AI settings showing all skills
- **Skill Form**: Create/edit page with Label (primary), Description,
and Content (markdown editor)
- **API Name**: Following existing patterns, name is derived from label
with advanced settings toggle for custom API names
- **Standard vs Custom**: Standard skills are read-only, custom skills
can be edited/deleted

## Key Design Decisions
- Skills are stored in the database (Salesforce-like approach) rather
than files
- Name is derived from Label by default (isLabelSyncedWithName pattern)
- Skills reference functions/files via @ mentions in markdown content
rather than explicit relations
- Standard skills are synced from code, custom skills are created via UI

## Screenshots
Skills table and form UI follow existing settings patterns.

## Testing
- [x] Lint passes
- [x] Typecheck passes
- [ ] CI tests
2026-01-02 14:22:01 +00:00
Félix Malfait f8fa709abf refactor: Migrate CRUD services to use Common API (#16869)
This PR migrates the workflow CRUD services to use the Common API
(CommonQueryRunners) instead of directly accessing TwentyORM.

## Changes

- Created CommonApiContextBuilderService to build context for Common API
- Migrated CreateRecordService to use CommonCreateOneQueryRunnerService
- Migrated UpdateRecordService to use CommonUpdateOneQueryRunnerService
- Migrated DeleteRecordService to use CommonDeleteOneQueryRunnerService
- Migrated FindRecordsService to use CommonFindManyQueryRunnerService
- Migrated UpsertRecordService to use Common API with upsert flag
- Removed unused get-selected-columns-from-restricted-fields.util.ts
- Updated module dependencies

## Benefits

- Consistent permission checking via Common API
- Query hooks (before/after execution)
- Automatic input transformation
- Same behavior as REST/GraphQL APIs
- Reduced code duplication
2025-12-31 10:24:50 +01:00
Félix Malfait 009e7e05f2 feat(workflow): use authContext in CRUD services for Common API migration (#16857)
## Summary

This PR migrates workflow CRUD operations to properly use the Common API
layer's authentication context, addressing the issues from the reverted
PR #15875.

The original PR was reverted because the Common API required passing
either a User or an API Key for authentication, which was problematic
for workflows. Since then, the "Application" concept was introduced in
the Common API layer, allowing for token injection in serverless
functions.

This PR leverages the "Twenty Standard Application" concept for
non-manual workflow triggers, providing a clean authentication path
without the issues of user impersonation.

## Changes

### Core Infrastructure
- **RecordCrudExecutionContext**: Replace `workspaceId` with full
`authContext`
- **WorkflowExecutionContext**: Add `authContext` field to carry
authentication info
- **ToolGeneratorContext/ToolSpecification**: Add optional `authContext`
support for tool generation

### Authentication Flow
- **WorkflowExecutionContextService**: Build appropriate auth context
based on trigger type:
- **Manual triggers**: Use user's workspace auth context with their role
permissions
- **Non-manual triggers**: Use Twenty Standard Application auth context
(bypasses permission checks or uses default serverless function role)
- **ApplicationService**: Add `findTwentyStandardApplicationOrThrow`
method to retrieve the system application
- **UserWorkspaceService**: Make relations configurable in
`getUserWorkspaceForUserOrThrow` to load only what's needed

### CRUD Services Migration
All 5 record CRUD services now receive `authContext` instead of
`workspaceId`:
- `CreateRecordService`
- `UpdateRecordService`
- `DeleteRecordService`
- `FindRecordsService`
- `UpsertRecordService`

### Workflow Actions
All record CRUD workflow actions pass `executionContext.authContext` to
the services:
- `CreateRecordWorkflowAction`
- `UpdateRecordWorkflowAction`
- `DeleteRecordWorkflowAction`
- `FindRecordsWorkflowAction`
- `UpsertRecordWorkflowAction`

### AI Agent Integration
- AI agent workflow action passes auth context to agent executor
- Tool provider and MCP protocol service support auth context
propagation

## Benefits
-  Proper authentication for workflow CRUD operations via Common API
-  Non-manual triggers use system application context (no user
impersonation issues)
-  Manual triggers preserve user permissions correctly
-  Foundation for better permission handling in automated workflows
-  Cleaner separation between user-initiated and system-initiated
operations

## Related
- Reverted PR: #15875
2025-12-30 21:36:15 +01:00
Félix Malfait 2e104c8e76 feat(ai): add code interpreter for AI data analysis (#16559)
## Summary

- Add code interpreter tool that enables AI to execute Python code for
data analysis, CSV processing, and chart generation
- Support for both local (development) and E2B (sandboxed production)
execution drivers
- Real-time streaming of stdout/stderr and generated files
- Frontend components for displaying code execution results with
expandable sections

## Code Quality Improvements

- Extract `getMimeType` to shared utility to reduce code duplication
between drivers
- Fix security issue: escape single quotes/backslashes in E2B driver env
variable injection
- Add `buildExecutionState` helper to reduce duplicated state object
construction
- Add `DEFAULT_CODE_INTERPRETER_TIMEOUT_MS` constant for consistency
- Fix lingui linting warning and TypeScript theme errors in frontend

## Test Plan

- [ ] Test code interpreter with local driver in development
- [ ] Test code interpreter with E2B driver in production environment
- [ ] Verify streaming output displays correctly in chat UI
- [ ] Verify generated files (charts, CSVs) are uploaded and
downloadable
- [ ] Test file upload flow (CSV, Excel) triggers code interpreter

<!-- CURSOR_SUMMARY -->
---

> [!NOTE]
> Updates generated i18n catalogs for Polish and pseudo-English, adding
strings for code execution/output (code interpreter) and various UI
messages, with minor text adjustments.
> 
> - **Localization**:
> - **Generated catalogs**: Refresh `locales/generated/pl-PL.ts` and
`locales/generated/pseudo-en.ts`.
> - Add strings for code execution/output (e.g., code, copy code/output,
running/waiting states, download files, generated files, Python code
execution).
> - Include new UI texts (errors, prompts, menus) and minor text
corrections.
>   - No changes to `pt-BR`; other files unchanged functionally.
> 
> <sup>Written by [Cursor
Bugbot](https://cursor.com/dashboard?tab=bugbot) for commit
befc13d02c21e5a6647bc1aa6daa2a89f60b7ef8. This will update automatically
on new commits. Configure
[here](https://cursor.com/dashboard?tab=bugbot).</sup>
<!-- /CURSOR_SUMMARY -->
2025-12-15 16:11:24 +01:00
Félix Malfait 5f4f4c0af8 feat(ai): add dashboard tools for AI chat (#16517)
## Summary

- Implements real tools for the dashboard-building skill to create and
manage dashboards through the AI chat interface
- Adds 6 new dashboard tools: `create_complete_dashboard`,
`list_dashboards`, `get_dashboard`, `add_dashboard_widget`,
`update_dashboard_widget`, `delete_dashboard_widget`
- Improves widget configuration robustness with typed Zod schemas and
discriminated unions for graph types

## Key Changes

**New Dashboard Tools:**
- `create_complete_dashboard` - Creates a dashboard with layout, tab,
and widgets in a single call
- `list_dashboards` - Lists all dashboards in the workspace
- `get_dashboard` - Gets full dashboard details including tabs and
widget configurations
- `add_dashboard_widget` - Adds a widget to an existing dashboard tab
- `update_dashboard_widget` - Updates widget properties or configuration
- `delete_dashboard_widget` - Removes a widget from a dashboard

**Widget Configuration Improvements:**
- Typed Zod schemas for each chart type (AGGREGATE, BAR, LINE, PIE)
- Discriminated union validation based on `graphType`
- Widget-level error handling for partial success when creating
dashboards
- Clear documentation about required `objectMetadataId` and field UUIDs

**Skill Documentation Updates:**
- Updated `dashboard-building.skill.ts` with critical guidance about
looking up field metadata first
- Added workflow instructions: use `list_object_metadata_items` before
creating GRAPH widgets
- Practical grid layout recommendations

## Test plan

- [ ] Create a new dashboard via AI chat
- [ ] Verify widgets display data correctly when proper field IDs are
provided
- [ ] Test adding/updating/deleting widgets on existing dashboards
- [ ] Verify error messages are helpful when configuration is incorrect
2025-12-12 07:35:14 +01:00
Félix Malfait 70a78aafe9 feat(ai): replace agent search with skills system (#16513)
## Summary

- Replace the agent search mechanism with a new skills-based system
- Add a `skills` module with predefined skill definitions that the AI
can load on demand
- Remove specialized agents (workflow-builder, data-manipulator,
dashboard-builder, metadata-builder, researcher), keeping only the
helper agent
- Add `recordReferences` to workflow creation tool for chip linking in
the UI

## Changes

### New Skills Module
- `skill-definitions.ts` - Contains 5 skill definitions with detailed
instructions
- `skills.service.ts` - Service to get skills by name
- `load-skill.tool.ts` - Tool for AI to load skills explicitly

### Removed
- `agent-search.tool.ts` - Replaced by skill loading
- Specialized agent definitions (converted to skills)

### Updated
- Chat execution now shows skill catalog in system prompt
- Workflow creation returns `recordReferences` for UI linking

## Test plan
- [ ] Verify AI can load skills using `load_skill` tool
- [ ] Verify skill content is returned correctly
- [ ] Verify workflow creation shows clickable chip in chat
- [ ] Verify helper agent still works
2025-12-12 06:51:25 +01:00
Félix Malfait 3cea19baf4 feat(ai): add view management tools for AI chat (#16495)
## Summary

Adds a new **VIEW** tool category for the AI chat, enabling it to work
with views:

- **get-views**: List views in the workspace, optionally filtered by
object metadata ID
- **get-view-query-parameters**: Convert a view's filters and sorts into
GraphQL query parameters that can be passed to existing `find_*` data
tools
- **create-view**, **update-view**, **delete-view**: CRUD operations for
view management

### Key design decisions

1. **No pagination duplication**: Instead of creating a
`find-records-from-view` tool that would duplicate pagination logic,
`get-view-query-parameters` returns filter/sort parameters that the AI
can pass to existing record-fetching tools.

2. **Permission model**:
- Read tools (get-views, get-view-query-parameters) are available to all
users
   - Write tools require the `VIEW` permission
   - UNLISTED views can only be modified by their creator

3. **Leverages existing utilities**: Uses
`computeRecordGqlOperationFilter` from `twenty-shared` for filter
conversion.

### Files changed

- Added `ViewToolProvider`, `ViewToolsFactory`, and
`ViewQueryParamsService`
- Added `VIEW` to `ToolCategory` enum and tool registry
- Updated `chat-execution.service.ts` to include view tools in the
catalog and pass viewId in browsing context
- Extracted shared `formatValidationErrors` utility to reduce
duplication

## Test plan

- [x] Unit tests for `ViewToolsFactory` 
- [x] Unit tests for `ViewQueryParamsService`
- [x] Lint and typecheck pass
2025-12-11 15:05:42 +01:00
Félix Malfait bc57b8ee4e feat(ai): add browsing context and fix tool loading (#16476)
## Summary

- Add `BrowsingContext` type to automatically pass what the user is
currently viewing (recordPage or listView) to the AI chat
- Simplify context architecture: remove toggleable context UI, make it
automatic and invisible to the user
- Fix tool loading: add `unionOf` handling in
`getDatabaseToolsForObject` and fix regex ordering so `find_one_*` tools
are properly registered
- Use plural names for find tools (`find_people` vs `find_one_person`)
for better semantics
- Clean up unused components and states

## Changes

### Frontend
- New `BrowsingContext` type and `useGetBrowsingContext` hook to gather
context from Recoil state
- Simplified `useAgentChat` to use the new browsing context
- Removed toggleable context UI components
(`AgentChatContextRecordPreview`, `SendMessageWithRecordsContextButton`,
etc.)
- Removed `isAgentChatCurrentContextActiveState`

### Backend
- New `BrowsingContextType` for recordPage and listView contexts
- Updated `ChatExecutionService` to build context from browsing context
- Fixed `tool-registry.service.ts`:
  - Added `unionOf` handling in permission config
- Fixed regex ordering (`find_one` before `find`) so tools load
correctly
- Use plural names for search tools (`find_people` instead of
`find_person`)

## Test plan

- [x] Typecheck passes
- [x] Lint passes
- [ ] Test AI chat on record page - should show context in system prompt
- [ ] Test AI chat on list view - should show view name and filters
- [ ] Test `find_one_*` tools now load correctly
- [ ] Test `find_*` tools use plural naming
2025-12-11 10:19:27 +01:00
Félix Malfait 5df8fd90c3 feat: simplify AI chat architecture and add record links (#16463)
## Summary

This PR significantly simplifies the AI chat architecture by removing
complex routing/planning mechanisms and introduces clickable record
links in AI responses.

## Changes

### AI Chat Architecture Simplification
- **Removed** the entire `ai-chat-router` module (~850 lines) including:
  - Strategy decider service
  - Plan generator service
  - Complex routing logic
- **Removed** agent execution planning services (~700 lines):
  - `agent-execution.service.ts`
  - `agent-plan-executor.service.ts`
  - `agent-tool-generator.service.ts`
- **Added** centralized `ToolRegistryService` for tool management:
  - Builds searchable tool index (database, action, workflow tools)
  - Provides tool lookup by name
  - Supports agent search for loading expertise
- **Added** `ChatExecutionService` as simple replacement:
  - Includes full tool catalog in system prompt
- Pre-loads common tools (find/create/update for company, person,
opportunity, task, note)
  - Uses `load_tools` mechanism for dynamic tool activation
  - Enables native web search by default

### Record References in AI Responses
- Added `recordReferences` field to tool outputs for create, find, and
update operations
- Implemented `[[record:objectName:recordId:displayName]]` syntax for AI
to reference records
- Created `RecordLink` component that renders clickable chips with
object icons
- Integrated record link parsing into the markdown renderer
- Users can now click directly on created/found records in AI responses

### Workflow Agent Fixes
- Fixed cache invalidation issue when creating agents in workflows
- Added default prompt for workflow-created agents to prevent validation
errors
- Relaxed agent validation to only check properties being updated (not
all required properties)

### Code Quality Improvements
- Extracted `getRecordDisplayName` utility that mirrors frontend's
`getLabelIdentifierFieldValue` logic
- Uses object metadata to determine the correct label identifier field
- Handles `FULL_NAME` composite type for person/workspaceMember objects
- Shared across create, find, and update record services

## Net Impact
- **~1,200 lines deleted** (complex routing/planning code)
- **~500 lines added** (simpler tool registry + record links)
- Significantly reduced code complexity
- Better tool discovery through full catalog in system prompt
- Improved UX with clickable record references

## Testing
- Typecheck passes
- Lint passes
- Manual testing of AI chat with record creation and linking
2025-12-10 15:14:12 +01:00
Weiko 859004f4fc Refactor global datasource part 2 (#16399)
## Context
Deprecating TwentyORMManager in favor of TwentyORMGlobalManager
(temporarily, as this will simplify the ultimate goal to later replace
all usages with the new TwentyORMGlobalManagerV2 which will have a
similar signature)
This means this PR had to refactor a bit of code to pass down the
workspaceId when not available directly as it is now a requirement,
meaning we also deprecated scopedWorkspaceContextFactory to have a less
obscure way to fetch the workspaceId and have something more
declarative.

Step 3 will be to update TwentyORMGlobalManager to use a featureFlag
toggling and use the new GlobalWorkspaceOrmManager internally using the
new cache service

Step 4 will be to remove the feature flag and pg_pool patch
2025-12-08 17:05:00 +01:00
martmull 7f1e69740a 1895 extensibility v1 application tokens (#16365)
First PR to implement application tokens
- add new application role in twenty-server
- move duplicated constants and types to twenty-shared
- will  add role configuration utils into twenty-sdk in another PR
2025-12-08 10:42:46 +01:00
Félix Malfait 223082a4da refactor(twenty-server): consolidate AI tool provider architecture (#16355)
## Summary

Consolidates the AI tool provider architecture by creating a single
`ToolProviderService` as the entry point for all tool generation. This
removes multiple intermediate services and simplifies the codebase.

## Changes

### New Architecture
- **`ToolProviderService`**: Single service for all tool generation
with:
  - `getTools(spec)` - Get tools by category with permissions
  - `getToolByType(type)` - Get specific tool for workflow execution
  
- **`ToolCategory` enum**: Declarative specification of tool types:
  - `DATABASE_CRUD` - Record CRUD operations
  - `ACTION` - HTTP requests, email sending, article search
  - `WORKFLOW` - Workflow management tools
  - `METADATA` - Object/field metadata tools
  - `NATIVE_MODEL` - Model-specific tools (e.g., web search)

- **`ToolSpecification` type**: Clean API for requesting tools with
permissions

### Removed
- `AiToolsModule` - No longer needed
- `ToolService` - Logic inlined into ToolProviderService
- `ToolAdapterService` - Logic inlined into ToolProviderService
- `ToolRegistryService` - Logic inlined into ToolProviderService

### Updated
- All consumers (agents, chat, MCP, workflows) now use
`ToolProviderService`
- Test files updated accordingly

## Stats
- **547 insertions, 1146 deletions** (net ~600 lines removed)
- 4 services deleted
- 1 module deleted

## Testing
- [x] Typecheck passes
- [x] Lint passes
2025-12-06 06:51:47 +01:00