e479d7538d9c7aa64ac0d8d4e1ad7facddf6b40b
31 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Raphaël Bosi
|
829ef9d8b9 |
Revert AI chat chips to the [[kind:...:label]] syntax (#23852)
Removes the `[[kind:...:label[[/kind]]` closing-tag syntax and goes back
to the simpler `[[kind:...:label]]` form for all four chip kinds
(record, object, field, view).
The parser is now a single regex pass instead of a two-pass scan with a
per-reference closing-tag search, a legacy fallback and surplus-bracket
handling. That removes 11 files. The label pattern excludes `[`, `]` and
newlines, which is what keeps an unclosed marker from swallowing the
text (and the marker) that follows it.
```mermaid
flowchart LR
subgraph before ["Before — two passes"]
O1["scan for marker openings"] --> O2["window each opening<br/>up to the next one"]
O2 --> O3["find that kind's closing tag<br/>inside the window"]
O3 --> O4["record only:<br/>bare-terminator fallback"]
O4 --> O5["consume surplus<br/>closing brackets"]
end
subgraph after ["After — one pass"]
N1["matchAll, one regex:<br/>object · field · view · record"] --> N2["map each match<br/>to a chip"]
end
before -.->|"11 files deleted"| after
```
Two things to know:
- Messages already stored with closing tags render as raw text instead
of chips.
- Malformed model output is no longer compensated for: a surplus `]`
after a chip stays in the text, and a display name containing brackets
does not chip. The system prompt tells the model to avoid both.
Rendering cost is unchanged for normal messages and noticeably lower on
long bracket runs, since the old opening pattern had to scan them.
<!-- This is an auto-generated description by cubic. -->
<a
href="https://cubic.dev/pr/twentyhq/twenty/pull/23852?utm_source=github"
target="_blank" rel="noopener noreferrer"
data-no-image-dialog="true"><picture><source
media="(prefers-color-scheme: dark)"
srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source
media="(prefers-color-scheme: light)"
srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img
alt="Review in cubic"
src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a>
<!-- End of auto-generated description by cubic. -->
|
||
|
Félix Malfait
|
29e68a7f87 |
Refactor outbound email content compilation (#23782)
## Integration status This is now the final landing PR for the reviewed editor/email architecture stack. | Order | Pull request | Scope | Status | | --- | --- | --- | --- | | 0 | #23657 | Advanced text editor capability presets | Merged into `main` | | 1 | **This PR** | Outbound email content compilation | Ready to land into `main` | | 2 | #23783 | Clean editor surface seam | Reviewed and merged into this branch | | 3 | #23790 | Shared editor block catalog | Reviewed and merged through #23783 | | 4 | #23791 | Canonical TipTap document persistence | Reviewed and merged through #23790 | The current branch tree contains the complete stack. Merging this PR lands all four follow-up layers. ## Architecture The stack establishes four reusable boundaries: 1. **Outbound compilation** — campaign, workflow, and one-to-one/tool email share one compiler, sanitizer policy, renderer, and plain-text derivation path. 2. **Editor surface profiles** — the generic editor owns rendering mechanics while each consuming surface declares chrome, extensions, and explicit compatibility readers. 3. **Shared block primitives** — sections, columns, HTML, images, buttons, and related commands live in the neutral advanced-editor catalog; email behavior is supplied by email schemas/rendering, not by relocating reusable blocks into an email editor. 4. **Canonical persistence** — Twenty-owned authoring persists complete, versioned TipTap JSON documents. HTML, Markdown, plain text, and BlockNote are projections or explicitly owned legacy boundaries. ## Compatibility boundaries Compatibility remains only where shipped data requires it: - workflow Send Email: versionless TipTap JSON, HTML, and plain text - inline email: HTML - AI instructions: Markdown - record rich text: BlockNote arrays and older Markdown/plain text Campaign is unshipped, so its editor, stored rows, sendability validation, and send-time compilation require the current canonical schema version. AI chat drafts are canonical-only local state; old or malformed drafts are rejected at hydration, and plain-text preprompts are converted at their entry point. ## Outbound compiler details The shared compiler owns: - strict structured email-document parsing - React-email rendering - one cached DOMPurify/JSDOM policy for structured and legacy HTML - plain-text derivation from sanitized HTML - single-pass structured-document binding resolution across text, variable tags, links, images, buttons, and raw HTML Resolved workflow values remain inert, legacy workflow and one-to-one HTML remain supported, and Campaign HTML/plain text come from the same compiled result. ## Verification - all automated standard/security reviews passed on the three merged upper PRs with no unresolved threads - shared TipTap/email codec tests: 20 passing - editor, AI draft, and workflow compatibility tests: 14 passing - campaign validation and compilation tests: 31 passing - full shared suite during development: 223 suites / 1,738 tests passing - twenty-front, twenty-shared, and twenty-server typechecks - changed-file type-aware lint and formatting checks |
||
|
Félix Malfait
|
5d88cf7f2f |
feat(server): add role management tools for AI chat (#23613)
Adds role management tools to the AI chat so the agent can create and configure roles, including row-level permissions. ## What A `RoleToolProvider` in `tool-provider/providers/`, mirroring `webhook-tool.provider.ts`, registered in `tool-provider.module.ts` and gated behind `PermissionFlagType.ROLES` via `PermissionsService.checkRolesPermissions` (same pattern as the VIEWS/WORKFLOWS gating). Tools: - `list_roles` — global record permissions, settings access, per-object overrides, permission flags, assignability; optionally includes row-level rules - `create_role`, `update_role`, `delete_role` - `assign_role_to_workspace_member` — via `UserRoleService.assignRoleToManyUserWorkspace`, which goes through role-target - `upsert_object_permissions` — per-object overrides, e.g. read-only on a given object - `upsert_row_level_permission_rules` — reuses `RowLevelPermissionPredicateService.upsertRowLevelPermissionPredicates` and the predicate-group service, so the agent can express rules like "members with this role only see records where the owner field matches the current user" (a predicate with `workspaceMemberFieldMetadataId` pointing at the workspaceMember `id` field, resolved to the current user at query time) Everything routes through the existing role services and DTOs (`RoleService`, `ObjectPermissionService`, `UserRoleService`, the row-level predicate services) rather than reimplementing them. A new `ToolCategory.ROLE` is added to `twenty-shared`, along with its label in the exhaustive switch in `build-tool-catalog-section.util.ts`. ## Safeguards - Any mutation on a role with `isEditable: false` is rejected. That covers the Admin role, which is created non-editable, and matches what Settings blocks. - Deleting the role the caller is currently acting under is rejected, since deletion would rebind them to the workspace default role. - Setting `canUpdateAllSettings: false` on the caller's own role is rejected unless that role keeps an explicit ROLES permission flag. - Changing your own role via `assign_role_to_workspace_member` is rejected, checked both by workspace member id and by resolved user workspace id. The tool-layer checks are deliberate pre-checks: the migration validators and services enforce the same rules downstream (`validate-role-is-editable.util.ts`, default-role deletion, last-admin unassignment, write-without-read consistency), but catching them early gives the model a named, actionable message instead of a build failure report. Where the deeper layer does reject, `formatValidationErrors` expands the migration exception so the underlying per-entity errors reach the model rather than a generic summary. Worth flagging for reviewers: the self-lockout protection currently lives only at the tool layer. A human admin can still strip settings access from their own role through Settings/GraphQL. Closing that would mean changing `RoleService`/`UserRoleService` behavior for the human path, which felt like a separate decision than what this change is scoped to. ## Notes `ToolCategory.ROLE` is intentionally left out of `WORKFLOW_AGENT_REGISTRY_TOOL_CATEGORIES`, so workflow agents don't get these tools; only the chat surface and the MCP/tool-index paths that share the registry do. ## Testing - 24 unit tests in `providers/__tests__/role-tool.provider.spec.ts`, covering permission gating, descriptor exposure, each safeguard, the N+1-free list path, and validation-error surfacing - 333 tests pass across the tool-provider, role, object-permission and ai suites - `npx nx lint:diff-with-main twenty-server` and `npx nx typecheck twenty-server` are clean --- _Generated by [Claude Code](https://claude.ai/code/session_0131sLKVsRuaDoaKCxFM8g4Z)_ <!-- This is an auto-generated description by cubic. --> <a href="https://cubic.dev/pr/twentyhq/twenty/pull/23613?utm_source=github" target="_blank" rel="noopener noreferrer" data-no-image-dialog="true"><picture><source media="(prefers-color-scheme: dark)" srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source media="(prefers-color-scheme: light)" srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img alt="Review in cubic" src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a> <!-- End of auto-generated description by cubic. --> |
||
|
Félix Malfait
|
087bee0036 |
fix(ai): notify all tabs when a pending question is answered (#22491)
## Rationale `resolvePendingQuestion` updates the question tool-part to `answered` and re-claims the thread — but publishes **nothing**. The answering tab converges via a local browser event; every other tab keeps rendering the question card as interactive until the resumed stream's first chunk happens to arrive. A second tab (or teammate view on shared context) can attempt to answer an already-answered question and hit a confusing `QUESTION_NOT_PENDING` error. ## Why this is the root cause, not a symptom patch Answering a question is a state transition every subscriber cares about — exactly like queue promotion, message persistence, and stream errors, all of which publish. This transition just never did. The fix publishes the existing refetch-trigger event (`queue-updated`, which every tab already handles by refetching messages + thread state) right after resolution — no new event type, no new client code path, consistent by construction with how every other transition converges tabs. A dedicated `question-answered` event carrying the answers would save one refetch round-trip; the audit's verdict was that's over-engineering for a rare interaction. Publishing *before* the resume-enqueue is deliberate: even if the enqueue fails, the question **is** answered server-side, and tabs should reflect server truth. ## User impact Second tabs stop offering an interactive question that will error when submitted; everyone sees the answered state within a refetch instead of whenever the stream resumes. ## Test plan - [ ] CI green - [ ] Manual: two tabs on one thread, answer the question in tab A → tab B's card flips to answered without interaction https://claude.ai/code/session_01Lyi6zTema2FMVVh8MD6c38 --- _Generated by [Claude Code](https://claude.ai/code/session_01Lyi6zTema2FMVVh8MD6c38)_ <!-- This is an auto-generated description by cubic. --> <a href="https://cubic.dev/pr/twentyhq/twenty/pull/22491?utm_source=github" target="_blank" rel="noopener noreferrer" data-no-image-dialog="true"><picture><source media="(prefers-color-scheme: dark)" srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source media="(prefers-color-scheme: light)" srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img alt="Review in cubic" src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a> <!-- End of auto-generated description by cubic. --> |
||
|
Félix Malfait
|
4aaf171d63 |
feat(ai): add ask_questions interactive clarifying-question tool (#22346)
## What & why Adds an `ask_questions` tool that lets the in-app **Ask AI** assistant **pause a turn to ask the user one or more multiple-choice questions** (per the [Figma design](https://www.figma.com/design/xt8O9mFeLl46C5InWwoMrN/Twenty?node-id=105959-117153)) and resume once answered — instead of guessing on ambiguous/consequential decisions. The tool is **harness-only**: an interactive question UI is meaningless without a user to answer it, so it must be absent from MCP and from head-less workflow agents. ## Design — true tool-result resume (not a synthetic user message) The user's answer is a **structured tool result bound to the `toolCallId`**, and the **same agent turn resumes** — exactly how Anthropic (`tool_result` by `tool_use_id`) and OpenAI (`function_call_output`) model human-in-the-loop. The naive form of this (leave the tool call in `input-available` to mean "pending") is **impossible** here: `finalizeDanglingToolParts` rewrites `input-available` → `output-error` ("Tool execution was interrupted") on both the persist path (`addMessage`) and the model-reload path (`chat-execution.service.ts`). That util is a load-bearing safety net, so weakening it is the wrong move. Instead: - `ask_questions` is an **inline, chat-only tool with an `execute` that returns a `status: 'pending'` result immediately**, so the tool part is always `output-available` and **immune to `finalizeDanglingToolParts`**. `stopWhen(hasToolCall('ask_questions'))` halts the turn right after the call (the model never sees the placeholder). - A nullable **`thread.pendingQuestionMessageId`** marker records that a turn is awaiting an answer. - The new **`answerAgentChatQuestion`** mutation atomically *claims* the question (clears the marker, marks the thread streaming), **writes the answer onto the same tool part** (`status: 'answered'`), and **re-enqueues the turn via the existing `existingTurnId` plumbing** (`isResume` bypasses the per-turn dedup guard). On resume `finalizeDanglingToolParts` leaves the `output-available` part untouched and `convertToModelMessages` emits `assistant(tool_use)` + `tool_result(answers)`, so the model continues. This achieves the platform-aligned semantics **without** weakening the finalize safety net or inventing a fragile new part state. ### Meets the two requirements - **Survives refresh, scoped per-thread** — the pending state is a normal persisted `output-available` part + the thread marker; the frontend card is derived per-thread from the loaded messages, so it re-appears on reload and only on its own thread. - **Takes priority over the queue** — a unified `isBlocked = activeStreamId || pendingQuestionMessageId` gate is applied in both `sendChatMessage` (new messages queue) and `flushNextQueuedMessage` (the drain). The queue cannot unpile until the question is answered and the resumed turn completes. ### Harness-only by construction `ask_questions` is added **only** to the chat's inline `activeTools` (like `learn_tools`/`execute_tool`/`load_skills`). It never enters the tool registry/catalog, so it is invisible to MCP and to workflow agents — no `MCP_EXCLUDED_TOOL_NAMES` entry needed. ## UX While a question is pending, the **composer is replaced by the question card** (matching the Figma): question title + pager (`1/2`), numbered option rows (`IconSquareNumber*`) with per-option info-icon descriptions and a "Recommended" badge, and the normal composer as the free-text fallback ("Type anything to do differently."). The transcript shows a compact "Asking questions…" status line that becomes an answered summary. ## Changes **twenty-shared** - `ai/types/AskQuestionsToolTypes.ts` — `AskQuestionItem/Option/Answer/Result`, `ASK_QUESTIONS_TOOL_NAME`. **twenty-server** - `ai-chat/tools/ask-questions.tool.ts` — inline tool factory (pending-result `execute`, zod schema, 1–4 questions × 2–4 options). - `chat-execution.service.ts` — add to `activeTools` + `preloadedToolNames`; `hasToolCall` in `stopWhen`. - `chat-system-prompts.const.ts` — when-to-use guidance. - `entities/agent-chat-thread.entity.ts` — `pendingQuestionMessageId` column. - `stream-agent-chat.job.ts` — set the marker on a question pause; bypass the dedup guard on resume; suppress the no-text warning for question pauses. - `agent-chat-streaming.service.ts` — gate `flushNextQueuedMessage`; `enqueueResumeStream`. - `agent-chat.resolver.ts` — gate `sendChatMessage`; `answerAgentChatQuestion` mutation. - `agent-chat.service.ts` — `resolvePendingQuestion` (atomic claim + write answer). - `dtos/agent-chat-question-answer.input.ts`, `ai.exception.ts` (`QUESTION_NOT_PENDING`), `utils/find-pending-question-part.util.ts`. **twenty-front** - `components/AiChatQuestionCard.tsx` — the interactive card (matches Figma tokens) + `__stories__/AiChatQuestionCard.stories.tsx`. - `components/AiChatEditorSection.tsx` — swap the composer for the card while pending. - `components/AiChatQuestionStatusRenderer.tsx` + branch in `AiChatAssistantMessageRenderer.tsx`. - `states/selectors/agentChatPendingQuestionComponentSelector.ts`, `types/AgentChatPendingQuestion.ts`. - `hooks/useSubmitQuestionAnswer.ts` + `utils/markQuestionAnswered.ts` (optimistic) + `graphql/mutations/answerAgentChatQuestion.ts`. A design doc lives at `packages/twenty-server/docs/ASK_USER_QUESTION_TOOL_PLAN.md`. ## Migration Adds a nullable `pendingQuestionMessageId` (uuid) column to `core.agentChatThread`. Needs a generated **fast instance command** (`database:migrate:generate --name addThreadPendingQuestion --type fast`) — see "Verification status". ## Tests - Server: `ask-questions.tool.spec.ts` (pending echo + schema bounds), `find-pending-question-part.util.spec.ts`. - Front: `markQuestionAnswered.test.ts`, plus the Storybook story. ## Verification status (please read) This branch was authored in an environment where the monorepo `yarn install` repeatedly failed on transient TLS resets from the package registry, so I could **not** locally run the mechanical gates. The logic was reviewed by hand and the `ai@6.0.97` exports used (`hasToolCall`, `stepCountIs`, `generateId`) were confirmed against the package's type defs. Still **TODO** (will rely on CI / a follow-up once deps install): - [ ] `nx run twenty-shared:generateBarrels` (the `ai/index.ts` export was added by hand; regen to reconcile) - [ ] `nx run twenty-front:graphql:generate` (new mutation + input type) - [ ] generate the fast instance command (migration) for the new column - [ ] `typecheck` + `lint:diff-with-main` (front + server) — expect minor import-ordering autofixes - [ ] run the unit tests **Screenshots:** reproducing the live flow needs an AI provider API key (to get the model to actually call `ask_questions`), which isn't available here. The card can be screenshotted from its **Storybook story** (`AiChatQuestionCard.stories.tsx`) with no API key — I'll add that image once deps install, or a reviewer can run `nx storybook twenty-front`. 🤖 Generated with [Claude Code](https://claude.com/claude-code) https://claude.ai/code/session_01AArS8H3y3Z1Qwm763xhPLB --- _Generated by [Claude Code](https://claude.ai/code/session_01AArS8H3y3Z1Qwm763xhPLB)_ <!-- This is an auto-generated description by cubic. --> <a href="https://cubic.dev/pr/twentyhq/twenty/pull/22346?utm_source=github" target="_blank" rel="noopener noreferrer" data-no-image-dialog="true"><picture><source media="(prefers-color-scheme: dark)" srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source media="(prefers-color-scheme: light)" srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img alt="Review in cubic" src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a> <!-- End of auto-generated description by cubic. --> |
||
|
Félix Malfait
|
da6a2ee300 |
fix(ai-chat): keep streams alive on silent SSE death + make the stream job idempotent (#22201)
## Problem In production, an AI-chat assistant response sometimes freezes mid-stream (partial text, looks hung), then "picks up again on its own" later without the user resending and without a known worker restart. Root cause: the **agent-chat SSE subscription has no keepalive and no silent-death detection**. - Delivery is fire-and-forget Redis pub/sub (`SubscriptionService.publishToAgentChat`) and the resolver returns the **raw** iterator — unlike `EventStreamResolver`, which heartbeats every 30s via `wrapAsyncIteratorWithLifecycle`. - During a quiet model/tool gap the connection sends no bytes, so a proxy/LB/NAT can silently drop it mid-stream. `graphql-sse` neither surfaces an error nor resumes with `Last-Event-ID`, and **nothing re-pulls the existing Redis chunk catch-up on reconnect** (it only runs on thread (re)mount / `message-persisted` refetch). - So the live view freezes; recovery only happens when the terminal `message-persisted` fires a full refetch from the DB — the observed "self-recovery". This is the **same silent-SSE-death class fixed for the DB event stream in #21061**, which was never applied to the agent-chat path. The symptom also matches #21096 (worker logs the job finishing, client never updates, reload shows the message). It is **not** queue prioritization, and it is **not** addressed by #22193 (which only stabilizes the assistant message id and removes end-of-stream flicker). A secondary, independent self-recovery path also existed: BullMQ stalled-job re-run (default 30s `lockDuration`, no idempotency guard) re-streaming the whole turn → duplicate assistant messages / double billing. ## Changes ### Commit 1 — keepalive + silent-death recovery (ports the #21061 pattern to agent chat) - **Shared:** new `keepalive` variant on `AgentChatSubscriptionEvent`. - **Server:** wrap the agent-chat subscription iterator with `wrapAsyncIteratorWithLifecycle` — emit a `keepalive` on connect and every `APPLICATION_KEEPALIVE_INTERVAL_MS` (30s) so the connection keeps flushing bytes and a dead connection becomes detectable. - **Client:** track the last received event timestamp (refreshed on every chunk/keepalive in the SSE `next` sink); new `AgentChatStreamKeepAliveEffect` forces a resubscribe + messages refetch after 90s of silence, so the durable Redis chunk list backfills the gap (`firstLiveSeq` is reset on resubscribe). ### Commit 2 — stream-job idempotency + lockDuration - Thread a `lockDuration` option through `MessageQueueWorkerOptions` + the BullMQ driver; set `aiStreamQueue` to 10 min so long streams aren't falsely stalled. - Guard `StreamAgentChatJob.handle` with a `streamId`-scoped Redis lock (`SET NX PX` + compare-and-delete release) so a stalled re-run is skipped instead of double-processing. ## Verification ⚠️ I could **not run typecheck/lint locally** — `yarn install` could not complete in this environment (transient registry network aborts before the link step, so `node_modules` never populated). **Please rely on CI for type/lint verification.** The changes are written to match existing conventions; the points most worth a reviewer's eye are the resolver's iterator typing and the ioredis `set(..., 'PX', ttl, 'NX')` overload. How to confirm the root cause in prod: a frozen client with the worker logging `StreamAgentChatJob processed in …ms` and no `[AI_CHAT_NO_TEXT]` is the silent-death signature (check reverse-proxy idle/buffering). For the secondary path, watch `aiStreamQueue` `stalled`/re-processed metrics and duplicate turns around worker restarts. ## Notes / trade-offs - The 10-min `lockDuration` means a genuinely crashed worker's job isn't reclaimed for up to 10 min; the client-side keepalive/catch-up recovers the view independently, and the idempotency lock prevents duplicates. Faster dead-worker recovery could be a follow-up. - Touches `useAgentChatSubscription.ts` / `AgentChatRuntimeEffects.tsx` / `stream-agent-chat.job.ts`, which #22193 also touches — trivial rebase expected. Opened as **draft** pending CI. https://claude.ai/code/session_018dF82A1VcsuWMxPLmdY3dm --- _Generated by [Claude Code](https://claude.ai/code/session_018dF82A1VcsuWMxPLmdY3dm)_ <!-- This is an auto-generated description by cubic. --> <a href="https://cubic.dev/pr/twentyhq/twenty/pull/22201?utm_source=github" target="_blank" rel="noopener noreferrer" data-no-image-dialog="true"><picture><source media="(prefers-color-scheme: dark)" srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source media="(prefers-color-scheme: light)" srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img alt="Review in cubic" src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a> <!-- End of auto-generated description by cubic. --> |
||
|
Etienne
|
5ca41d55fb |
feat(ai): humanize tool-call (#21976)
# Humanize tool-call labels cc: https://github.com/twentyhq/twenty/pull/21462 ## Preview <img width="459" height="156" alt="Screenshot 2026-06-22 at 19 13 11" src="https://github.com/user-attachments/assets/e7a2f5f5-cd09-4ec6-920b-5eb16b98285c" /> <img width="461" height="156" alt="Screenshot 2026-06-22 at 19 14 54" src="https://github.com/user-attachments/assets/c2114d2e-2aa8-499a-9801-68e3bb7c45f8" /> <img width="461" height="505" alt="Screenshot 2026-06-22 at 19 15 01" src="https://github.com/user-attachments/assets/ee9ca5d0-8e79-4c63-a2ff-ed5e359a9a9c" /> ## Why In the AI chat, tool steps were displayed using raw tool identifiers (`find_many_companies`, `create_one_task`, `send_email`...) and labels were partially reconstructed/humanized on the frontend. This was hard to localize and inconsistent across tool categories. This PR makes the **backend the single source of truth for human-readable, localized tool labels**, exposes them through `getToolIndex`, and reduces the frontend to a thin resolver that picks the right label for the current status (in-progress / completed). ## What changed ### Backend - `ToolIndexEntry` (and the `getToolIndex` GraphQL DTO) now carry `label`, `inProgressLabel?`, `completedLabel?`. - New `getCrudToolLabels(operation, objectLabel, i18nService, locale)` builds CRUD labels from a verb table (Search / Find / Group / Create / Update / Upsert / Delete × imperative / in-progress / completed) + the (translated, lowercased) object label. - New `translate-tool-label.util.ts` translates a source label via `I18nService` (`generateMessageId` → fallback to source when no translation exists). - Action tools: labels extracted to the `ACTION_TOOL_LABELS` constant (`msg` + `i18nLabel`) and translated in `ActionToolProvider.buildDescriptor`. - Logic-function tools use the function name as label; `toolSetToDescriptors` (workflow / view / metadata / dashboard) accepts an optional `labels` map and falls back to a humanized tool name. - Labels are localized server-side using the request locale (`@RequestLocale` → `buildToolIndex` → `context.locale`, threaded through `ToolContext` / `ToolProviderContext`). - `code_interpreter` schema now asks the model for `loadingMessage` (present tense) and `completedMessage` (past tense), so its status text is model-generated. - Removed the old generic `loadingMessage` injection mechanism (`wrap-tool-for-execution.util.ts` deleted; `wrapJsonSchemaForExecution` / `stripLoadingMessage` no longer wrap every tool). ### Frontend - New `useToolLabelMap()` hook builds a `Map<name, { label, inProgressLabel, completedLabel }>` from `getToolIndex`. - `getToolDisplayMessage` → `resolveToolDisplayMessage({ input, toolName, isFinished, labelMap, output })`: a small resolver registry keyed by tool name (`execute_tool`, `web_search`, `learn_tools`, `load_skills`, `code_interpreter`, default). - Default resolver prefers backend `completedLabel` / `inProgressLabel`, falling back to `Ran X` / `Running X`. - `learn_tools` / `load_skills` resolve their inner tool/skill names to labels (label map → tool output labels via `getToolOutputLabelEntries` → raw name). - `code_interpreter` step is now expandable to show the code even while running. ## How tool labelling flows (BE → FE) ```text BACKEND ┌───────────────────────────────────────────────────────────────────────────┐ │ Tool providers (per category) → ToolIndexEntry │ │ │ │ DatabaseToolProvider │ │ getCrudToolLabels(operation, object.labelPlural/Singular, i18n, locale) │ │ verb table (Search/Create/Update/Delete…) + translateToolLabel(object) │ │ → { label, inProgressLabel, completedLabel } │ │ │ │ ActionToolProvider │ │ ACTION_TOOL_LABELS[toolId] (msg) → translateToolLabel(…, locale) │ │ → { label, inProgressLabel?, completedLabel? } │ │ │ │ LogicFunctionToolProvider → label = logicFunction.name │ │ toolSetToDescriptors → label = labels[name] ?? humanize(name) │ │ (workflow / view / metadata / dashboard) │ └───────────────────────────────────────────────────────────────────────────┘ │ ▼ ┌───────────────────────────────────────────────────────────────────────────┐ │ GraphQL Query getToolIndex : [ToolIndexEntry] │ │ { name, label, inProgressLabel, completedLabel, description, │ │ category, objectName, icon } │ └───────────────────────────────────────────────────────────────────────────┘ │ ▼ FRONTEND ─ resolve the right label for the current status ┌───────────────────────────────────────────────────────────────────────────┐ │ useGetToolIndex() → useToolLabelMap() │ │ Map<name, { label, inProgressLabel?, completedLabel? }> │ └───────────────────────────────────────────────────────────────────────────┘ │ ▼ ┌───────────────────────────────────────────────────────────────────────────┐ │ resolveToolDisplayMessage({ input, toolName, isFinished, labelMap, output })│ │ │ │ TOOL_LABEL_RESOLVERS[toolName] ?? defaultResolver │ │ ├─ execute_tool → unwrap { toolName, arguments } then re-resolve │ │ ├─ web_search → "Searching/Searched the web for <query>" │ │ ├─ learn_tools → "Learning/Learned <labels>" │ │ ├─ load_skills → "Loading/Loaded <labels>" │ │ │ inner names resolved via: labelMap → output labels → raw name │ │ ├─ code_interpreter → model's loadingMessage / completedMessage │ │ └─ default → isFinished │ │ ? completedLabel ?? "Ran <label>" │ │ : inProgressLabel ?? "Running <label>" │ └───────────────────────────────────────────────────────────────────────────┘ │ ▼ Rendered by ThinkingStepsDisplay / ToolStepRenderer ``` ## Localization notes - Standard object labels and action/CRUD verbs are translated server-side via `I18nService` using the requester's locale. - Custom object labels are not translated unless a workspace custom translation exists (matched by `generateMessageId`); otherwise the source label is used as-is. ## Tests - **FE:** `resolveToolDisplayMessage` / `getToolOutputLabelEntries` (status selection, inner-name resolution, `code_interpreter` model labels, fallbacks). - **BE:** `toolSetToDescriptors` (label map + humanized fallback) and `database-tool.provider` label generation. <!-- This is an auto-generated description by cubic. --> <a href="https://cubic.dev/pr/twentyhq/twenty/pull/21976?utm_source=github" target="_blank" rel="noopener noreferrer" data-no-image-dialog="true"><picture><source media="(prefers-color-scheme: dark)" srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source media="(prefers-color-scheme: light)" srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img alt="Review in cubic" src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a> <!-- End of auto-generated description by cubic. --> |
||
|
Charles Bochet
|
0064ff6741 |
fix(ai): validate AI agent output field names against schema-key constraint (#21834)
## Problem
On a self-hosted instance, an AI Agent workflow action fails at run time
with an opaque model error:
```
The model returned the following errors: tools.0.custom.input_schema.properties:
Property keys should match pattern '^[a-zA-Z0-9_.-]{1,64}$'
```
This is Anthropic's validation on tool `input_schema` **property keys**.
An AI Agent's structured **Output** fields are turned into a JSON schema
and passed to the model as a tool; each output **variable name** becomes
a property key. Anthropic rejects any key that does not match
`^[a-zA-Z0-9_.-]{1,64}$` — most commonly a name containing a **space**
(e.g. `meetings brief`), but also names over 64 characters or with other
symbols.
Until now nothing validated this: `fieldsToSchema` writes
`properties[field.name]` verbatim, so a bad name only failed once the
workflow executed, with an error that gives the user no idea what to
fix. It doesn't reproduce on every instance — it depends purely on how
the workflow's output variables happen to be named.
## Fix
Introduce a single shared check,
`isValidAgentResponseSchemaPropertyKey`, and enforce it in two places:
- **Backend** — `validateAgentResponseFormat` now rejects invalid output
field names at agent **save time** with a clear `userFriendlyMessage`,
instead of letting the broken schema reach the model. This also gates
agents created via the API and re-saves of existing bad data.
- **Frontend** — the output schema builder shows an inline error on the
Variable Name field as soon as an invalid name is entered.
## Tests
- Unit test for the shared validity check (valid + invalid cases:
spaces, leading space, empty, > 64 chars, symbols, unicode).
- Unit test for `validateAgentResponseFormat` covering text/json
formats, valid names, a space in a name, an over-length name, and
reporting multiple invalid names at once.
## Notes for the reporter
The immediate unblock for an affected workflow is to rename the output
variable to remove the space (e.g. `meetings brief` → `meetings_brief`)
and retry the run. With this change the bad name is caught up front with
an explanation rather than failing mid-run.
<!-- This is an auto-generated description by cubic. -->
<a
href="https://cubic.dev/pr/twentyhq/twenty/pull/21834?utm_source=github"
target="_blank" rel="noopener noreferrer"
data-no-image-dialog="true"><picture><source
media="(prefers-color-scheme: dark)"
srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source
media="(prefers-color-scheme: light)"
srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img
alt="Review in cubic"
src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a>
<!-- End of auto-generated description by cubic. -->
|
||
|
martmull
|
c2ca90c255 |
feat(sdk): add runAgent() to run app agents from logic functions (#21157)
<img width="948" height="593" alt="image" src="https://github.com/user-attachments/assets/d990fa98-3cfd-469d-ab7f-0b2d4ccf3afc" /> <img width="1361" height="802" alt="image" src="https://github.com/user-attachments/assets/1091f598-49f3-4c16-92ea-1e1c200181e2" /> ## Add `runAgent()` to the Logic Function SDK Lets an app's logic function run one of its own AI agents server-side and get the result back synchronously — reusing the existing agent executor instead of a new bespoke transport. ### Backend - New **`runAgent` GraphQL mutation** (metadata schema) in `ai-agent-execution`, wrapping the existing `AgentAsyncExecutorService.executeAgent`. Scopes the agent lookup to the calling application and runs it under an application auth context. - New `@AuthApplication()` param decorator (mirrors `@AuthWorkspace()`) — first GraphQL resolver authenticated by an **application access token**. - Guarded by `WorkspaceAuthGuard` + `SettingsPermissionGuard(PermissionFlagType.AI)`: the app's role must grant the `AI` permission flag. ### SDK - `runAgent({ agentUniversalIdentifier, prompt })` posts the mutation to `/metadata` with the app token via a new runtime GraphQL transport. Returns `{ result, hasNoMoreAvailableCredits }`. - Refactored the connections helpers onto a shared `postAppEndpoint` util (removes duplicated transport logic). ### Frontend - App install permission modal now shows an explicit consent line — _"Run AI agents and bill AI credits to your workspace"_ — when the app's role requests the `AI` flag. ### Docs - Documented `runAgent` and its `AI` permission-flag requirement in _Skills & Agents_. - Fixed outdated role-permission examples in _Roles & Permissions_ (`permissionFlags` → `permissionFlagUniversalIdentifiers`, `PermissionFlag` → `SystemPermissionFlag`). ### Test plan - [x] SDK unit tests (`run-agent.spec.ts`) — request shape, GraphQL/HTTP error handling, missing env vars - [x] `twenty-server`, `twenty-front`, `twenty-shared` typecheck + lint - [ ] Manual: install an app granting the `AI` flag, call `runAgent()` from a logic function, confirm the agent runs and credits are billed --------- Co-authored-by: cubic-dev-ai[bot] <191113872+cubic-dev-ai[bot]@users.noreply.github.com> |
||
|
Félix Malfait
|
de044f4b45 |
feat(ai-chat): add navigation menu item + webhook tool providers (#20759)
## Summary
Exposes two Twenty primitives to the AI chat that it could not
previously manage:
- **Navigation menu items** — workspace nav and personal favorites
(favorites are just nav items with `scope: 'user'`).
- **Webhooks** — full CRUD with a structured operations input (record +
metadata events).
Page layouts and workflow runs were originally in this PR but have been
split out — they touch heavier surfaces (21 widget configurations and
the workflow runner cycle, respectively) and deserve their own focused
PRs.
### Tool inventory (8 new tools across 2 providers)
| Provider | Tools |
|---|---|
| NavigationMenuItem | `list_`, `create_`, `update_`,
`delete_navigation_menu_item` |
| Webhook | `list_`, `create_`, `update_`, `delete_webhook` |
### Design notes
- Both providers follow the established **view-style pattern**: tool
workspace service lives in the entity module's `tools/` folder, is
provided + exported by the entity module, and `ToolProviderModule`
imports the entity module. No `@Global()` modules or injection tokens
introduced.
- `create_navigation_menu_item` uses a Zod `discriminatedUnion` on
`type` (`FOLDER` / `LINK` / `OBJECT` / `VIEW` / `RECORD` /
`PAGE_LAYOUT`). `scope: 'workspace' | 'user'` switches between shared
nav and personal favorites — the underlying
`NavigationMenuItemAccessService` enforces LAYOUTS for workspace writes.
- Webhook operations accept both record events (`{kind:'record', object,
event}` → `<object>.<event>`) and metadata events (`{kind:'metadata',
metadataName, operation}` → `metadata.<metadataName>.<operation>`).
- Permissions reuse existing flags (`LAYOUTS`, `API_KEYS_AND_WEBHOOKS`).
No new permission flags, no migrations.
### Category cleanup
- New: `ToolCategory.NAVIGATION_MENU_ITEM`, `ToolCategory.WEBHOOK`.
- `ToolCategory.VIEW_FIELD` → folded into `VIEW`. Same permission gate,
same domain — separate category was organizational drift.
- `navigate_app` action stays in `ToolCategory.ACTION` where it belongs.
### System prompt addition
[chat-system-prompts.const.ts](packages/twenty-server/src/engine/metadata-modules/ai/ai-chat/constants/chat-system-prompts.const.ts)
now teaches the AI:
- Favorites are nav items with `scope: 'user'`.
- A default OBJECT nav item is auto-created with
`create_object_metadata` — don't double-create.
### One file = one export
Every new schema / type / util file has exactly one top-level export.
## Test plan
- [ ] `npx nx typecheck twenty-server` — passes
- [ ] Spin up locally and exercise via AI chat:
- [ ] "Pin the Companies view to my favorites in a folder called
Important." → `create_navigation_menu_item` (FOLDER, user) then (VIEW,
user, folderId)
- [ ] "Register a webhook to https://example.com firing when any person
is created or updated." → `create_webhook` with discriminated operations
- [ ] Verify workspace-scoped nav writes are denied for a user without
LAYOUTS permission
- [ ] Verify user-scoped nav writes work without LAYOUTS permission
## Follow-ups (separate PRs)
- Page layout tools (record-page, record-index, standalone) — needs
widget-config strategy.
- Workflow run tools (list, get, run, stop) — uses the workflow-runner
cycle path.
- Dashboard / page-layout tool unification —
`DashboardToolWorkspaceService` and a future
`PageLayoutToolWorkspaceService` both inject the same trio
(PageLayout/Tab/Widget services).
- Webhook Settings page reads from raw Apollo query — switch to the
metadata store so it refreshes when the AI mutates webhooks.
|
||
|
Etienne
|
aecfe699f4 |
feat(ai-chat) - Stop ai thinking if credits exhausted (#20526)
Billing is now decremented per-step, not per-turn. The onStepFinish callback in chat-execution.service.ts calls a new decrementAndCheckAvailableCredits method on each model step, so Redis is debited incrementally as the agent runs rather than all at once at the end. Credit exhaustion stops the stream mid-run. When a step depletes the remaining credits, a hasNoMoreAvailableCredits flag is set and passed into the stopWhen predicate of streamText, causing the agent to halt before starting the next step. A new credits-exhausted event is introduced. After the stream drains and the response is persisted, if credits ran out the job publishes a dedicated credits-exhausted event to the frontend instead of the normal message-persisted event. The frontend handles this new event. useAgentChatSubscription has a new credits-exhausted case that sets a BILLING_CREDITS_EXHAUSTED-coded error on the atom, closes the writer, and stops the streaming state — triggering the existing AiChatCreditsExhaustedMessage UI. |
||
|
Etienne
|
91124a3cb8 |
AI - Add azure foundry provider (#20170)
[Merge this before](https://github.com/twentyhq/twenty-infra/pull/655) Co-authored-by: Félix Malfait <felix.malfait@gmail.com> |
||
|
nitin
|
6c1c0737b0 |
Clarify registry tools vs native model tool binding (#20022)
## Intent This is a small foundation cleanup for the tool architecture. The main decision is: registry tools and native SDK/model tools are different things. - Registry tools have descriptors, schemas, catalog entries, and execute through `ToolExecutorService` - Native model tools are opaque AI SDK objects, bound directly into the model `ToolSet` - Surfaces still own their policy: chat, MCP, and workflow agents decide what they expose ## What changed - Removed `NATIVE_MODEL` from `ToolCategory` - Kept `ToolRegistryService` focused on registry-backed tools only - Moved native model tool binding through `NativeToolBinderService` - Reused native binding from chat instead of duplicating provider-specific web-search logic - Kept MCP local execution exclusions in a dedicated constant - Moved surface-specific constants into dedicated constant files ## What comes next - Move hardcoded chat app preloads, like Exa web search, into app/manifest metadata - Decide a clearer policy for local runtime tools like code interpreter and HTTP request - Gradually document the three tool shapes: registry tools, native model tools, and local runtime tools --------- Co-authored-by: claude[bot] <41898282+claude[bot]@users.noreply.github.com> Co-authored-by: Félix Malfait <FelixMalfait@users.noreply.github.com> Co-authored-by: Félix Malfait <felix.malfait@gmail.com> |
||
|
Félix Malfait
|
6117a1d6c0 |
refactor: standardize AI acronym to Ai (PascalCase) across internal identifiers (#19837)
## Summary
The "AI" acronym was rendered inconsistently across the codebase. The
backend AI module had settled on PascalCase `Ai` (`AiAgentModule`,
`AiBillingService`, `AiChatModule`, `AiModelRegistryService`, etc.),
while frontend components, several DTOs, a few types, and shared
identifiers still used all-caps `AI` (`AIChatTab`,
`AISystemPromptPreviewDTO`, `SettingsPath.AIPrompts`, ...). CLAUDE.md
specifies PascalCase for classes; this PR normalizes everything internal
to `Ai`.
**This is a pure internal rename.** The GraphQL schema is untouched —
`@ObjectType` decorator string arguments, resolver method names (which
become Query/Mutation field names), gql template contents, and the
`generated-metadata/graphql.ts` file are preserved verbatim. The only
visible change is TypeScript identifiers and file names.
## Also folded in (adjacent cleanups)
- **`AgentModelConfigService` → `AiModelConfigService`**. Lives in
`ai-models/` and is used by multiple AI code paths, not just the Agent
entity. The "Agent" prefix was misleading.
- **`generate-text-input.dto.ts` → `generate-text.input.ts`**. The
`ai-agent/dtos/` folder already uses `<entity>.input.ts` convention for
Input classes (`create-agent.input.ts` etc.); the old path mixed
`.dto.ts` file extension with a class that has no DTO suffix. File
rename only; class stays `GenerateTextInput`.
- **Removed stale TODO** in `ai-model-config.type.ts` that asked for the
`AiModelConfig` rename that this PR performs.
## Rename methodology
Bulk rename via perl with anchored regex
`(?<!['"])(?<![A-Z.])AI([A-Z])(?=[a-z])/Ai$1/g`:
- **Lookbehind for non-uppercase** skips adjacent acronyms (`MOSAIC`,
`OIDCSSO`) and leaves `AIRBNB_ID` alone.
- **Lookbehind for non-quote** protects most string literals.
- **Lookahead for lowercase** restricts matches to PascalCase
identifiers (`AIChatTab`), leaving SCREAMING_SNAKE constants untouched.
Strict file-scope exclusions: `generated-metadata/**`, `generated/**`,
`locales/**`, `migrations/**`, `illustrations/**`, `halftone/**`, and
the two gql template files (`queries/getAISystemPromptPreview.ts`,
`mutations/uploadAIChatFile.ts`).
Post-rename reverts for identifiers where the regex was too eager:
- Backend resolver method names kept: `getAISystemPromptPreview`,
`uploadAIChatFile` (they are GraphQL field names).
- `@ObjectType('AdminAIModels')` / `('AISystemPromptPreview')` /
`('AISystemPromptSection')` kept as-is.
- Backend classes `ClientAIModelConfig` / `AdminAIModelConfig` kept
as-is (they use `@ObjectType()` with no argument, so the class name IS
the schema name).
- External-library symbols restored: `OpenAIProvider`,
`createOpenAICompatible`, `vercelAIIntegration`.
File renames use a two-step rename to work on macOS case-insensitive
filesystems: `git mv X.tsx X.tsx.tmp && git mv X.tsx.tmp renamed.tsx`.
## Diff audit
- 0 changes to migrations
- 0 changes to locale `.po` / `.ts` files
- 0 changes to `generated-metadata/graphql.ts`
- 0 changes to website illustration files (base64 blobs preserved)
- 0 renames inside user-facing translation strings (`t\`…\``,
`msg\`…\``, `<Trans>…</Trans>`)
## Test plan
- [x] `npx nx typecheck twenty-server` — PASS
- [x] `npx nx typecheck twenty-front` — PASS
- [x] `npx jest ai-model admin agent-role` — 79/79 PASS
- [x] `npx oxlint --type-aware` on 118 changed files — 0 errors
- [x] `npx prettier --check` on 118 changed files — clean
- [ ] CI
|
||
|
Abdul Rahman
|
7f2b853ae1 | feat: add message compaction for AI chats (#19205) | ||
|
nitin
|
3c067f072c | [AI] Improve tools tab (#19221) | ||
|
Félix Malfait
|
fd7387928c |
feat: queue messages + replace AI SDK with GraphQL SSE subscription (#19203)
## Summary - **Queue messages while streaming**: Messages sent during active AI streaming are queued server-side and auto-flushed when the current stream completes. Frontend renders queued messages optimistically in a dedicated queue UI. - **Drop `@ai-sdk/react` + `resumable-stream`**: Replace the dual HTTP SSE + AI SDK client architecture with a single GraphQL SSE subscription per thread. All events (token chunks, message persistence, queue updates, errors) flow through Redis PubSub → GraphQL subscription. - **Server-driven architecture**: The server decides whether to queue or stream (via `POST /:threadId/message`). The frontend mirrors this decision for optimistic rendering but defers to the server response. - **Reuse AI SDK accumulation logic**: `readUIMessageStream` from the `ai` package handles chunk-to-message accumulation on the frontend, avoiding a custom 780-line accumulator. ## Key files **Backend:** - `agent-chat-event-publisher.service.ts` — publishes events to Redis PubSub - `agent-chat-subscription.resolver.ts` — GraphQL subscription resolver - `stream-agent-chat.job.ts` — publishes chunks via PubSub instead of resumable-stream - `agent-chat.controller.ts` — unified `POST /:threadId/message` endpoint **Frontend:** - `useAgentChatSubscription.ts` — subscribes to `onAgentChatEvent`, bridges to `readUIMessageStream` - `useAgentChat.ts` — send/stop/optimistic rendering (no more AI SDK) - `AgentChatStreamSubscriptionEffect.tsx` — replaces `AgentChatAiSdkStreamEffect.tsx` ## Test plan - [ ] Send message on new thread → optimistic render, streaming response appears - [ ] Send message while streaming → queued instantly (no flash in main thread) - [ ] Queued message auto-flushes after current stream completes - [ ] Remove queued message via queue UI - [ ] Stop streaming mid-response - [ ] Leave chat idle for several minutes → streaming still works after (SSE client recycling) - [ ] Token refresh during session → requests succeed (authenticated fetch) - [ ] Switch threads while streaming → clean subscription handoff Made with [Cursor](https://cursor.com) --------- Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com> |
||
|
Félix Malfait
|
908aefe7c1 |
feat: replace hardcoded AI model constants with JSON seed catalog (#18818)
## Summary - Replaces per-provider TypeScript constant files (`openai-models.const.ts`, `anthropic-models.const.ts`, etc.) with a single `ai-providers.json` catalog as the source of truth - Adds runtime model discovery via AI SDK for self-hosted providers, with `models.dev` enrichment for pricing/capabilities - Introduces composite model IDs (`provider/modelId`) for canonical, conflict-free identification - Simplifies provider configuration: API keys are injected from environment variables (e.g., `OPENAI_API_KEY`) - Adds admin panel UI for provider management (add/remove/test), model discovery, recommended model configuration, and default fast/smart model selection per workspace - Removes deprecated config variables (`AI_DISABLED_MODEL_IDS`, `AUTO_ENABLE_NEW_AI_MODELS`, etc.) - Adds database migration for composite model ID format ## Test plan - [ ] Server typecheck passes - [ ] Frontend typecheck passes - [ ] Server unit tests pass - [ ] Frontend unit tests pass - [ ] CI pipeline green - [ ] Admin panel AI tab loads correctly - [ ] Provider discovery works for configured providers - [ ] Model recommendation toggles persist - [ ] Default fast/smart model selection works Made with [Cursor](https://cursor.com) |
||
|
Lucas Bordeau
|
cb3e32df86 |
Fix AI demo workspace skill (#18575)
This PR fixes what allows to have a working demo workspace skill. - Skill updated many times into something that works - Fixed infinite loop in AI chat by memoizing ai-sdk output - Finished navigateToView implementation - Increased MAX_STEPS to 300 so the chat don't quit in the middle of a long running skill - Added CreateManyRelationFields |
||
|
Lucas Bordeau
|
2a82df7073 |
AI tools to create a demo workspace (#18236)
This PR adds the necessary tool to create a demo workspace with : relevant custom objects and fields, mock data and a real dashboard with graph widgets. It is still a bit under-optimized and slow but it works. This PR also adds an AI tool that allows to see what happens in real time, it navigates the app and waits when necessary. --------- Co-authored-by: Félix Malfait <felix.malfait@gmail.com> |
||
|
Etienne
|
26f0a416a1 |
File storage cleaning (#18381)
- Remove feature flag - Remove legacy methods in file-upload and file-service - Migrate AI Chat to new file management --------- Co-authored-by: Charles Bochet <charles@twenty.com> |
||
|
Abdullah.
|
09e1684300 |
feat: show auto-generated conversation title for AI chat. (#17922)
## Summary Replaces the static "Ask AI" header in the command menu with the conversation’s auto-generated title once it’s set after the first message. ## Changes - **Backend:** Title is generated after the first user message (existing behavior). - **Frontend:** After the first stream completes, we fetch the thread title and sync it to: - `currentAIChatThreadTitleState` (persists across command menu close/reopen) - Command menu page info and navigation stack (so the title survives back navigation) - **Entry points:** Opening Ask AI from the left nav or command center uses the same title resolution (explicit `pageTitle` → current thread title → "Ask AI" fallback). - **Race fix:** Title sync only runs when the thread that finished streaming is still the active thread, so switching threads mid-stream doesn’t overwrite the current thread’s title. --------- Co-authored-by: Félix Malfait <felix@twenty.com> Co-authored-by: Cursor <cursoragent@cursor.com> |
||
|
Félix Malfait
|
3216b634a3 |
feat: improve AI chat - system prompt, tool output, context window display (#17769)
⚠️ **AI-generated PR — not ready for review** ⚠️ cc @FelixMalfait --- ## Changes ### System prompt improvements - Explicit skill-before-tools workflow to prevent the model from calling tools without loading the matching skill first - Data efficiency guidance (default small limits, use filters) - Pluralized `load_skill` → `load_skills` for consistency with `load_tools` ### Token usage reduction - Output serialization layer: strips null/undefined/empty values from tool results - Lowered default `find_*` limit from 100 → 10, max from 1000 → 100 ### System object tool generation - System objects (calendar events, messages, etc.) now generate AI tools - Only workflow-related and favorite-related objects are excluded ### Context window display fix - **Bug**: UI compared cumulative tokens (sum of all turns) against single-request context window → showed 100% after a few turns - **Fix**: Track `conversationSize` (last step's `inputTokens`) which represents the actual conversation history size sent to the model - New `conversationSize` column on thread entity with migration ### Workspace AI instructions - Support for custom workspace-level AI instructions --------- Co-authored-by: claude[bot] <41898282+claude[bot]@users.noreply.github.com> |
||
|
Félix Malfait
|
2e104c8e76 |
feat(ai): add code interpreter for AI data analysis (#16559)
## Summary - Add code interpreter tool that enables AI to execute Python code for data analysis, CSV processing, and chart generation - Support for both local (development) and E2B (sandboxed production) execution drivers - Real-time streaming of stdout/stderr and generated files - Frontend components for displaying code execution results with expandable sections ## Code Quality Improvements - Extract `getMimeType` to shared utility to reduce code duplication between drivers - Fix security issue: escape single quotes/backslashes in E2B driver env variable injection - Add `buildExecutionState` helper to reduce duplicated state object construction - Add `DEFAULT_CODE_INTERPRETER_TIMEOUT_MS` constant for consistency - Fix lingui linting warning and TypeScript theme errors in frontend ## Test Plan - [ ] Test code interpreter with local driver in development - [ ] Test code interpreter with E2B driver in production environment - [ ] Verify streaming output displays correctly in chat UI - [ ] Verify generated files (charts, CSVs) are uploaded and downloadable - [ ] Test file upload flow (CSV, Excel) triggers code interpreter <!-- CURSOR_SUMMARY --> --- > [!NOTE] > Updates generated i18n catalogs for Polish and pseudo-English, adding strings for code execution/output (code interpreter) and various UI messages, with minor text adjustments. > > - **Localization**: > - **Generated catalogs**: Refresh `locales/generated/pl-PL.ts` and `locales/generated/pseudo-en.ts`. > - Add strings for code execution/output (e.g., code, copy code/output, running/waiting states, download files, generated files, Python code execution). > - Include new UI texts (errors, prompts, menus) and minor text corrections. > - No changes to `pt-BR`; other files unchanged functionally. > > <sup>Written by [Cursor Bugbot](https://cursor.com/dashboard?tab=bugbot) for commit befc13d02c21e5a6647bc1aa6daa2a89f60b7ef8. This will update automatically on new commits. Configure [here](https://cursor.com/dashboard?tab=bugbot).</sup> <!-- /CURSOR_SUMMARY --> |
||
|
Félix Malfait
|
4f91b48470 |
feat(ai): add context usage display to AI chat (BREAKING: deploy server first) (#16518)
## Summary - Add a context usage indicator to the AI chat interface inspired by Vercel's AI SDK Context component - Display token consumption, context window utilization percentage, and estimated cost in credits - Show a circular progress ring with percentage, revealing detailed breakdown on hover ## Changes ### Backend - Stream usage metadata (tokens, model config) via `messageMetadata` callback in `agent-chat-streaming.service.ts` - Return model config from `chat-execution.service.ts` - Add usage and model types to `ExtendedUIMessage` metadata ### Frontend - New `ContextUsageProgressRing` component - circular SVG progress indicator - New `AIChatContextUsageButton` component with hover card showing: - Progress bar with used/total tokens - Input/output token counts with credit costs - Total credits consumed - Track cumulative usage in Recoil state (`agentChatUsageState`) - Reset usage when creating new chat thread - Integrate button into `AIChatTab` ## Test plan - [ ] Open AI chat and send a message - [ ] Verify the context usage button appears with percentage - [ ] Hover over the button to see detailed breakdown - [ ] Verify credits are calculated correctly - [ ] Create a new chat thread and verify usage resets to 0 |
||
|
Félix Malfait
|
e7ebf51e50 |
Replace agent handoff system with planning-based router (#16003)
## Overview This PR replaces the dynamic agent handoff system with a more predictable planning-based router that decides upfront how to handle multi-agent coordination. ## Major Changes ### 🔄 Architecture Shift: Handoffs → Planning **Removed:** - `AgentHandoffEntity` and handoff tracking system - `AgentHandoffService` and `AgentHandoffExecutorService` - Dynamic agent-to-agent transfers during execution - Handoff tool generation and description templates **Added:** - `AiRouterService` with two strategies: `simple` (single agent) and `planned` (multi-agent) - `AgentPlanExecutorService` for executing multi-step plans - Plan validation (cycle detection, dependency resolution) - `UnifiedRouterResult` type with discriminated union ### 🤖 New Standard Agents Added two new specialized agents: - **Researcher Agent**: Web search, fact-finding, competitive intelligence - **Code Agent**: TypeScript function generation for serverless workflows ### 🏗️ Router Refactoring (Latest) Split router responsibilities into focused services: - `AiRouterStrategyDeciderService`: Decides simple vs planned strategy - `AiRouterPlanGeneratorService`: Generates and validates execution plans - `AiRouterService`: Coordinates between services (reduced from 426→275 lines) ### ⚙️ Configuration Improvements - Added `outputStrategy` to agent definitions (`direct` vs `synthesize`) - Removed hardcoded special cases for workflow-builder - Added `plannerModel` field to workspace entity - Increased `MAX_STEPS` from 10 to 25 for complex workflows ### 📝 Agent Prompt Refinements Significantly simplified prompts for better clarity: - Workflow Builder: 51→36 lines - Helper: 49→28 lines - Data Manipulator: Enhanced with sorting guidance ### 🔍 Enhanced Debugging - Plan reasoning and step count in data message parts - Router debug info with token usage tracking - Better logging throughout execution pipeline ## Benefits 1. **Simpler Mental Model**: Router decides upfront vs dynamic transfers 2. **Better Predictability**: Users see the plan before execution 3. **Cleaner Architecture**: SRP with focused services 4. **Configuration Over Code**: Agent behavior via config, not hardcoded logic 5. **Plan Validation**: Catches invalid dependencies and cycles ## Migration Notes - Database migration removes `agentHandoff` table - Adds `plannerModel` column to workspace table - No API breaking changes (agent endpoints unchanged) ## Testing - Integration tests updated to remove handoff dependencies - Agent tool test utilities simplified - Plan validation covered by new logic ## Next Steps (Future PRs) - Parallel execution of independent plan steps - Dynamic re-planning based on results - Plan caching for common routing patterns - Error recovery strategies in plan executor |
||
|
Félix Malfait
|
a281f2a773 |
feat: add configurable response format for AI agents (text/JSON) (#15953)
## Summary
This PR adds configurable response format support for AI agents,
allowing them to return either plain text or structured JSON data based
on a defined schema.
## Key Features
### 1. Agent Response Format Configuration
- Added `AgentResponseFormat` type supporting:
- `text`: Returns plain text responses (default)
- `json`: Returns structured JSON based on defined schema
- New `AgentResponseSchema` type moved to `twenty-shared/ai` for sharing
between frontend/backend
### 2. Settings UI
- New `SettingsAgentResponseFormat` component for configuring response
format
- Visual schema builder for defining JSON output structure
- Real-time validation and preview
- Integrated into agent settings tab
### 3. Workflow Integration
- AI Agent workflow action automatically uses agent's configured
response format
- Output schema dynamically generated from agent's response format
- Workflow variable picker shows structured fields for JSON responses
- Backward compatible with existing text-only agents
### 4. Backend Implementation
- Added `convertAgentSchemaToZod` utility to validate JSON responses
- Agent executor service handles both text and JSON generation
- Automatic agent creation/cloning when adding AI agent steps to
workflows
- Unique agent naming with conflict resolution
### 5. Database Migration
- Migration `1763622159656-update-agent-response-format.ts`
- Sets default `responseFormat` to `{"type":"text"}` for existing agents
- Updated all standard agents with proper response format
## Changes by Module
### Frontend (`twenty-front`)
- 🆕 `AgentResponseFormat` type
- 🆕 `SettingsAgentResponseFormat` component
- ✏️ Updated `WorkflowEditActionAiAgent` to support response format
configuration
- 🗑️ Removed deprecated `useAiAgentOutputSchema` hook and
`AiAgentOutputSchema` type
### Backend (`twenty-server`)
- 🆕 `AgentResponseFormat` type in agent entity
- 🆕 `convertAgentSchemaToZod` utility for schema validation
- ✏️ Updated `AiAgentExecutorService` to handle both text and JSON
generation
- ✏️ Updated `WorkflowSchemaWorkspaceService` to generate output schema
from agent config
- ✏️ Enhanced `WorkflowVersionStepOperationsWorkspaceService` with agent
creation/cloning
- 🆕 Agent naming constants for conflict resolution
### Shared (`twenty-shared`)
- 🆕 `AgentResponseSchema` type
- 🆕 `ModelConfiguration` type moved to shared package
- Updated exports in `ai/index.ts`
## Code Quality
- Removed useless comments following code style guidelines
- All linter checks passed
- Type-safe implementation with proper TypeScript types
## Testing
- ✅ Database migration tested
- ✅ Agent creation/cloning in workflows verified
- ✅ Response format switching (text ↔ JSON) validated
- ✅ Backward compatibility with existing agents confirmed
## Migration Notes
- Existing agents will have `responseFormat: {type: 'text'}` set
automatically
- No breaking changes - all existing functionality preserved
- Agents can be updated to use JSON format through settings UI
|
||
|
Félix Malfait
|
f9c61833ec |
Add debug info in AI chat (#15758)
## 🐛 Critical Bug Fix ### Cost Calculation Error (1000x undercharge) - **Fixed**: Cost conversion utility was calculating credits at 1/1000th of actual value - **Before**: `cents * 10` ❌ - **After**: `(cents / 100) * DOLLAR_TO_CREDIT_MULTIPLIER` ✅ - **Impact**: Users were being undercharged by 1000x - Example: 0.75 cents should = 7,500 credits - Bug calculated it as 7.5 credits --- ## 🎯 Code Centralization & DRY ### Unified Cost Calculation - Centralized all cost conversions to use `convertCentsToBillingCredits` utility - Refactored 3 different implementations into 1 single source of truth - Files updated: - `ai-billing.service.ts` - `agent-streaming.service.ts` (2 usages) **Before** (multiple implementations): ```typescript // Wrong implementation const credits = cents * 10; // Verbose implementation const costInDollars = costInCents / 100; const creditsUsed = Math.round(costInDollars * DOLLAR_TO_CREDIT_MULTIPLIER); ``` **After** (unified): ```typescript const creditsUsed = Math.round(convertCentsToBillingCredits(costInCents)); ``` --- ## ✨ UI Component Refactoring ### RoutingDebugDisplay.tsx - **Reduced from 118 lines to 34 lines** (71% reduction) - Extracted `renderTimingRow` helper to eliminate 15 repetitive JSX blocks - Added `formatTokenBreakdown` helper for token display logic - Much easier to add new debug metrics **Before**: 15 nearly-identical blocks of repetitive JSX **After**: Clean, DRY implementation with reusable helpers --- ## 🧹 Code Quality Improvements ### Removed Debug Code - Removed `console.log` accidentally left in `RoutingStatusDisplay.tsx` ### Cleaned Up Comments (18+ removed) Removed redundant comments that stated the obvious: - ❌ "Calculate routing cost if we have token usage" - ❌ "Send the updated routing status with execution metrics to the client" - ❌ "Count tool calls in the response" - ❌ "AI SDK's LanguageModelUsage uses inputTokens/outputTokens" - And 14+ more... Kept meaningful comments: - ✅ "Timing is optional, ignore errors" (explains catch block) - ✅ Type definition grouping comments --- ## 📊 Statistics **Files Modified**: 10 - `convert-cents-to-billing-credits.util.ts` (fixed formula) - `ai-billing.service.ts` (use centralized utility) - `agent-streaming.service.ts` (use utility, remove comments) - `agent-execution.service.ts` (remove comments) - `ai-router.service.ts` (remove comments) - `RoutingStatusDisplay.tsx` (remove debug code) - `RoutingDebugDisplay.tsx` (major refactor) ⭐ - `isDebugModeState.ts` (new file) - `DataMessagePart.ts` (type extensions) - `useClientConfig.ts` (debug mode support) **Impact**: - Lines removed: ~130 (redundant code + comments) - Lines added: ~45 (helper functions) - **Net reduction**: ~85 lines - **Bug fixes**: 1 critical (1000x cost error) - **Centralizations**: 3 locations now using shared utility - **Major refactors**: 1 UI component (71% reduction) --- ## ✅ Verification - ✅ All linter checks pass - ✅ All tests pass (`ai-billing.service.spec.ts` verified) - ✅ No `any` types in affected code - ✅ No TODO/FIXME markers --- ## 🎯 Principles Applied 1. ✅ **Fix Root Causes, Not Symptoms** - Fixed utility function, then used it everywhere 2. ✅ **DRY (Don't Repeat Yourself)** - Centralized cost calculation and UI rendering 3. ✅ **Single Source of Truth** - One place for cost conversion formula 4. ✅ **Code as Documentation** - Removed comments that repeated what code says 5. ✅ **Composability** - Created reusable helper functions 6. ✅ **Type Safety** - Maintained strict typing throughout |
||
|
Abdul Rahman
|
32558673c6 |
feat: Implement AI Router for Dynamic Agent Selection (#15227)
Adds intelligent routing system that automatically selects the best agent for user queries based on conversation context. ### Changes: - Added `routerModel` column to workspace table for configurable router LLM selection - Implemented `RouterService` with conversation history analysis and agent matching logic - Created router settings UI in AI Settings page with model dropdown - Removed agent-specific thread associations - threads are now agent-agnostic - Added real-time routing status notification in chat UI with shimmer effect - Removed automatic default assistant agent creation - Renamed GraphQL operations from agent-specific to generic (e.g., `agentChatThreads` → `chatThreads`) --------- Co-authored-by: Félix Malfait <felix.malfait@gmail.com> Co-authored-by: Félix Malfait <felix@twenty.com> |
||
|
Abdul Rahman
|
2685f4a5b9 |
Restructure agent chat messages with parts-based architecture (#14749)
Co-authored-by: Félix Malfait <felix@twenty.com> |
||
|
Abdul Rahman
|
216d72b5d7 |
AI SDK v5 migration (#14549)
Co-authored-by: Félix Malfait <felix@twenty.com> |
