8a4bcd14453eaf2ca25329ea3d9abb11b02aa06b
89 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Félix Malfait
|
9f4efa57ff |
Expose sent message identifiers in workflow send-email step output (#22520)
## Context
First step toward thread-continuity / follow-up email steps in workflows
(email sequences). The outbound send pipeline already knows the sent
email's RFC-822 Message-ID, the provider thread id, and the persisted
message/thread records — but none of it was surfaced in the send-email
step output, so a later step had no way to reference the email that was
sent.
## What changed
- `saveMessagesWithinTransaction` also returns a `messageExternalId →
messageThreadId` map, and `saveMessagesAndEnqueueContactCreation`
returns the message/thread id maps (both other call sites ignore the
return value)
- `SentMessagePersistenceService.persistSentMessage` and
`SendEmailService.persistSentMessage` return the persisted `{ messageId,
messageThreadId }` (`undefined` when persistence is skipped or fails —
sending still succeeds)
- `SendEmailTool` result now includes `headerMessageId`,
`threadExternalId`, `messageId` and `messageThreadId`
- SEND_EMAIL step output schema (server + frontend) declares
`headerMessageId`/`messageId`/`messageThreadId` so they show up in the
variable picker; DRAFT_EMAIL keeps its success-only schema since draft
creation returns no identifiers yet
This already enables manual thread continuity today: wire
`{{sendEmailStep.headerMessageId}}` into a later email step's
In-Reply-To advanced field — the composer resolves the References chain
and provider thread from it.
## Tests
- New `send-email-tool.spec.ts` covering identifiers in the result,
persistence disabled, and persistence failure
- Extended save-messages spec with the new map, updated frontend
`computeStepOutputSchema` tests
<!-- This is an auto-generated description by cubic. -->
<a
href="https://cubic.dev/pr/twentyhq/twenty/pull/22520?utm_source=github"
target="_blank" rel="noopener noreferrer"
data-no-image-dialog="true"><picture><source
media="(prefers-color-scheme: dark)"
srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source
media="(prefers-color-scheme: light)"
srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img
alt="Review in cubic"
src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a>
<!-- End of auto-generated description by cubic. -->
|
||
|
Etienne
|
56deba351b |
feat(ai): reliable bulk data import via code-interpreter (#22209)
## Summary Makes AI-assisted bulk data import (CSV/Excel/spreadsheets) reliable and token-efficient by letting an entire import run inside a single code-interpreter call, with a persistent sandbox session and server-side bulk helpers. Also includes supporting improvements to attachment handling, upsert reporting, and field-permission error messages. ## Changes ### Code interpreter - **Persistent per-session kernel** in `LocalDriver`: a long-lived Python process per `sessionId` keeps variables, imports, and files alive across calls (matching E2B behavior). Falls back to the existing ephemeral per-call path when no session is provided. CAN BE REMOVED, INTERESTING FOR DEV X - Idle watchdog that self-terminates the kernel, configurable via the new `CODE_INTERPRETER_IDLE_TIMEOUT_MS` config variable; the process also exits on parent shutdown (EOF on control fd). CAN BE REMOVED, INTERESTING FOR DEV X - New `bulk_upsert` and `lookup_by` helpers on the sandbox `twenty` object for idempotent batched writes (≤200/batch) and bounded relation-ID resolution. ### Records - `upsert_many_*` now reports a `created` / `updated` / `total` split in its result and log line (new `isFreshlyCreatedRecord` util). ### AI chat - `replaceUnsupportedFileParts`: user-attached files whose MIME type the model can't handle natively (and that aren't code-interpreter-supported) are downgraded to a descriptive text note instead of being sent as unsupported file parts. Modality→MIME mapping drives native support detection. - Finalize dangling tool parts before `convertToModelMessages` to avoid malformed model messages. - Extracted shared types/constants for code-interpreter file extraction. ### Permissions - Field permission-denied exceptions now include the field name and entity name for easier debugging. ### Skill docs - Added the bulk-import recipe ## To do in following PR - [ ] Skill command migration ## Test plan - [x] Unit tests for `getNativeMimeTypesForModalities` and `replaceUnsupportedFileParts` pass - [x] Run a bulk import (>50 rows) end-to-end through the code interpreter and verify a single sandbox call handles read → resolve relations → upsert → summary - [x] Verify session persistence: define a variable in one call, use it in the next within the same session - [x] Verify the kernel self-terminates after `CODE_INTERPRETER_IDLE_TIMEOUT_MS` - [x] Verify unsupported attachments are replaced with a text note for models lacking the modality - [x] Verify `upsert_many_*` returns correct created/updated counts - [x] Verify field-restricted role triggers a permission error naming the field and entity - [ ] Test with [hotel_business.xlsx](https://github.com/user-attachments/files/29376307/hotel_business.xlsx) and simple "import record" prompt <!-- This is an auto-generated description by cubic. --> <a href="https://cubic.dev/pr/twentyhq/twenty/pull/22209?utm_source=github" target="_blank" rel="noopener noreferrer" data-no-image-dialog="true"><picture><source media="(prefers-color-scheme: dark)" srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source media="(prefers-color-scheme: light)" srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img alt="Review in cubic" src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a> <!-- End of auto-generated description by cubic. --> |
||
|
Etienne
|
02d62c4175 |
fix(ai-tools): make navigate_app tool schema a valid object root for direct model binding (#22284)
## Problem
Using an AI Agent step in a workflow fails with:
> Invalid schema for function 'navigate_app': schema must be a JSON
Schema of 'type: "object"', got 'type: "None"'.
## Root cause
The `navigate_app` tool declared its `inputSchema` as a top-level
`z.discriminatedUnion('type', [...])`. When serialized via
`toToolJsonSchema`, a discriminated union produces a **root-level
`anyOf`** with no top-level `"type"`:
```json
{ "anyOf": [ { "type": "object", ... }, ... ] }
<!-- This is an auto-generated description by cubic. -->
<a href="https://cubic.dev/pr/twentyhq/twenty/pull/22284?utm_source=github" target="_blank" rel="noopener noreferrer" data-no-image-dialog="true"><picture><source media="(prefers-color-scheme: dark)" srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source media="(prefers-color-scheme: light)" srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img alt="Review in cubic" src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a>
<!-- End of auto-generated description by cubic. -->
|
||
|
Félix Malfait
|
0e22ae0521 |
feat: create calendar events on Google and Microsoft accounts (#22231)
## Context Twenty can import calendar events and send emails, but cannot create calendar events. This adds calendar event creation on connected **Google** and **Microsoft** accounts, mirroring the existing email-send architecture (`message-outbound-manager`). ## What it adds The capability is exposed three ways, all backed by the same composer → driver → persist pipeline: - **GraphQL mutation** `createCalendarEvent` (metadata API) - **AI agent tool** `create_calendar_event` (flows to MCP automatically), gated by a new `CREATE_CALENDAR_EVENT_TOOL` permission flag - **Workflow builder node** "Create Calendar Event" in the **Core** section, with a full settings form (variable interpolation supported) CalDAV/IMAP is intentionally out of scope for now (different long pole). ## Design notes - **Reuse over reinvention** — the created event is run through the existing inbound formatters (`formatGoogleCalendarEvents` / `formatMicrosoftCalendarEvents`) and persisted immediately via the existing `CalendarSaveEventsService`, so it appears in Twenty right away and is reconciled by the next provider sync (dedup on external id). Persistence is best-effort. - **OAuth scopes** — Google already requests `calendar.events` (read+write), so no change there. Microsoft moves `Calendars.Read` → `Calendars.ReadWrite`; existing Microsoft accounts must re-consent (surfaced as a clear "reconnect" error via a missing-scope check). - **Deliberate invitation semantics** — `sendInvitations` is off by default. When off, the event is created with **no attendees** on either provider, so creating an event never silently emails external people. When on, attendees are attached and notified (Google `sendUpdates: all`, Microsoft's default). This sidesteps Microsoft Graph having no per-request suppression. - **Timezone correctness** — Microsoft Graph interprets `dateTime` as wall-clock in the supplied `timeZone` and ignores the offset, so the absolute instant is converted to its wall-clock form before sending (Google honors the offset directly). Both providers end up scheduling the same instant. - **Conferencing** — optional Google Meet (`conferenceData.createRequest`, with a follow-up `events.get` to resolve the async link) / Microsoft Teams (`isOnlineMeeting`). - Attendees are a comma-separated string everywhere (tool input, GraphQL DTO, workflow input), consistent with `send_email` recipients; the composer parses to its internal list. ## Test plan - **Unit**: 45 tests covering the composer (validation, all-day boundaries, offset enforcement, timezone, scope checks, default-account resolution), both provider drivers, the dispatcher, and the workflow step-log builder. - **Integration**: `createCalendarEvent` on the `/metadata` API fails closed with a structured error for a non-existent account (the auth/ownership/validation path that doesn't require provider mocking). - **Manual**: verified the workflow node appears in the Core section, the settings form renders and round-trips (edit → autosave → reload), and the live mutation returns a structured failure for a bogus account. ## Open question for reviewers The metadata mutation `createCalendarEvent` shares a name with the core schema's auto-generated `createCalendarEvent(data:)` CRUD mutation for the CalendarEvent object — they live on different endpoints (`/metadata` vs `/graphql`) so there's no runtime conflict, but it's a potential point of confusion for API consumers. Happy to rename (e.g. `createCalendarEventOnConnectedAccount`) if preferred. ## Out of scope / follow-ups - CalDAV/IMAP support - Event update/delete and recurrence - Existing Microsoft accounts need re-consent for the widened scope <!-- This is an auto-generated description by cubic. --> <a href="https://cubic.dev/pr/twentyhq/twenty/pull/22231?utm_source=github" target="_blank" rel="noopener noreferrer" data-no-image-dialog="true"><picture><source media="(prefers-color-scheme: dark)" srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source media="(prefers-color-scheme: light)" srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img alt="Review in cubic" src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a> <!-- End of auto-generated description by cubic. --> --------- Co-authored-by: neo773 <neo773@protonmail.com> |
||
|
Thomas des Francs
|
eedd838189 |
Fix threaded draft email replies (#22175)
## Summary Fixes Gmail and Microsoft draft replies so workflow-created drafts stay attached to the existing provider thread. Fixes twentyhq/core-team-issues#2597. ## Root cause The email composer already resolved `threadExternalId` and `references` from `inReplyTo`, but `DraftEmailTool` only forwarded `inReplyTo` to the outbound draft service. Gmail therefore created a raw draft without `message.threadId`, which lets the draft appear as a standalone compose instead of an inline thread reply. For Microsoft, the draft path used Graph `createReply`, but parent lookup filtered on a URL-encoded `internetMessageId`. That can miss the parent message and fall back to creating a new draft message instead of a reply draft. ## Changes - Forward `threadExternalId` and `references` from `DraftEmailTool` to outbound draft creation. - Set Gmail draft `message.threadId` when `threadExternalId` is available. - Make Microsoft parent lookup use Graph request query builders with OData string escaping, so `createReply` is reached reliably. - Add targeted Jest coverage for the Draft Email tool, Gmail draft threading, and Microsoft reply-draft creation. ## Validation - `NX_DAEMON=false /Users/thomascolasdesfrancs/.cache/codex-runtimes/codex-primary-runtime/dependencies/node/bin/node ../../node_modules/nx/dist/bin/nx.js jest twenty-server -- --runTestsByPath src/engine/core-modules/tool/tools/email-tool/__tests__/draft-email-tool.spec.ts src/modules/messaging/message-outbound-manager/drivers/gmail/services/__tests__/gmail-message-outbound.service.spec.ts src/modules/messaging/message-outbound-manager/drivers/microsoft/services/__tests__/microsoft-message-outbound.service.spec.ts --runInBand` - `NX_DAEMON=false /Users/thomascolasdesfrancs/.cache/codex-runtimes/codex-primary-runtime/dependencies/node/bin/node ./node_modules/nx/dist/bin/nx.js lint:diff-with-main twenty-server` <!-- This is an auto-generated description by cubic. --> <a href="https://cubic.dev/pr/twentyhq/twenty/pull/22175?utm_source=github" target="_blank" rel="noopener noreferrer" data-no-image-dialog="true"><picture><source media="(prefers-color-scheme: dark)" srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source media="(prefers-color-scheme: light)" srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img alt="Review in cubic" src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a> <!-- End of auto-generated description by cubic. --> --------- Co-authored-by: claude[bot] <41898282+claude[bot]@users.noreply.github.com> Co-authored-by: Félix Malfait <FelixMalfait@users.noreply.github.com> |
||
|
Etienne
|
5ca41d55fb |
feat(ai): humanize tool-call (#21976)
# Humanize tool-call labels cc: https://github.com/twentyhq/twenty/pull/21462 ## Preview <img width="459" height="156" alt="Screenshot 2026-06-22 at 19 13 11" src="https://github.com/user-attachments/assets/e7a2f5f5-cd09-4ec6-920b-5eb16b98285c" /> <img width="461" height="156" alt="Screenshot 2026-06-22 at 19 14 54" src="https://github.com/user-attachments/assets/c2114d2e-2aa8-499a-9801-68e3bb7c45f8" /> <img width="461" height="505" alt="Screenshot 2026-06-22 at 19 15 01" src="https://github.com/user-attachments/assets/ee9ca5d0-8e79-4c63-a2ff-ed5e359a9a9c" /> ## Why In the AI chat, tool steps were displayed using raw tool identifiers (`find_many_companies`, `create_one_task`, `send_email`...) and labels were partially reconstructed/humanized on the frontend. This was hard to localize and inconsistent across tool categories. This PR makes the **backend the single source of truth for human-readable, localized tool labels**, exposes them through `getToolIndex`, and reduces the frontend to a thin resolver that picks the right label for the current status (in-progress / completed). ## What changed ### Backend - `ToolIndexEntry` (and the `getToolIndex` GraphQL DTO) now carry `label`, `inProgressLabel?`, `completedLabel?`. - New `getCrudToolLabels(operation, objectLabel, i18nService, locale)` builds CRUD labels from a verb table (Search / Find / Group / Create / Update / Upsert / Delete × imperative / in-progress / completed) + the (translated, lowercased) object label. - New `translate-tool-label.util.ts` translates a source label via `I18nService` (`generateMessageId` → fallback to source when no translation exists). - Action tools: labels extracted to the `ACTION_TOOL_LABELS` constant (`msg` + `i18nLabel`) and translated in `ActionToolProvider.buildDescriptor`. - Logic-function tools use the function name as label; `toolSetToDescriptors` (workflow / view / metadata / dashboard) accepts an optional `labels` map and falls back to a humanized tool name. - Labels are localized server-side using the request locale (`@RequestLocale` → `buildToolIndex` → `context.locale`, threaded through `ToolContext` / `ToolProviderContext`). - `code_interpreter` schema now asks the model for `loadingMessage` (present tense) and `completedMessage` (past tense), so its status text is model-generated. - Removed the old generic `loadingMessage` injection mechanism (`wrap-tool-for-execution.util.ts` deleted; `wrapJsonSchemaForExecution` / `stripLoadingMessage` no longer wrap every tool). ### Frontend - New `useToolLabelMap()` hook builds a `Map<name, { label, inProgressLabel, completedLabel }>` from `getToolIndex`. - `getToolDisplayMessage` → `resolveToolDisplayMessage({ input, toolName, isFinished, labelMap, output })`: a small resolver registry keyed by tool name (`execute_tool`, `web_search`, `learn_tools`, `load_skills`, `code_interpreter`, default). - Default resolver prefers backend `completedLabel` / `inProgressLabel`, falling back to `Ran X` / `Running X`. - `learn_tools` / `load_skills` resolve their inner tool/skill names to labels (label map → tool output labels via `getToolOutputLabelEntries` → raw name). - `code_interpreter` step is now expandable to show the code even while running. ## How tool labelling flows (BE → FE) ```text BACKEND ┌───────────────────────────────────────────────────────────────────────────┐ │ Tool providers (per category) → ToolIndexEntry │ │ │ │ DatabaseToolProvider │ │ getCrudToolLabels(operation, object.labelPlural/Singular, i18n, locale) │ │ verb table (Search/Create/Update/Delete…) + translateToolLabel(object) │ │ → { label, inProgressLabel, completedLabel } │ │ │ │ ActionToolProvider │ │ ACTION_TOOL_LABELS[toolId] (msg) → translateToolLabel(…, locale) │ │ → { label, inProgressLabel?, completedLabel? } │ │ │ │ LogicFunctionToolProvider → label = logicFunction.name │ │ toolSetToDescriptors → label = labels[name] ?? humanize(name) │ │ (workflow / view / metadata / dashboard) │ └───────────────────────────────────────────────────────────────────────────┘ │ ▼ ┌───────────────────────────────────────────────────────────────────────────┐ │ GraphQL Query getToolIndex : [ToolIndexEntry] │ │ { name, label, inProgressLabel, completedLabel, description, │ │ category, objectName, icon } │ └───────────────────────────────────────────────────────────────────────────┘ │ ▼ FRONTEND ─ resolve the right label for the current status ┌───────────────────────────────────────────────────────────────────────────┐ │ useGetToolIndex() → useToolLabelMap() │ │ Map<name, { label, inProgressLabel?, completedLabel? }> │ └───────────────────────────────────────────────────────────────────────────┘ │ ▼ ┌───────────────────────────────────────────────────────────────────────────┐ │ resolveToolDisplayMessage({ input, toolName, isFinished, labelMap, output })│ │ │ │ TOOL_LABEL_RESOLVERS[toolName] ?? defaultResolver │ │ ├─ execute_tool → unwrap { toolName, arguments } then re-resolve │ │ ├─ web_search → "Searching/Searched the web for <query>" │ │ ├─ learn_tools → "Learning/Learned <labels>" │ │ ├─ load_skills → "Loading/Loaded <labels>" │ │ │ inner names resolved via: labelMap → output labels → raw name │ │ ├─ code_interpreter → model's loadingMessage / completedMessage │ │ └─ default → isFinished │ │ ? completedLabel ?? "Ran <label>" │ │ : inProgressLabel ?? "Running <label>" │ └───────────────────────────────────────────────────────────────────────────┘ │ ▼ Rendered by ThinkingStepsDisplay / ToolStepRenderer ``` ## Localization notes - Standard object labels and action/CRUD verbs are translated server-side via `I18nService` using the requester's locale. - Custom object labels are not translated unless a workspace custom translation exists (matched by `generateMessageId`); otherwise the source label is used as-is. ## Tests - **FE:** `resolveToolDisplayMessage` / `getToolOutputLabelEntries` (status selection, inner-name resolution, `code_interpreter` model labels, fallbacks). - **BE:** `toolSetToDescriptors` (label map + humanized fallback) and `database-tool.provider` label generation. <!-- This is an auto-generated description by cubic. --> <a href="https://cubic.dev/pr/twentyhq/twenty/pull/21976?utm_source=github" target="_blank" rel="noopener noreferrer" data-no-image-dialog="true"><picture><source media="(prefers-color-scheme: dark)" srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source media="(prefers-color-scheme: light)" srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img alt="Review in cubic" src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a> <!-- End of auto-generated description by cubic. --> |
||
|
Etienne
|
0f4c4e69a9 |
fix(ai-tool): make search_output a raw-text occurrence search (#22034)
## Summary
`search_output` (the spilled-output navigation tool) was built around a
JSON-centric, line-based model that breaks for the data it actually
receives. Spilled outputs are written as compact
`JSON.stringify(output)` (single line, escaped newlines), so the tool's
line-by-line matching collapsed to at most one match, and its schema
described searching "the indented JSON representation" even though it
falls back to raw text for non-JSON. It also ran arbitrary,
model-supplied regexes through the native engine with no ReDoS
protection.
This reworks the tool into a `grep -o` style search over the raw file
bytes: it finds every occurrence of a pattern regardless of newlines and
returns a character window around each hit. It works uniformly for
compact/pretty JSON, CSV, HTML, and plain text.
## Changes
- **Occurrence-based matching** (`search-output.util.ts`): search the
raw content for every match via a global-regex `exec` loop (with a
zero-width-match guard), bounded by `offset + maxMatches`. Results are
now `{ charOffset, match, context }` with a character window around each
occurrence and a centered-ellipsis cap for very long single matches. The
line model (`split`, line numbers, line context) is removed.
- **ReDoS hardening**: matching now uses `re2` (already a dependency)
with the global flag, guaranteeing linear-time matching. Unsupported
regex features (lookahead/backreferences) and invalid patterns fall back
to escaped-literal search instead of throwing.
- **No more reserialization** (`search-output-tool.ts`): the
`JSON.stringify(JSON.parse(...))` round-trip is gone; the tool searches
the exact bytes on disk, so there is no coordinate divergence with
`extract_json_paths`.
- **API** (`search-output-tool.schema.ts`): `contextLines` →
`contextChars` (default 100, max 2000); honest descriptions reflecting
raw-text occurrence search and the regex-or-literal fallback. The result
message reports occurrence counts.
- **Cleanup**: removed unused constants
(`default-search-output-context-lines`,
`search-output-max-line-length`); added
`default-search-output-context-chars` and
`search-output-max-match-length`.
`extract_json_paths` and the spill service are untouched.
## Tradeoff
Results use character offsets/windows rather than line numbers and line
context. For an LLM extracting values from a spilled blob this is more
robust (works on single-line content); the cost is no line-based context
for genuinely line-structured content.
## Test plan
- [x] `search-output.util.spec.ts` rewritten for occurrence semantics:
multiple hits on a single newline-free line, zero-width-pattern
termination, catastrophic-backtracking pattern stays fast (RE2),
lookahead/invalid-regex literal fallback, char-window clipping, offset
pagination, long-match truncation. 12/12 pass.
- [x] `npx nx typecheck twenty-server` clean.
- [x] `npx nx lint:diff-with-main twenty-server` clean (lint + format).
## Deploy note
`re2` is a native addon. It was declared in `package.json` but never
imported/built before this PR, so its binary may be absent in some
environments (local install required `npm rebuild re2`). Confirm the
install/build pipeline (CI, Docker images) compiles native modules so
the tool doesn't throw `Cannot find module 're2.node'` at runtime.
<!-- This is an auto-generated description by cubic. -->
<a
href="https://cubic.dev/pr/twentyhq/twenty/pull/22034?utm_source=github"
target="_blank" rel="noopener noreferrer"
data-no-image-dialog="true"><picture><source
media="(prefers-color-scheme: dark)"
srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source
media="(prefers-color-scheme: light)"
srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img
alt="Review in cubic"
src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a>
<!-- End of auto-generated description by cubic. -->
|
||
|
Etienne
|
7b45380777 |
feat(ai): large tool output handling + navigation tools (#21982)
## Summary
Large tool outputs (e.g. a workflow run that serializes to ~70k tokens)
blow the chat context budget and force per-tool "raw" variants. This PR
handles oversized outputs generically in one place:
1. **Producer:** when a tool result exceeds a byte budget, it is spilled
to a `FileFolder.AgentChat` file and replaced with a compact `{ spilled,
outputRef, shape, hint }` envelope.
2. **Consumer:** two bounded, in-server navigation tools —
`extract_json_path` and `search_output` — let the model dig into the
spilled file by `fileId` without spinning up `code_interpreter`.
Together they add a fast, auditable middle tier between "truncated
inline preview" and "full code_interpreter relay," and enable an
enterprise "restricted" mode (spill + navigation, no sandbox).
## Data flow
```mermaid
flowchart TD
exec["resolveAndExecute / hydrateToolSet closure"] --> compact[compactToolOutput]
compact --> enabled{"spillLargeOutput enabled? (chat only)"}
enabled -->|no| inlineRaw["inline raw (MCP, workflow, sandbox bridge)"]
enabled -->|yes| size{"bytes > MAX_INLINE_TOOL_OUTPUT_BYTES?"}
size -->|no| inline["inline result"]
size -->|yes| skeleton["jsonShapeSkeleton + largeOutputHint"]
skeleton --> write["writeFile(AgentChat)"]
write --> envelope["return { spilled, outputRef, shape, hint }"]
envelope --> model[Model]
model --> nav["extract_json_path / search_output / code_interpreter (by fileId)"]
```
## Part 1 — Navigation tools (consumer)
- `extract_json_path`: extracts a sub-tree from a spilled JSON file by a
JSONPath-lite expression (dot/bracket access, array slicing,
single-level wildcard), with `maxItems`/`maxDepth` bounding. No filters
or recursive descent — those belong to `code_interpreter`.
- `search_output`: grep-like line search with context lines and
stateless `offset` pagination (`{ matches, totalMatches, hasMore }`).
- Both read from `FileFolder.AgentChat` by `fileId`, enforce their own
output byte cap, and are registered in `ActionToolProvider` (always
available; read-only).
## Part 2 — Spill producer
- Spilling slots in right after the existing `compactToolOutput` step at
the two seams in `ToolRegistryService` (`resolveAndExecute` and the
`hydrateToolSet` execute closure).
- `ToolOutputSpillService.spillIfTooLarge()` measures
`Buffer.byteLength`; over `MAX_INLINE_TOOL_OUTPUT_BYTES` (16 KB ≈ 4k
tokens) it writes the full payload and returns the envelope. Spill
failures never block the call (inline + warning).
- `jsonShapeSkeleton` computes a bounded structural map (depth 4, arrays
as `"array[N] of <type>"`, id-keyed maps collapsed, long leaves as size
markers, hard-capped at 1024 bytes) so the model knows the key paths in
one pass.
- Optional per-tool `largeOutputHint` (on the `Tool` type, threaded via
the descriptor) is used as the hint when present, else a generic hint.
The `shape` is always computed generically.
## Surfaces
Spilling is an opt-in flag (`spillLargeOutput`) mirroring
`compactOutput`:
| Surface | `spillLargeOutput` | Behavior |
| --- | --- | --- |
| AI chat / agent | `true` (in `chat-execution.service.ts`) | Spill on;
nav tools + `code_interpreter` in catalog |
| External MCP clients | unset | Raw output |
| Workflow agents | unset | Raw output |
| `code_interpreter` sandbox bridge | unset (it's an MCP call) | Raw
output |
The sandbox bridge inherits "no spill" for free via the MCP path — no
header sniffing, no `ToolContext.source` field.
## Design constraints (anti-micro-OS)
Exactly two navigation tools, no composition/piping, read-only, bounded
output. The boundary is: expressible as a single path lookup or text
search → nav tool; aggregation/correlation/transform →
`code_interpreter`.
## Notes / deviations from the plan
- `jsonShapeSkeleton` and `ToolOutputSpillService` live under the `tool`
module (not `tool-provider/output-transforms`) to avoid a `tool →
tool-provider` import cycle.
- Spill files use `{ isTemporaryFile: false, toDelete: false }` (same as
`code_interpreter`); `isTemporaryFile` here means files-field promotion,
not a TTL.
## Test plan
- [x] `extract-json-path` + `search-output` util unit tests (23 cases)
- [x] `jsonShapeSkeleton` unit tests (6) and `ToolOutputSpillService`
unit tests (4)
- [x] oxlint + oxfmt clean on changed files; `twenty-server` typecheck
clean (pre-existing unrelated errors aside)
- [ ] Manual: trigger an oversized tool result in chat, confirm the
envelope is returned and `extract_json_path` / `search_output` read the
spilled file by `fileId`
## Why no automated e2e
Spilling is chat-only and the chat path runs a live model, so the
black-box MCP integration harness can't deterministically trigger a
spill (MCP intentionally doesn't spill). The seam is small, explicit
flag-threading mirrored on `compactOutput`, covered by the unit suites.
<!-- This is an auto-generated description by cubic. -->
<a
href="https://cubic.dev/pr/twentyhq/twenty/pull/21982?utm_source=github"
target="_blank" rel="noopener noreferrer"
data-no-image-dialog="true"><picture><source
media="(prefers-color-scheme: dark)"
srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source
media="(prefers-color-scheme: light)"
srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img
alt="Review in cubic"
src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a>
<!-- End of auto-generated description by cubic. -->
|
||
|
Thomas Trompette
|
8130fa1c45 |
feat(workflow): use workspace member as variable sender for emails (#21582)
## Summary
Lets the email workflow node sender be driven by a variable: the
connected-account field accepts a `{{variable}}`, and the backend
resolves it to a connected account at run time.
<img width="436" height="195" alt="Capture d’écran 2026-06-16 à 11 59
27"
src="https://github.com/user-attachments/assets/18eee21e-aed6-4447-9bf4-5cb0e2cfc371"
/>
### Email sender by variable
- The connected-account field now accepts a `{{variable}}` via the
variable picker (uses `FormSelectFieldInput` with
`WorkflowVariablePicker`), with a hint to pick a connected account or
set a workspace member as a variable.
- The email workflow action resolves the stored sender value explicitly:
if it is a `workspaceMemberId` (a UUID matching a workspace member), it
resolves that member's first connected account; otherwise the value is
used directly as a `connectedAccountId`.
- Resolution lives in `EmailWorkflowActionBase` and applies to both
`SEND_EMAIL` and `DRAFT_EMAIL`. If a matching member has no connected
account, the run fails fast with a clear message (no silent fallback).
- `DRAFT_EMAIL` also fails fast when the resolved connected account is
missing the required OAuth scopes (`gmail.compose` / `Mail.Send`), via a
server-side `getMissingDraftEmailScopes` util that mirrors the front-end
check.
- Existing workflows with a hardcoded `connectedAccountId` keep working
unchanged (no migration needed).
> Note: exposing the running workspace member as a manual-trigger
variable (`_metadata.workspaceMemberId`) is split into a follow-up PR.
## Test plan
- [x] Backend unit tests for `draft-email-tool`,
`get-missing-draft-email-scopes`, and the `send-email` / `draft-email`
workflow actions (incl. workspace-member sender resolution)
- [x] Lints clean on all changed files
- [x] Manual: configure an email node with a workspace-member variable
sender and confirm it resolves and drafts/sends
- [x] Manual: confirm a member lacking compose permission fails the run
with the permission message
<!-- This is an auto-generated description by cubic. -->
<a
href="https://cubic.dev/pr/twentyhq/twenty/pull/21582?utm_source=github"
target="_blank" rel="noopener noreferrer"
data-no-image-dialog="true"><picture><source
media="(prefers-color-scheme: dark)"
srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source
media="(prefers-color-scheme: light)"
srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img
alt="Review in cubic"
src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a>
<!-- End of auto-generated description by cubic. -->
---
### Update — scoped to Draft Email only
The sender variable picker is now exposed **only on the Draft Email
node**. The Send Email node keeps a plain account select (no variable
picker, no variable hint) until we enable it there in a follow-up.
Backend resolution still lives in `EmailWorkflowActionBase` and remains
generic, so enabling the picker for Send Email later requires no backend
change.
|
||
|
Félix Malfait
|
bb6da7b7d1 |
feat(code-interpreter): reuse a warm sandbox per conversation (E2B) (#21664)
## What
The E2B code-interpreter driver created a **fresh sandbox on every
execution** and killed it in `finally`, so every call in a conversation
paid full cold-start and started blank. This PR keeps **one warm sandbox
per conversation** and, on idle, **pauses** it rather than killing it.
## How
- **Discovery without a registry:** the sandbox is tagged with the chat
`threadId` (scoped `workspaceId:threadId`) via E2B **metadata**, found
with `Sandbox.list({ query: { state: ['running','paused'], metadata }
})` and resumed with `Sandbox.connect()` (which auto-resumes a paused
sandbox). E2B is the source of truth — no Redis/DB mapping.
- **Pause/resume (E2B 2.x):** session sandboxes are created with
`lifecycle: { onTimeout: 'pause', autoResume: true }`. When idle they
**pause** — compute billing stops, filesystem **and** kernel/memory
state are preserved — and resume in ~1s on the next call. This replaces
the earlier keepalive approach.
- **No premature pause mid-run:** the sandbox is kept alive for
`max(execution timeout, idle window)`, so a long execution is never
paused underneath itself.
- **Tenant isolation:** discovery filters by the `twentySessionId` tag
and **re-checks it client-side**, so a loose server-side match can never
hand one conversation's warm sandbox (with its files, kernel state,
token) to another.
- **Concurrency:** executions sharing a session are serialized
in-process (one active stream per thread, run as a single job — the chat
resolver queues concurrent messages), so parallel tool calls can't race
the shared kernel.
- **Output isolation:** `/home/user/output` is reset at the start of
each reused run, so a call only returns the artifacts it actually
produced; durable state lives elsewhere and persists.
## SDK upgrade
`@e2b/code-interpreter` **`^1.0.4` → `^2.6.0`** (pulls `e2b@2.x`). The
typed pause/resume API, `lifecycle`, and the `state`/`metadata` list
filter only exist in the 2.x line; 1.x exposed them only as untyped
OpenAPI internals. `Sandbox.list()` is now a paginator (handled).
## Config
| Var | Default | Purpose |
|---|---|---|
| `CODE_INTERPRETER_TIMEOUT_MS` | `300000` | Max single-execution
duration. |
| `CODE_INTERPRETER_IDLE_TIMEOUT_MS` | `300000` | Idle window before the
warm sandbox auto-pauses. |
Reuse is always-on when a session id is present (chat path). The
workflow-agent path and the dev-only `LocalDriver` are unaffected.
## ⚠️ Open item before merge: paused-sandbox GC
E2B retains paused sandboxes **indefinitely** (no TTL). Unlike the old
keepalive path (which auto-killed on idle), pause means a conversation's
sandbox persists after the chat ends — so without garbage collection,
paused sandboxes accumulate (≈ one per historical conversation) and
consume storage. A GC policy is required; the approach + retention
window are being decided (see PR discussion). Also: the E2B runtime path
can't run in CI, so this still needs a **live smoke test** (reuse hit,
idle→pause, resume) and confirmation of paused-storage pricing before
rollout.
## Tests / checks
- Resolver unit tests (`getOrCreateSessionSandbox`): reuse+extend,
create-when-absent, duplicate reaping, connect-failure fallback,
keep-first-connectable-when-earlier-dead, **ignore cross-tenant
metadata**, and **kill-on-timeout-refresh-failure**.
- `nx typecheck twenty-server` (against e2b 2.x), `oxlint --type-aware`,
`oxfmt --check` all clean.
---------
Co-authored-by: Claude <noreply@anthropic.com>
|
||
|
neo773
|
1ad919955a |
Support variables file email attachment (#21613)
<!-- This is an auto-generated description by cubic. --> <a href="https://cubic.dev/pr/twentyhq/twenty/pull/21613?utm_source=github" target="_blank" rel="noopener noreferrer" data-no-image-dialog="true"><picture><source media="(prefers-color-scheme: dark)" srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source media="(prefers-color-scheme: light)" srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img alt="Review in cubic" src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a> <!-- End of auto-generated description by cubic. --> |
||
|
Charles Bochet
|
7c0136b97b |
feat(deps): migrate frontend to React 19 (#21531)
## What Migrates the frontend stack from **React 18.3 → 19.2**. The website, sdk, companion and emails packages were already on React 19; this brings the remaining holdouts (`twenty-front`, `twenty-ui`, `twenty-ui-deprecated`, `twenty-front-component-renderer`) and `twenty-server`'s email rendering onto 19, and pins a single React version repo-wide. ## Why React 18.x is now the legacy line. Staying current keeps us on the patched/maintained branch and unblocks downstream library majors (react-router 7, mantine 9, etc.) that require React 19 peers. ## Dependency bumps (required by React 19 peers / removed APIs) | Package | From | To | Reason | |---|---|---|---| | react / react-dom | 18.3.1 | 19.2.3 | core | | @hello-pangea/dnd | 16 | 18 | peer `^18 \|\| ^19` | | react-datepicker | 6 | 9 | v<7 used removed `findDOMNode`; drops `@types/react-datepicker` | | react-data-grid | beta.13 | beta.59 | peer `^19.2`; new render API | | graphiql (+ @graphiql/react, plugin-explorer) | 3 / 0.23 / 1 | 5 / 0.37 / 5.1 | peer `^18 \|\| ^19` | | react-helmet-async | 1.3 | **@dr.pogodin/react-helmet** 3.2 | upstream caps peer at `^18`; drop-in React 19 fork | A `resolutions` pin enforces a single React (19.2.3) + `@types/react` (19.2.14) across the monorepo to avoid duplicate copies / type-identity splits. Versions are the aged lockfile patches (clears the `npmMinimalAgeGate`). ## Code changes - **Global `JSX` shim** (`react-jsx-global.d.ts` per package): React 19 moved the `JSX` namespace under `React.JSX`; several deps' published types (notably `@linaria/react`'s `styled.d.ts`, which types every `styled.x` via `keyof JSX.IntrinsicElements`) still reference the global namespace. Without the shim, every styled component degrades to `any` props. - **Ref nullability**: `useRef<T>(null)` now returns `RefObject<T | null>`; widened consumer prop/hook ref types accordingly (incl. the shared `useListenClickOutside`). - **react-datepicker v9**: `onChange`/`onSelect` accept `Date | null`, `calendarStartDay` typing, `ReactDatePickerProps`→`DatePickerProps`, relaxed the dynamic `selectsMultiple` discriminated union. - **react-data-grid beta.59**: `formatter`→`renderCell`, `editor`→`renderEditCell`, `headerRenderer`→`renderHeaderCell`, `components`→`renderers`, `onRowClick`→`onCellClick`, object-shaped `useRowSelection`, Set-based selection. - **dnd style cast**: `@radix-ui/react-popper` augments `CSSProperties` with a `--radix-*` index signature that dnd's closed `DraggingStyle` doesn't satisfy → cast at the spread. ## Status / testing - ✅ `typecheck` green: twenty-front, twenty-ui, twenty-ui-deprecated, twenty-front-component-renderer, twenty-server - ⏳ build / lint / unit tests / storybook+argos / runtime smoke-test in progress Draft until local + CI verification completes. Notable behavior to QA manually: spreadsheet import (data-grid), date pickers, drag-and-drop boards/lists, GraphQL playground, page titles/favicon. <!-- This is an auto-generated description by cubic. --> <a href="https://cubic.dev/pr/twentyhq/twenty/pull/21531?utm_source=github" target="_blank" rel="noopener noreferrer" data-no-image-dialog="true"><picture><source media="(prefers-color-scheme: dark)" srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source media="(prefers-color-scheme: light)" srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img alt="Review in cubic" src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a> <!-- End of auto-generated description by cubic. --> |
||
|
neo773
|
5d892bdfd0 |
[WIP] Feat/marketing emails (#21173)
Marketing/campaign emails on top of the emailing-domain (SES) feature:
send a broadcast to a hand-picked list, with per-customer-domain
unsubscribe links and opt-out-only **unsubscribe topics**.
## Model
Standard objects (workspace schema, flat-metadata):
- `messageCampaign` — a campaign send (subject, body template, from
address, status, list, optional unsubscribe topic).
- `messageList` + `messageListMember` — the hand-picked audience (person
↔ list join). A campaign's recipients are its list's members; everyone
is sendable unless suppressed.
Core entities (`core` schema, workspace-scoped — readable by the public
unsubscribe flow without a workspace context):
- `unsubscribeTopic` — an opt-out-only category (name, description,
visibility). There is no opt-in subscription state.
- `messageSuppression` — the single consent store: a row with
`unsubscribeTopicId` NULL is a global block; a row with an
`unsubscribeTopicId` and reason `UNSUBSCRIBE` is a per-topic opt-out.
Two partial unique indexes dedupe global vs per-topic rows (Postgres
treats NULLs as distinct).
- `emailingDomain` — the workspace's SES sending domain,
auto-provisioned when an email channel is added (and cleaned up when its
last channel is removed), with verification status + DNS records.
Campaign messages reuse the existing `message` / `messageThread` /
`messageParticipant` model — one outbound `message` per recipient with a
`deliveryStatus` state machine.
## Sending
- `sendMessageCampaign` resolves the audience **under the caller's
permissions**, creates the campaign, and enqueues a single fan-out job
(the request never materializes per-recipient rows or jobs).
- The fan-out job materializes one QUEUED message per recipient
(deterministic ids → idempotent re-runs, reconciles crash-orphaned rows)
and fans out per-recipient send jobs carrying **only ids**.
- Each send job renders per-recipient `{{variable}}` merge fields and
sends via `EmailingDomainSenderService`, which applies suppression
(global + per-topic) and the unsubscribe footer/headers. Suppressed
recipients are recorded `SKIPPED`.
- The campaign finalizes `SENT`, or `SENT_WITH_ERRORS` if any recipient
terminally failed.
- `previewMessageCampaignAudience` returns a pre-send breakdown (total /
without-email / duplicate / globally-unsubscribed / topic-unsubscribed /
sendable), shown as a hint under the composer pickers.
## Unsubscribe
- Encrypted (AES-256-GCM) token carrying workspaceId, address, optional
`unsubscribeTopicId`, `issuedAt`, and a `preview` flag.
- One-click POST (RFC 8058) + `mailto:` — topic-scoped when the token
carries a topic, global otherwise.
- Preferences page: a checkbox per visible topic (checked = still
receiving); submitting creates per-topic opt-outs for unchecked topics
and lifts re-checked ones (UNSUBSCRIBE only — never
`BOUNCE`/`COMPLAINT`, never a global block).
- A **Preview** action in settings opens the live page via a
preview-claim token; opt-out POSTs are no-ops for preview tokens, so
previewing never mutates state.
- SES webhooks: inbound unsubscribe + outbound bounce/complaint →
suppression (race-safe against at-least-once delivery, with reason
escalation that never downgrades).
- Per-customer unsubscribe hostname (Cloudflare DNS); sends are gated on
it being active, except in LOG/demo mode.
## Architecture
Campaign orchestration, suppression, the sender, the unsubscribe
controller, and the SES webhook handlers live in `src/modules/emailing`
+ `src/modules/messaging-webhooks` (the workspace-feature layer).
`core-modules/emailing-domain` keeps the SES driver, domain
provisioning, the `unsubscribeTopic` / `messageSuppression` core
entities, and the unsubscribe token/hostname plumbing. Domain creation
is validated (`CreateEmailingDomainInput` — domain-format regex,
lowercased) before any value reaches SES or the unsubscribe hostname.
## Frontend
- Campaign composer side panel (from / list / unsubscribe topic /
subject / body) with a live audience-preview hint.
- Email settings: email channels each showing their auto-provisioned
sending domain in a single section (status + DNS records + a "Check
verification" action), plus an **Unsubscribe Topics** section to
create/manage topics and preview the recipient page. A demo-mode banner
is shown when the LOG driver is active.
---------
Co-authored-by: Félix Malfait <felix@twenty.com>
Co-authored-by: Félix Malfait <felix.malfait@gmail.com>
|
||
|
Félix Malfait
|
76f69efb43 |
Keep synced messages and events when removing a workspace member (#21443)
## Context Removing a workspace member deletes their connected accounts, which cascades into deleting every message and calendar event those accounts synced. For a CRM, losing the email history of departed teammates is a big deal. ## What this does Connected accounts are now kept and reassigned instead of deleted when a member is removed: - Ownership moves to the acting user (whoever removed the member). When members remove themselves (leave workspace, account deletion), it falls back to the oldest admin. - OAuth tokens are revoked, credentials wiped, message/calendar channels get `isSyncEnabled = false`, and the account is stamped with a new `archivedAt` column (fast instance command included). - Synced messages, threads and calendar events stay in the workspace. Channel visibility settings keep applying as before, since channels and associations survive. - The reassigned account appears in the new owner's Settings → Accounts, where it can still be deleted (with its data) like any other account. The transfer happens synchronously during removal, while the member's userWorkspace row still exists. This also removes `DeleteWorkspaceMemberConnectedAccountsCleanupJob` and its listener: the async job had to reconstruct the account-owner link from rows the removal flow had just deleted, which was race-prone (see 2181fb541e). Archived accounts are excluded from the workflow send-email default account resolution, and both removal confirmation modals now mention what happens to synced data. --------- Co-authored-by: claude[bot] <41898282+claude[bot]@users.noreply.github.com> |
||
|
neo773
|
186d5b8faa | revert #21177 (#21284) | ||
|
Paul Rastoin
|
3d49642d12 |
[AUDIT] Run knip over twenty-server (#21159)
# Introduction Run [knip](https://knip.dev/) over twenty-server Used config: ```json { "$schema": "https://unpkg.com/knip@5/schema.json", "workspaces": { "packages/twenty-server": { "entry": [ "src/main.ts", "src/command/command.ts", "src/queue-worker/queue-worker.ts", "src/database/scripts/setup-db.ts", "src/database/scripts/truncate-db.ts", "src/database/clickHouse/migrations/run-migrations.ts", "src/database/clickHouse/seeds/run-seeds.ts", "src/instrument.ts", "lingui.config.ts", "test/integration/graphql/codegen/index.ts", "test/integration/utils/setup-test.ts", "test/integration/utils/teardown-test.ts", "scripts/**/*.ts", "**/*.spec.ts", "**/*.integration-spec.ts" ], "project": ["src/**/*.ts", "test/**/*.ts", "scripts/**/*.ts"], "ignore": [ "src/database/typeorm/**/migrations/**", "src/database/typeorm/**/*.entity.ts", "**/*.workspace-entity.ts", "**/logic-function-resource/constants/seed-project/**" ], "ignoreDependencies": ["@types/psl", "@types/aws-lambda"], "ignoreBinaries": ["nest", "lingui", "typeorm"] } } } ``` |
||
|
Etienne
|
15eaabdbc1 |
fix(ai) - optimize crud tools (#21133)
- **Add delete many**, `delete_many_{object}` added alongside the
existing `delete_one_{object}`.
- **Uniformize naming**, crud module, type names, and MCP helper
constants renamed for consistency.
- **Optimize tool schema (learn phase)**
- `find_many(_companies)`: **7 158 → 2 700 tokens**
- `find_one(_company)`: **280 → 126 tokens**
- ....
- Main mechanism: `reused: 'ref'` (line 7 of
`to-tool-json-schema.util.ts`). Zod walks the schema tree, tracks which
Zod schema instances appear more than once, and emits each reused
instance exactly once in `$defs`, replacing all subsequent occurrences
with a `$ref`. Works because filter and value schemas are now extracted
as shared objects.
- **Optimize system prompt (tool catalog)**, DATABASE_CRUD section
restructured to list operation patterns (`find_many_{object}`, …) once +
objects once, instead of the full N×M cross-product of tool names.
- **Optimize execute_tool**, shared record-properties schema (same
`$defs` deduplication applies at call time); introduced `upsert_many`;
added `selectedFields` to `find_*` so the agent only fetches the fields
it needs.
|
||
|
Etienne
|
dd0039ca1c |
feat(mcp) - optimize instruction prompt and hide get_tool_catalog (#21183)
Workspace-aware initialize.instructions - Deleted the static mcp-server-instructions.const.ts - Created build-mcp-server-instructions.util.ts — a comprehensive system prompt with identity, object list, tool grammar, routing decision tree, intent mapping, skills vs tools, safety constraints, and data efficiency guidelines - Created McpInstructionBuilderService — fetches workspace-specific object names + skill names and injects them into the instructions Hide/deprecate get_tool_catalog Benefit : skip first MCP call (tools are included in instruction) |
||
|
Marie
|
4b15b949f3 |
Provide additional logsobservability to workflow runs (per node) (#21142)
Surfaces per-step "Logs" tabs in the workflow run side panel so users can see what each step actually did (model + tokens + tool calls for AI, console output for serverless functions, request/response for HTTP, recipients/body for Email). <img width="546" height="501" alt="ai_agent_without_websearch" src="https://github.com/user-attachments/assets/c6ca3518-9489-4484-a570-3d0569ff3b03" /> ## Storage - New `stepLogs` JSONB column on the `workflowRun` workspace entity, typed as `Record<string, WorkflowRunStepLog>` (keyed by step id). - Schema lives in `twenty-shared`: `workflowRunStepLogSchema` with a discriminated `details.type` union for `AI_AGENT | CODE | HTTP_REQUEST | EMAIL` — frontends and backends consume the same Zod-inferred type. - Field is added to existing workspaces via a workspace upgrade command (`2-9 add-workflow-run-step-logs-field`); the standard-object metadata declares it for new workspaces. - Writes happen atomically per step in `WorkflowRunStepLogWorkspaceService.setStepLog` using `jsonb_set`. That lets concurrent steps in the same run write their own keys without contending with the existing lock around `workflowRun.state`. - Per-step payload is hard-capped at 256 KB; anything larger is dropped with a `logger.warn`, so a pathological tool call can never bloat a row. See below for more information. ## How logs are produced **Aalmost everything was already being collected; this PR mostly persists and renders it.** - **AI agent** — `AgentAsyncExecutorService` already tracked token usage, model id, native web-search count, and the AI SDK's `steps[]`. We map those into the log via `mapAiStepsToToolCallLogs` (`searchVector` stripped from record outputs, per-call input/output capped at 32/64 KB, max 200 tool calls per step). The only new measurement is a wall-clock `durationMs` taken around `executeAgent`, and we now fold native web-search cost into the displayed `totalCostInDollars` (it was already billed, just not shown). - **Code / serverless function** — reuses the `console.log` output the function runner already returns (`logsByLevel`); `build-code-step-log.util` only repackages it. - **HTTP request** — built from the action's existing input/output via `build-http-request-step-log.util`. No new signals collected. - **Email (send / draft)** — added `sanitizedHtmlBody` + `plainTextBody` to the existing tool outputs (a small additive change), then `build-email-step-log.util` consumes them. No additional AI inference or external calls are made for logging — the cost is a small CPU overhead per step plus the JSONB write. ## Security The log surface intentionally shows whatever the workflow touched, which made redaction and sanitization the main design concern. - **HTTP — secrets in headers**: existing `SENSITIVE_HEADER_NAMES` set (Authorization, Cookie, …) replaced with `[redacted]` in both request and response. - **HTTP — secrets in URLs**: `SENSITIVE_URL_PARAM_NAMES` (e.g. `api_key`, `token`, `access_token`) replaced in the query string via `URL`-based parsing. - **HTTP — secrets in bodies**: `SENSITIVE_BODY_KEY_REGEX` deep-walks JSON request/response bodies (object input or stringified JSON) and redacts matching keys. Applied to the `error` field too, since transport-layer errors sometimes embed structured payloads. - **Email — XSS risk in body preview**: tool outputs now expose a server-side `sanitizedHtmlBody`; the log builder prefers it over the raw user-authored `input.body`, with `plainTextBody` as a second fallback. The original raw body is only used if sanitization didn't happen (e.g. tool failed before composing). - **AI — internal/noisy data**: `searchVector` (Postgres tsvector strings) is stripped from record outputs returned by Twenty tools to avoid leaking internal full-text-search payloads. - **DB bloat / runaway agents**: 256 KB per-step cap + 32 KB / 64 KB per-tool-call input/output cap + 200 tool calls per step. <img width="547" height="307" alt="logic_function" src="https://github.com/user-attachments/assets/dd4a3d16-67f2-434b-95b3-bdcaf9ed053d" /> ## More details on Log size & truncation Logs are stored in `workflowRun.stepLogs` (JSONB), keyed by `stepId`. ### Per-step cap Each step's log is hard-capped at **256 KB** (`MAX_STEP_LOG_BYTES` in `WorkflowRunStepLogWorkspaceService.setStepLog`). For ~99% of workflows this is roomy — typical real-world sizes: - Code / serverless function: 1–20 KB - HTTP request: 5–70 KB - Email: 5–30 KB - AI agent (a handful of tool calls): 5–50 KB ### Two layers of bounding 1. **Per-field truncation** in each builder (before writing): - **Code**: ≤ 500 entries, ≤ 4 KB per message, ≤ 8 KB stack trace - **HTTP**: ≤ 32 KB per body (request + response), UTF-8 byte-aware - **Email**: ≤ 8 KB body preview, UTF-8 byte-aware - **AI agent**: ≤ 32 KB tool input, ≤ 64 KB tool output, ≤ 200 tool calls/step 2. **Global per-step safety net** at write time: if the assembled `stepLog` still exceeds 256 KB, the write is **dropped entirely** with a `logger.warn`. The workflow itself keeps running unaffected. ### What this means in practice - **Safe**: workflow execution, step results, downstream steps — never blocked by log size. - **Safe**: iterators (each iteration overwrites the previous log for that `stepId`, so they can't accumulate). - **Safe**: step retries (same `stepId` is overwritten, not appended). - **Possible**: an AI agent step with many large tool outputs (e.g., 50+ heavy `web_search` calls) can exceed 256 KB → the **entire** step's log is dropped, side panel shows "No logs were recorded for this step". The user has no explicit signal that the log was dropped due to size (only server-side warn). - **Possible** (theoretical): a workflow with hundreds of distinct steps could push the row toward Postgres's internal ~256 MB jsonb limit. Beyond that, individual `jsonb_set` writes would error and be swallowed by the action's try/catch — workflow still completes. ### Possible future hardening (not in this PR) - Replace "drop entire log" with a stub that preserves the summary card (cost, duration, status) and marks `truncated.reason = 'size_cap'`. - Surface size-drops in the UI (similar to the existing `<StyledTruncatedNotice>`). - Emit a metric so dropped logs are observable in dashboards. |
||
|
Paul Rastoin
|
164a5b1e8d |
Refactor email composer (#21177)
# Introduction Gate what connected account can be ingested in case of ai mcp user workspace agnostic funnel to only the workspace shared connected account Added a quick win intregration tests on seeded connected accounts ( that wasn't covered but already protected fix impacts only the mcp ) Refactored the API slightly too ## Notice This mean there's a breaking change in the product behavior Whereas before a non user workspace related mcp interaction would might have fallback on any private user connected account it will now only search for workspace visible listed ones |
||
|
Félix Malfait
|
b338a7a1d2 |
feat(settings): discovery hero rollout + ephemeral playground token (#21072)
## Summary
Two intertwined streams of work:
### UI — discovery hero pattern, settings shell, AI/API redesign
- **Generalize `SettingsDiscoveryHeroCard`** and use it on Layout, Data
Model, Apps, AI, API/Webhooks, Members. Drops 4 per-page wrapper files
(`SettingsObjectCoverImage`, `SettingsLayoutCoverImage`,
`SettingsLayoutCustomizeVideoModal`,
`SettingsDataModelVisualizeVideoModal`). Each page now supplies cover
src, modal id, and tab list.
- **Modal**: swap `<video>` placeholder for the Vimeo iframe pattern
from `twenty-docs`, per-tab `vimeoId`. Drop the parallel border-bottom
on the header (TabList draws its own baseline) and the grey background
behind the video. Note: Vimeo's embed allowlist applies — the iframes
load with the correct URL on `localhost` but the player itself requires
the video owner to allow the dev/staging domains in Vimeo settings.
- **AI page** rebuilt into a Cockpit pattern (Overview / Models / Skills
/ Tools / Usage). New `SettingsAiOverviewTab` with default Smart/Fast
pickers, at-a-glance stats, and an MCP signpost that deep-links to
`/settings/api-webhooks#mcp`. System Prompt link moved under Models.
Advanced tab removed.
- **API & Webhooks** now has 4 tabs (Playground / MCP / API Keys /
Webhooks). Hero card above tabs. Playground tab inverted to "Core API" /
"Metadata API" sections, each containing REST + GraphQL cards — schema
is the meaningful axis, protocol is secondary. Hash deep-link sync
delegated to the shared `TabListFromUrlOptionalEffect`.
- **Settings shell**: unified drawer outer padding (kill `isSettings`
branch), extract `CollapsibleNavigationDrawerSection`, add `iconColor`
on settings nav items, fix Exit Settings button alignment, 880px content
cap.
### Backend — strategy C: ephemeral playground token
The legacy paste-your-API-key flow is replaced by an on-demand
short-lived token scoped to the calling user's permissions. No shared
"Playground" API key to manage or revoke.
- New `JwtTokenTypeEnum.PLAYGROUND`. `PlaygroundTokenJwtPayload =
Omit<AccessTokenJwtPayload, 'type' | impersonation fields>` so any
future ACCESS claim flows through automatically.
- `AccessTokenService.generatePlaygroundToken` signs an access-shaped
JWT with `type: PLAYGROUND` and a configurable short TTL. A shared
private `resolveTokenSubject` helper parallelizes the user / workspace /
userWorkspace lookups for both generators.
- `JwtAuthStrategy.validateAccessToken` widened to accept
`AccessTokenJwtPayload | PlaygroundTokenJwtPayload`; impersonation gated
on `payload.type === ACCESS` so the union narrows without `as unknown
as` casts. The two branches in `validate()` collapse into one.
- New `PLAYGROUND_TOKEN_EXPIRES_IN` config var (default `2h`).
- New `generatePlaygroundToken` mutation (`WorkspaceAuthGuard`, no args,
returns `AuthToken`).
- Frontend `useOpenPlayground` hook centralizes mint → atom write →
navigate, with Apollo `onError` snackbar and a "use cached PLAYGROUND
token if still fresh" short-circuit (decodes via `jwt-decode`, checks
both `type` AND `exp`). Old API_KEY tokens left in localStorage from the
prior paste-form flow are rejected on `type` alone and force a re-mint —
this is what was causing the "This API Key is revoked" symptom on stale
browsers.
### Drive-by cleanups
- `PlaygroundToken` DTO removed (identical shape to `AuthToken` already
in use).
- 5 `customize-sidebar.webm` imports and the dead placeholder pipeline
removed.
## Test plan
### Discovery hero
- [ ] `/settings/layout`, `/settings/data-model`,
`/settings/applications`, `/settings/ai`, `/settings/api-webhooks`,
`/settings/members` each render the discovery hero card with its
illustration + play button + tabbed modal
- [ ] Modal tabs show the correct Vimeo embed URL per tab; aspect ratio
stays at 1440/900; no parallel border-bottom jog at the tab baseline
- [ ] AI Overview tab shows Smart/Fast model pickers + stats grid + MCP
signpost card; the MCP card lands on `/settings/api-webhooks#mcp` with
the MCP tab active
### API playground (ephemeral token)
- [ ] With an empty `playgroundApiKeyState` in localStorage, clicking
REST or GraphQL playground card opens the playground and the cached
token has `type: "PLAYGROUND"` with ~2h exp
- [ ] Clicking the card again within the freshness window does **not**
re-mint (`iat` / fingerprint stable across visits)
- [ ] Planting a fake API_KEY-shaped JWT in localStorage and clicking
the card forces a fresh mint (old token rejected on `type`)
- [ ] `GET /rest/companies?limit=1` with the cached token returns 200 +
real data
- [ ] `POST /graphql { __typename }` returns 200
### Settings shell
- [ ] Settings nav matches main app drawer padding; sections collapse;
Exit Settings button aligns with the workspace links above
- [ ] Active nav items have a right-gap (cleaner active state)
- [ ] Content area capped at 880px
### Verify
- [ ] `npx nx typecheck twenty-front` passes
- [ ] `npx nx typecheck twenty-server` passes
- [ ] `npx nx lint:diff-with-main twenty-front` passes
- [ ] `npx nx lint:diff-with-main twenty-server` passes
|
||
|
Félix Malfait
|
4797d2f270 |
feat(twenty-orm): introduce WorkspaceScopedRepository for core/metadata workspace-scoped entities (#20953)
## Summary Adds a third tenancy enforcement layer for entities that live in shared schemas (`core`, `metadata`) and carry a `workspaceId` column — previously the only safeguard at this layer was developer discipline (remembering to put `workspaceId` in every WHERE clause). ### The three layers, after this PR | Layer | Scope | How it's enforced | |---|---|---| | 1. Workspace data | per-workspace schema (companies, people, custom objects) | `twentyORMManager.getRepository(workspace, E)` — physical isolation (own data source) | | 2. Metadata | shared `metadata` schema (objectMetadata, fieldMetadata, views, roles…) | Flat-entity-maps cache — workspace-scoped in-memory map, lookups by id within it | | 3. Core (new) | shared `core` schema (agent threads/turns/messages, app tokens, etc.) | `WorkspaceScopedRepository<T>` — `workspaceId` is a required positional argument on every read/write | ## What's in the PR ### The wrapper (`packages/twenty-server/src/engine/twenty-orm/workspace-scoped-repository/`) - `WorkspaceScopedRepository<T extends WorkspaceScopedEntity>` — wraps a TypeORM `Repository<T>`, requires `workspaceId` on every `find`/`findOne`/`findOneOrFail`/`update`/`delete`/`softDelete`/`insert`/`save`/`count` call, merging it into the WHERE or stamping it on the entity. `createQueryBuilder` is an explicit escape hatch (caller scopes manually). - Provided via Nest DI with `@InjectWorkspaceScopedRepository(EntityClass)` and the `provideWorkspaceScopedRepository(EntityClass)` provider factory. - 19 unit tests cover the merge behavior, override-on-conflict, and the array-where (OR) case. ### Lint enforcement (`packages/twenty-oxlint-rules/rules/prefer-workspace-scoped-repository.ts`) - New `twenty/prefer-workspace-scoped-repository` rule (level: **error**). - Blacklist of entity names: raw `@InjectRepository(E)` is rejected if `E` is on the list. - Initial list: `AgentTurnEntity`, `AgentMessageEntity`, `AgentMessagePartEntity`, `AgentChatThreadEntity`, `AgentTurnEvaluationEntity`, `AgentEntity`. - Designed to grow over time as more consumers are migrated. - 5 rule tests. ### Migration in this PR All consumers of the six blacklisted entities, including: - AI agent / chat / monitor resolvers, services, and jobs - `AgentService`, `AiAgentRoleService`, `AiAgentWorkflowAction`, `ApplicationService`, `WorkspaceFlatAgentMapCacheService` - Admin-panel chat (migrated where the lookup is workspace-known; one documented `eslint-disable` on the threadId-discovery lookup that necessarily precedes the `allowImpersonation` permission check) - `AiAgentRoleService` unit spec updated to mock the scoped wrapper ## Future work (deliberately not in this PR) A standalone audit identified ~14 additional `core`/`metadata` entities with `workspaceId` that currently use raw `@InjectRepository` and could be added to the blacklist. Notable candidates: `UserWorkspaceEntity` (42 sites), `AppTokenEntity` (10), `FileEntity` (7), `BillingCustomerEntity`/`BillingSubscriptionEntity` (~22 combined). Each should be its own PR — the migration is mechanical but the surface is wide. ## Test plan - [x] `npx nx typecheck twenty-server` — clean - [x] `npx nx lint twenty-server` — 0 warnings, 0 errors - [x] `npx jest workspace-scoped-repository` — 19/19 pass - [x] `npx nx test twenty-oxlint-rules` — 215/215 pass - [x] `npx jest src/engine/metadata-modules/ai` — 44/44 pass - [ ] Manual smoke: end-to-end AI agent chat send/receive (reviewer) - [ ] Manual smoke: AI agent monitor — list turns, run evaluation (reviewer) - [ ] Manual smoke: admin-panel chat thread inspection (reviewer) |
||
|
Paul Rastoin
|
b8de469f37 |
Refactor and centralize file mimeType integrity check and sanitization (#20889)
# Introduction closes https://github.com/twentyhq/private-issues/issues/484 This PR refactors the writeFile API to never expect to be passed a mimetype, its extract is done programmatically low level so any callers will pass through Same for the file sanitization ## IANA override Disclaimer for consistency we existing behavior we wanted to always have `application/typescript` - should we rather consider fallbacking to octect-steam instead ? - Any pulbic assets that has .ts will now also fallback to `application/typescript` instead of the official IANA ## Integration Added coverage |
||
|
Paul Rastoin
|
076c05cbd0 |
File service uniformize not found behavior and stream management (#20891)
# Introduction closes https://github.com/twentyhq/private-issues/issues/485 |
||
|
neo773
|
4554dbe3c9 |
make connectedAccount clients self contained (#20827)
Replace OAuth2ClientManagerService with per provider each loading their own entity and resolving tokens internally Removes the ugly spread pattern of sprinkling tokens everywhere, this caused downtime of messaging when we migrated to encrypted tokens |
||
|
neo773
|
1e2ae5342b |
Send Email UI IMAP/SMTP message threading fix (#20784)
**Problem:** When using Twenty Send Email UI IMAP/SMTP message threading is broken on Twenty side as well as recipient email client **Twenty side fix:** - SMTP has no concept of `externalThreadId` sendEmail resolver always returns null, this breaks threading Fix is to pass `parentThreadExternalId` to `resolveOutboundThreadExternalId` for SMTP/IMAP path **Recipient email client fix:** - Fetch associated threads as per RFC spec to write `References` header ``` From: johndoe@domain.com To: janedoe@domain.com Subject: Test References: <root@...> <mid1@...> <parent@...> ``` |
||
|
Paul Rastoin
|
643ec121a7 |
[twenty-server] no-misused-promise lint (#20529)
# Introduction
Adding `no-miused-promise` lint rule to the twenty-server
In order to flag such pattern
```ts
// ❌ Flagged — forEach doesn't await the callback
items.forEach(async (item) => {
await process(item);
});
```
## What happened
- Refactored the code-interpreter driver to have a async onResult (
which is also expected by e2b )
- Workspace manager still dirty solution including force cast
- Basic fixes
|
||
|
Charles Bochet
|
ac653182b2 |
feat(server): migrate all remaining JWT token types to ES256 (#20513)
## Summary Extends the asymmetric signing work from #20467 to cover **every remaining `JwtTokenTypeEnum` value**: `LOGIN`, `WORKSPACE_AGNOSTIC`, `FILE`, `API_KEY`, `APPLICATION_ACCESS`, `APPLICATION_REFRESH`, `APP_OAUTH_STATE`, plus the ACCESS-shaped session token issued by the code interpreter tool. After this PR, every JWT the server signs is ES256 with a `kid` pointing at the current `core."signingKey"` row, while legacy HS256 tokens (no `kid` header) remain verifiable indefinitely through the existing fallback in `JwtWrapperService.resolveVerificationKey`. No new entity / migration / config: this is a pure routing change on top of the infrastructure that already shipped. ## Why `#20467` only flipped `ACCESS` and `REFRESH` to ES256. Every other JWT type was still HS256-signed against the global `APP_SECRET`, which kept the original blast radius (a leaked `APP_SECRET` invalidates *every* JWT type forever). Migrating the rest unifies the sign path on rotatable per-server private keys without forcing any token reissue. ## Mechanical changes ### Sign side (8 services) - `LoginTokenService.generateLoginToken` - `TransientTokenService.generateTransientToken` - `WorkspaceAgnosticTokenService.generateWorkspaceAgnosticToken` - `ApplicationTokenService.signApplicationToken` (`APPLICATION_ACCESS` + `APPLICATION_REFRESH`) - `ApiKeyService.generateApiKeyToken` - `FileUrlService.signFileByIdUrl` / `signWorkspaceLogoUrl` - `ConnectionProviderOAuthFlowService.signState` (`APP_OAUTH_STATE`) - `CodeInterpreterTool.generateSessionToken` Each call site swaps `jwtWrapperService.sign(payload, { secret: generateAppSecret(...), ... })` for `await jwtWrapperService.signAsync(payload, { expiresIn, [jwtid] })`. The `generateAppSecret` calls on the sign side are dropped (verifier-side `generateAppSecret` stays in `resolveVerificationKey` for the HS256 fallback). ### Verifier side - `WorkspaceAgnosticTokenService.validateToken` now goes through `verifyJwtToken` instead of the bespoke `verify({ secret })` path, so new ES256 tokens are accepted while the legacy HS256 fallback inside `resolveVerificationKey` still serves the old shape. - `JwtWrapperService.sign()` is kept (legacy compat / tests) but is now strictly deprecated — there are no remaining production callers. ### Async ripple (`signFileByIdUrl` was synchronous) - `FileUrlService.signFileByIdUrl` and `signWorkspaceLogoUrl` are now `async`; the `signUrl` callback used by `getRecordImageIdentifier` is widened to accept `Promise<string | null>`. - Every direct/indirect caller is updated: admin panel (user lookup + statistics + top workspaces), search service (`computeSearchObjectResults`, `getImageIdentifierValue`), workspace resolver (`logo` resolver, public workspace by domain/id), `WorkspaceMemberTranspiler` (now `async toWorkspaceMemberDto[s]` / `toDeletedWorkspaceMemberDto[s]` / `generateSignedAvatarUrl`), `UserService.loadSignedAvatarUrlsByUserId`, `UserWorkspaceService.castWorkspaceToAvailableWorkspace`, workspace-invitation, approved-access-domain, agent-chat-streaming, agent-message-part resolver, navigation-menu-item record identifier, file-ai-chat / file-core-picture / file-email-attachment / file-workflow / files-field services, rich-text & files-field query result getters, and the code-interpreter tool. ## Backward compatibility - **Legacy HS256 tokens (no `kid`)** keep verifying via `resolveVerificationKey` → `extractAppSecretBody` → `generateAppSecret` for both `workspaceId`-bearing and `userId`-bearing payloads. - The `API_KEY` HS256-via-ACCESS-secret fallback (#16504) still kicks in inside `verifyJwtToken` for pre-2025-12-12 API keys. - No payload shape changes, no DB writes, no env var changes — old tokens issued by `main` continue to authenticate. ## Tests ### Unit (all green locally — 63/63) Updated specs for every migrated service to mock `signAsync` instead of `sign` and assert the new option shape: - `login-token.service.spec.ts`, `transient-token.service.spec.ts`, `workspace-agnostic-token.service.spec.ts`, `application-token.service.spec.ts`, `api-key.service.spec.ts`, `connection-provider-oauth-flow.service.spec.ts`. ### Integration (`jwt-key-rotation.integration-spec.ts`) - Existing ACCESS coverage (current key, legacy HS256 fallback, rotated-out key, revoked key, unknown kid) is preserved. - New `it.each` assertion: `REFRESH`, `WORKSPACE_AGNOSTIC`, and `LOGIN` tokens emitted by the real signUp → signUpInNewWorkspace → getAuthTokensFromLoginToken pipeline are ES256 with a `kid` matching the current signing key — proves end-to-end that the migration didn't regress those flows. ## Open question (separate decision) This PR keeps the legacy HS256 verification fallback **forever**. We may eventually want to sunset it for `API_KEY` once telemetry shows pre-migration tokens are gone, but that's a separate product/security decision and not part of this change. ## Test plan - [ ] CI green - [ ] `npx nx lint:diff-with-main twenty-server` passes - [ ] `npx nx typecheck twenty-server` passes - [ ] `jwt-key-rotation` integration suite passes (new + existing assertions) - [ ] Manually verify: signing in issues an ES256 ACCESS / REFRESH token, generating an API key issues an ES256 token with `kid`, signed file URL JWT is ES256 with `kid` - [ ] Pre-existing HS256 tokens still authenticate (covered by integration test, but worth a manual check with a token from `main`) |
||
|
Krzysztof Woś
|
276d4f6e84 |
fix(code-interpreter): three correctness fixes for the TwentyMCP helper + tool output shape (#20103)
## Summary
Three independently-useful correctness fixes for the `code_interpreter`
tool, all surfaced while standing up a self-hosted code interpreter
against MCP. Each is isolated to one bug and applies regardless of
`CODE_INTERPRETER_TYPE`.
### 1. UI: unwrap `execute_tool` envelope when rendering
`code_interpreter` output
When `code_interpreter` is invoked through MCP's `execute_tool`
meta-tool, the result arrives wrapped: `{success, result: {stdout,
exitCode, files, ...}, ...}`. `ToolStepRenderer` reads `exitCode` at the
top level, which is `undefined` → the step renders as "Failed" even on a
clean `exitCode === 0`. Symmetric to the input-side unwrap that already
exists; the fix lifts `outputObj.result` when `rawToolName ===
'execute_tool'`.
### 2. Helper: route `TwentyMCP.call_tool` through `execute_tool` for
catalog tools
The `TwentyMCP` helper injected into every code-interpreter sandbox
exposes a `call_tool(name, arguments)` method. Direct MCP calls only
work for the 5 meta-tools (`get_tool_catalog`, `learn_tools`,
`execute_tool`, `load_skills`, `search_help_center`); the 250+ catalog
tools are accessed through `execute_tool`. Today
`twenty.call_tool('find_companies', {...})` raises "Unknown tool". This
commit detects catalog tools and auto-routes them through
`execute_tool`. It also flattens the nested `{catalog: {category:
[...]}}` shape returned by `list_tools()` and propagates `{success:
false}` envelopes as explicit exceptions (they were being silently
returned as dicts).
### 3. Prompt: stop the agent from hallucinating \`import twenty\`
Despite the helper being pre-injected, models frequently emitted
\`import twenty\` and crashed with \`ModuleNotFoundError\`. Two
contributing sources: the helper docstring did not explicitly say "do
not import," and the code-interpreter skill template's example block
referenced placeholder tool names. Fix: explicit "DO NOT import twenty"
block in the helper + rewritten skill examples using real tool names and
real response shapes.
## Test plan
- [ ] Existing \`code_interpreter\` tests still pass.
- [ ] Run a chat turn that invokes \`code_interpreter\` indirectly via
MCP \`execute_tool\` and confirm the UI no longer flips to "Failed" on
\`exitCode === 0\`.
- [ ] From inside the sandbox, run \`twenty.call_tool('find_companies',
{limit: 5})\` and confirm records return (was raising "Unknown tool").
- [ ] Confirm \`twenty.list_tools()\` returns a flat list, not the
nested \`{catalog: {...}}\` envelope.
- [ ] Trigger a tool error from inside the sandbox and confirm it raises
rather than returning a \`{success: false}\` dict.
- [ ] Ask an LLM to use \`code_interpreter\`; confirm it does not emit
\`import twenty\`.
---------
Co-authored-by: Félix Malfait <felix@twenty.com>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Co-authored-by: Félix Malfait <felix.malfait@gmail.com>
|
||
|
Félix Malfait
|
4f938aa097 |
feat(app): infrastructure for pre-installed apps (#19973)
**PR 1 of 2.** Follow-up PR ships the Exa app, sets it as a default
pre-installed app, and removes the current `WebSearchTool` /
`WebSearchService` / `ExaDriver`. This PR adds the plumbing; no
user-visible change yet.
## Summary
- Server admins can declare a list of npm app packages to auto-install
on every new workspace and backfill onto existing workspaces via CLI.
- Server-level secrets (like Exa's API key) live on the
`ApplicationRegistration` (one row per server, encrypted) and are
injected into logic function execution env at runtime. No more
per-workspace storage of global secrets.
- A generic `POST /app/billing/charge` endpoint lets app logic functions
emit workspace usage events for metered features. Exa uses it in PR 2;
future apps (call recorder, etc.) reuse it.
- `LogicFunctionToolProvider` tool name prefix changes `logic_function_`
→ `app_`. Shorter, accurate (they come from installed apps).
## What's in this PR
**Logic function executor — server-level variables**
- `LogicFunctionExecutorService.getExecutionEnvVariables` now resolves
env vars in the order: hardcoded defaults →
`ApplicationRegistrationVariable[]` (server-level) →
`ApplicationVariable[]` (workspace-level override). The manifest
`serverVariables` schema has existed; this closes the loop.
**Config**
- `PRE_INSTALLED_APPS` — comma-separated list of npm packages. Default:
empty.
**\`PreInstalledAppsService\`** (new module)
- \`onApplicationBootstrap()\` — fetches each package's manifest from
the app registry CDN, upserts an \`ApplicationRegistration\`, and seeds
declared \`serverVariables\` from matching env vars (e.g.
\`EXA_API_KEY\` env → encrypted registration variable).
- \`installOnWorkspace(workspaceId)\` — installs all pre-installed apps
on a single workspace. Tolerates per-app failures.
**Auto-install on new workspace activation**
- \`WorkspaceService.prefillCreatedWorkspaceRecords\` invokes
\`installOnWorkspace\` after prefilling standard records. Non-blocking
on failure.
**Backfill CLI command**
- \`install-pre-installed-apps\` — iterates active and suspended
workspaces, installs pre-installed apps that aren't yet installed.
Idempotent. Run after changing \`PRE_INSTALLED_APPS\`.
**App billing endpoint**
- \`POST /app/billing/charge\`. Authenticated via \`APPLICATION_ACCESS\`
token (already injected into logic function execution env as
\`DEFAULT_APP_ACCESS_TOKEN\`). Body: \`{ creditsUsedMicro, quantity,
unit, operationType, resourceContext? }\`. Emits \`USAGE_RECORDED\` with
\`applicationId\` as \`resourceId\`. Generic — reusable by any app.
**Tool name prefix**
- \`LogicFunctionToolProvider.buildLogicFunctionToolName\` now produces
\`app_<name>\` instead of \`logic_function_<name>\`. Only affects tools
sourced from logic functions; other tool providers unchanged.
## Stats
- 16 files, +501 / −2
- 7 new files (1 command, 1 service × 2, 1 controller, 1 DTO, 2 modules)
- Typecheck: 7 pre-existing errors, zero new
- Prettier clean
## Behavior deltas
- **\`PRE_INSTALLED_APPS\` default = empty**: existing servers see no
change on merge.
- **\`ApplicationRegistrationVariable\` is now read by the executor**:
apps that were using manifest \`serverVariables\` but expecting them to
be ignored by the executor will now see them injected. No apps ship with
\`isTool: true\` logic functions today, so this is latent — first
consumer is Exa in PR 2.
- **Tool prefix**: currently no logic-function tools are named
\`logic_function_*\` in any production flow. The prefix change affects
only future tools emitted by \`LogicFunctionToolProvider\`.
## Risks
- **CDN unavailability at startup**: if the app registry CDN is down,
\`ensureRegistrationsExist\` logs warnings but doesn't block server
start. Installation on new workspaces during this window will find no
registrations and log a non-blocking error. Backfill command can retry
after CDN recovers.
- **Cold-start overhead**: \`ensureRegistrationsExist\` is called once
per process on bootstrap. Current configurable default is empty, so zero
overhead. When an admin sets \`PRE_INSTALLED_APPS\`, they accept one
HTTP call per package at boot.
- **Server-level variables flow**:
\`ApplicationRegistrationVariable.encryptedValue\` is shared by all
workspaces of a server. Appropriate for a single-tenant Exa key. Not
appropriate for per-tenant keys — those go in workspace-level
\`ApplicationVariable\` and override.
## Test plan
- [ ] \`npx nx typecheck twenty-server\` passes (verified: 7
pre-existing unrelated errors, zero new)
- [ ] Set \`PRE_INSTALLED_APPS=@twenty-apps/hello-world\` (or any real
npm-published app), \`HELLO_WORLD_API_KEY=xxx\`, restart server:
\`ApplicationRegistration\` row is upserted,
\`ApplicationRegistrationVariable\` for HELLO_WORLD_API_KEY is populated
(encrypted).
- [ ] Create a new workspace: the app is auto-installed,
\`ApplicationEntity\` row created, \`LogicFunctionEntity\` rows created.
- [ ] Existing workspace: run \`yarn nx run twenty-server:command
install-pre-installed-apps\`: apps install across all workspaces,
idempotent on re-run.
- [ ] Trigger a logic function that reads
\`process.env.HELLO_WORLD_API_KEY\`: value resolves from the
server-level \`ApplicationRegistrationVariable\`.
- [ ] Log a charge from the handler: \`POST /app/billing/charge\` with
\`Authorization: Bearer \$DEFAULT_APP_ACCESS_TOKEN\` body
\`{creditsUsedMicro: 1000, quantity: 1, unit: "INVOCATION",
operationType: "WEB_SEARCH"}\` → returns \`{success: true}\`,
\`USAGE_RECORDED\` event emitted with correct
\`resourceId=applicationId\`.
- [ ] Tool name generated by \`LogicFunctionToolProvider\` starts with
\`app_\`.
## What's NOT in this PR (PR 2 scope)
- The Exa app itself (\`packages/twenty-apps/...\` directory)
- Removing \`WebSearchTool\`, \`WebSearchService\`, \`ExaDriver\`,
\`web-search\` module
- Removing \`WEB_SEARCH_DRIVER\` config var
- Removing the current \`exa_web_search\` entry in
\`ActionToolProvider\`
- Chat preload list updated to \`app_exa_web_search\`
- Frontend \`getToolDisplayMessage\` branch for \`app_exa_web_search\`
- Setting \`PRE_INSTALLED_APPS\` default to include \`@twenty-apps/exa\`
---------
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
Félix Malfait
|
0c929e7903 |
refactor(tool-provider): rename web_search to exa_web_search, drop XOR toggle (#19969)
## Summary
- Today `WEB_SEARCH_PREFER_NATIVE` forces a **mutual exclusion**: either
the custom Exa tool preloads as `web_search` or the SDK-native
`web_search` binds. Same name, different backends.
- This PR lets them **coexist**. Custom Exa becomes `exa_web_search`;
native keeps `web_search`. The model picks based on tool descriptions.
- `WEB_SEARCH_PREFER_NATIVE` and `shouldUseNativeSearch()` are deleted.
Exa enablement follows `WEB_SEARCH_DRIVER` (existing). Native enablement
follows the agent's `modelConfiguration.webSearch.enabled` (existing).
## Key changes
**Config / service**
- Deleted `WEB_SEARCH_PREFER_NATIVE` (config-variables.ts)
- Deleted `WebSearchService.shouldUseNativeSearch()`
- `WebSearchService.isEnabled()` unchanged — still gates Exa
availability
**Custom tool rename**
- `ActionToolProvider.toolMap`: `'web_search'` → `'exa_web_search'`
- Descriptor name matches
- `WebSearchTool.description` rewritten to position Exa as
structured/entity-aware, complementary to native
**Native tool binder**
- `NativeToolBinder.bind()` drops the `shouldUseNativeSearch` gate.
Per-agent `modelConfiguration.webSearch.enabled` (inside
`getNativeModelTools`) stays authoritative.
**Chat**
- Preload list now always includes `exa_web_search` —
`ActionToolProvider` silently skips the descriptor when Exa is disabled,
so `getToolsByName` degrades gracefully
- Native tools always attempted; returns empty ToolSet when the model
doesn't support them
- `directTools = { ...preloadedTools, ...nativeSearchTools }` — both
present when both enabled
- `billNativeWebSearchUsage` called unconditionally (the function
already short-circuits on count ≤ 0)
**Workflow agent**
- Same unconditional billing pattern
- `WebSearchService` dependency removed
**System prompt**
- Dropped the special-cased `web_search` branch. Preloaded tools list
uniformly now.
**Frontend**
- `exa_web_search` reuses the same "Searching the web for X" display as
native
- Test coverage added
## Billing isolation (verified)
- `countNativeWebSearchCallsFromSteps` counts `toolName ===
'web_search'` only. After the rename, only native calls match. Exa calls
(`exa_web_search`) are billed separately via
`WebSearchService.emitUsageEvent` inside `search()`.
- No double-billing path.
## Behavior deltas (intended)
| Scenario | Before | After |
|---|---|---|
| Anthropic model + Exa enabled + PREFER_NATIVE=true | native only |
**both** |
| Anthropic + Exa enabled + PREFER_NATIVE=false | Exa only (as
`web_search`) | **both** |
| Non-native model + Exa enabled | Exa as `web_search` | Exa as
`exa_web_search` |
| Any model + Exa disabled + native supported | native only | native
only |
| Workflow agent with `webSearch.enabled=true` + Anthropic + Exa enabled
| native only | **both** |
## Known regression (accepted)
Customers who set `WEB_SEARCH_PREFER_NATIVE=false` to force Exa-only
will now **also** see native `web_search` if the model supports it.
There's no chat-level kill switch after this PR. Per discussion, this is
accepted — future model-level capability gating (in the model JSON) will
be the right place for that control.
## Stats
- 10 files, +63 / −73 (net deletion)
- Typecheck clean (server: 7 pre-existing unrelated, front: 13
pre-existing unrelated — zero new either side)
- Prettier clean
## Test plan
- [ ] `npx nx typecheck twenty-server` and `npx nx typecheck
twenty-front` pass
- [ ] With Anthropic + Exa enabled: chat shows both `web_search` and
`exa_web_search` in preloaded list; model can call either
- [ ] With Anthropic + Exa disabled: chat shows only native `web_search`
- [ ] With non-native model + Exa enabled: chat shows only
`exa_web_search`
- [ ] Workflow agent with `modelConfiguration.webSearch.enabled=true` +
Exa enabled: both available
- [ ] Billing: native calls billed via `billNativeWebSearchUsage`; Exa
calls billed via `WebSearchService.emitUsageEvent`; no double-billing
- [ ] Frontend: `exa_web_search` renders "Searching the web for X" the
same as `web_search`
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
neo773
|
a1de37e424 | Fix Email composer rich text to HTML conversion (#19872) | ||
|
neo773
|
68746e22a0 |
Send Email Tool: Don't persist message on SMTP only connections (#19756)
Previously this blocked users who only had SMTP configured to send outbound emails, this fixes it by making messageChannel and persist layer conditional |
||
|
Félix Malfait
|
67e7f05a68 |
feat: email attachments and open-in-app click action (#19485)
## Changes ### Email Attachments - Added `EmailAttachmentsField` component for uploading and managing email attachments - New `useUploadEmailAttachment` hook for handling file uploads with size validation - New `UPLOAD_EMAIL_ATTACHMENT_FILE` mutation for backend file persistence - Integrated attachments into email composer with file validation - Added `EmailRecipientLimits` constant to enforce max recipients (100) on frontend ### Open in App Click Action - New `useOpenEmailInAppOrFallback` hook to open emails in the in-app composer - Email fields now default to "Open in app" action instead of "Open as link" - New `SettingsDataModelFieldOnClickActionForm` support for `OPEN_IN_APP` action - Email secondary table cell button now offers in-app composer as alternative action - `AttachmentChip` component moved from advanced-text-editor to file module for reuse ### Refactoring & New Utilities - Extracted `useComposeEmailForTargetRecord` hook for consistent email composer opening - New `useResolveDefaultEmailRecipient` hook to resolve recipient based on record type - New `getPrimaryEmailFromRecord` utility for safe email field access - New `EmptyInboxPlaceholder` component with CTA button - Simplified `ComposeEmailButton` using new hooks - Enhanced `ComposeEmailCommand` to support bulk Person selections - Updated `useSendEmail` to accept and forward attachments - Recipient count validation with warning in composer footer ### Backend - New `FileEmailAttachmentModule` with resolver and service - New `file-email-attachment.command` for record selection menu items - Updated `SendEmailInput` GraphQL type to include `files` field - Email tool constants and exceptions updated ### Type Updates - Added `SendEmailAttachmentInput` GraphQL type - Added `FileFolder.EMAIL_ATTACHMENT` to file folder interface --------- Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com> Co-authored-by: claude[bot] <41898282+claude[bot]@users.noreply.github.com> |
||
|
Félix Malfait
|
83d30f8b76 |
feat: Send email from UI — inline reply composer & SendEmail mutation (#19363)
## Summary - **Inline email reply**: Replace external email client redirects (Gmail/Outlook deeplinks) with an in-app email composer. Users can reply to email threads directly from the email thread widget or via the command menu. - **SendEmail GraphQL mutation**: New backend mutation that reuses `EmailComposerService` for body sanitization, recipient validation, and SMTP dispatch via the existing outbound messaging infrastructure. - **Side panel compose page**: Command menu "Reply" action now opens a side-panel compose email page with pre-filled To, Subject, and In-Reply-To fields. ### Backend - `SendEmailResolver` with `SendEmailInput` / `SendEmailOutputDTO` - `SendEmailModule` wired into `CoreEngineModule` - Reuses `EmailComposerService` + `MessagingMessageOutboundService` ### Frontend - `EmailComposer` / `EmailComposerFields` components - `useSendEmail`, `useReplyContext`, `useEmailComposerState` hooks - `useOpenComposeEmailInSidePanel` + `SidePanelComposeEmailPage` - `EmailThreadWidget` inline Reply bar with toggle composer - `ReplyToEmailThreadCommand` now opens side-panel instead of external links ### Seeds - Added `handle` field to message participant seeds for realistic email addresses - Seed `connectedAccount` and `messageChannel` in correct batch order ## Test plan - [ ] Open an email thread on a person/company record → verify "Reply..." bar appears below the last message - [ ] Click "Reply..." → composer opens inline with pre-filled To and Subject - [ ] Type a message and click Send → email is sent via SMTP, composer closes - [ ] Use command menu Reply action → side panel opens with compose email page - [ ] Verify Send/Cancel buttons work correctly in side panel - [ ] Test with Cc/Bcc toggle in composer fields - [ ] Verify error handling: invalid recipients, missing connected account Made with [Cursor](https://cursor.com) --------- Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com> |
||
|
Félix Malfait
|
ea572975d8 |
feat: generic web search driver abstraction with Exa support and billing (#19341)
## Summary - Introduces a pluggable `WebSearchDriver` abstraction (interface, factory, service, module) so web search is no longer tied to native provider tools (Anthropic/OpenAI) - **Exa** is the first driver implementation with support for category-filtered search (company, people, news, research paper, etc.) — particularly useful for CRM workflows - Per-query billing for both Exa ($0.007/query) and native provider surcharges ($0.01/query for Anthropic/OpenAI) via the existing `USAGE_RECORDED` pipeline - New config variables: `WEB_SEARCH_DRIVER` (EXA/DISABLED), `EXA_API_KEY`, `WEB_SEARCH_PREFER_NATIVE` (default false — prefers Exa over native when both available) - `WEB_SEARCH` operation type added for usage tracking and Stripe metering ### Architecture ``` WebSearchDriver (interface) ├── ExaDriver — Exa neural search with category support └── DisabledDriver — throws when search is disabled WebSearchDriverFactory (extends DriverFactoryBase) └── creates driver based on WEB_SEARCH_DRIVER config WebSearchService (facade) ├── search(query, options?, billingContext?) ├── isEnabled() └── emits USAGE_RECORDED events per query WebSearchTool (Tool implementation) └── registered in ActionToolProvider, available via tool catalog ``` ### Native search billing gap fixed Anthropic and OpenAI both charge $0.01/search on top of token costs. The token costs were already billed, but the per-call surcharge was not. Added `countNativeWebSearchCallsFromSteps` utility + `billNativeWebSearchUsage` to `AiBillingService`, wired into both chat and workflow agent paths. ## Test plan - [ ] Set `WEB_SEARCH_DRIVER=EXA` + `EXA_API_KEY=...` and verify AI chat can search the web - [ ] Verify category parameter works (ask about a specific company/person) - [ ] Set `WEB_SEARCH_DRIVER=DISABLED` and verify search tool is not exposed - [ ] Set `WEB_SEARCH_PREFER_NATIVE=true` with Anthropic model and verify native search is used - [ ] Verify usage events are emitted in ClickHouse for both Exa and native search paths - [ ] Verify existing billing tests pass (`npx jest ai-billing.service.spec.ts`) Made with [Cursor](https://cursor.com) --------- Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com> Co-authored-by: claude[bot] <41898282+claude[bot]@users.noreply.github.com> |
||
|
neo773
|
d3f0162cf5 |
Remove connected account feature flag (#19286)
Co-authored-by: martmull <martmull@hotmail.fr> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: github-actions <github-actions@twenty.com> Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com> Co-authored-by: Charles Bochet <charles@twenty.com> |
||
|
Félix Malfait
|
563e27831b |
Clean up tool output architecture: remove wrappers, enforce ToolOutput everywhere (#19321)
## Summary
- **Remove `ExecuteToolResult` wrapper** — `execute_tool` is now a
transparent dispatcher that returns the raw `ToolOutput` from underlying
tools. No more `{ toolName, result }` envelope.
- **Type the entire execution chain as `Promise<ToolOutput>`** — from
`ToolExecutorService.dispatch()` through `resolveAndExecute()` to
`execute_tool.execute()`. Zero `Promise<unknown>` remaining in the tool
layer.
- **Use `Extract<ToolExecutionRef, ...>`** for dispatch methods,
enabling exhaustive switch checking and removing `as never` casts.
- **Relax `ToolOutput.result` to accept `null`** — removes `??
undefined` hacks at the boundary with logic function results.
- **Enforce 1-export-per-file** across tool type/interface files (split
`tool-descriptor.type.ts`, `tool-provider.interface.ts`, `tool.type.ts`,
`tool-output.type.ts`, `tool-executor.service.ts`).
- **Simplify error handling** — `wrapWithErrorHandler` and all
meta-errors (tool not found, tool excluded) now return consistent
`ToolOutput` shape with `error` as a plain string.
- **Frontend reads output directly** — removed `unwrapToolOutput`
utility; `ToolStepRenderer` and `ThinkingStepsDisplay` extract
`message`/`error` from the raw output with simple type guards.
- **Add permission error detection** for email tools via
`isInsufficientPermissionsError`, guiding the AI model to suggest
account reconnection instead of hallucinating about visibility settings.
## Test plan
- [ ] AI chat tool calls return visible output (not "null") in the UI
- [ ] Tool errors display correctly in the JSON tree
- [ ] Email draft/send tools return actionable permission errors
- [ ] Code interpreter output renders correctly
- [ ] Thinking steps display tool outputs properly
Made with [Cursor](https://cursor.com)
---------
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
|
||
|
Félix Malfait
|
bb3d556799 |
Refined demo workspace creation skill (rebased, review fixes) (#19274)
## Summary Rebased version of #19051 with all review comments addressed. Clean branch on latest main, lint/typecheck/tests passing. ### Changes from original PR - AI can now create, update, and delete **view filters** (`ViewFilterToolsFactory`) and **view sorts** (`ViewSortToolsFactory`) - `create_view` now accepts `calendarFieldName`, `calendarLayout`, and `fieldNames` to configure views at creation time - Three new standard skills: `view-building`, `view-filters-and-sorts`, `custom-objects-cleanup` - `workspace-demo-seeding` skill reworked to keep standard objects and enrich them with custom fields - Cache invalidation for nav menu items when object `isActive` changes - Dashboard tool descriptions improved (RECORD_TABLE widget workflow) ### Review comments addressed (all 10 from #19051) 1. **Sentry + Cubic**: Calendar field DATE/DATE_TIME validation — added `resolveCalendarFieldMetadataId` using `isFieldMetadataDateKind` 2. **Cubic**: "navigate tool" → "navigate_app tool" in skill metadata (all 7 occurrences) 3. **Copilot**: KANBAN views now require `mainGroupByFieldName` — throws clear error if missing 4. **Copilot**: CALENDAR views now require both `calendarFieldName` and `calendarLayout` — validated before DB call 5. **Copilot**: Mock field fixtures include `type` property (DATE_TIME, TEXT, SELECT) 6. **Copilot**: `ViewFilterValue` type assertion instead of unsafe `as string` casts (3 locations) 7. **FelixMalfait**: Removed `NavigationMenuItemObjectDeactivationListener` — replaced with cache invalidation 8. **FelixMalfait**: Consolidated `ViewFilterToolProvider` and `ViewSortToolProvider` into single `ViewToolProvider` 9. Removed `VIEW_FILTER` and `VIEW_SORT` from `ToolCategory` enum (merged into `VIEW`) 10. Removed stale `existingFeatureFlagsMap` param incompatible with current main ## Test plan - [x] `npx nx lint:diff-with-main twenty-server` — passes - [x] `npx nx typecheck twenty-server` — passes - [x] `view-tools.factory.spec.ts` — all 20 tests pass (including 3 new validation tests) Supersedes #19051 https://claude.ai/code/session_01QPV74NU6vzmJb32e4i899E --------- Co-authored-by: Claude <noreply@anthropic.com> |
||
|
Félix Malfait
|
6f4f7a1198 |
fix: add security headers to file serving endpoints to prevent stored XSS (#18857)
## Summary - File serving endpoints (`GET file/:fileFolder/:id` and `GET public-assets/...`) were piping S3/local file streams directly to the response without any HTTP headers, allowing a stored XSS attack via uploaded HTML files rendered inline on the CRM origin. - Adds `Content-Type`, `Content-Disposition`, and `X-Content-Type-Options: nosniff` headers to all file serving responses. Only known-safe MIME types (images, PDF, plain text, audio, video) are served inline; everything else (HTML, SVG, XML, etc.) forces `Content-Disposition: attachment` to trigger download instead of rendering. - New `setFileResponseHeaders` utility with an explicit allowlist of inline-safe MIME types. ## Test plan - [x] Unit tests pass (9 tests including 2 new ones: header assertions and attachment-disposition for HTML) - [x] Lint clean (`lint:diff-with-main`) - [x] Typecheck clean (`nx typecheck twenty-server`) - [ ] Manual: upload an HTML file via `uploadWorkflowFile`, access the returned URL — should download instead of rendering - [ ] Manual: upload a PNG image, access the URL — should render inline with correct `Content-Type: image/png` - [ ] Manual: verify `X-Content-Type-Options: nosniff` header is present on all file responses Made with [Cursor](https://cursor.com) --------- Co-authored-by: claude[bot] <41898282+claude[bot]@users.noreply.github.com> Co-authored-by: Etienne <etiennejouan@users.noreply.github.com> Co-authored-by: Etienne <45695613+etiennejouan@users.noreply.github.com> |
||
|
neo773
|
8ef32c4781 |
Add In-Reply-To to Email Workflow Node (#18641)
Co-authored-by: github-actions <github-actions@twenty.com> Co-authored-by: Charles Bochet <charles@twenty.com> |
||
|
Charles Bochet
|
cee4cf6452 |
feat: migrate ConnectedAccount infrastructure entities to metadata schema (#18784)
## Summary - Migrates 4 entities (`connectedAccount`, `messageChannel`, `calendarChannel`, `messageFolder`) from per-workspace schemas to the shared `core` metadata schema - Introduces a `IS_CONNECTED_ACCOUNT_MIGRATED` feature flag to control the migration: when enabled, reads come from core metadata and all writes are dual-written to both workspace and core - Extracts 12 enums from workspace entity files to `twenty-shared` for reuse across frontend and backend - Creates new TypeORM entities, metadata services, GraphQL resolvers/DTOs, and exception interceptors per entity - Each entity owns its own data access module (`ConnectedAccountDataAccessModule`, `MessageChannelDataAccessModule`, `CalendarChannelDataAccessModule`, `MessageFolderDataAccessModule`) — no umbrella infrastructure module - Adds a 1.20 upgrade command that backfills data from workspace schemas to core (preserving UUIDs) and enables the feature flag - Replaces direct repository access with data access service calls across ~50 files in messaging, calendar, and connected-account modules - Adds `lastSignedInAt` and `oidcTokenClaims` fields to the new `ConnectedAccountEntity` - Drops unused `lastSyncHistoryId` field from the migrated connected account entity ## Test plan - [x] Lint passes (`npx nx lint:diff-with-main twenty-server`) - [x] Typecheck passes (`npx nx typecheck twenty-server`) - [x] All unit tests pass (477 suites, 4267 tests, 0 failures) - [ ] Manual test: verify messaging sync works with feature flag disabled (existing behavior) - [ ] Manual test: run upgrade command on a workspace, verify data backfilled to core tables - [ ] Manual test: verify messaging/calendar sync works with feature flag enabled (dual-write path) - [ ] Manual test: verify GraphQL metadata resolvers return correct data when flag enabled |
||
|
Lucas Bordeau
|
cb3e32df86 |
Fix AI demo workspace skill (#18575)
This PR fixes what allows to have a working demo workspace skill. - Skill updated many times into something that works - Fixed infinite loop in AI chat by memoizing ai-sdk output - Finished navigateToView implementation - Increased MAX_STEPS to 300 so the chat don't quit in the middle of a long running skill - Added CreateManyRelationFields |
||
|
Félix Malfait
|
05c2da2d0f |
Improve SSRF IP validation and add protocol allowlist (#18518)
## Summary - Replace regex-based private IP detection in `isPrivateIp` with Node.js `net.BlockList` for CIDR-based range checking, which properly handles all IPv4-mapped IPv6 representations (both dotted-decimal and hex forms) - Add missing non-routable IP ranges: carrier-grade NAT (`100.64.0.0/10`), IANA special purpose, documentation networks, benchmarking, multicast, and reserved ranges - Add protocol allowlist (http/https only) as an axios request interceptor in `SecureHttpClientService` and as a Zod refinement in the HTTP tool schema ## Test plan - [x] All 100 existing + new tests pass across 4 secure-http-client test suites - [x] New tests cover carrier-grade NAT range boundaries (100.64.0.0 – 100.127.255.255) - [x] New tests cover documentation, benchmarking, multicast, and reserved ranges - [x] New tests cover hex-form IPv4-mapped IPv6 addresses (the form Node.js URL parser actually produces) - [x] New tests verify protocol interceptor blocks `ftp:` and `file:` schemes - [x] New tests verify protocol interceptor is only active when safe mode is enabled Made with [Cursor](https://cursor.com) |
||
|
Lucas Bordeau
|
2a82df7073 |
AI tools to create a demo workspace (#18236)
This PR adds the necessary tool to create a demo workspace with : relevant custom objects and fields, mock data and a real dashboard with graph widgets. It is still a bit under-optimized and slow but it works. This PR also adds an AI tool that allows to see what happens in real time, it navigates the app and waits when necessary. --------- Co-authored-by: Félix Malfait <felix.malfait@gmail.com> |
||
|
Félix Malfait
|
0e89c96170 |
feat: add npm and tarball app distribution with upgrade mechanism (#18358)
## Summary - **npm + tarball app distribution**: Apps can be installed from the npm registry (public or private) or uploaded as `.tar.gz` tarballs, with `AppRegistrationSourceType` tracking the origin - **Upgrade mechanism**: `AppUpgradeService` checks for newer versions, supports rollback for npm-sourced apps, and a cron job runs every 6 hours to update `latestAvailableVersion` on registrations - **Security hardening**: Tarball extraction uses path traversal protection, and `enableScripts: false` in `.yarnrc.yml` disables all lifecycle scripts during `yarn install` to prevent RCE - **Frontend**: "Install from npm" and "Upload tarball" modals, upgrade button on app detail page, blue "Update" badge on installed apps table when a newer version is available - **Marketplace catalog sync**: Hourly cron job syncs a hardcoded catalog index into `ApplicationRegistration` entities - **Integration tests**: Coverage for install, upgrade, tarball upload, and catalog sync flows ## Backend changes | Area | Files | |------|-------| | Entity & migration | `ApplicationRegistrationEntity` (sourceType, sourcePackage, latestAvailableVersion), `ApplicationEntity` (applicationRegistrationId), migration | | Services | `AppPackageResolverService`, `ApplicationInstallService`, `AppUpgradeService`, `MarketplaceCatalogSyncService` | | Cron jobs | `MarketplaceCatalogSyncCronJob` (hourly), `AppVersionCheckCronJob` (every 6h) | | REST endpoint | `AppRegistrationUploadController` — tarball upload with secure extraction | | Resolver | `MarketplaceResolver` — simplified `installMarketplaceApp` (removed redundant `sourcePackage` arg) | | Security | `.yarnrc.yml` — `enableScripts: false` to block postinstall RCE | ## Frontend changes | Area | Files | |------|-------| | Modals | `SettingsInstallNpmAppModal`, `SettingsUploadTarballModal`, `SettingsAppModalLayout` | | Hooks | `useUploadAppTarball`, `useInstallMarketplaceApp` (cleaned up) | | Upgrade UI | `SettingsApplicationVersionContainer`, `SettingsApplicationDetailAboutTab` | | Badge | `SettingsApplicationTableRow` — blue "Update" tag, `SettingsApplicationsInstalledTab` — fetches registrations for version comparison | | Styling | Migrated to Linaria (matching main) | ## Test plan - [ ] Install an app from npm via the "Install from npm" modal - [ ] Upload a `.tar.gz` tarball via the "Upload tarball" modal - [ ] Verify upgrade badge appears when `latestAvailableVersion > version` - [ ] Verify upgrade flow from app detail page - [ ] Run integration tests: `app-distribution.integration-spec.ts`, `marketplace-catalog-sync.integration-spec.ts` - [ ] Verify `enableScripts: false` blocks postinstall scripts during yarn install Made with [Cursor](https://cursor.com) |
||
|
Etienne
|
26f0a416a1 |
File storage cleaning (#18381)
- Remove feature flag - Remove legacy methods in file-upload and file-service - Migrate AI Chat to new file management --------- Co-authored-by: Charles Bochet <charles@twenty.com> |
||
|
Raphaël Bosi
|
b11f77df2a |
[FRONT COMPONENTS] Introduce conditionalAvailabilityExpression to command menu items (#18319)
## PR Description - Uses `expr-eval` to enable front components (SDK plugins) to define conditional availability as declarative expressions. - Moves shared types and constants to `twenty-shared` - Introduces a `conditionalAvailabilityExpression` field on `CommandMenuItemEntity`, allowing command menu items to store an `expr-eval` compatible expression string that is evaluated against a CommandMenuContext to determine if the item should be shown. - Creates an esbuild transform plugin `conditional-availability-transform-plugin` in `twenty-sdk` that converts TypeScript conditional availability expressions into `expr-eval` compatible syntax at build time, so SDK developers can write natural TS expressions that get transformed to evaluable strings. - Removes deprecated `forceRegisteredActionsByKey` state and its usage. - Creates `useCommandMenuContext` hook that builds the full `CommandMenuContext` object from React state, which is then passed to `useCommandMenuItemFrontComponentActions` for evaluating conditional availability expressions. |
||
|
Félix Malfait
|
f4a61f26c0 |
Replace generic "Unknown error" messages with descriptive error details (#18019)
## Summary - Replace all generic `"Unknown error"` fallback messages across the server codebase with messages that include the actual error details - The most impactful change is in `guard-redirect.service.ts`, which handles OAuth redirect errors — non-`AuthException` errors (e.g., passport state verification failures) now show `"Authentication error: <actual message>"` instead of the opaque `"Unknown error"` - Gmail/Google error handler services now include the error message in the thrown exception instead of discarding it - Other catch blocks (workflow delay resume, migration runner rollback, code interpreter, marketplace) now use `String(error)` for non-Error objects instead of a static fallback Fixes the class of issues reported in https://github.com/twentyhq/twenty/issues/17812, where a user saw "Unknown error" during Google OAuth and had no way to diagnose the root cause (which turned out to be a session cookie / SSL configuration issue). ## Test plan - [ ] Verify OAuth error flows (e.g., Google Auth with misconfigured callback URL) now display the actual error message on the `/verify` page instead of "Unknown error" - [ ] Verify Gmail sync error handling still correctly classifies and re-throws errors with descriptive messages - [ ] Verify workflow delay resume failures include the error details in the workflow run status Made with [Cursor](https://cursor.com) Co-authored-by: Cursor <cursoragent@cursor.com> |
||
|
Etienne
|
163c1175cb |
File - Migrate core pictures (workspace and member logo) + workflow attachments (#17924)
- Create a common file-by-id download controller - Create core picture module with resolver and logic to handle workspaceLogo and workspaceMemberProfilePicture update - Create workflow file module (same) - Data migration |
