houseassassin/twenty
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
Files
85406a58fb05d64e6c36f2183dfbdd74b0775da9
twenty/packages/twenty-server/package.json
T
Etienne 7b45380777 feat(ai): large tool output handling + navigation tools (#21982) 
## Summary

Large tool outputs (e.g. a workflow run that serializes to ~70k tokens)
blow the chat context budget and force per-tool "raw" variants. This PR
handles oversized outputs generically in one place:

1. **Producer:** when a tool result exceeds a byte budget, it is spilled
to a `FileFolder.AgentChat` file and replaced with a compact `{ spilled,
outputRef, shape, hint }` envelope.
2. **Consumer:** two bounded, in-server navigation tools —
`extract_json_path` and `search_output` — let the model dig into the
spilled file by `fileId` without spinning up `code_interpreter`.

Together they add a fast, auditable middle tier between "truncated
inline preview" and "full code_interpreter relay," and enable an
enterprise "restricted" mode (spill + navigation, no sandbox).

## Data flow

```mermaid
flowchart TD
  exec["resolveAndExecute / hydrateToolSet closure"] --> compact[compactToolOutput]
  compact --> enabled{"spillLargeOutput enabled? (chat only)"}
  enabled -->|no| inlineRaw["inline raw (MCP, workflow, sandbox bridge)"]
  enabled -->|yes| size{"bytes > MAX_INLINE_TOOL_OUTPUT_BYTES?"}
  size -->|no| inline["inline result"]
  size -->|yes| skeleton["jsonShapeSkeleton + largeOutputHint"]
  skeleton --> write["writeFile(AgentChat)"]
  write --> envelope["return { spilled, outputRef, shape, hint }"]
  envelope --> model[Model]
  model --> nav["extract_json_path / search_output / code_interpreter (by fileId)"]
```

## Part 1 — Navigation tools (consumer)

- `extract_json_path`: extracts a sub-tree from a spilled JSON file by a
JSONPath-lite expression (dot/bracket access, array slicing,
single-level wildcard), with `maxItems`/`maxDepth` bounding. No filters
or recursive descent — those belong to `code_interpreter`.
- `search_output`: grep-like line search with context lines and
stateless `offset` pagination (`{ matches, totalMatches, hasMore }`).
- Both read from `FileFolder.AgentChat` by `fileId`, enforce their own
output byte cap, and are registered in `ActionToolProvider` (always
available; read-only).

## Part 2 — Spill producer

- Spilling slots in right after the existing `compactToolOutput` step at
the two seams in `ToolRegistryService` (`resolveAndExecute` and the
`hydrateToolSet` execute closure).
- `ToolOutputSpillService.spillIfTooLarge()` measures
`Buffer.byteLength`; over `MAX_INLINE_TOOL_OUTPUT_BYTES` (16 KB ≈ 4k
tokens) it writes the full payload and returns the envelope. Spill
failures never block the call (inline + warning).
- `jsonShapeSkeleton` computes a bounded structural map (depth 4, arrays
as `"array[N] of <type>"`, id-keyed maps collapsed, long leaves as size
markers, hard-capped at 1024 bytes) so the model knows the key paths in
one pass.
- Optional per-tool `largeOutputHint` (on the `Tool` type, threaded via
the descriptor) is used as the hint when present, else a generic hint.
The `shape` is always computed generically.

## Surfaces

Spilling is an opt-in flag (`spillLargeOutput`) mirroring
`compactOutput`:

| Surface | `spillLargeOutput` | Behavior |
| --- | --- | --- |
| AI chat / agent | `true` (in `chat-execution.service.ts`) | Spill on;
nav tools + `code_interpreter` in catalog |
| External MCP clients | unset | Raw output |
| Workflow agents | unset | Raw output |
| `code_interpreter` sandbox bridge | unset (it's an MCP call) | Raw
output |

The sandbox bridge inherits "no spill" for free via the MCP path — no
header sniffing, no `ToolContext.source` field.

## Design constraints (anti-micro-OS)

Exactly two navigation tools, no composition/piping, read-only, bounded
output. The boundary is: expressible as a single path lookup or text
search → nav tool; aggregation/correlation/transform →
`code_interpreter`.

## Notes / deviations from the plan

- `jsonShapeSkeleton` and `ToolOutputSpillService` live under the `tool`
module (not `tool-provider/output-transforms`) to avoid a `tool →
tool-provider` import cycle.
- Spill files use `{ isTemporaryFile: false, toDelete: false }` (same as
`code_interpreter`); `isTemporaryFile` here means files-field promotion,
not a TTL.

## Test plan

- [x] `extract-json-path` + `search-output` util unit tests (23 cases)
- [x] `jsonShapeSkeleton` unit tests (6) and `ToolOutputSpillService`
unit tests (4)
- [x] oxlint + oxfmt clean on changed files; `twenty-server` typecheck
clean (pre-existing unrelated errors aside)
- [ ] Manual: trigger an oversized tool result in chat, confirm the
envelope is returned and `extract_json_path` / `search_output` read the
spilled file by `fileId`

## Why no automated e2e

Spilling is chat-only and the chat path runs a live model, so the
black-box MCP integration harness can't deterministically trigger a
spill (MCP intentionally doesn't spill). The seam is small, explicit
flag-threading mirrored on `compactOutput`, covered by the unit suites.

<!-- This is an auto-generated description by cubic. -->
<a
href="https://cubic.dev/pr/twentyhq/twenty/pull/21982?utm_source=github"
target="_blank" rel="noopener noreferrer"
data-no-image-dialog="true"><picture><source
media="(prefers-color-scheme: dark)"
srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source
media="(prefers-color-scheme: light)"
srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img
alt="Review in cubic"
src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a>
<!-- End of auto-generated description by cubic. -->
2026-06-23 10:46:06 +00:00

235 lines
7.8 KiB
JSON
Raw Blame History

{
"name": "twenty-server",
"description": "",
"author": "",
"private": true,
"license": "AGPL-3.0",
"scripts": {
"nx": "NX_DEFAULT_PROJECT=twenty-server node ../../node_modules/nx/bin/nx.js",
"start:prod": "node dist/main",
"command:prod": "node dist/command/command",
"worker:prod": "node dist/queue-worker/queue-worker",
"database:init:prod": "node dist/database/scripts/setup-db.js && yarn database:migrate:prod --force --include-slow",
"database:migrate:prod": "node dist/command/command run-instance-commands",
"clickhouse:migrate:prod": "node dist/database/clickHouse/migrations/run-migrations.js",
"typeorm": "../../node_modules/typeorm/.bin/typeorm"
},
"dependencies": {
"@ai-sdk/amazon-bedrock": "^4.0.116",
"@ai-sdk/anthropic": "^3.0.83",
"@ai-sdk/azure": "^3.0.55",
"@ai-sdk/google": "^3.0.81",
"@ai-sdk/mistral": "^3.0.38",
"@ai-sdk/openai": "^3.0.70",
"@ai-sdk/openai-compatible": "^2.0.49",
"@ai-sdk/provider-utils": "^4.0.28",
"@ai-sdk/xai": "^3.0.94",
"@aws-sdk/client-lambda": "3.1001.0",
"@aws-sdk/client-s3": "3.1001.0",
"@aws-sdk/client-sesv2": "3.1001.0",
"@aws-sdk/client-sts": "3.1001.0",
"@aws-sdk/credential-providers": "3.1001.0",
"@aws-sdk/s3-request-presigner": "3.1001.0",
"@azure/msal-node": "^5.2.3",
"@blocknote/server-util": "^0.51.4",
"@clickhouse/client": "^1.18.1",
"@e2b/code-interpreter": "^2.6.0",
"@envelop/core": "4.0.3",
"@faker-js/faker": "9.8.0",
"@file-type/pdf": "^0.2.0",
"@graphql-tools/schema": "10.0.4",
"@graphql-tools/utils": "9.2.1",
"@graphql-yoga/nestjs": "2.1.0",
"@jrmdayn/googleapis-batcher": "^0.10.1",
"@lifeomic/axios-fetch": "^3.1.0",
"@lingui/conf": "5.9.5",
"@lingui/core": "^5.9.5",
"@lingui/format-po": "5.9.5",
"@microsoft/microsoft-graph-client": "3.0.7",
"@microsoft/microsoft-graph-types": "^2.40.0",
"@nestjs/axios": "3.1.2",
"@nestjs/cache-manager": "^2.3.0",
"@nestjs/common": "11.1.24",
"@nestjs/config": "^4.0.4",
"@nestjs/core": "^11.1.24",
"@nestjs/event-emitter": "2.1.0",
"@nestjs/graphql": "patch:@nestjs/graphql@npm%3A13.4.2#~/.yarn/patches/@nestjs-graphql-npm-13.4.2-a41c900c19.patch",
"@nestjs/jwt": "11.0.1",
"@nestjs/passport": "11.0.5",
"@nestjs/platform-express": "11.1.24",
"@nestjs/schedule": "^6.0.1",
"@nestjs/serve-static": "5.0.5",
"@nestjs/terminus": "11.0.0",
"@nestjs/typeorm": "11.0.0",
"@node-saml/node-saml": "5.1.0",
"@node-saml/passport-saml": "^5.1.0",
"@opentelemetry/api": "^1.9.0",
"@opentelemetry/exporter-metrics-otlp-http": "^0.200.0",
"@opentelemetry/exporter-prometheus": "^0.217.0",
"@opentelemetry/sdk-metrics": "^2.0.0",
"@ptc-org/nestjs-query-core": "^9.4.0",
"@ptc-org/nestjs-query-graphql": "patch:@ptc-org/nestjs-query-graphql@npm%3A9.4.0#~/.yarn/patches/@ptc-org-nestjs-query-graphql-npm-9.4.0-8e6f7894e1.patch",
"@ptc-org/nestjs-query-typeorm": "patch:@ptc-org/nestjs-query-typeorm@npm%3A9.4.0#~/.yarn/patches/@ptc-org-nestjs-query-typeorm-npm-9.4.0-ca3414967e.patch",
"@react-email/render": "^1.2.3",
"@sentry/nestjs": "^10.51.0",
"@sentry/node": "^10.51.0",
"@sentry/profiling-node": "^10.51.0",
"@sniptt/guards": "0.2.0",
"addressparser": "1.0.1",
"ai": "6.0.97",
"archiver": "7.0.1",
"axios": "^1.16.0",
"axios-retry": "^4.5.0",
"bcrypt": "^6.0.0",
"bullmq": "5.78.0",
"bytes": "3.1.2",
"cache-manager": "^5.4.0",
"cache-manager-redis-yet": "^4.1.2",
"chalk": "4.1.2",
"class-transformer": "0.5.1",
"class-validator": "0.14.0",
"cloudflare": "^4.5.0",
"connect-redis": "^7.1.1",
"cron-parser": "5.1.1",
"dataloader": "2.2.2",
"date-fns": "4.4.0",
"digest-fetch": "^3.1.1",
"dompurify": "^3.4.11",
"dotenv": "16.4.5",
"email-reply-parser": "^2.3.5",
"express": "5.2.1",
"express-session": "^1.18.2",
"file-type": "^21.3.2",
"fuse.js": "^7.1.0",
"gaxios": "7.1.5",
"googleapis": "173.0.0",
"graphql": "16.8.1",
"graphql-fields": "2.0.3",
"graphql-redis-subscriptions": "2.7.0",
"graphql-scalars": "1.23.0",
"graphql-subscriptions": "2.0.0",
"graphql-tag": "2.12.6",
"graphql-type-json": "0.3.2",
"graphql-upload": "16.0.2",
"graphql-yoga": "4.0.5",
"html-to-text": "^9.0.5",
"imapflow": "1.4.2",
"ioredis": "5.10.1",
"jsdom": "^29.1.1",
"json-schema": "0.4.0",
"jsonwebtoken": "9.0.2",
"libphonenumber-js": "1.11.5",
"lodash.camelcase": "4.3.0",
"lodash.chunk": "4.2.0",
"lodash.compact": "3.0.1",
"lodash.differencewith": "^4.5.0",
"lodash.groupby": "4.6.0",
"lodash.isempty": "4.4.0",
"lodash.isequal": "4.5.0",
"lodash.isobject": "3.0.2",
"lodash.kebabcase": "4.1.1",
"lodash.merge": "^4.6.2",
"lodash.omit": "4.5.0",
"lodash.snakecase": "4.1.1",
"lodash.uniq": "^4.5.0",
"lodash.uniqby": "^4.7.0",
"microdiff": "1.4.0",
"mrmime": "^2.0.1",
"ms": "2.1.3",
"nest-commander": "^3.19.1",
"node-ical": "^0.21.0",
"nodemailer": "^9.0.1",
"openapi-types": "12.1.3",
"openid-client": "^5.7.0",
"otplib": "^12.0.1",
"passport": "^0.7.0",
"passport-google-oauth20": "2.0.0",
"passport-jwt": "4.0.1",
"passport-microsoft": "2.1.0",
"path-to-regexp": "^8.4.0",
"pg": "8.12.0",
"planer": "1.2.0",
"pluralize": "8.0.0",
"postal-mime": "^2.6.1",
"psl": "^1.9.0",
"re2": "^1.25.0",
"react": "^19.2.0",
"react-dom": "^19.2.0",
"redis": "^4.7.0",
"reflect-metadata": "0.2.2",
"rxjs": "7.8.1",
"semver": "7.6.3",
"sns-payload-validator": "^2.1.0",
"stripe": "20.4.1",
"tar": "^7.5.9",
"temporal-polyfill": "^0.3.0",
"tsconfig-paths": "^4.2.0",
"tsdav": "^2.2.0",
"tslib": "2.8.1",
"type-fest": "4.10.1",
"typeorm": "patch:typeorm@0.3.29#./patches/typeorm+0.3.29.patch",
"unzipper": "^0.12.3",
"uuid": "^11.1.1",
"zod": "^4.1.11"
},
"devDependencies": {
"@lingui/cli": "^5.9.5",
"@nestjs/cli": "^11.0.16",
"@nestjs/schematics": "^11.0.9",
"@nestjs/testing": "11.1.24",
"@swc/cli": "^0.8.1",
"@swc/core": "^1.15.11",
"@swc/jest": "^0.2.39",
"@types/addressparser": "^1.0.3",
"@types/archiver": "^6.0.0",
"@types/aws-lambda": "^8.10.147",
"@types/bcrypt": "^5.0.0",
"@types/bytes": "^3.1.1",
"@types/express": "^4.17.13",
"@types/express-session": "^1.18.0",
"@types/graphql-fields": "^1.3.6",
"@types/graphql-upload": "^16.0.7",
"@types/html-to-text": "^9.0.4",
"@types/jest": "^30.0.0",
"@types/lodash.chunk": "^4.2.9",
"@types/lodash.compact": "^3.0.9",
"@types/lodash.differencewith": "^4.5.9",
"@types/lodash.groupby": "^4.6.9",
"@types/lodash.isempty": "^4.4.7",
"@types/lodash.isequal": "^4.5.8",
"@types/lodash.isobject": "^3.0.7",
"@types/lodash.merge": "^4.6.9",
"@types/lodash.omit": "^4.5.9",
"@types/lodash.snakecase": "^4.1.7",
"@types/lodash.uniq": "^4.5.9",
"@types/lodash.uniqby": "^4.7.9",
"@types/ms": "^0.7.31",
"@types/node": "^24.0.0",
"@types/nodemailer": "^7.0.3",
"@types/passport-google-oauth20": "^2.0.11",
"@types/passport-jwt": "^3.0.8",
"@types/passport-microsoft": "^2.1.0",
"@types/pluralize": "^0.0.33",
"@types/psl": "^1.1.3",
"@types/react": "^19.2.0",
"@types/supertest": "^2.0.11",
"@types/unzipper": "^0",
"@types/uuid": "^9.0.2",
"jest": "29.7.0",
"jest-environment-node": "^29.4.1",
"prettier": "^3.1.1",
"rimraf": "^5.0.5",
"supertest": "^6.1.3",
"ts-jest": "^29.1.1",
"ts-node": "10.9.1",
"twenty-client-sdk": "workspace:*",
"twenty-emails": "workspace:*",
"twenty-shared": "workspace:*"
},
"engines": {
"node": "^24.5.0",
"npm": "please-use-yarn",
"yarn": "^4.0.2"
}
}
Reference in New Issue View Git Blame Copy Permalink