Etienne
0f4c4e69a9
## Summary
`search_output` (the spilled-output navigation tool) was built around a
JSON-centric, line-based model that breaks for the data it actually
receives. Spilled outputs are written as compact
`JSON.stringify(output)` (single line, escaped newlines), so the tool's
line-by-line matching collapsed to at most one match, and its schema
described searching "the indented JSON representation" even though it
falls back to raw text for non-JSON. It also ran arbitrary,
model-supplied regexes through the native engine with no ReDoS
protection.
This reworks the tool into a `grep -o` style search over the raw file
bytes: it finds every occurrence of a pattern regardless of newlines and
returns a character window around each hit. It works uniformly for
compact/pretty JSON, CSV, HTML, and plain text.
## Changes
- **Occurrence-based matching** (`search-output.util.ts`): search the
raw content for every match via a global-regex `exec` loop (with a
zero-width-match guard), bounded by `offset + maxMatches`. Results are
now `{ charOffset, match, context }` with a character window around each
occurrence and a centered-ellipsis cap for very long single matches. The
line model (`split`, line numbers, line context) is removed.
- **ReDoS hardening**: matching now uses `re2` (already a dependency)
with the global flag, guaranteeing linear-time matching. Unsupported
regex features (lookahead/backreferences) and invalid patterns fall back
to escaped-literal search instead of throwing.
- **No more reserialization** (`search-output-tool.ts`): the
`JSON.stringify(JSON.parse(...))` round-trip is gone; the tool searches
the exact bytes on disk, so there is no coordinate divergence with
`extract_json_paths`.
- **API** (`search-output-tool.schema.ts`): `contextLines` →
`contextChars` (default 100, max 2000); honest descriptions reflecting
raw-text occurrence search and the regex-or-literal fallback. The result
message reports occurrence counts.
- **Cleanup**: removed unused constants
(`default-search-output-context-lines`,
`search-output-max-line-length`); added
`default-search-output-context-chars` and
`search-output-max-match-length`.
`extract_json_paths` and the spill service are untouched.
## Tradeoff
Results use character offsets/windows rather than line numbers and line
context. For an LLM extracting values from a spilled blob this is more
robust (works on single-line content); the cost is no line-based context
for genuinely line-structured content.
## Test plan
- [x] `search-output.util.spec.ts` rewritten for occurrence semantics:
multiple hits on a single newline-free line, zero-width-pattern
termination, catastrophic-backtracking pattern stays fast (RE2),
lookahead/invalid-regex literal fallback, char-window clipping, offset
pagination, long-match truncation. 12/12 pass.
- [x] `npx nx typecheck twenty-server` clean.
- [x] `npx nx lint:diff-with-main twenty-server` clean (lint + format).
## Deploy note
`re2` is a native addon. It was declared in `package.json` but never
imported/built before this PR, so its binary may be absent in some
environments (local install required `npm rebuild re2`). Confirm the
install/build pipeline (CI, Docker images) compiles native modules so
the tool doesn't throw `Cannot find module 're2.node'` at runtime.
<!-- This is an auto-generated description by cubic. -->
<a
href="https://cubic.dev/pr/twentyhq/twenty/pull/22034?utm_source=github"
target="_blank" rel="noopener noreferrer"
data-no-image-dialog="true"><picture><source
media="(prefers-color-scheme: dark)"
srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source
media="(prefers-color-scheme: light)"
srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img
alt="Review in cubic"
src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a>
<!-- End of auto-generated description by cubic. -->
The #1 Open-Source CRM
Website · 
Documentation · 
Roadmap · 
Discord · 
Figma
Why Twenty
Twenty gives technical teams the building blocks for a custom CRM that meets complex business needs and quickly adapts as the business evolves. Twenty is the CRM you build, ship, and version like the rest of your stack.
Learn more about why we built Twenty
Installation
Cloud
The fastest way to get started. Sign up at twenty.com and spin up a workspace in under a minute, with no infrastructure to manage and always up to date.
Build an app
Scaffold a new app with the Twenty CLI:
npx create-twenty-app my-app
Define objects, fields, and views as code:
import { defineObject, FieldType } from 'twenty-sdk/define';
export default defineObject({
nameSingular: 'deal',
namePlural: 'deals',
labelSingular: 'Deal',
labelPlural: 'Deals',
fields: [
{ name: 'name', label: 'Name', type: FieldType.TEXT },
{ name: 'amount', label: 'Amount', type: FieldType.CURRENCY },
{ name: 'closeDate', label: 'Close Date', type: FieldType.DATE_TIME },
],
});
Then ship it to your workspace:
npx twenty app:publish --private
See the app development guide for objects, views, agents, and logic functions.
Self-hosting
Run Twenty on your own infrastructure with Docker Compose, or contribute locally via the local setup guide.
Everything you need
Twenty gives you the building blocks of a modern CRM (objects, views, workflows, and agents) and lets you extend them as code. Here's a tour of what's in the box.
Want to go deeper? Read the 
User Guide for product walkthroughs, or the Documentation for developer reference.
|
|
|
|
|
|
Learn more about version control in doc
Stack
TypeScript
Nx
NestJS, with BullMQ, 
PostgreSQL, 
Redis
React, with Jotai, Linaria and Lingui
Thanks
Thanks to these amazing services that we use and recommend for code review (Greptile), catching bugs (Sentry) and translating (Crowdin).
Join the Community
Star the repo · Discord · 
Feature requests · Releases · 
X · 
LinkedIn · 
Crowdin · 
Contribute