## Summary
`search_output` (the spilled-output navigation tool) was built around a
JSON-centric, line-based model that breaks for the data it actually
receives. Spilled outputs are written as compact
`JSON.stringify(output)` (single line, escaped newlines), so the tool's
line-by-line matching collapsed to at most one match, and its schema
described searching "the indented JSON representation" even though it
falls back to raw text for non-JSON. It also ran arbitrary,
model-supplied regexes through the native engine with no ReDoS
protection.
This reworks the tool into a `grep -o` style search over the raw file
bytes: it finds every occurrence of a pattern regardless of newlines and
returns a character window around each hit. It works uniformly for
compact/pretty JSON, CSV, HTML, and plain text.
## Changes
- **Occurrence-based matching** (`search-output.util.ts`): search the
raw content for every match via a global-regex `exec` loop (with a
zero-width-match guard), bounded by `offset + maxMatches`. Results are
now `{ charOffset, match, context }` with a character window around each
occurrence and a centered-ellipsis cap for very long single matches. The
line model (`split`, line numbers, line context) is removed.
- **ReDoS hardening**: matching now uses `re2` (already a dependency)
with the global flag, guaranteeing linear-time matching. Unsupported
regex features (lookahead/backreferences) and invalid patterns fall back
to escaped-literal search instead of throwing.
- **No more reserialization** (`search-output-tool.ts`): the
`JSON.stringify(JSON.parse(...))` round-trip is gone; the tool searches
the exact bytes on disk, so there is no coordinate divergence with
`extract_json_paths`.
- **API** (`search-output-tool.schema.ts`): `contextLines` →
`contextChars` (default 100, max 2000); honest descriptions reflecting
raw-text occurrence search and the regex-or-literal fallback. The result
message reports occurrence counts.
- **Cleanup**: removed unused constants
(`default-search-output-context-lines`,
`search-output-max-line-length`); added
`default-search-output-context-chars` and
`search-output-max-match-length`.
`extract_json_paths` and the spill service are untouched.
## Tradeoff
Results use character offsets/windows rather than line numbers and line
context. For an LLM extracting values from a spilled blob this is more
robust (works on single-line content); the cost is no line-based context
for genuinely line-structured content.
## Test plan
- [x] `search-output.util.spec.ts` rewritten for occurrence semantics:
multiple hits on a single newline-free line, zero-width-pattern
termination, catastrophic-backtracking pattern stays fast (RE2),
lookahead/invalid-regex literal fallback, char-window clipping, offset
pagination, long-match truncation. 12/12 pass.
- [x] `npx nx typecheck twenty-server` clean.
- [x] `npx nx lint:diff-with-main twenty-server` clean (lint + format).
## Deploy note
`re2` is a native addon. It was declared in `package.json` but never
imported/built before this PR, so its binary may be absent in some
environments (local install required `npm rebuild re2`). Confirm the
install/build pipeline (CI, Docker images) compiles native modules so
the tool doesn't throw `Cannot find module 're2.node'` at runtime.
<!-- This is an auto-generated description by cubic. -->
<a
href="https://cubic.dev/pr/twentyhq/twenty/pull/22034?utm_source=github"
target="_blank" rel="noopener noreferrer"
data-no-image-dialog="true"><picture><source
media="(prefers-color-scheme: dark)"
srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source
media="(prefers-color-scheme: light)"
srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img
alt="Review in cubic"
src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a>
<!-- End of auto-generated description by cubic. -->
Etienne
0f4c4e69a9
