## What this is
close https://github.com/twentyhq/core-team-issues/issues/2622
A **POC / discussion branch** implementing the runner-scoped alternative
to the `__warmedUpCache` design in
[core-team-issues#2622](https://github.com/twentyhq/core-team-issues/issues/2622).
Not for merge as-is — meant to diff against that plan.
## Problem
`deriveSearchVectorAsExpressionForTsVectorField` scans the entire
`flatSearchFieldMetadataMaps` (`Object.values(...).filter(...)`) once
per object created in a migration. On install that's `O(objectsCreated ×
totalSearchFields)` — the quadratic #2622 targets.
Only **one** of the three call sites is actually hot:
- `create-object` (runner) — global maps, called per created object →
the quadratic
- export DDL — maps already built **per object** (O(k))
- `update-field` rebuild — one field, gated on `rebuildSearchVector`
## Approach
Instead of a private `__warmedUpCache` side-channel on
`FlatEntityMaps<T>` + drain-on-hydration, this keeps the index in the
**consumer**:
1. `derive` now takes `targetSearchFieldMetadatas` (already scoped to
the tsVector field) instead of scanning the map itself.
2. The runner builds a `Map<tsVectorFieldMetadataId, searchFields[]>`
**once per migration**, lazily, and threads it through the action
context. Safe because `searchFieldMetadata` creates are ordered before
`objectMetadata` creates (`computeOrderedMigrationActions`), so the map
is complete on first use. → `O(totalSearchFields)`.
3. `getTargetSearchFieldMetadatasForTsVectorField` (O(total) filter)
stays as the fallback for the one-off callers (export, field-update) and
when the accessor isn't provided.
## Why this over `__warmedUpCache`
- **No `FlatEntityMaps<T>` type widening**, no convention-only privacy,
no id/universalIdentifier drain to keep in sync.
- **No referential-integrity obligation.** The index only ever contains
entities present in the map, so the "search field created-then-deleted
before its object hydrates" case (deferred as an edge in #2622) can't
put a stale id into an aggregator and crash `derive` via the `-orThrow`
lookup.
- **One `derive` path**, not "aggregator + direct-filter fallback for
export".
- Blast radius: ~220 lines, mostly a new util + test.
## Benchmark (micro, isolated function)
Median of 7 trials, 10 search fields per object, running the real
shipped utils — old = `getTargetSearchFieldMetadatasForTsVectorField`
once per object (identical to the old inline scan), new =
`buildSearchFieldMetadatasByTsVectorFieldId` once + N lookups (both
assert they resolve the same fields):
| objects | total search fields | old (scan/obj) | new (index once) |
speedup |
|--------:|--------------------:|---------------:|-----------------:|--------:|
| 50 | 500 | 2.08 ms | 0.06 ms | 33× |
| 100 | 1,000 | 9.26 ms | 0.12 ms | 77× |
| 200 | 2,000 | 36.2 ms | 0.23 ms | 160× |
| 400 | 4,000 | 151 ms | 0.40 ms | 379× |
| 800 | 8,000 | 701 ms | 0.92 ms | 766× |
Confirms the old path is quadratic (~4× per doubling of object count)
and the new path is linear (sub-ms throughout).
**Caveats — read these before trusting the speedup:**
- This is the **isolated derivation function**, no DB / DDL / inserts.
In a real `create-object` action the derive is a small fraction of
per-action cost, so the end-to-end win is far smaller than the ratios
above.
- A default workspace has ~20–30 objects, where the **old** code already
costs only ~1–2 ms total across the whole install. The quadratic only
becomes material (>50 ms, the runner's slow-action threshold) around
**200–400 objects**.
- The measurement that should actually gate this — `[install-perf]
create:objectMetadata` on a real install against a real DB with a few
hundred objects — has **not** been run yet. The micro-benchmark bounds
the upside and locates the knee of the curve; it does not prove
end-to-end payoff.
## Not done on purpose
- **No end-to-end benchmark yet** — step 0 should still be measuring
`[install-perf] create:objectMetadata` on a real large install to
confirm the quadratic is worth removing at all.
- Relies on the ordering invariant (commented at the build site). The
fully self-contained variant is to put the object's search fields on
`FlatCreateObjectAction` (builder change) — deliberately left out to
keep this runner-scoped.
## Checks
`nx typecheck twenty-server`, `nx lint:diff-with-main twenty-server`,
new util spec + existing `generate-workspace-schema-ddl` spec all green.
Paul Rastoin
2b1417770e