## Context
Prod investigation (Sentry, last 7 days) traced the current slowness to
the per-pod workspace cache. Every recompute of a `localDataOnly` key
(`ORMEntityMetadatas`, `flatWorkspaceMemberMaps`) published a fresh
`crypto.randomUUID()` as the shared Redis validation hash. Because these
keys recover from a hash mismatch by recomputing (their data never
enters Redis), one miss on one pod invalidated the local copy on every
other pod; each of their recomputes minted yet another hash,
re-invalidating everyone else. The fleet never converges.
Measured impact in prod:
- The `ORMEntityMetadatas` rebuild (full `objectMetadata` +
`fieldMetadata` + `application` queries, ~220ms combined, plus
`EntityMetadataBuilder.build`) ran **~963k times in 24h** (~11/s),
roughly 58h of cumulative Postgres time per day.
- The hottest single workspace recomputed its schema metadata 51k
times/day (once per 1.7s).
- Second-order effects: `POST /metadata` averaged 26.6s (p95 2.3s, so a
tail hangs for minutes on pool/event-loop starvation), GraphQL p95 went
846ms (v2.20.0) to 1744ms (v2.21.0), `Query read timeout` on trivial
cron queries at 18x baseline.
The random hash was correct in the original design (#15962): it is a
generation token, and Redis-backed keys recover absorptively by adopting
hash+data from Redis. #16287 added `localOnly` keys (EntityMetadata[] is
not serializable) whose recovery is generative, which silently broke the
invariant later documented in #18649 ("hashes change only on
invalidateAndRecompute").
## What this does
- Recovery recomputes now **adopt** the hash already present in Redis
instead of minting a new one, and write nothing back. A miss costs one
recompute on one pod instead of an unbounded fleet-wide loop.
- Minting is reserved for `invalidateAndRecompute` (real metadata
changes, propagation semantics unchanged, including the frontend
collectionHashes contract) and the bootstrap case where Redis has no
hash.
- The bootstrap write uses **SET NX** (new
`CacheStorageService.setIfAbsent`) instead of a plain overwrite: a slow
bootstrap recompute could otherwise land after a concurrent
`invalidateAndRecompute` mint and clobber it with a hash of
pre-migration data. Under the old code that clobber self-healed via the
cascade; with adopt semantics it would pin stale data, so the bootstrap
write must lose that race. A losing pod keeps its result locally as
provisional and converges on the winning hash at its next revalidation
(covered by a dedicated race test).
Redis-backed keys are untouched: same fetch-on-mismatch recovery, same
mint-and-write on `missingInRedis`.
## Expected effect and how to verify
`FieldMetadataEntity`/`ObjectMetadataEntity`/`ApplicationEntity`
full-workspace query counts in Sentry should collapse from ~1M/day to
the true metadata-change rate, and with them the DB pool pressure behind
the `/metadata` latency tail. This also makes local-cache eviction
(`MAX_LOCAL_CACHE_ENTRIES`, #22946) cheap: the cap can be tuned purely
for RAM.
Complementary to, not competing with, the planned Redis pub/sub
invalidation: a version token in Redis is still needed for restart
catch-up, and this PR gives it sound semantics.
---
_Generated by [Claude
Code](https://claude.ai/code/session_01T3JUHwXJHPmZDZTrv6YTDi)_
<!-- This is an auto-generated description by cubic. -->
<a
href="https://cubic.dev/pr/twentyhq/twenty/pull/22980?utm_source=github"
target="_blank" rel="noopener noreferrer"
data-no-image-dialog="true"><picture><source
media="(prefers-color-scheme: dark)"
srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source
media="(prefers-color-scheme: light)"
srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img
alt="Review in cubic"
src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a>
<!-- End of auto-generated description by cubic. -->
Félix Malfait
d4ac6e752b
