fix(server): mitigate integration-test OOM flakiness (#21588)

## Problem

`server-integration-test` shards have been failing intermittently across
unrelated PRs with a distinctive signature: the shard exits code 1 with
**no jest assertion failure, no `Test Suites:` summary, and no V8
`JavaScript heap out of memory` error** — the process just dies mid-run.
Failures hit random shards and clear on re-run (e.g. an unrelated branch
failed shard 6 once, then passed 3× on identical code), while the
`merge_group` gate stays green.

### Root cause

Each shard runs a **single in-band jest process** that boots one shared
NestJS app (`globalSetup` → `app.listen`) and holds it for the entire
shard, driving heavy metadata migrations + cache rebuilds in that one
process. `NODE_OPTIONS=--max-old-space-size=12288` let V8 grow to 12 GB
— *above* the `ubuntu-latest` runner's available RAM (16 GB, shared with
Postgres/Redis/ClickHouse). V8 therefore deferred aggressive GC and grew
past physical memory, so the **OS OOM-killer killed the process before
V8 hit its own ceiling** — which is why there's no heap error and no
jest summary, just a silent exit.

## Changes (CI/test-only — prod runtime untouched)

- **Lower the integration jest heap cap `12288` → `6144`** so V8
self-limits below physical headroom instead of being OS-killed.
Counterintuitively safer: a real leak now surfaces as a *visible* heap
error naming the test, rather than a silent death. (`database:reset`
keeps 12288 — it runs alone, before jest.)
- **Add `--logHeapUsage`** to the integration jest runs to expose the
per-file heap trend for confirming/pinpointing the growth.
- **Split integration tests across 16 shards (was 10)** to lower the
peak working set per shard.
- **Make perf logging a first-class `LoggerService` tool** (per
@prastoin's review): add `LoggerService.perf()` and unify the existing
`time()`/`timeEnd()` helpers into `perfTime()`/`perfTimeEnd()` (now
routed through the driver), all gated by a new `PERF_LOG_ENABLED` config
var. It **defaults on** so real environments keep emitting the
install-perf logs, and `.env.test` sets it `false` to mute the
per-action flood in integration tests. The `application-manifest` and
`validate-build` services were moved from the built-in `Logger` to
`LoggerService` to use it.

## Notes

- `--max-old-space-size` lives only in the `test:integration` nx target;
it is **not** the prod server heap setting, so prod is unaffected.
- This is mitigation. If `--logHeapUsage` shows monotonic growth across
files, there's a real accumulation in the long-lived app (retained
flat-maps / metadata cache) worth a follow-up heap-snapshot fix.

<!-- This is an auto-generated description by cubic. -->
<a
href="https://cubic.dev/pr/twentyhq/twenty/pull/21588?utm_source=github"
target="_blank" rel="noopener noreferrer"
data-no-image-dialog="true"><picture><source
media="(prefers-color-scheme: dark)"
srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source
media="(prefers-color-scheme: light)"
srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img
alt="Review in cubic"
src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a>
<!-- End of auto-generated description by cubic. -->
This commit is contained in:
Charles Bochet
2026-06-15 15:54:51 +02:00
committed by GitHub
parent 88b9294afd
commit 97871131a1
16 changed files with 191 additions and 73 deletions
+2 -2
View File
@@ -302,7 +302,7 @@ jobs:
strategy:
fail-fast: false
matrix:
shard: [1, 2, 3, 4, 5, 6, 7, 8, 9, 10]
shard: [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16]
services:
postgres:
image: postgres:18
@@ -338,7 +338,7 @@ jobs:
ANALYTICS_ENABLED: true
CLICKHOUSE_URL: "http://default:clickhousePassword@localhost:8123/twenty"
CLICKHOUSE_PASSWORD: clickhousePassword
SHARD_COUNTER: 10
SHARD_COUNTER: 16
steps:
- name: Fetch custom Github Actions and base branch history
uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5 # v4.3.1