perf(server): raise workspace local cache size and meter evictions (#21954)

## Context

The per-pod in-process workspace metadata cache
(`WorkspaceCacheService`) evicts by a fixed **1,000-entry count**. Each
workspace's cached metadata is ~1 MB (dominated by the flat
`field-metadata` map) over ~10–13 entries, so 1,000 entries ≈ only a few
dozen workspaces per pod. On a multi-tenant instance with far more
active workspaces, the L1 cache thrashes — LRU-evicting and re-fetching
the ~1 MB of maps from Redis on misses — and since the cache sits in one
AZ while pods span both, ~half of that transfer is billed cross-AZ. (In
prod this cache node serves ~2.6 TB/day.)

## What this does

- **Raise `MAX_LOCAL_CACHE_ENTRIES` 1,000 → 7,500** (~500 workspaces at
~1 MB each; server pods are 4 GiB / `--max-old-space-size=3500`, so this
stays well within the heap).
- **Add a `workspace-metadata-cache/local-eviction` counter**
(incremented by the number of entries dropped each time the cache hits
capacity) so we can see capacity-driven evictions in metrics and tune
the limit from real data rather than guessing.

Eviction stays **batched** (`MIN_EVICT_KEYS`), so the sort runs about
once per 100 inserts at steady state rather than on every write.

### Why count, not bytes
An earlier iteration bounded by measured bytes, but that required
`JSON.stringify`-ing every cached value (incl. the ~1 MB field-metadata
maps) on every write — meaningful CPU/GC overhead on the fill path. A
raised count cap avoids that entirely; the new eviction metric gives us
the signal to right-size it.

No change to cache semantics, hashing, or the Redis format.
This commit is contained in:
Charles Bochet
2026-06-22 17:00:29 +02:00
committed by GitHub
parent 4dd9253d01
commit 5f94ee3e02
4 changed files with 19 additions and 2 deletions
@@ -53,4 +53,5 @@ export enum MetricsKeys {
AiChatTurnLatencyMs = 'ai-chat/turn-latency-ms',
AiChatStepLatencyMs = 'ai-chat/step-latency-ms',
AiChatTtftMs = 'ai-chat/ttft-ms',
WorkspaceMetadataCacheLocalEviction = 'workspace-metadata-cache/local-eviction',
}
@@ -5,6 +5,7 @@ import { WorkspaceCacheProvider } from 'src/engine/workspace-cache/interfaces/wo
import { type CacheStorageService } from 'src/engine/core-modules/cache-storage/services/cache-storage.service';
import { CacheStorageNamespace } from 'src/engine/core-modules/cache-storage/types/cache-storage-namespace.enum';
import { MetricsService } from 'src/engine/core-modules/metrics/metrics.service';
import { WORKSPACE_CACHE_KEY } from 'src/engine/workspace-cache/decorators/workspace-cache.decorator';
import { WorkspaceCacheService } from 'src/engine/workspace-cache/services/workspace-cache.service';
@@ -61,6 +62,12 @@ describe('WorkspaceCacheService', () => {
get: jest.fn(),
},
},
{
provide: MetricsService,
useValue: {
incrementCounterBy: jest.fn(),
},
},
],
}).compile();
@@ -10,6 +10,8 @@ import { WorkspaceCacheProvider } from 'src/engine/workspace-cache/interfaces/wo
import { InjectCacheStorage } from 'src/engine/core-modules/cache-storage/decorators/cache-storage.decorator';
import { CacheStorageService } from 'src/engine/core-modules/cache-storage/services/cache-storage.service';
import { CacheStorageNamespace } from 'src/engine/core-modules/cache-storage/types/cache-storage-namespace.enum';
import { MetricsService } from 'src/engine/core-modules/metrics/metrics.service';
import { MetricsKeys } from 'src/engine/core-modules/metrics/types/metrics-keys.type';
import { PromiseMemoizer } from 'src/engine/twenty-orm/storage/promise-memoizer.storage';
import {
WORKSPACE_CACHE_KEY,
@@ -33,7 +35,7 @@ const LOCAL_ENTRY_TTL_MS = 30 * 60 * 1000; // 30 minutes
const MEMOIZER_TTL_MS = 10_000; // 10 seconds
const STALE_VERSION_TTL_MS = 5_000; // 5 seconds
const MAX_LOCAL_STALE_VERSIONS = 5; // 5 stale versions
const MAX_LOCAL_CACHE_ENTRIES = 1_000;
const MAX_LOCAL_CACHE_ENTRIES = 7_500;
const MIN_EVICT_KEYS = 100;
type CacheDataType = WorkspaceCacheDataMap[WorkspaceCacheKeyName];
@@ -60,6 +62,7 @@ export class WorkspaceCacheService implements OnModuleInit {
private readonly cacheStorage: CacheStorageService,
private readonly discoveryService: DiscoveryService,
private readonly reflector: Reflector,
private readonly metricsService: MetricsService,
) {}
async onModuleInit() {
@@ -444,6 +447,11 @@ export class WorkspaceCacheService implements OnModuleInit {
for (const [key] of toEvict) {
this.localCache.delete(key);
}
this.metricsService.incrementCounterBy({
key: MetricsKeys.WorkspaceMetadataCacheLocalEviction,
amount: toEvict.length,
});
}
private cleanupStaleVersions(
@@ -2,10 +2,11 @@ import { Module } from '@nestjs/common';
import { DiscoveryModule } from '@nestjs/core';
import { CacheStorageModule } from 'src/engine/core-modules/cache-storage/cache-storage.module';
import { MetricsModule } from 'src/engine/core-modules/metrics/metrics.module';
import { WorkspaceCacheService } from 'src/engine/workspace-cache/services/workspace-cache.service';
@Module({
imports: [CacheStorageModule, DiscoveryModule],
imports: [CacheStorageModule, DiscoveryModule, MetricsModule],
providers: [WorkspaceCacheService],
exports: [WorkspaceCacheService],
})