perf(server): raise workspace local cache size and meter evictions (#21954)
## Context The per-pod in-process workspace metadata cache (`WorkspaceCacheService`) evicts by a fixed **1,000-entry count**. Each workspace's cached metadata is ~1 MB (dominated by the flat `field-metadata` map) over ~10–13 entries, so 1,000 entries ≈ only a few dozen workspaces per pod. On a multi-tenant instance with far more active workspaces, the L1 cache thrashes — LRU-evicting and re-fetching the ~1 MB of maps from Redis on misses — and since the cache sits in one AZ while pods span both, ~half of that transfer is billed cross-AZ. (In prod this cache node serves ~2.6 TB/day.) ## What this does - **Raise `MAX_LOCAL_CACHE_ENTRIES` 1,000 → 7,500** (~500 workspaces at ~1 MB each; server pods are 4 GiB / `--max-old-space-size=3500`, so this stays well within the heap). - **Add a `workspace-metadata-cache/local-eviction` counter** (incremented by the number of entries dropped each time the cache hits capacity) so we can see capacity-driven evictions in metrics and tune the limit from real data rather than guessing. Eviction stays **batched** (`MIN_EVICT_KEYS`), so the sort runs about once per 100 inserts at steady state rather than on every write. ### Why count, not bytes An earlier iteration bounded by measured bytes, but that required `JSON.stringify`-ing every cached value (incl. the ~1 MB field-metadata maps) on every write — meaningful CPU/GC overhead on the fill path. A raised count cap avoids that entirely; the new eviction metric gives us the signal to right-size it. No change to cache semantics, hashing, or the Redis format.
This commit is contained in:
+9
-1
@@ -10,6 +10,8 @@ import { WorkspaceCacheProvider } from 'src/engine/workspace-cache/interfaces/wo
|
||||
import { InjectCacheStorage } from 'src/engine/core-modules/cache-storage/decorators/cache-storage.decorator';
|
||||
import { CacheStorageService } from 'src/engine/core-modules/cache-storage/services/cache-storage.service';
|
||||
import { CacheStorageNamespace } from 'src/engine/core-modules/cache-storage/types/cache-storage-namespace.enum';
|
||||
import { MetricsService } from 'src/engine/core-modules/metrics/metrics.service';
|
||||
import { MetricsKeys } from 'src/engine/core-modules/metrics/types/metrics-keys.type';
|
||||
import { PromiseMemoizer } from 'src/engine/twenty-orm/storage/promise-memoizer.storage';
|
||||
import {
|
||||
WORKSPACE_CACHE_KEY,
|
||||
@@ -33,7 +35,7 @@ const LOCAL_ENTRY_TTL_MS = 30 * 60 * 1000; // 30 minutes
|
||||
const MEMOIZER_TTL_MS = 10_000; // 10 seconds
|
||||
const STALE_VERSION_TTL_MS = 5_000; // 5 seconds
|
||||
const MAX_LOCAL_STALE_VERSIONS = 5; // 5 stale versions
|
||||
const MAX_LOCAL_CACHE_ENTRIES = 1_000;
|
||||
const MAX_LOCAL_CACHE_ENTRIES = 7_500;
|
||||
const MIN_EVICT_KEYS = 100;
|
||||
|
||||
type CacheDataType = WorkspaceCacheDataMap[WorkspaceCacheKeyName];
|
||||
@@ -60,6 +62,7 @@ export class WorkspaceCacheService implements OnModuleInit {
|
||||
private readonly cacheStorage: CacheStorageService,
|
||||
private readonly discoveryService: DiscoveryService,
|
||||
private readonly reflector: Reflector,
|
||||
private readonly metricsService: MetricsService,
|
||||
) {}
|
||||
|
||||
async onModuleInit() {
|
||||
@@ -444,6 +447,11 @@ export class WorkspaceCacheService implements OnModuleInit {
|
||||
for (const [key] of toEvict) {
|
||||
this.localCache.delete(key);
|
||||
}
|
||||
|
||||
this.metricsService.incrementCounterBy({
|
||||
key: MetricsKeys.WorkspaceMetadataCacheLocalEviction,
|
||||
amount: toEvict.length,
|
||||
});
|
||||
}
|
||||
|
||||
private cleanupStaleVersions(
|
||||
|
||||
Reference in New Issue
Block a user