perf(server): raise workspace local cache size and meter evictions (#21954)
## Context The per-pod in-process workspace metadata cache (`WorkspaceCacheService`) evicts by a fixed **1,000-entry count**. Each workspace's cached metadata is ~1 MB (dominated by the flat `field-metadata` map) over ~10–13 entries, so 1,000 entries ≈ only a few dozen workspaces per pod. On a multi-tenant instance with far more active workspaces, the L1 cache thrashes — LRU-evicting and re-fetching the ~1 MB of maps from Redis on misses — and since the cache sits in one AZ while pods span both, ~half of that transfer is billed cross-AZ. (In prod this cache node serves ~2.6 TB/day.) ## What this does - **Raise `MAX_LOCAL_CACHE_ENTRIES` 1,000 → 7,500** (~500 workspaces at ~1 MB each; server pods are 4 GiB / `--max-old-space-size=3500`, so this stays well within the heap). - **Add a `workspace-metadata-cache/local-eviction` counter** (incremented by the number of entries dropped each time the cache hits capacity) so we can see capacity-driven evictions in metrics and tune the limit from real data rather than guessing. Eviction stays **batched** (`MIN_EVICT_KEYS`), so the sort runs about once per 100 inserts at steady state rather than on every write. ### Why count, not bytes An earlier iteration bounded by measured bytes, but that required `JSON.stringify`-ing every cached value (incl. the ~1 MB field-metadata maps) on every write — meaningful CPU/GC overhead on the fill path. A raised count cap avoids that entirely; the new eviction metric gives us the signal to right-size it. No change to cache semantics, hashing, or the Redis format.
This commit is contained in:
@@ -53,4 +53,5 @@ export enum MetricsKeys {
|
||||
AiChatTurnLatencyMs = 'ai-chat/turn-latency-ms',
|
||||
AiChatStepLatencyMs = 'ai-chat/step-latency-ms',
|
||||
AiChatTtftMs = 'ai-chat/ttft-ms',
|
||||
WorkspaceMetadataCacheLocalEviction = 'workspace-metadata-cache/local-eviction',
|
||||
}
|
||||
|
||||
+7
@@ -5,6 +5,7 @@ import { WorkspaceCacheProvider } from 'src/engine/workspace-cache/interfaces/wo
|
||||
|
||||
import { type CacheStorageService } from 'src/engine/core-modules/cache-storage/services/cache-storage.service';
|
||||
import { CacheStorageNamespace } from 'src/engine/core-modules/cache-storage/types/cache-storage-namespace.enum';
|
||||
import { MetricsService } from 'src/engine/core-modules/metrics/metrics.service';
|
||||
import { WORKSPACE_CACHE_KEY } from 'src/engine/workspace-cache/decorators/workspace-cache.decorator';
|
||||
import { WorkspaceCacheService } from 'src/engine/workspace-cache/services/workspace-cache.service';
|
||||
|
||||
@@ -61,6 +62,12 @@ describe('WorkspaceCacheService', () => {
|
||||
get: jest.fn(),
|
||||
},
|
||||
},
|
||||
{
|
||||
provide: MetricsService,
|
||||
useValue: {
|
||||
incrementCounterBy: jest.fn(),
|
||||
},
|
||||
},
|
||||
],
|
||||
}).compile();
|
||||
|
||||
|
||||
+9
-1
@@ -10,6 +10,8 @@ import { WorkspaceCacheProvider } from 'src/engine/workspace-cache/interfaces/wo
|
||||
import { InjectCacheStorage } from 'src/engine/core-modules/cache-storage/decorators/cache-storage.decorator';
|
||||
import { CacheStorageService } from 'src/engine/core-modules/cache-storage/services/cache-storage.service';
|
||||
import { CacheStorageNamespace } from 'src/engine/core-modules/cache-storage/types/cache-storage-namespace.enum';
|
||||
import { MetricsService } from 'src/engine/core-modules/metrics/metrics.service';
|
||||
import { MetricsKeys } from 'src/engine/core-modules/metrics/types/metrics-keys.type';
|
||||
import { PromiseMemoizer } from 'src/engine/twenty-orm/storage/promise-memoizer.storage';
|
||||
import {
|
||||
WORKSPACE_CACHE_KEY,
|
||||
@@ -33,7 +35,7 @@ const LOCAL_ENTRY_TTL_MS = 30 * 60 * 1000; // 30 minutes
|
||||
const MEMOIZER_TTL_MS = 10_000; // 10 seconds
|
||||
const STALE_VERSION_TTL_MS = 5_000; // 5 seconds
|
||||
const MAX_LOCAL_STALE_VERSIONS = 5; // 5 stale versions
|
||||
const MAX_LOCAL_CACHE_ENTRIES = 1_000;
|
||||
const MAX_LOCAL_CACHE_ENTRIES = 7_500;
|
||||
const MIN_EVICT_KEYS = 100;
|
||||
|
||||
type CacheDataType = WorkspaceCacheDataMap[WorkspaceCacheKeyName];
|
||||
@@ -60,6 +62,7 @@ export class WorkspaceCacheService implements OnModuleInit {
|
||||
private readonly cacheStorage: CacheStorageService,
|
||||
private readonly discoveryService: DiscoveryService,
|
||||
private readonly reflector: Reflector,
|
||||
private readonly metricsService: MetricsService,
|
||||
) {}
|
||||
|
||||
async onModuleInit() {
|
||||
@@ -444,6 +447,11 @@ export class WorkspaceCacheService implements OnModuleInit {
|
||||
for (const [key] of toEvict) {
|
||||
this.localCache.delete(key);
|
||||
}
|
||||
|
||||
this.metricsService.incrementCounterBy({
|
||||
key: MetricsKeys.WorkspaceMetadataCacheLocalEviction,
|
||||
amount: toEvict.length,
|
||||
});
|
||||
}
|
||||
|
||||
private cleanupStaleVersions(
|
||||
|
||||
@@ -2,10 +2,11 @@ import { Module } from '@nestjs/common';
|
||||
import { DiscoveryModule } from '@nestjs/core';
|
||||
|
||||
import { CacheStorageModule } from 'src/engine/core-modules/cache-storage/cache-storage.module';
|
||||
import { MetricsModule } from 'src/engine/core-modules/metrics/metrics.module';
|
||||
import { WorkspaceCacheService } from 'src/engine/workspace-cache/services/workspace-cache.service';
|
||||
|
||||
@Module({
|
||||
imports: [CacheStorageModule, DiscoveryModule],
|
||||
imports: [CacheStorageModule, DiscoveryModule, MetricsModule],
|
||||
providers: [WorkspaceCacheService],
|
||||
exports: [WorkspaceCacheService],
|
||||
})
|
||||
|
||||
Reference in New Issue
Block a user