Rekey metadata caches when flat map hashes change (#23164)
## Context
The `/metadata` GraphQL response cache (`ObjectMetadataItems`,
`FindAllViews`) and the workspace SDL cache were keyed on
`workspace.metadataVersion`, an integer bumped on every object/field
migration. That mechanism is legacy (the migration runner literally
calls it `getLegacyCacheInvalidationPromises`): the data plane already
moved to `WorkspaceCacheService`, which versions each flat entity map
with its own hash minted on invalidation.
Version keying had two concrete costs: the version bump was the only
proactive invalidation for `ObjectMetadataItems`, and keeping
`FindAllViews` fresh required `flushGraphQLOperation`, a full Redis
keyspace SCAN on every relevant migration. It is also the main blocker
for deprecating `metadataVersion` entirely.
This PR re-keys both caches on the flat-map hashes instead.
## What changed
**Response cache** (`use-cached-metadata.ts`): the key is now
`{operation}:{workspaceId}:{combinedDependencyHash}[:{userWorkspaceId}]:{locale}:{queryHash}`.
Each cached operation declares which flat maps its resolvers read
(`metadata-graphql-operations-to-cache.constant.ts`) plus a scope:
`ObjectMetadataItems` stays workspace-shared, `FindAllViews` is per-user
because unlisted-view visibility depends on the caller. When any
declared map changes, its hash rotates and the key rotates with it; no
flush needed. The key is resolved once per request and reused in
`onResponse`, so a rotation mid-request can never cache a response under
a fresher key than the data it was built from. If hash resolution fails,
the request is served uncached (Sentry-captured).
This also fixes three pre-existing key soundness gaps: `FindAllViews`
ignored locale although view names are translated server-side, the query
hash ignored GraphQL variables (`$viewTypes`), and mid-request version
rotation could re-key between request and response.
**SDL cache** (`workspace-graphql-schema-sdl.service.ts`): keyed on the
combined hash of the four maps the schema is generated from, taken from
the same `getOrRecomputeWithHashes` call that returns the data, so key
and content cannot skew. The `metadataVersion` read/seed block there is
gone; the Redis seed moved to `middleware.service.ts` so the
`X-Schema-Version` "refresh the page" check keeps working after the
Redis key's TTL expires.
**`WorkspaceCacheService`**: the internal pipeline now threads `{data,
hashes}` through every stage (local hit, hash validation, Redis fetch,
recompute) and the memoizer stores the pair, so returned hashes are
always consistent with returned data. New public
`getOrRecomputeWithHashes` and `getOrRecomputeCombinedHash`
(hashes-first: one MGET of the small `:hash` keys, full pipeline only
for missing ones, so cold pods never pull map payloads just to build a
key).
**Atomic pair writes** (`cache-storage.service.ts`): `mset` on the Redis
driver now delegates to the store's own `mset` (a MULTI of `SET ... PX`,
or native `MSET`), grouped by TTL. Previously it was a `Promise.all` of
independent SETs, so two concurrent recomputes could interleave and
leave one recompute's `:data` next to the other's `:hash`; with
hash-keyed caches that torn pair could persist a stale response under a
live key. `CoreEntityCacheService` writes through the same method and is
fixed for free.
**Cleanup**: `flush()` lost its `metadataVersion` parameter (always
pattern-flush per key on workspace deletion),
`METADATA_VERSIONED_WORKSPACE_CACHE_KEY` became
`HASH_KEYED_WORKSPACE_CACHE_KEYS` with the `MetadataVersion` key
relocated to `WORKSPACE_CACHE_KEYS` and the dead `ORMEntitySchemas`
entry removed.
## Deliberately unchanged
- `incrementMetadataVersion` and all its callers stay: the version still
feeds the `X-Schema-Version` check and the pinned upgrade commands.
Deprecating the column is a later stage.
- The runner's `FindAllViews` pattern-flush is kept for exactly one
release: view-only migrations never bump `metadataVersion`, so old pods
in a rolling deploy have no other invalidation signal for their
version-keyed entries. It gets deleted next release, which removes the
SCAN entirely.
- Old-shape cache entries are not migrated; they expire via the 7-day
TTL.
## Known limitations (follow-ups, not regressions)
- The plugin reads dependency hashes Redis-fresh while resolvers can
serve up to 10s-old memoized data, so a request landing right after a
migration can cache a pre-rotation response under the new key. Same
shape existed under `metadataVersion`; closing it needs request-scoped
snapshot plumbing.
- Concurrent recomputes are last-writer-wins (lost update). Fencing with
a conditional write is a follow-up.
## Validation
- Unit: response-cache plugin behavior (scope, key stash, serve-uncached
on failure, prototype-name guard), atomic `mset` batching, existing
`WorkspaceCacheService` spec passing unchanged.
- Integration: a new drift-guard spec runs the real
`ObjectMetadataItems`/`FindAllViews` operations with full frontend
selection sets against the in-process app, spies on
`WorkspaceCacheService`, and fails if resolvers read a flat map missing
from the declared dependency lists, so the constant cannot silently
drift.
- Manual against a live server: creating a field rotates the field-map
hash and the very next `ObjectMetadataItems` response contains it (hash
rotation is now its only invalidation path); warm hits are ~5ms; SDL
entries appear under hash-shaped keys via introspection.
## Suggested reading order
1. `workspace-cache.service.ts`, `workspace-cache-key.type.ts`,
`combine-cache-hashes.util.ts` (the `{data, hashes}` pipeline)
2. `use-cached-metadata.ts`,
`metadata-graphql-operations-to-cache.constant.ts`,
`metadata.module-factory.ts` (response cache)
3. `workspace-graphql-schema-sdl.service.ts`,
`workspace-cache-storage.service.ts` (SDL cache and renames)
4. `middleware.service.ts` (metadata version seed relocation)
5. `cache-storage.service.ts` (atomic writes)
6. Tests
This commit is contained in:
Weiko
+99
-35
@@ -29,8 +29,10 @@ import {
|
||||
WorkspaceCacheKeyName,
|
||||
type WorkspaceCacheDataMap,
|
||||
type WorkspaceCacheResult,
|
||||
type WorkspaceCacheResultWithHashes,
|
||||
} from 'src/engine/workspace-cache/types/workspace-cache-key.type';
|
||||
import { type WorkspaceLocalCacheEntry } from 'src/engine/workspace-cache/types/workspace-local-cache-entry.type';
|
||||
import { combineCacheHashes } from 'src/engine/workspace-cache/utils/combine-cache-hashes.util';
|
||||
|
||||
const LOCAL_TTL_MS = 100; // 100ms
|
||||
const LOCAL_ENTRY_TTL_MS = 30 * 60 * 1000; // 30 minutes
|
||||
@@ -43,6 +45,11 @@ const MIN_EVICT_KEYS = 100;
|
||||
|
||||
type CacheDataType = WorkspaceCacheDataMap[WorkspaceCacheKeyName];
|
||||
|
||||
type CacheEntriesResult = {
|
||||
data: Partial<WorkspaceCacheDataMap>;
|
||||
hashes: Partial<Record<WorkspaceCacheKeyName, string>>;
|
||||
};
|
||||
|
||||
type RecomputeHashResolution =
|
||||
| { strategy: 'mint' }
|
||||
| {
|
||||
@@ -61,9 +68,9 @@ export class WorkspaceCacheService implements OnModuleInit {
|
||||
WorkspaceCacheProvider<CacheDataType>
|
||||
>();
|
||||
private readonly localDataOnlyKeys = new Set<WorkspaceCacheKeyName>();
|
||||
private readonly memoizer = new PromiseMemoizer<
|
||||
Partial<WorkspaceCacheDataMap>
|
||||
>(MEMOIZER_TTL_MS);
|
||||
private readonly memoizer = new PromiseMemoizer<CacheEntriesResult>(
|
||||
MEMOIZER_TTL_MS,
|
||||
);
|
||||
|
||||
private readonly logger = new Logger(WorkspaceCacheService.name);
|
||||
|
||||
@@ -114,18 +121,22 @@ export class WorkspaceCacheService implements OnModuleInit {
|
||||
workspaceId: string,
|
||||
cacheKeyNames: K,
|
||||
): Promise<WorkspaceCacheResult<K>> {
|
||||
this.evictExpiredLocalEntries();
|
||||
const { data } = await this.getOrRecomputeWithHashes(
|
||||
workspaceId,
|
||||
cacheKeyNames,
|
||||
);
|
||||
|
||||
if (
|
||||
!isDefined(workspaceId) ||
|
||||
cacheKeyNames.length === 0 ||
|
||||
!isValidUuid(workspaceId)
|
||||
) {
|
||||
throw new WorkspaceCacheException(
|
||||
'Invalid parameters: workspace ID and cache key names are required',
|
||||
WorkspaceCacheExceptionCode.INVALID_PARAMETERS,
|
||||
);
|
||||
}
|
||||
return data;
|
||||
}
|
||||
|
||||
public async getOrRecomputeWithHashes<
|
||||
const K extends WorkspaceCacheKeyName[],
|
||||
>(
|
||||
workspaceId: string,
|
||||
cacheKeyNames: K,
|
||||
): Promise<WorkspaceCacheResultWithHashes<K>> {
|
||||
this.evictExpiredLocalEntries();
|
||||
this.assertValidCacheParameters(workspaceId, cacheKeyNames);
|
||||
|
||||
const memoKey =
|
||||
`${workspaceId}-${[...cacheKeyNames].sort().join(',')}` as const;
|
||||
@@ -138,10 +149,10 @@ export class WorkspaceCacheService implements OnModuleInit {
|
||||
workspaceId,
|
||||
cacheKeyNames,
|
||||
);
|
||||
const freshData = this.getFromLocalCache(workspaceId, freshKeys);
|
||||
const freshEntries = this.getFromLocalCache(workspaceId, freshKeys);
|
||||
|
||||
if (staleKeys.length === 0) {
|
||||
return freshData;
|
||||
return freshEntries;
|
||||
}
|
||||
|
||||
// Stage 2: Validate ttl stale keys against Redis hash
|
||||
@@ -154,32 +165,66 @@ export class WorkspaceCacheService implements OnModuleInit {
|
||||
workspaceId,
|
||||
staleKeys,
|
||||
);
|
||||
const validatedData = this.getFromLocalCache(workspaceId, validKeys);
|
||||
const validatedEntries = this.getFromLocalCache(workspaceId, validKeys);
|
||||
|
||||
// Stage 3: Fetch data from Redis
|
||||
const { redisData, missingInRedis } = await this.fetchDataFromRedis(
|
||||
const { redisEntries, missingInRedis } = await this.fetchDataFromRedis(
|
||||
workspaceId,
|
||||
keysNeedingDataFromRedis,
|
||||
);
|
||||
|
||||
// Stage 4: Recompute remaining
|
||||
const keysToRecompute = [...keysNeedingRecompute, ...missingInRedis];
|
||||
const recomputedData = await this.recomputeDataFromProvider(
|
||||
const recomputedEntries = await this.recomputeDataFromProvider(
|
||||
workspaceId,
|
||||
keysToRecompute,
|
||||
{ strategy: 'recover', adoptableHashes },
|
||||
);
|
||||
|
||||
return {
|
||||
...freshData,
|
||||
...validatedData,
|
||||
...redisData,
|
||||
...recomputedData,
|
||||
data: {
|
||||
...freshEntries.data,
|
||||
...validatedEntries.data,
|
||||
...redisEntries.data,
|
||||
...recomputedEntries.data,
|
||||
},
|
||||
hashes: {
|
||||
...freshEntries.hashes,
|
||||
...validatedEntries.hashes,
|
||||
...redisEntries.hashes,
|
||||
...recomputedEntries.hashes,
|
||||
},
|
||||
};
|
||||
},
|
||||
);
|
||||
|
||||
return result as WorkspaceCacheResult<K>;
|
||||
return result as WorkspaceCacheResultWithHashes<K>;
|
||||
}
|
||||
|
||||
public async getOrRecomputeCombinedHash(
|
||||
workspaceId: string,
|
||||
cacheKeyNames: WorkspaceCacheKeyName[],
|
||||
): Promise<string> {
|
||||
this.assertValidCacheParameters(workspaceId, cacheKeyNames);
|
||||
|
||||
const cachedHashes = await this.getCacheHashes(workspaceId, cacheKeyNames);
|
||||
const missingKeys = cacheKeyNames.filter(
|
||||
(cacheKeyName) => !isDefined(cachedHashes[cacheKeyName]),
|
||||
);
|
||||
|
||||
if (missingKeys.length === 0) {
|
||||
return combineCacheHashes(cachedHashes, cacheKeyNames);
|
||||
}
|
||||
|
||||
const { hashes: recomputedHashes } = await this.getOrRecomputeWithHashes(
|
||||
workspaceId,
|
||||
missingKeys,
|
||||
);
|
||||
|
||||
return combineCacheHashes(
|
||||
{ ...cachedHashes, ...recomputedHashes },
|
||||
cacheKeyNames,
|
||||
);
|
||||
}
|
||||
|
||||
public async invalidateAndRecompute(
|
||||
@@ -242,6 +287,22 @@ export class WorkspaceCacheService implements OnModuleInit {
|
||||
this.deleteFromLocalCache(workspaceId, cacheKeyNames);
|
||||
}
|
||||
|
||||
private assertValidCacheParameters(
|
||||
workspaceId: string,
|
||||
cacheKeyNames: WorkspaceCacheKeyName[],
|
||||
): void {
|
||||
if (
|
||||
!isDefined(workspaceId) ||
|
||||
cacheKeyNames.length === 0 ||
|
||||
!isValidUuid(workspaceId)
|
||||
) {
|
||||
throw new WorkspaceCacheException(
|
||||
'Invalid parameters: workspace ID and cache key names are required',
|
||||
WorkspaceCacheExceptionCode.INVALID_PARAMETERS,
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
private checkLocalTTL<K extends WorkspaceCacheKeyName>(
|
||||
workspaceId: string,
|
||||
cacheKeyNames: readonly K[],
|
||||
@@ -328,14 +389,14 @@ export class WorkspaceCacheService implements OnModuleInit {
|
||||
workspaceId: string,
|
||||
cacheKeyNames: WorkspaceCacheKeyName[],
|
||||
): Promise<{
|
||||
redisData: Partial<WorkspaceCacheDataMap>;
|
||||
redisEntries: CacheEntriesResult;
|
||||
missingInRedis: WorkspaceCacheKeyName[];
|
||||
}> {
|
||||
const redisData: Partial<WorkspaceCacheDataMap> = {};
|
||||
const redisEntries: CacheEntriesResult = { data: {}, hashes: {} };
|
||||
const missingInRedis: WorkspaceCacheKeyName[] = [];
|
||||
|
||||
if (cacheKeyNames.length === 0) {
|
||||
return { redisData, missingInRedis };
|
||||
return { redisEntries, missingInRedis };
|
||||
}
|
||||
|
||||
// Interleave data and hash keys for atomic fetch: [data1, hash1, data2, hash2, ...]
|
||||
@@ -354,22 +415,23 @@ export class WorkspaceCacheService implements OnModuleInit {
|
||||
const hash = allValues[index * 2 + 1] as string | undefined;
|
||||
|
||||
if (isDefined(data) && isDefined(hash)) {
|
||||
Object.assign(redisData, { [keyName]: data });
|
||||
Object.assign(redisEntries.data, { [keyName]: data });
|
||||
redisEntries.hashes[keyName] = hash;
|
||||
this.setInLocalCache(workspaceId, keyName, data, hash);
|
||||
} else {
|
||||
missingInRedis.push(keyName);
|
||||
}
|
||||
}
|
||||
|
||||
return { redisData, missingInRedis };
|
||||
return { redisEntries, missingInRedis };
|
||||
}
|
||||
|
||||
private async recomputeDataFromProvider(
|
||||
workspaceId: string,
|
||||
cacheKeyNames: WorkspaceCacheKeyName[],
|
||||
hashResolution: RecomputeHashResolution,
|
||||
): Promise<Partial<WorkspaceCacheDataMap>> {
|
||||
const result: Partial<WorkspaceCacheDataMap> = {};
|
||||
): Promise<CacheEntriesResult> {
|
||||
const result: CacheEntriesResult = { data: {}, hashes: {} };
|
||||
|
||||
if (cacheKeyNames.length === 0) {
|
||||
return result;
|
||||
@@ -412,7 +474,8 @@ export class WorkspaceCacheService implements OnModuleInit {
|
||||
const bootstrapHashEntries: Array<{ key: string; value: string }> = [];
|
||||
|
||||
for (const { keyName, data, hash, isAdopted } of computed) {
|
||||
Object.assign(result, { [keyName]: data });
|
||||
Object.assign(result.data, { [keyName]: data });
|
||||
result.hashes[keyName] = hash;
|
||||
|
||||
const baseKey = this.buildCacheKey(workspaceId, keyName);
|
||||
const isLocalDataOnly = this.localDataOnlyKeys.has(keyName);
|
||||
@@ -453,8 +516,8 @@ export class WorkspaceCacheService implements OnModuleInit {
|
||||
private getFromLocalCache(
|
||||
workspaceId: string,
|
||||
workspaceCacheKeyNames: WorkspaceCacheKeyName[],
|
||||
): Partial<WorkspaceCacheDataMap> {
|
||||
const result: Partial<WorkspaceCacheDataMap> = {};
|
||||
): CacheEntriesResult {
|
||||
const result: CacheEntriesResult = { data: {}, hashes: {} };
|
||||
|
||||
for (const keyName of workspaceCacheKeyNames) {
|
||||
const localKey = this.buildCacheKey(workspaceId, keyName);
|
||||
@@ -463,7 +526,8 @@ export class WorkspaceCacheService implements OnModuleInit {
|
||||
|
||||
if (isDefined(entry) && isDefined(version)) {
|
||||
version.lastReadAt = Date.now();
|
||||
Object.assign(result, { [keyName]: version.data });
|
||||
Object.assign(result.data, { [keyName]: version.data });
|
||||
result.hashes[keyName] = entry.latestHash;
|
||||
this.cleanupStaleVersions(entry);
|
||||
}
|
||||
}
|
||||
|
||||
Reference in New Issue
Block a user