## Context
`Lambda invocation failed for function '<id>' during build: Failed to
acquire lock for key: lambda-build:<id>` fires ~1000 times/day in
production.
## Root cause
`LambdaExecutorManagerService.buildExecutor` re-checks `canSkip` inside
the `lambda-build:<functionId>` lock, but the re-check reuses the
`flatApplication` snapshot captured when the request started. `canSkip`
depends on `!flatApplication.isSdkLayerStale`, so:
1. An app sync/install regenerates the SDK client and sets
`isSdkLayerStale = true`
2. All in-flight executions of the function fail `canSkip` and queue on
the lock
3. The first holder rebuilds and `markSdkLayerFresh` clears the flag in
DB + workspace cache
4. Queued waiters can't see that fix — their in-memory snapshot still
says stale — so **each waiter redoes the full rebuild serially**
(download SDK archive, delete + republish layer, update function config,
wait for update)
5. The lock is held back-to-back for minutes; everyone deeper in the
queue exhausts the 120s retry budget and throws
The local driver already handles this correctly
(`LocalLayerManagerService` refreshes the flat application from the
workspace cache inside its lock); the lambda driver missed it.
## Fix
- Refresh `flatApplication` from the workspace cache inside the lock
before re-checking `canSkip`, so waiters skip in ~100ms once the first
holder finishes
- Degrade gracefully on lock-acquisition timeout: re-check build status
with fresh data and proceed with the invocation if the executor is
already usable, instead of failing the run (introduces a typed
`CacheLockAcquisitionError` so only that case is caught)
## Test plan
- [x] `cache-lock.service.spec.ts` passes
- [x] `lint:diff-with-main` + typecheck pass
- [ ] Monitor `Failed to acquire lock for key: lambda-build:*` error
rate in production after deploy
Thomas Trompette
0ac4f237c0