## Problem
In production, an AI-chat assistant response sometimes freezes
mid-stream (partial text, looks hung), then "picks up again on its own"
later without the user resending and without a known worker restart.
Root cause: the **agent-chat SSE subscription has no keepalive and no
silent-death detection**.
- Delivery is fire-and-forget Redis pub/sub
(`SubscriptionService.publishToAgentChat`) and the resolver returns the
**raw** iterator — unlike `EventStreamResolver`, which heartbeats every
30s via `wrapAsyncIteratorWithLifecycle`.
- During a quiet model/tool gap the connection sends no bytes, so a
proxy/LB/NAT can silently drop it mid-stream. `graphql-sse` neither
surfaces an error nor resumes with `Last-Event-ID`, and **nothing
re-pulls the existing Redis chunk catch-up on reconnect** (it only runs
on thread (re)mount / `message-persisted` refetch).
- So the live view freezes; recovery only happens when the terminal
`message-persisted` fires a full refetch from the DB — the observed
"self-recovery".
This is the **same silent-SSE-death class fixed for the DB event stream
in #21061**, which was never applied to the agent-chat path. The symptom
also matches #21096 (worker logs the job finishing, client never
updates, reload shows the message).
It is **not** queue prioritization, and it is **not** addressed by
#22193 (which only stabilizes the assistant message id and removes
end-of-stream flicker).
A secondary, independent self-recovery path also existed: BullMQ
stalled-job re-run (default 30s `lockDuration`, no idempotency guard)
re-streaming the whole turn → duplicate assistant messages / double
billing.
## Changes
### Commit 1 — keepalive + silent-death recovery (ports the #21061
pattern to agent chat)
- **Shared:** new `keepalive` variant on `AgentChatSubscriptionEvent`.
- **Server:** wrap the agent-chat subscription iterator with
`wrapAsyncIteratorWithLifecycle` — emit a `keepalive` on connect and
every `APPLICATION_KEEPALIVE_INTERVAL_MS` (30s) so the connection keeps
flushing bytes and a dead connection becomes detectable.
- **Client:** track the last received event timestamp (refreshed on
every chunk/keepalive in the SSE `next` sink); new
`AgentChatStreamKeepAliveEffect` forces a resubscribe + messages refetch
after 90s of silence, so the durable Redis chunk list backfills the gap
(`firstLiveSeq` is reset on resubscribe).
### Commit 2 — stream-job idempotency + lockDuration
- Thread a `lockDuration` option through `MessageQueueWorkerOptions` +
the BullMQ driver; set `aiStreamQueue` to 10 min so long streams aren't
falsely stalled.
- Guard `StreamAgentChatJob.handle` with a `streamId`-scoped Redis lock
(`SET NX PX` + compare-and-delete release) so a stalled re-run is
skipped instead of double-processing.
## Verification
⚠️ I could **not run typecheck/lint locally** — `yarn install` could not
complete in this environment (transient registry network aborts before
the link step, so `node_modules` never populated). **Please rely on CI
for type/lint verification.** The changes are written to match existing
conventions; the points most worth a reviewer's eye are the resolver's
iterator typing and the ioredis `set(..., 'PX', ttl, 'NX')` overload.
How to confirm the root cause in prod: a frozen client with the worker
logging `StreamAgentChatJob processed in …ms` and no `[AI_CHAT_NO_TEXT]`
is the silent-death signature (check reverse-proxy idle/buffering). For
the secondary path, watch `aiStreamQueue` `stalled`/re-processed metrics
and duplicate turns around worker restarts.
## Notes / trade-offs
- The 10-min `lockDuration` means a genuinely crashed worker's job isn't
reclaimed for up to 10 min; the client-side keepalive/catch-up recovers
the view independently, and the idempotency lock prevents duplicates.
Faster dead-worker recovery could be a follow-up.
- Touches `useAgentChatSubscription.ts` / `AgentChatRuntimeEffects.tsx`
/ `stream-agent-chat.job.ts`, which #22193 also touches — trivial rebase
expected.
Opened as **draft** pending CI.
https://claude.ai/code/session_018dF82A1VcsuWMxPLmdY3dm
---
_Generated by [Claude
Code](https://claude.ai/code/session_018dF82A1VcsuWMxPLmdY3dm)_
<!-- This is an auto-generated description by cubic. -->
<a
href="https://cubic.dev/pr/twentyhq/twenty/pull/22201?utm_source=github"
target="_blank" rel="noopener noreferrer"
data-no-image-dialog="true"><picture><source
media="(prefers-color-scheme: dark)"
srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source
media="(prefers-color-scheme: light)"
srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img
alt="Review in cubic"
src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a>
<!-- End of auto-generated description by cubic. -->
Félix Malfait
da6a2ee300
