## Why
Today the assistant chat message is written to the DB exactly **once**,
at `onFinish` (`handleStreamFinish`). The per-step hook (`onStepFinish`
in `chat-execution.service.ts`) only does billing/metrics — no
persistence. Content streams to the client live over Redis, but the
durable record only lands at the end.
The graceful paths already cover partials: normal completion, user
cancel, and the shutdown-abort from #22517 all fire `onFinish` with
`isAborted` and persist whatever parts exist. The gap is a **true
SIGKILL** — OOM, node loss, or a grace-period overrun — where `onFinish`
never runs. When that happens mid-turn, the assistant message vanishes
from the thread even though its tool calls already executed real CRM
mutations. That's the worst failure shape: side effects persisted, the
record of them didn't.
This closes that gap by materializing the assistant message
progressively, so a hard kill leaves the tools-already-run on the
thread. It's the app-level piece behind the earlier discussion on #22518
— with this, a retried/interrupted turn also resumes from its own
partial (the model reloads history and continues) instead of re-doing
completed steps.
Scope is **chat only** — the workflow agent path
(`AgentAsyncExecutorService`, blocking `generateText`) is a different
model with its own step-log persistence and workflow-engine resumption,
and is deliberately out of scope here.
## What
- `AgentChatService.upsertAssistantMessage`: idempotent message+parts
write keyed on the deterministic `uuidv5(streamId)` id (upsert the row,
replace its parts), reusing the existing `mapUIMessagePartsToDBParts` /
`finalizeDanglingToolParts`.
- `stream-agent-chat.job.ts`: tee the assembled UI stream — one branch
keeps publishing chunks unchanged; the other drives the SDK's own
`readUIMessageStream` and, throttled to
`AGENT_CHAT_CHECKPOINT_INTERVAL_MS` (2s), fires a serialized
fire-and-forget `upsertAssistantMessage`. No chunk re-assembly — the
parts come straight from the SDK assembler, identical to what `onFinish`
produces.
- `handleStreamFinish` now upserts (authoritative) instead of
insert-then-skip. Two ordering/idempotency guards:
- Checkpoints are serialized through one promise chain and gated off
(`isFinalizingPersist`) before the final write, which drains the chain
first — so the authoritative write always lands last and never races a
checkpoint on the parts table.
- The old `hasMessageById`→skip protected the thread-totals accumulation
from double-counting on a re-executed job. Since checkpoints now make
the row exist mid-stream, that signal is captured **once at stream
start** (`assistantMessageExistedAtStart`) and used to gate the totals
update — preserving the exact prior idempotency while allowing
progressive writes.
`readUIMessageStream` runs with `terminateOnError: false` and the
checkpoint consumer swallows errors: checkpoints are best-effort and
must never affect the stream or the authoritative persist.
## User impact
A worker that dies hard mid-turn no longer erases the assistant message.
Combined with #22434's Retry, the user sees the partial turn (including
executed tools) and can continue, rather than a turn that silently
disappeared while its side effects stuck.
Note: this is insurance against true SIGKILL specifically — graceful
shutdown (#22514/#22517) already persists partials — so it's most
valuable for OOM/node-loss/grace-overrun. Framed that way deliberately;
happy to drop it if you'd rather not touch this path for that scope.
## Validation
- Unit (`stream-agent-chat.job.spec.ts`, 59 green): the success path
persists via `upsertAssistantMessage` with the assembled parts + turnId;
a re-executed job whose message existed at start still upserts but does
**not** re-apply thread totals; all existing flows (mid-stream error,
user cancel, shutdown-abort, missing workspace) unchanged.
- Local runtime (isolated instance, real OpenAI stream, checkpoint
interval shortened for the test):
- **SIGKILL mid-stream** (no graceful onFinish) → the assistant message
row (deterministic id) is present afterward with a partial text part
(~381 chars) that would otherwise have been lost.
- **Normal completion** → the final upsert converges to the full message
("Hello, Tim."), `activeStreamId` cleared, `totalOutputTokens` applied
exactly once.
<!-- This is an auto-generated description by cubic. -->
<a
href="https://cubic.dev/pr/twentyhq/twenty/pull/22524?utm_source=github"
target="_blank" rel="noopener noreferrer"
data-no-image-dialog="true"><picture><source
media="(prefers-color-scheme: dark)"
srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source
media="(prefers-color-scheme: light)"
srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img
alt="Review in cubic"
src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a>
<!-- End of auto-generated description by cubic. -->
Félix Malfait
9496a98aa3
