Etienne 5c825e8712 fix(ai-chat): write the stream heartbeat before the DB claim (#23198)
## Problem

Answering an `ask_questions` (select) prompt sometimes killed the turn
with
"Failed to get response. The response was interrupted before it could
finish."
The answer was swallowed and Retry rewound the whole turn. It was
intermittent,
worse on long threads and when coming back from another tab.

Reported in [discord quality
issue](https://discord.com/channels/1130383047699738754/1526875783170097172).
Confirmed in prod: ~28
`ai_chat_turn_failed_total{failure_phase="interrupted"}`
over the last 7 days (the only failure phase firing), plus matching
`the thread no longer holds this claim` worker logs around the report
time.

## Root cause

A stream is tracked by two records: the claim (`activeStreamId` in
Postgres) and
the heartbeat (a Redis key refreshed while the worker runs).
`reapDeadStream`
treats "claim set but no heartbeat" as a crashed worker and kills the
turn.

On the answer path the ordering left a window where that was falsely
true:

1. `resolvePendingQuestion` writes `activeStreamId` to Postgres (claim
set)
2. `enqueueResumeStream` reloads the thread and runs
`loadMessagesFromDB`
(reads every message and part, signs a URL per file, hundreds of ms on
long threads)
3. only then `markClaimed` writes the heartbeat

Between 1 and 3 the thread looks dead to the reaper. Worse,
`question-answered`
was published inside that window, so the client refetched, and the
refetch's
`chatStreamCatchupChunks` query runs the reaper, racing the server into
its own
setup window. The keepalive reap tick could land there too.

## Fix

Enforce one invariant everywhere: the heartbeat exists before any DB row
carries
the `activeStreamId`, so "claim without heartbeat" can only ever mean a
genuinely
dead worker.

- New `answerPendingQuestionAndResumeStream` owns the answer flow:
`markClaimed`
first, then the DB claim, then enqueue, then publish `question-answered`
(moved
after the enqueue so client refetches can't race the setup, and so we
don't tell
  the client "answered" when the enqueue failed and rolled back).
- Both failure paths clean up: clear the heartbeat if resolving fails;
restore the
  pending question and clear the heartbeat if enqueueing fails.
- `tryClaimStream` (send / retry / queue-flush) reordered the same way:
heartbeat
  before the claim, cleared if the claim is lost.
- `releaseStreamClaim` now also clears the heartbeat so failed claims
leave no orphan key.

No grace period or schema change needed: the ordering closes the race
structurally.
The Retry-rewinds-the-turn behavior is unrelated and left as a separate
follow-up.

## Testing

- New `agent-chat-streaming.service.answer.spec.ts`: heartbeat marked
before the
claim, publish only after enqueue, both failure paths restore state and
clear the key.
- Extended `agent-chat-streaming.service.claim.spec.ts`:
heartbeat-before-claim
  ordering and key cleanup on lost claim / failed enqueue.
- Full ai-chat suite green (74 tests), lint and typecheck clean.

After deploy,
`sum(increase(ai_chat_turn_failed_total{failure_phase="interrupted"}[1d]))`
trending to zero confirms the fix.

<!-- This is an auto-generated description by cubic. -->
<a
href="https://cubic.dev/pr/twentyhq/twenty/pull/23198?utm_source=github"
target="_blank" rel="noopener noreferrer"
data-no-image-dialog="true"><picture><source
media="(prefers-color-scheme: dark)"
srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source
media="(prefers-color-scheme: light)"
srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img
alt="Review in cubic"
src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a>
<!-- End of auto-generated description by cubic. -->
2026-07-23 14:17:50 +02:00
2026-06-11 11:02:28 +02:00

Twenty logo

The #1 Open-Source CRM

Website · Documentation · Roadmap · Discord · Figma

Twenty banner


Why Twenty

Twenty gives technical teams the building blocks for a custom CRM that meets complex business needs and quickly adapts as the business evolves. Twenty is the CRM you build, ship, and version like the rest of your stack.

Learn more about why we built Twenty


Installation

Cloud

The fastest way to get started. Sign up at twenty.com and spin up a workspace in under a minute, with no infrastructure to manage and always up to date.

Build an app

Scaffold a new app with the Twenty CLI:

npx create-twenty-app my-app

Define objects, fields, and views as code:

import { defineObject, FieldType } from 'twenty-sdk/define';

export default defineObject({
  nameSingular: 'deal',
  namePlural: 'deals',
  labelSingular: 'Deal',
  labelPlural: 'Deals',
  fields: [
    { name: 'name', label: 'Name', type: FieldType.TEXT },
    { name: 'amount', label: 'Amount', type: FieldType.CURRENCY },
    { name: 'closeDate', label: 'Close Date', type: FieldType.DATE_TIME },
  ],
});

Then ship it to your workspace:

npx twenty app:publish --private

See the app development guide for objects, views, agents, and logic functions.

Self-hosting

Run Twenty on your own infrastructure with Docker Compose, or contribute locally via the local setup guide.



Everything you need

Twenty gives you the building blocks of a modern CRM (objects, views, workflows, and agents) and lets you extend them as code. Here's a tour of what's in the box.

Want to go deeper? Read the User Guide for product walkthroughs, or the Documentation for developer reference.

Create your apps

Learn more about apps in doc

Stay on top with version control

Learn more about version control in doc

All the tools you need to build anything

Learn more about primitives in doc

Customize your layouts

Learn more about layouts in doc

AI agents and chats

Learn more about AI in doc

Plus all the tools of a good CRM

Learn more about CRM features in doc


Stack

Thanks

Greptile      Sentry      Crowdin

Thanks to these amazing services that we use and recommend for code review (Greptile), catching bugs (Sentry) and translating (Crowdin).

Join the Community

Star the repo · Discord · Feature requests · Releases · X · LinkedIn · Crowdin · Contribute

S
Description
The open alternative to Salesforce, designed for AI.
Readme AGPL-3.0 1.4 GiB
Languages
TypeScript 79.6%
MDX 17.3%
JavaScript 2.7%
Python 0.2%
SCSS 0.1%