feat(ai): add ask_questions interactive clarifying-question tool (#22346)

## What & why

Adds an `ask_questions` tool that lets the in-app **Ask AI** assistant
**pause a turn to ask the user one or more multiple-choice questions**
(per the [Figma
design](https://www.figma.com/design/xt8O9mFeLl46C5InWwoMrN/Twenty?node-id=105959-117153))
and resume once answered — instead of guessing on
ambiguous/consequential decisions.

The tool is **harness-only**: an interactive question UI is meaningless
without a user to answer it, so it must be absent from MCP and from
head-less workflow agents.

## Design — true tool-result resume (not a synthetic user message)

The user's answer is a **structured tool result bound to the
`toolCallId`**, and the **same agent turn resumes** — exactly how
Anthropic (`tool_result` by `tool_use_id`) and OpenAI
(`function_call_output`) model human-in-the-loop.

The naive form of this (leave the tool call in `input-available` to mean
"pending") is **impossible** here: `finalizeDanglingToolParts` rewrites
`input-available` → `output-error` ("Tool execution was interrupted") on
both the persist path (`addMessage`) and the model-reload path
(`chat-execution.service.ts`). That util is a load-bearing safety net,
so weakening it is the wrong move.

Instead:

- `ask_questions` is an **inline, chat-only tool with an `execute` that
returns a `status: 'pending'` result immediately**, so the tool part is
always `output-available` and **immune to `finalizeDanglingToolParts`**.
`stopWhen(hasToolCall('ask_questions'))` halts the turn right after the
call (the model never sees the placeholder).
- A nullable **`thread.pendingQuestionMessageId`** marker records that a
turn is awaiting an answer.
- The new **`answerAgentChatQuestion`** mutation atomically *claims* the
question (clears the marker, marks the thread streaming), **writes the
answer onto the same tool part** (`status: 'answered'`), and
**re-enqueues the turn via the existing `existingTurnId` plumbing**
(`isResume` bypasses the per-turn dedup guard). On resume
`finalizeDanglingToolParts` leaves the `output-available` part untouched
and `convertToModelMessages` emits `assistant(tool_use)` +
`tool_result(answers)`, so the model continues.

This achieves the platform-aligned semantics **without** weakening the
finalize safety net or inventing a fragile new part state.

### Meets the two requirements

- **Survives refresh, scoped per-thread** — the pending state is a
normal persisted `output-available` part + the thread marker; the
frontend card is derived per-thread from the loaded messages, so it
re-appears on reload and only on its own thread.
- **Takes priority over the queue** — a unified `isBlocked =
activeStreamId || pendingQuestionMessageId` gate is applied in both
`sendChatMessage` (new messages queue) and `flushNextQueuedMessage` (the
drain). The queue cannot unpile until the question is answered and the
resumed turn completes.

### Harness-only by construction

`ask_questions` is added **only** to the chat's inline `activeTools`
(like `learn_tools`/`execute_tool`/`load_skills`). It never enters the
tool registry/catalog, so it is invisible to MCP and to workflow agents
— no `MCP_EXCLUDED_TOOL_NAMES` entry needed.

## UX

While a question is pending, the **composer is replaced by the question
card** (matching the Figma): question title + pager (`1/2`), numbered
option rows (`IconSquareNumber*`) with per-option info-icon descriptions
and a "Recommended" badge, and the normal composer as the free-text
fallback ("Type anything to do differently."). The transcript shows a
compact "Asking questions…" status line that becomes an answered
summary.

## Changes

**twenty-shared**
- `ai/types/AskQuestionsToolTypes.ts` —
`AskQuestionItem/Option/Answer/Result`, `ASK_QUESTIONS_TOOL_NAME`.

**twenty-server**
- `ai-chat/tools/ask-questions.tool.ts` — inline tool factory
(pending-result `execute`, zod schema, 1–4 questions × 2–4 options).
- `chat-execution.service.ts` — add to `activeTools` +
`preloadedToolNames`; `hasToolCall` in `stopWhen`.
- `chat-system-prompts.const.ts` — when-to-use guidance.
- `entities/agent-chat-thread.entity.ts` — `pendingQuestionMessageId`
column.
- `stream-agent-chat.job.ts` — set the marker on a question pause;
bypass the dedup guard on resume; suppress the no-text warning for
question pauses.
- `agent-chat-streaming.service.ts` — gate `flushNextQueuedMessage`;
`enqueueResumeStream`.
- `agent-chat.resolver.ts` — gate `sendChatMessage`;
`answerAgentChatQuestion` mutation.
- `agent-chat.service.ts` — `resolvePendingQuestion` (atomic claim +
write answer).
- `dtos/agent-chat-question-answer.input.ts`, `ai.exception.ts`
(`QUESTION_NOT_PENDING`), `utils/find-pending-question-part.util.ts`.

**twenty-front**
- `components/AiChatQuestionCard.tsx` — the interactive card (matches
Figma tokens) + `__stories__/AiChatQuestionCard.stories.tsx`.
- `components/AiChatEditorSection.tsx` — swap the composer for the card
while pending.
- `components/AiChatQuestionStatusRenderer.tsx` + branch in
`AiChatAssistantMessageRenderer.tsx`.
- `states/selectors/agentChatPendingQuestionComponentSelector.ts`,
`types/AgentChatPendingQuestion.ts`.
- `hooks/useSubmitQuestionAnswer.ts` + `utils/markQuestionAnswered.ts`
(optimistic) + `graphql/mutations/answerAgentChatQuestion.ts`.

A design doc lives at
`packages/twenty-server/docs/ASK_USER_QUESTION_TOOL_PLAN.md`.

## Migration

Adds a nullable `pendingQuestionMessageId` (uuid) column to
`core.agentChatThread`. Needs a generated **fast instance command**
(`database:migrate:generate --name addThreadPendingQuestion --type
fast`) — see "Verification status".

## Tests

- Server: `ask-questions.tool.spec.ts` (pending echo + schema bounds),
`find-pending-question-part.util.spec.ts`.
- Front: `markQuestionAnswered.test.ts`, plus the Storybook story.

## Verification status (please read)

This branch was authored in an environment where the monorepo `yarn
install` repeatedly failed on transient TLS resets from the package
registry, so I could **not** locally run the mechanical gates. The logic
was reviewed by hand and the `ai@6.0.97` exports used (`hasToolCall`,
`stepCountIs`, `generateId`) were confirmed against the package's type
defs. Still **TODO** (will rely on CI / a follow-up once deps install):

- [ ] `nx run twenty-shared:generateBarrels` (the `ai/index.ts` export
was added by hand; regen to reconcile)
- [ ] `nx run twenty-front:graphql:generate` (new mutation + input type)
- [ ] generate the fast instance command (migration) for the new column
- [ ] `typecheck` + `lint:diff-with-main` (front + server) — expect
minor import-ordering autofixes
- [ ] run the unit tests

**Screenshots:** reproducing the live flow needs an AI provider API key
(to get the model to actually call `ask_questions`), which isn't
available here. The card can be screenshotted from its **Storybook
story** (`AiChatQuestionCard.stories.tsx`) with no API key — I'll add
that image once deps install, or a reviewer can run `nx storybook
twenty-front`.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

https://claude.ai/code/session_01AArS8H3y3Z1Qwm763xhPLB

---
_Generated by [Claude
Code](https://claude.ai/code/session_01AArS8H3y3Z1Qwm763xhPLB)_

<!-- This is an auto-generated description by cubic. -->
<a
href="https://cubic.dev/pr/twentyhq/twenty/pull/22346?utm_source=github"
target="_blank" rel="noopener noreferrer"
data-no-image-dialog="true"><picture><source
media="(prefers-color-scheme: dark)"
srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source
media="(prefers-color-scheme: light)"
srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img
alt="Review in cubic"
src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a>
<!-- End of auto-generated description by cubic. -->
This commit is contained in:
Félix Malfait
2026-07-02 15:32:18 +02:00
committed by GitHub
parent 198f1e4916
commit 4aaf171d63
42 changed files with 1911 additions and 72 deletions
@@ -3380,6 +3380,7 @@ type Mutation {
createChatThread: AgentChatThread!
sendChatMessage(threadId: UUID!, text: String!, messageId: UUID!, browsingContext: JSON, modelId: String, fileAttachments: [FileAttachmentInput!]): SendChatMessageResult!
retryChatMessage(threadId: UUID!, modelId: String): SendChatMessageResult!
answerAgentChatQuestion(threadId: UUID!, messageId: UUID!, answers: [AgentChatQuestionAnswerInput!]!, modelId: String): SendChatMessageResult!
stopAgentChatStream(threadId: UUID!): Boolean!
renameChatThread(id: UUID!, title: String!): AgentChatThread!
archiveChatThread(id: UUID!): AgentChatThread!
@@ -4486,6 +4487,12 @@ input FileAttachmentInput {
filename: String!
}
input AgentChatQuestionAnswerInput {
questionIndex: Int!
selectedOptionIndices: [Int!]!
freeText: String
}
input CreateSkillInput {
id: UUID
name: String!
@@ -2912,6 +2912,7 @@ export interface Mutation {
createChatThread: AgentChatThread
sendChatMessage: SendChatMessageResult
retryChatMessage: SendChatMessageResult
answerAgentChatQuestion: SendChatMessageResult
stopAgentChatStream: Scalars['Boolean']
renameChatThread: AgentChatThread
archiveChatThread: AgentChatThread
@@ -6088,6 +6089,7 @@ export interface MutationGenqlSelection{
createChatThread?: AgentChatThreadGenqlSelection
sendChatMessage?: (SendChatMessageResultGenqlSelection & { __args: {threadId: Scalars['UUID'], text: Scalars['String'], messageId: Scalars['UUID'], browsingContext?: (Scalars['JSON'] | null), modelId?: (Scalars['String'] | null), fileAttachments?: (FileAttachmentInput[] | null)} })
retryChatMessage?: (SendChatMessageResultGenqlSelection & { __args: {threadId: Scalars['UUID'], modelId?: (Scalars['String'] | null)} })
answerAgentChatQuestion?: (SendChatMessageResultGenqlSelection & { __args: {threadId: Scalars['UUID'], messageId: Scalars['UUID'], answers: AgentChatQuestionAnswerInput[], modelId?: (Scalars['String'] | null)} })
stopAgentChatStream?: { __args: {threadId: Scalars['UUID']} }
renameChatThread?: (AgentChatThreadGenqlSelection & { __args: {id: Scalars['UUID'], title: Scalars['String']} })
archiveChatThread?: (AgentChatThreadGenqlSelection & { __args: {id: Scalars['UUID']} })
@@ -6509,6 +6511,8 @@ export interface UpdateCalendarChannelInputUpdates {visibility?: (CalendarChanne
export interface FileAttachmentInput {id: Scalars['UUID'],filename: Scalars['String']}
export interface AgentChatQuestionAnswerInput {questionIndex: Scalars['Int'],selectedOptionIndices: Scalars['Int'][],freeText?: (Scalars['String'] | null)}
export interface CreateSkillInput {id?: (Scalars['UUID'] | null),name: Scalars['String'],label: Scalars['String'],icon?: (Scalars['String'] | null),description?: (Scalars['String'] | null),content: Scalars['String']}
export interface UpdateSkillInput {id: Scalars['UUID'],name?: (Scalars['String'] | null),label?: (Scalars['String'] | null),icon?: (Scalars['String'] | null),description?: (Scalars['String'] | null),content?: (Scalars['String'] | null),isActive?: (Scalars['Boolean'] | null)}
@@ -81,8 +81,8 @@ export default {
349,
356,
439,
484,
493
485,
494
],
"types": {
"BillingProductDTO": {
@@ -8318,6 +8318,26 @@ export default {
]
}
],
"answerAgentChatQuestion": [
318,
{
"threadId": [
3,
"UUID!"
],
"messageId": [
3,
"UUID!"
],
"answers": [
475,
"[AgentChatQuestionAnswerInput!]!"
],
"modelId": [
1
]
}
],
"stopAgentChatStream": [
6,
{
@@ -8380,7 +8400,7 @@ export default {
311,
{
"input": [
475,
476,
"CreateSkillInput!"
]
}
@@ -8389,7 +8409,7 @@ export default {
311,
{
"input": [
476,
477,
"UpdateSkillInput!"
]
}
@@ -8447,7 +8467,7 @@ export default {
243,
{
"input": [
477,
478,
"GetAuthorizationUrlForSSOInput!"
]
}
@@ -8613,7 +8633,7 @@ export default {
246,
{
"input": [
478
479
]
}
],
@@ -8768,7 +8788,7 @@ export default {
6,
{
"input": [
479,
480,
"UpdateWorkspaceMemberSettingsInput!"
]
}
@@ -8802,7 +8822,7 @@ export default {
221,
{
"input": [
480,
481,
"SetupOIDCSsoInput!"
]
}
@@ -8811,7 +8831,7 @@ export default {
221,
{
"input": [
481,
482,
"SetupSAMLSsoInput!"
]
}
@@ -8820,7 +8840,7 @@ export default {
217,
{
"input": [
482,
483,
"DeleteSsoInput!"
]
}
@@ -8829,7 +8849,7 @@ export default {
218,
{
"input": [
483,
484,
"EditSsoInput!"
]
}
@@ -8858,7 +8878,7 @@ export default {
307,
{
"type": [
484,
485,
"AnalyticsType!"
],
"name": [
@@ -8898,7 +8918,7 @@ export default {
297,
{
"input": [
485,
486,
"CreateCalendarEventInput!"
]
}
@@ -8907,7 +8927,7 @@ export default {
306,
{
"input": [
486,
487,
"SendEmailInput!"
]
}
@@ -8929,7 +8949,7 @@ export default {
"String!"
],
"connectionParameters": [
488,
489,
"EmailAccountConnectionParameters!"
],
"id": [
@@ -8941,7 +8961,7 @@ export default {
167,
{
"input": [
490,
491,
"UpdateLabPublicFeatureFlagInput!"
]
}
@@ -8981,7 +9001,7 @@ export default {
74,
{
"input": [
491,
492,
"CreateOneAppTokenInput!"
]
}
@@ -9059,7 +9079,7 @@ export default {
"String!"
],
"fileFolder": [
493,
494,
"FileFolder!"
],
"filePath": [
@@ -11443,6 +11463,20 @@ export default {
1
]
},
"AgentChatQuestionAnswerInput": {
"questionIndex": [
21
],
"selectedOptionIndices": [
21
],
"freeText": [
1
],
"__typename": [
1
]
},
"CreateSkillInput": {
"id": [
3
@@ -11649,7 +11683,7 @@ export default {
1
],
"files": [
487
488
],
"__typename": [
1
@@ -11668,13 +11702,13 @@ export default {
},
"EmailAccountConnectionParameters": {
"IMAP": [
489
490
],
"SMTP": [
489
490
],
"CALDAV": [
489
490
],
"__typename": [
1
@@ -11713,7 +11747,7 @@ export default {
},
"CreateOneAppTokenInput": {
"appToken": [
492
493
],
"__typename": [
1
@@ -11742,7 +11776,7 @@ export default {
231,
{
"input": [
495,
496,
"LogicFunctionLogsInput!"
]
}
@@ -57,6 +57,12 @@ export type AgentChatEvent = {
threadId: Scalars['String']['output'];
};
export type AgentChatQuestionAnswerInput = {
freeText?: InputMaybe<Scalars['String']['input']>;
questionIndex: Scalars['Int']['input'];
selectedOptionIndices: Array<Scalars['Int']['input']>;
};
export type AgentChatThread = {
__typename?: 'AgentChatThread';
contextWindowTokens?: Maybe<Scalars['Int']['output']>;
@@ -2454,6 +2460,7 @@ export type Mutation = {
activateSkill: Skill;
activateWorkspace: Workspace;
addQueryToEventStream: Scalars['Boolean']['output'];
answerAgentChatQuestion: SendChatMessageResult;
archiveChatThread: AgentChatThread;
assignRoleToAgent: Scalars['Boolean']['output'];
assignRoleToApiKey: Scalars['Boolean']['output'];
@@ -2693,6 +2700,14 @@ export type MutationAddQueryToEventStreamArgs = {
};
export type MutationAnswerAgentChatQuestionArgs = {
answers: Array<AgentChatQuestionAnswerInput>;
messageId: Scalars['UUID']['input'];
modelId?: InputMaybe<Scalars['String']['input']>;
threadId: Scalars['UUID']['input'];
};
export type MutationArchiveChatThreadArgs = {
id: Scalars['UUID']['input'];
};
@@ -6373,6 +6388,16 @@ export type ActivateSkillMutationVariables = Exact<{
export type ActivateSkillMutation = { __typename?: 'Mutation', activateSkill: { __typename?: 'Skill', id: string, name: string, label: string, description?: string | null, icon?: string | null, content: string, isCustom: boolean, isActive: boolean, createdAt: string, updatedAt: string } };
export type AnswerAgentChatQuestionMutationVariables = Exact<{
threadId: Scalars['UUID']['input'];
messageId: Scalars['UUID']['input'];
answers: Array<AgentChatQuestionAnswerInput> | AgentChatQuestionAnswerInput;
modelId?: InputMaybe<Scalars['String']['input']>;
}>;
export type AnswerAgentChatQuestionMutation = { __typename?: 'Mutation', answerAgentChatQuestion: { __typename?: 'SendChatMessageResult', messageId: string, queued: boolean, streamId?: string | null } };
export type ArchiveChatThreadMutationVariables = Exact<{
id: Scalars['UUID']['input'];
}>;
@@ -8711,6 +8736,7 @@ export const UnsubscribeTopicsDocument = {"kind":"Document","definitions":[{"kin
export const SendEmailDocument = {"kind":"Document","definitions":[{"kind":"OperationDefinition","operation":"mutation","name":{"kind":"Name","value":"SendEmail"},"variableDefinitions":[{"kind":"VariableDefinition","variable":{"kind":"Variable","name":{"kind":"Name","value":"input"}},"type":{"kind":"NonNullType","type":{"kind":"NamedType","name":{"kind":"Name","value":"SendEmailInput"}}}}],"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"sendEmail"},"arguments":[{"kind":"Argument","name":{"kind":"Name","value":"input"},"value":{"kind":"Variable","name":{"kind":"Name","value":"input"}}}],"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"success"}},{"kind":"Field","name":{"kind":"Name","value":"error"}},{"kind":"Field","name":{"kind":"Name","value":"messageThreadId"}}]}}]}}]} as unknown as DocumentNode<SendEmailMutation, SendEmailMutationVariables>;
export const SendMessageCampaignDocument = {"kind":"Document","definitions":[{"kind":"OperationDefinition","operation":"mutation","name":{"kind":"Name","value":"SendMessageCampaign"},"variableDefinitions":[{"kind":"VariableDefinition","variable":{"kind":"Variable","name":{"kind":"Name","value":"input"}},"type":{"kind":"NonNullType","type":{"kind":"NamedType","name":{"kind":"Name","value":"SendMessageCampaignInput"}}}}],"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"sendMessageCampaign"},"arguments":[{"kind":"Argument","name":{"kind":"Name","value":"input"},"value":{"kind":"Variable","name":{"kind":"Name","value":"input"}}}],"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"campaignId"}},{"kind":"Field","name":{"kind":"Name","value":"queuedCount"}},{"kind":"Field","name":{"kind":"Name","value":"skipped"},"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"noEmail"}},{"kind":"Field","name":{"kind":"Name","value":"deduped"}},{"kind":"Field","name":{"kind":"Name","value":"overCap"}}]}}]}}]}}]} as unknown as DocumentNode<SendMessageCampaignMutation, SendMessageCampaignMutationVariables>;
export const ActivateSkillDocument = {"kind":"Document","definitions":[{"kind":"OperationDefinition","operation":"mutation","name":{"kind":"Name","value":"ActivateSkill"},"variableDefinitions":[{"kind":"VariableDefinition","variable":{"kind":"Variable","name":{"kind":"Name","value":"id"}},"type":{"kind":"NonNullType","type":{"kind":"NamedType","name":{"kind":"Name","value":"UUID"}}}}],"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"activateSkill"},"arguments":[{"kind":"Argument","name":{"kind":"Name","value":"id"},"value":{"kind":"Variable","name":{"kind":"Name","value":"id"}}}],"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"FragmentSpread","name":{"kind":"Name","value":"SkillFields"}}]}}]}},{"kind":"FragmentDefinition","name":{"kind":"Name","value":"SkillFields"},"typeCondition":{"kind":"NamedType","name":{"kind":"Name","value":"Skill"}},"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"id"}},{"kind":"Field","name":{"kind":"Name","value":"name"}},{"kind":"Field","name":{"kind":"Name","value":"label"}},{"kind":"Field","name":{"kind":"Name","value":"description"}},{"kind":"Field","name":{"kind":"Name","value":"icon"}},{"kind":"Field","name":{"kind":"Name","value":"content"}},{"kind":"Field","name":{"kind":"Name","value":"isCustom"}},{"kind":"Field","name":{"kind":"Name","value":"isActive"}},{"kind":"Field","name":{"kind":"Name","value":"createdAt"}},{"kind":"Field","name":{"kind":"Name","value":"updatedAt"}}]}}]} as unknown as DocumentNode<ActivateSkillMutation, ActivateSkillMutationVariables>;
export const AnswerAgentChatQuestionDocument = {"kind":"Document","definitions":[{"kind":"OperationDefinition","operation":"mutation","name":{"kind":"Name","value":"AnswerAgentChatQuestion"},"variableDefinitions":[{"kind":"VariableDefinition","variable":{"kind":"Variable","name":{"kind":"Name","value":"threadId"}},"type":{"kind":"NonNullType","type":{"kind":"NamedType","name":{"kind":"Name","value":"UUID"}}}},{"kind":"VariableDefinition","variable":{"kind":"Variable","name":{"kind":"Name","value":"messageId"}},"type":{"kind":"NonNullType","type":{"kind":"NamedType","name":{"kind":"Name","value":"UUID"}}}},{"kind":"VariableDefinition","variable":{"kind":"Variable","name":{"kind":"Name","value":"answers"}},"type":{"kind":"NonNullType","type":{"kind":"ListType","type":{"kind":"NonNullType","type":{"kind":"NamedType","name":{"kind":"Name","value":"AgentChatQuestionAnswerInput"}}}}}},{"kind":"VariableDefinition","variable":{"kind":"Variable","name":{"kind":"Name","value":"modelId"}},"type":{"kind":"NamedType","name":{"kind":"Name","value":"String"}}}],"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"answerAgentChatQuestion"},"arguments":[{"kind":"Argument","name":{"kind":"Name","value":"threadId"},"value":{"kind":"Variable","name":{"kind":"Name","value":"threadId"}}},{"kind":"Argument","name":{"kind":"Name","value":"messageId"},"value":{"kind":"Variable","name":{"kind":"Name","value":"messageId"}}},{"kind":"Argument","name":{"kind":"Name","value":"answers"},"value":{"kind":"Variable","name":{"kind":"Name","value":"answers"}}},{"kind":"Argument","name":{"kind":"Name","value":"modelId"},"value":{"kind":"Variable","name":{"kind":"Name","value":"modelId"}}}],"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"messageId"}},{"kind":"Field","name":{"kind":"Name","value":"queued"}},{"kind":"Field","name":{"kind":"Name","value":"streamId"}}]}}]}}]} as unknown as DocumentNode<AnswerAgentChatQuestionMutation, AnswerAgentChatQuestionMutationVariables>;
export const ArchiveChatThreadDocument = {"kind":"Document","definitions":[{"kind":"OperationDefinition","operation":"mutation","name":{"kind":"Name","value":"ArchiveChatThread"},"variableDefinitions":[{"kind":"VariableDefinition","variable":{"kind":"Variable","name":{"kind":"Name","value":"id"}},"type":{"kind":"NonNullType","type":{"kind":"NamedType","name":{"kind":"Name","value":"UUID"}}}}],"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"archiveChatThread"},"arguments":[{"kind":"Argument","name":{"kind":"Name","value":"id"},"value":{"kind":"Variable","name":{"kind":"Name","value":"id"}}}],"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"id"}},{"kind":"Field","name":{"kind":"Name","value":"deletedAt"}},{"kind":"Field","name":{"kind":"Name","value":"updatedAt"}}]}}]}}]} as unknown as DocumentNode<ArchiveChatThreadMutation, ArchiveChatThreadMutationVariables>;
export const AssignRoleToAgentDocument = {"kind":"Document","definitions":[{"kind":"OperationDefinition","operation":"mutation","name":{"kind":"Name","value":"AssignRoleToAgent"},"variableDefinitions":[{"kind":"VariableDefinition","variable":{"kind":"Variable","name":{"kind":"Name","value":"agentId"}},"type":{"kind":"NonNullType","type":{"kind":"NamedType","name":{"kind":"Name","value":"UUID"}}}},{"kind":"VariableDefinition","variable":{"kind":"Variable","name":{"kind":"Name","value":"roleId"}},"type":{"kind":"NonNullType","type":{"kind":"NamedType","name":{"kind":"Name","value":"UUID"}}}}],"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"assignRoleToAgent"},"arguments":[{"kind":"Argument","name":{"kind":"Name","value":"agentId"},"value":{"kind":"Variable","name":{"kind":"Name","value":"agentId"}}},{"kind":"Argument","name":{"kind":"Name","value":"roleId"},"value":{"kind":"Variable","name":{"kind":"Name","value":"roleId"}}}]}]}}]} as unknown as DocumentNode<AssignRoleToAgentMutation, AssignRoleToAgentMutationVariables>;
export const CreateChatThreadDocument = {"kind":"Document","definitions":[{"kind":"OperationDefinition","operation":"mutation","name":{"kind":"Name","value":"CreateChatThread"},"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"createChatThread"},"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"id"}},{"kind":"Field","name":{"kind":"Name","value":"title"}},{"kind":"Field","name":{"kind":"Name","value":"createdAt"}},{"kind":"Field","name":{"kind":"Name","value":"updatedAt"}}]}}]}}]} as unknown as DocumentNode<CreateChatThreadMutation, CreateChatThreadMutationVariables>;
@@ -4,13 +4,17 @@ import { RoutingStatusDisplay } from '@/ai/components/RoutingStatusDisplay';
import { ThinkingStepsDisplay } from '@/ai/components/ThinkingStepsDisplay';
import { IconDotsVertical } from 'twenty-ui/icon';
import { AiChatQuestionStatusRenderer } from '@/ai/components/AiChatQuestionStatusRenderer';
import { LazyMarkdownRenderer } from '@/ai/components/LazyMarkdownRenderer';
import { ToolStepRenderer } from '@/ai/components/ToolStepRenderer';
import { groupContiguousThinkingStepParts } from '@/ai/utils/groupContiguousThinkingStepParts';
import { isCodeInterpreterToolPart } from '@/ai/utils/isCodeInterpreterToolPart';
import { styled } from '@linaria/react';
import { isToolUIPart } from 'ai';
import { type ExtendedUIMessagePart } from 'twenty-shared/ai';
import { getToolName, isToolUIPart } from 'ai';
import {
ASK_QUESTIONS_TOOL_NAME,
type ExtendedUIMessagePart,
} from 'twenty-shared/ai';
import { useContext } from 'react';
import { ThemeContext, themeCssVariables } from 'twenty-ui/theme-constants';
@@ -76,6 +80,15 @@ const MessagePartRenderer = ({
);
default:
if (isToolUIPart(part)) {
if (getToolName(part) === ASK_QUESTIONS_TOOL_NAME) {
return (
<AiChatQuestionStatusRenderer
toolPart={part}
isStreaming={isStreaming}
/>
);
}
return <ToolStepRenderer toolPart={part} isStreaming={isStreaming} />;
}
return null;
@@ -3,8 +3,11 @@ import { EditorContent } from '@tiptap/react';
import { useLingui } from '@lingui/react/macro';
import { themeCssVariables } from 'twenty-ui/theme-constants';
import { isDefined } from 'twenty-shared/utils';
import { AiChatBanner } from '@/ai/components/AiChatBanner';
import { AiChatEmptyState } from '@/ai/components/AiChatEmptyState';
import { AiChatQuestionCard } from '@/ai/components/AiChatQuestionCard';
import { AIChatNoMoreBillingCreditsBanner } from '@/ai/components/AIChatNoMoreBillingCreditsBanner';
import { AiChatStandaloneError } from '@/ai/components/AiChatStandaloneError';
import { AgentChatContextPreview } from '@/ai/components/internal/AgentChatContextPreview';
@@ -18,8 +21,10 @@ import { useAiChatEditor } from '@/ai/hooks/useAiChatEditor';
import { useAiModelOptions } from '@/ai/hooks/useAiModelOptions';
import { useWorkspaceAiModelAvailability } from '@/ai/hooks/useWorkspaceAiModelAvailability';
import { agentChatUserSelectedModelState } from '@/ai/states/agentChatUserSelectedModelState';
import { agentChatPendingQuestionComponentSelector } from '@/ai/states/selectors/agentChatPendingQuestionComponentSelector';
import { Select } from '@/ui/input/components/Select';
import { useIsMobile } from '@/ui/utilities/responsive/hooks/useIsMobile';
import { useAtomComponentSelectorValue } from '@/ui/utilities/state/jotai/hooks/useAtomComponentSelectorValue';
import { useAtomStateValue } from '@/ui/utilities/state/jotai/hooks/useAtomStateValue';
import { useSetAtomState } from '@/ui/utilities/state/jotai/hooks/useSetAtomState';
import { hasReachedCurrentBillingPeriodCapSelector } from '@/workspace/states/hasReachedCurrentBillingPeriodCapSelector';
@@ -137,6 +142,10 @@ export const AiChatEditorSection = () => {
const { editor, handleSendAndClear } = useAiChatEditor();
const pendingQuestion = useAtomComponentSelectorValue(
agentChatPendingQuestionComponentSelector,
);
return (
<>
<AiChatEditorFocusEffect editor={editor} />
@@ -155,35 +164,39 @@ export const AiChatEditorSection = () => {
{hasReachedCurrentBillingPeriodCap && (
<AIChatNoMoreBillingCreditsBanner />
)}
<StyledInputBox>
<StyledEditorWrapper>
<EditorContent editor={editor} />
</StyledEditorWrapper>
<StyledButtonsContainer>
<StyledLeftButtonsContainer>
<AgentChatFileUploadButton />
<AiChatContextUsageButton />
</StyledLeftButtonsContainer>
<StyledRightButtonsContainer>
<Select
dropdownId="ai-chat-smart-model-select"
value={selectedModelId}
onChange={setAgentChatUserSelectedModel}
options={smartModelOptions}
pinnedOption={defaultPinnedOption}
disabled={hasNoEnabledModels}
selectSizeVariant="small"
showContextualTextInControl={false}
withSearchInput
dropdownOffset={{ x: 0, y: 8 }}
/>
<SendMessageButton
onSend={handleSendAndClear}
isDisabled={hasNoEnabledModels}
/>
</StyledRightButtonsContainer>
</StyledButtonsContainer>
</StyledInputBox>
{isDefined(pendingQuestion) ? (
<AiChatQuestionCard pendingQuestion={pendingQuestion} />
) : (
<StyledInputBox>
<StyledEditorWrapper>
<EditorContent editor={editor} />
</StyledEditorWrapper>
<StyledButtonsContainer>
<StyledLeftButtonsContainer>
<AgentChatFileUploadButton />
<AiChatContextUsageButton />
</StyledLeftButtonsContainer>
<StyledRightButtonsContainer>
<Select
dropdownId="ai-chat-smart-model-select"
value={selectedModelId}
onChange={setAgentChatUserSelectedModel}
options={smartModelOptions}
pinnedOption={defaultPinnedOption}
disabled={hasNoEnabledModels}
selectSizeVariant="small"
showContextualTextInControl={false}
withSearchInput
dropdownOffset={{ x: 0, y: 8 }}
/>
<SendMessageButton
onSend={handleSendAndClear}
isDisabled={hasNoEnabledModels}
/>
</StyledRightButtonsContainer>
</StyledButtonsContainer>
</StyledInputBox>
)}
</StyledInputArea>
</>
);
@@ -0,0 +1,481 @@
import { styled } from '@linaria/react';
import { useLingui } from '@lingui/react/macro';
import { type KeyboardEvent, useContext, useMemo, useState } from 'react';
import { type AskQuestionAnswer, type AskQuestionItem } from 'twenty-shared/ai';
import { isDefined } from 'twenty-shared/utils';
import { AppTooltip, TooltipDelay } from 'twenty-ui/surfaces';
import {
IconArrowUp,
IconChevronLeft,
IconChevronRightPipe,
IconInfoCircle,
type IconComponent,
IconSquareNumber1,
IconSquareNumber2,
IconSquareNumber3,
IconSquareNumber4,
IconSquareNumber5,
IconSquareNumber6,
IconSquareNumber7,
IconSquareNumber8,
IconSquareNumber9,
} from 'twenty-ui/icon';
import {
LightIconButton,
RoundedIconButton,
type SelectOption,
} from 'twenty-ui/input';
import { ThemeContext, themeCssVariables } from 'twenty-ui/theme-constants';
import { AgentChatFileUploadButton } from '@/ai/components/internal/AgentChatFileUploadButton';
import { AiChatContextUsageButton } from '@/ai/components/internal/AiChatContextUsageButton';
import { useAgentChatModelId } from '@/ai/hooks/useAgentChatModelId';
import { useAiModelOptions } from '@/ai/hooks/useAiModelOptions';
import { useSubmitQuestionAnswer } from '@/ai/hooks/useSubmitQuestionAnswer';
import { useWorkspaceAiModelAvailability } from '@/ai/hooks/useWorkspaceAiModelAvailability';
import { agentChatUserSelectedModelState } from '@/ai/states/agentChatUserSelectedModelState';
import { type AgentChatPendingQuestion } from '@/ai/types/AgentChatPendingQuestion';
import { Select } from '@/ui/input/components/Select';
import { useSetAtomState } from '@/ui/utilities/state/jotai/hooks/useSetAtomState';
const NUMBER_ICONS: IconComponent[] = [
IconSquareNumber1,
IconSquareNumber2,
IconSquareNumber3,
IconSquareNumber4,
IconSquareNumber5,
IconSquareNumber6,
IconSquareNumber7,
IconSquareNumber8,
IconSquareNumber9,
];
const StyledCard = styled.div`
background-color: ${themeCssVariables.background.transparent.lighter};
border: 1px solid ${themeCssVariables.border.color.medium};
border-radius: ${themeCssVariables.border.radius.sm};
box-sizing: border-box;
display: flex;
flex-direction: column;
width: 100%;
`;
const StyledQuestionSection = styled.div`
display: flex;
flex-direction: column;
gap: ${themeCssVariables.spacing[3]};
padding: ${themeCssVariables.spacing[2]};
`;
const StyledQuestionHeaderRow = styled.div`
align-items: center;
display: flex;
gap: ${themeCssVariables.spacing[2]};
justify-content: space-between;
min-height: 24px;
padding-left: ${themeCssVariables.spacing[1]};
`;
const StyledQuestionText = styled.p`
color: ${themeCssVariables.font.color.primary};
flex: 1 0 0;
font-size: ${themeCssVariables.font.size.md};
font-weight: ${themeCssVariables.font.weight.medium};
line-height: 1.4;
margin: 0;
min-width: 0;
overflow-wrap: anywhere;
`;
const StyledPager = styled.div`
align-items: center;
display: flex;
flex-shrink: 0;
gap: ${themeCssVariables.spacing[1]};
`;
const StyledPagerLabel = styled.span`
color: ${themeCssVariables.font.color.light};
font-size: ${themeCssVariables.font.size.sm};
`;
const StyledOptionsList = styled.div`
display: flex;
flex-direction: column;
gap: ${themeCssVariables.spacing['0.5']};
`;
const StyledOptionRow = styled.div<{ isHighlighted: boolean }>`
align-items: center;
background: ${({ isHighlighted }) =>
isHighlighted
? themeCssVariables.background.transparent.light
: 'transparent'};
border-radius: ${themeCssVariables.border.radius.sm};
box-sizing: border-box;
cursor: pointer;
display: flex;
gap: ${themeCssVariables.spacing[2]};
height: 32px;
justify-content: space-between;
overflow: hidden;
padding: 0 ${themeCssVariables.spacing[1]};
&:hover {
background: ${themeCssVariables.background.transparent.light};
}
`;
const StyledOptionLeft = styled.div`
align-items: center;
display: flex;
flex: 1 0 0;
gap: ${themeCssVariables.spacing[2]};
min-width: 0;
overflow: hidden;
`;
const StyledOptionLabel = styled.span`
color: ${themeCssVariables.font.color.secondary};
flex-shrink: 0;
font-size: ${themeCssVariables.font.size.md};
line-height: 1.4;
max-width: 100%;
overflow: hidden;
text-overflow: ellipsis;
white-space: nowrap;
`;
const StyledRecommended = styled.span`
color: ${themeCssVariables.font.color.light};
flex-shrink: 0;
font-size: ${themeCssVariables.font.size.md};
line-height: 1.4;
`;
const StyledDivider = styled.div`
background: ${themeCssVariables.border.color.light};
height: 1px;
width: 100%;
`;
const StyledComposerSection = styled.div`
box-sizing: border-box;
display: flex;
flex-direction: column;
gap: ${themeCssVariables.spacing[2]};
min-height: 80px;
padding: ${themeCssVariables.spacing[2]};
`;
const StyledFreeTextArea = styled.textarea`
background: transparent;
border: none;
color: ${themeCssVariables.font.color.primary};
flex: 1 0 0;
font-family: inherit;
font-size: ${themeCssVariables.font.size.md};
line-height: 1.4;
min-height: 24px;
outline: none;
resize: none;
&::placeholder {
color: ${themeCssVariables.font.color.light};
font-weight: ${themeCssVariables.font.weight.medium};
}
`;
const StyledActionsRow = styled.div`
align-items: center;
display: flex;
justify-content: space-between;
width: 100%;
`;
const StyledLeftActions = styled.div`
align-items: center;
display: flex;
gap: ${themeCssVariables.spacing['0.5']};
`;
const StyledRightActions = styled.div`
align-items: center;
display: flex;
gap: ${themeCssVariables.spacing[1]};
`;
const areAllQuestionsAnswered = (
questions: AskQuestionItem[],
selectedByQuestion: Record<number, number[]>,
freeTextByQuestion: Record<number, string>,
) =>
questions.every(
(_, index) =>
(selectedByQuestion[index]?.length ?? 0) > 0 ||
(freeTextByQuestion[index] ?? '').trim().length > 0,
);
type AiChatQuestionCardProps = {
pendingQuestion: AgentChatPendingQuestion;
};
export const AiChatQuestionCard = ({
pendingQuestion,
}: AiChatQuestionCardProps) => {
const { t } = useLingui();
const { theme } = useContext(ThemeContext);
const { messageId, toolCallId, questions } = pendingQuestion;
const [currentIndex, setCurrentIndex] = useState(0);
const [selectedByQuestion, setSelectedByQuestion] = useState<
Record<number, number[]>
>({});
const [freeTextByQuestion, setFreeTextByQuestion] = useState<
Record<number, string>
>({});
const [isSubmitting, setIsSubmitting] = useState(false);
const { submitAnswer } = useSubmitQuestionAnswer();
const { options: modelOptions, pinnedOption } = useAiModelOptions({
variant: 'pinned-default',
});
const { enabledModels } = useWorkspaceAiModelAvailability();
const hasNoEnabledModels = enabledModels.length === 0;
const { selectedModelId } = useAgentChatModelId();
const setAgentChatUserSelectedModel = useSetAtomState(
agentChatUserSelectedModelState,
);
const defaultPinnedOption: SelectOption<string | null> | undefined =
pinnedOption ? { ...pinnedOption, value: null } : undefined;
const currentQuestion = questions[currentIndex];
const hasMultipleQuestions = questions.length > 1;
const isLastQuestion = currentIndex === questions.length - 1;
const buildAnswers = (
selected: Record<number, number[]>,
): AskQuestionAnswer[] =>
questions.map((_, index) => {
const trimmedFreeText = (freeTextByQuestion[index] ?? '').trim();
return {
questionIndex: index,
selectedOptionIndices: selected[index] ?? [],
freeText: trimmedFreeText.length > 0 ? trimmedFreeText : undefined,
};
});
const submit = async (answers: AskQuestionAnswer[]) => {
if (isSubmitting) {
return;
}
setIsSubmitting(true);
await submitAnswer({ messageId, toolCallId, answers });
setIsSubmitting(false);
};
const handleSelectOption = (optionIndex: number) => {
if (currentQuestion.allowMultiSelect === true) {
setSelectedByQuestion((previous) => {
const current = previous[currentIndex] ?? [];
const next = current.includes(optionIndex)
? current.filter((value) => value !== optionIndex)
: [...current, optionIndex];
return { ...previous, [currentIndex]: next };
});
return;
}
const nextSelected = {
...selectedByQuestion,
[currentIndex]: [optionIndex],
};
setSelectedByQuestion(nextSelected);
if (!isLastQuestion) {
setCurrentIndex(currentIndex + 1);
return;
}
if (areAllQuestionsAnswered(questions, nextSelected, freeTextByQuestion)) {
void submit(buildAnswers(nextSelected));
}
};
const allQuestionsAnswered = useMemo(
() =>
areAllQuestionsAnswered(
questions,
selectedByQuestion,
freeTextByQuestion,
),
[questions, selectedByQuestion, freeTextByQuestion],
);
const handleSend = () => {
if (!allQuestionsAnswered) {
return;
}
void submit(buildAnswers(selectedByQuestion));
};
const handleKeyDown = (event: KeyboardEvent<HTMLTextAreaElement>) => {
if (event.key === 'Enter' && !event.shiftKey) {
event.preventDefault();
handleSend();
}
};
return (
<StyledCard>
<StyledQuestionSection>
<StyledQuestionHeaderRow>
<StyledQuestionText>{currentQuestion.question}</StyledQuestionText>
{hasMultipleQuestions && (
<StyledPager>
<LightIconButton
Icon={IconChevronLeft}
size="small"
disabled={currentIndex === 0}
onClick={() =>
setCurrentIndex((index) => Math.max(0, index - 1))
}
/>
<StyledPagerLabel>
{currentIndex + 1}/{questions.length}
</StyledPagerLabel>
<LightIconButton
Icon={IconChevronRightPipe}
size="small"
disabled={isLastQuestion}
onClick={() =>
setCurrentIndex((index) =>
Math.min(questions.length - 1, index + 1),
)
}
/>
</StyledPager>
)}
</StyledQuestionHeaderRow>
<StyledOptionsList>
{currentQuestion.options.map((option, optionIndex) => {
const NumberIcon =
NUMBER_ICONS[optionIndex] ??
NUMBER_ICONS[NUMBER_ICONS.length - 1];
const isSelected = (
selectedByQuestion[currentIndex] ?? []
).includes(optionIndex);
const hasSelection =
(selectedByQuestion[currentIndex] ?? []).length > 0;
const isHighlighted =
isSelected || (!hasSelection && option.isRecommended === true);
const tooltipId = `ask-question-option-${toolCallId}-${currentIndex}-${optionIndex}`;
return (
<StyledOptionRow
key={optionIndex}
isHighlighted={isHighlighted}
role="button"
tabIndex={0}
onClick={() => handleSelectOption(optionIndex)}
onKeyDown={(event) => {
if (event.target !== event.currentTarget) {
return;
}
if (event.key === 'Enter' || event.key === ' ') {
event.preventDefault();
handleSelectOption(optionIndex);
}
}}
>
<StyledOptionLeft>
<NumberIcon
size={theme.icon.size.sm}
color={themeCssVariables.font.color.tertiary}
/>
<StyledOptionLabel>{option.label}</StyledOptionLabel>
{option.isRecommended === true && (
<StyledRecommended>· {t`Recommended`}</StyledRecommended>
)}
</StyledOptionLeft>
{isDefined(option.description) && (
<>
<span
id={tooltipId}
onClick={(event) => event.stopPropagation()}
>
<LightIconButton
Icon={IconInfoCircle}
size="small"
accent="tertiary"
/>
</span>
<AppTooltip
anchorSelect={`#${tooltipId}`}
content={option.description}
delay={TooltipDelay.shortDelay}
place="left"
/>
</>
)}
</StyledOptionRow>
);
})}
</StyledOptionsList>
</StyledQuestionSection>
<StyledDivider />
<StyledComposerSection>
<StyledFreeTextArea
value={freeTextByQuestion[currentIndex] ?? ''}
placeholder={t`Type anything to do differently.`}
onChange={(event) =>
setFreeTextByQuestion((previous) => ({
...previous,
[currentIndex]: event.target.value,
}))
}
onKeyDown={handleKeyDown}
autoFocus
/>
<StyledActionsRow>
<StyledLeftActions>
<AgentChatFileUploadButton />
<AiChatContextUsageButton />
</StyledLeftActions>
<StyledRightActions>
<Select
dropdownId="ai-chat-question-model-select"
value={selectedModelId}
onChange={setAgentChatUserSelectedModel}
options={modelOptions}
pinnedOption={defaultPinnedOption}
disabled={hasNoEnabledModels}
selectSizeVariant="small"
showContextualTextInControl={false}
withSearchInput
dropdownOffset={{ x: 0, y: 8 }}
/>
<RoundedIconButton
Icon={IconArrowUp}
size="medium"
onClick={handleSend}
disabled={!allQuestionsAnswered || isSubmitting}
/>
</StyledRightActions>
</StyledActionsRow>
</StyledComposerSection>
</StyledCard>
);
};
@@ -0,0 +1,112 @@
import { styled } from '@linaria/react';
import { useLingui } from '@lingui/react/macro';
import { isNonEmptyString } from '@sniptt/guards';
import { type DynamicToolUIPart, type ToolUIPart } from 'ai';
import { useContext } from 'react';
import { type AskQuestionsToolResult } from 'twenty-shared/ai';
import { IconHelpCircle } from 'twenty-ui/icon';
import { ThemeContext, themeCssVariables } from 'twenty-ui/theme-constants';
import { ShimmeringText } from '@/ai/components/ShimmeringText';
const StyledContainer = styled.div`
align-items: flex-start;
color: ${themeCssVariables.font.color.tertiary};
display: flex;
gap: ${themeCssVariables.spacing[1]};
padding: ${themeCssVariables.spacing[1]} 0;
svg {
flex-shrink: 0;
margin-top: 1px;
}
`;
const StyledContent = styled.div`
display: flex;
flex-direction: column;
gap: ${themeCssVariables.spacing['0.5']};
min-width: 0;
`;
const StyledMessage = styled.span`
color: ${themeCssVariables.font.color.tertiary};
font-size: ${themeCssVariables.font.size.md};
font-weight: ${themeCssVariables.font.weight.medium};
`;
const StyledAnswerLine = styled.span`
color: ${themeCssVariables.font.color.tertiary};
font-size: ${themeCssVariables.font.size.sm};
`;
const StyledAnswerValue = styled.span`
color: ${themeCssVariables.font.color.secondary};
`;
export const AiChatQuestionStatusRenderer = ({
toolPart,
isStreaming,
}: {
toolPart: ToolUIPart | DynamicToolUIPart;
isStreaming: boolean;
}) => {
const { t } = useLingui();
const { theme } = useContext(ThemeContext);
const result = (toolPart.output as { result?: AskQuestionsToolResult } | null)
?.result;
const questions = result?.questions ?? [];
const status = result?.status ?? 'pending';
if (status === 'pending') {
const label = t`Asking questions...`;
return (
<StyledContainer>
<IconHelpCircle size={theme.icon.size.sm} />
{isStreaming ? (
<ShimmeringText>
<StyledMessage>{label}</StyledMessage>
</ShimmeringText>
) : (
<StyledMessage>{label}</StyledMessage>
)}
</StyledContainer>
);
}
const answers = result?.answers ?? [];
return (
<StyledContainer>
<IconHelpCircle size={theme.icon.size.sm} />
<StyledContent>
<StyledMessage>{t`Questions answered`}</StyledMessage>
{questions.map((question, index) => {
const answer = answers.find(
(candidate) => candidate.questionIndex === index,
);
const selectedLabels = (answer?.selectedOptionIndices ?? [])
.map((optionIndex) => question.options[optionIndex]?.label)
.filter(isNonEmptyString);
const freeTextAnswer = answer?.freeText ?? '';
const value =
freeTextAnswer.length > 0
? freeTextAnswer
: selectedLabels.join(', ');
if (value.length === 0) {
return null;
}
return (
<StyledAnswerLine key={index}>
{question.header}: <StyledAnswerValue>{value}</StyledAnswerValue>
</StyledAnswerLine>
);
})}
</StyledContent>
</StyledContainer>
);
};
@@ -0,0 +1,124 @@
import { type Meta, type StoryObj } from '@storybook/react-vite';
import { useStore } from 'jotai';
import { type ReactNode, useEffect } from 'react';
import { ComponentDecorator } from 'twenty-ui/testing';
import { AiChatQuestionCard } from '@/ai/components/AiChatQuestionCard';
import { AgentChatComponentInstanceContext } from '@/ai/contexts/AgentChatComponentInstanceContext';
import { agentChatDisplayedThreadState } from '@/ai/states/agentChatDisplayedThreadState';
import { currentAiChatThreadState } from '@/ai/states/currentAiChatThreadState';
import { type AgentChatPendingQuestion } from '@/ai/types/AgentChatPendingQuestion';
import { styled } from '@linaria/react';
import { RootDecorator } from '~/testing/decorators/RootDecorator';
import { SnackBarDecorator } from '~/testing/decorators/SnackBarDecorator';
const StyledContainer = styled.div`
max-width: 400px;
padding: 24px;
`;
const INSTANCE_ID = 'agentChatQuestionCardStory';
const singleQuestion: AgentChatPendingQuestion = {
messageId: 'assistant-1',
toolCallId: 'call-1',
questions: [
{
header: 'Email type',
question: 'What type of emails would you like to send?',
options: [
{
label: 'A welcome email',
description: 'A short, friendly note to introduce yourself.',
isRecommended: true,
},
{ label: 'A presentation of Twenty' },
{ label: 'An offer for a potential partnership' },
],
},
],
};
const multipleQuestions: AgentChatPendingQuestion = {
messageId: 'assistant-1',
toolCallId: 'call-2',
questions: [
singleQuestion.questions[0],
{
header: 'Tone',
question: 'Which tone should the email use?',
options: [
{ label: 'Friendly', isRecommended: true },
{ label: 'Formal' },
],
},
],
};
const longQuestion: AgentChatPendingQuestion = {
messageId: 'assistant-1',
toolCallId: 'call-3',
questions: [
{
header: 'Improvement',
question:
'What is the one improvement you would make to the "Send follow-up emails to stale opportunities" workflow before we roll it out to the whole team?',
options: [
{ label: 'Wording clarity' },
{ label: 'Better layout' },
{ label: 'More flexibility', isRecommended: true },
{ label: 'Fewer steps' },
],
},
multipleQuestions.questions[1],
],
};
const StoreSeeder = ({ children }: { children: ReactNode }) => {
const store = useStore();
useEffect(() => {
store.set(currentAiChatThreadState.atom, 'thread-1');
store.set(agentChatDisplayedThreadState.atom, 'thread-1');
}, [store]);
return <>{children}</>;
};
const meta: Meta<typeof AiChatQuestionCard> = {
title: 'Modules/AiChat/AiChatQuestionCard',
component: AiChatQuestionCard,
decorators: [
(Story) => (
<AgentChatComponentInstanceContext.Provider
value={{ instanceId: INSTANCE_ID }}
>
<StoreSeeder>
<StyledContainer>
<Story />
</StyledContainer>
</StoreSeeder>
</AgentChatComponentInstanceContext.Provider>
),
SnackBarDecorator,
ComponentDecorator,
RootDecorator,
],
};
export default meta;
type Story = StoryObj<typeof AiChatQuestionCard>;
export const SingleQuestion: Story = {
args: { pendingQuestion: singleQuestion },
};
export const MultipleQuestions: Story = {
args: { pendingQuestion: multipleQuestions },
};
export const LongQuestion: Story = {
args: { pendingQuestion: longQuestion },
};
@@ -0,0 +1,21 @@
import { gql } from '@apollo/client';
export const ANSWER_AGENT_CHAT_QUESTION = gql`
mutation AnswerAgentChatQuestion(
$threadId: UUID!
$messageId: UUID!
$answers: [AgentChatQuestionAnswerInput!]!
$modelId: String
) {
answerAgentChatQuestion(
threadId: $threadId
messageId: $messageId
answers: $answers
modelId: $modelId
) {
messageId
queued
streamId
}
}
`;
@@ -0,0 +1,81 @@
import { CombinedGraphQLErrors } from '@apollo/client/errors';
import { useApolloClient } from '@apollo/client/react';
import { useStore } from 'jotai';
import { useCallback } from 'react';
import { type AskQuestionAnswer } from 'twenty-shared/ai';
import { isDefined } from 'twenty-shared/utils';
import { AGENT_CHAT_INSTANCE_ID } from '@/ai/constants/AgentChatInstanceId';
import { AGENT_CHAT_REFETCH_MESSAGES_EVENT_NAME } from '@/ai/constants/AgentChatRefetchMessagesEventName';
import { ANSWER_AGENT_CHAT_QUESTION } from '@/ai/graphql/mutations/answerAgentChatQuestion';
import { useAgentChatModelId } from '@/ai/hooks/useAgentChatModelId';
import { agentChatDisplayedThreadState } from '@/ai/states/agentChatDisplayedThreadState';
import { agentChatMessagesComponentFamilyState } from '@/ai/states/agentChatMessagesComponentFamilyState';
import { markQuestionAnswered } from '@/ai/utils/markQuestionAnswered';
import { markQuestionPending } from '@/ai/utils/markQuestionPending';
import { dispatchBrowserEvent } from '@/browser-event/utils/dispatchBrowserEvent';
import { useSnackBar } from '@/ui/feedback/snack-bar-manager/hooks/useSnackBar';
export const useSubmitQuestionAnswer = () => {
const apolloClient = useApolloClient();
const store = useStore();
const { enqueueErrorSnackBar } = useSnackBar();
const { modelIdForRequest } = useAgentChatModelId();
const submitAnswer = useCallback(
async ({
messageId,
toolCallId,
answers,
}: {
messageId: string;
toolCallId: string;
answers: AskQuestionAnswer[];
}) => {
const threadId = store.get(agentChatDisplayedThreadState.atom);
if (!isDefined(threadId)) {
return;
}
const messagesAtom = agentChatMessagesComponentFamilyState.atomFamily({
instanceId: AGENT_CHAT_INSTANCE_ID,
familyKey: { threadId },
});
const previousMessages = store.get(messagesAtom);
store.set(
messagesAtom,
markQuestionAnswered(previousMessages, messageId, toolCallId, answers),
);
try {
await apolloClient.mutate({
mutation: ANSWER_AGENT_CHAT_QUESTION,
variables: {
threadId,
messageId,
answers,
modelId: modelIdForRequest,
},
});
dispatchBrowserEvent(AGENT_CHAT_REFETCH_MESSAGES_EVENT_NAME);
} catch (error) {
const currentMessages = store.get(messagesAtom);
store.set(
messagesAtom,
markQuestionPending(currentMessages, messageId, toolCallId),
);
enqueueErrorSnackBar({
apolloError: CombinedGraphQLErrors.is(error) ? error : undefined,
});
}
},
[apolloClient, store, enqueueErrorSnackBar, modelIdForRequest],
);
return { submitAnswer };
};
@@ -0,0 +1,57 @@
import { getToolName, isToolUIPart } from 'ai';
import {
ASK_QUESTIONS_TOOL_NAME,
type AskQuestionsToolResult,
} from 'twenty-shared/ai';
import { AgentChatComponentInstanceContext } from '@/ai/contexts/AgentChatComponentInstanceContext';
import { agentChatDisplayedThreadState } from '@/ai/states/agentChatDisplayedThreadState';
import { agentChatMessagesComponentFamilyState } from '@/ai/states/agentChatMessagesComponentFamilyState';
import { type AgentChatPendingQuestion } from '@/ai/types/AgentChatPendingQuestion';
import { createAtomComponentSelector } from '@/ui/utilities/state/jotai/utils/createAtomComponentSelector';
export const agentChatPendingQuestionComponentSelector =
createAtomComponentSelector<AgentChatPendingQuestion | null>({
key: 'agentChatPendingQuestionComponentSelector',
componentInstanceContext: AgentChatComponentInstanceContext,
get:
({ instanceId }) =>
({ get }) => {
const currentThreadId = get(agentChatDisplayedThreadState);
const messages = get(agentChatMessagesComponentFamilyState, {
instanceId,
familyKey: { threadId: currentThreadId },
});
const lastAssistantMessage = [...messages]
.reverse()
.find((message) => message.role === 'assistant');
if (!lastAssistantMessage) {
return null;
}
for (const part of lastAssistantMessage.parts) {
if (
!isToolUIPart(part) ||
getToolName(part) !== ASK_QUESTIONS_TOOL_NAME
) {
continue;
}
const result = (part.output as { result?: AskQuestionsToolResult })
?.result;
if (result?.status === 'pending') {
return {
messageId: lastAssistantMessage.id,
toolCallId: part.toolCallId,
questions: result.questions,
};
}
}
return null;
},
});
@@ -0,0 +1,7 @@
import { type AskQuestionItem } from 'twenty-shared/ai';
export type AgentChatPendingQuestion = {
messageId: string;
toolCallId: string;
questions: AskQuestionItem[];
};
@@ -0,0 +1,83 @@
import { type ExtendedUIMessage } from 'twenty-shared/ai';
import { markQuestionAnswered } from '@/ai/utils/markQuestionAnswered';
const buildMessages = (status: 'pending' | 'answered'): ExtendedUIMessage[] => [
{
id: 'assistant-1',
role: 'assistant',
parts: [
{
type: 'tool-ask_questions',
toolCallId: 'call-1',
state: 'output-available',
input: { questions: [] },
output: {
success: true,
message: 'x',
result: {
questions: [
{ header: 'Type', question: 'Which?', options: [{ label: 'A' }] },
],
status,
},
},
},
],
metadata: { createdAt: '2024-01-01T00:00:00.000Z' },
} as unknown as ExtendedUIMessage,
];
describe('markQuestionAnswered', () => {
it('flips the matching tool part to answered and stores the answers', () => {
const answers = [
{ questionIndex: 0, selectedOptionIndices: [0], freeText: undefined },
];
const result = markQuestionAnswered(
buildMessages('pending'),
'assistant-1',
'call-1',
answers,
);
const output = (result[0].parts[0] as { output?: { result?: unknown } })
.output;
expect(output).toMatchObject({
result: { status: 'answered', answers },
});
});
it('preserves the original questions on the resolved part', () => {
const result = markQuestionAnswered(
buildMessages('pending'),
'assistant-1',
'call-1',
[{ questionIndex: 0, selectedOptionIndices: [0] }],
);
const output = (
result[0].parts[0] as {
output?: { result?: { questions?: unknown[] } };
}
).output;
expect(output?.result?.questions).toHaveLength(1);
});
it('leaves non-matching tool calls untouched', () => {
const result = markQuestionAnswered(
buildMessages('pending'),
'assistant-1',
'other-call',
[{ questionIndex: 0, selectedOptionIndices: [0] }],
);
const output = (
result[0].parts[0] as { output?: { result?: { status?: string } } }
).output;
expect(output?.result?.status).toBe('pending');
});
});
@@ -0,0 +1,48 @@
import { isToolUIPart } from 'ai';
import {
type AskQuestionAnswer,
type AskQuestionsToolResult,
type ExtendedUIMessage,
type ExtendedUIMessagePart,
} from 'twenty-shared/ai';
import { isDefined } from 'twenty-shared/utils';
export const markQuestionAnswered = (
messages: ExtendedUIMessage[],
messageId: string,
toolCallId: string,
answers: AskQuestionAnswer[],
): ExtendedUIMessage[] =>
messages.map((message) => {
if (message.id !== messageId) {
return message;
}
return {
...message,
parts: message.parts.map((part) => {
if (!isToolUIPart(part) || part.toolCallId !== toolCallId) {
return part;
}
const previousOutput = isDefined(part.output)
? (part.output as Record<string, unknown>)
: {};
const previousResult = previousOutput.result as
| AskQuestionsToolResult
| undefined;
return {
...part,
output: {
...previousOutput,
result: {
questions: previousResult?.questions ?? [],
status: 'answered',
answers,
} satisfies AskQuestionsToolResult,
},
} as ExtendedUIMessagePart;
}),
};
});
@@ -0,0 +1,45 @@
import { isToolUIPart } from 'ai';
import {
type AskQuestionsToolResult,
type ExtendedUIMessage,
type ExtendedUIMessagePart,
} from 'twenty-shared/ai';
import { isDefined } from 'twenty-shared/utils';
export const markQuestionPending = (
messages: ExtendedUIMessage[],
messageId: string,
toolCallId: string,
): ExtendedUIMessage[] =>
messages.map((message) => {
if (message.id !== messageId) {
return message;
}
return {
...message,
parts: message.parts.map((part) => {
if (!isToolUIPart(part) || part.toolCallId !== toolCallId) {
return part;
}
const previousOutput = isDefined(part.output)
? (part.output as Record<string, unknown>)
: {};
const previousResult = previousOutput.result as
| AskQuestionsToolResult
| undefined;
return {
...part,
output: {
...previousOutput,
result: {
questions: previousResult?.questions ?? [],
status: 'pending',
} satisfies AskQuestionsToolResult,
},
} as ExtendedUIMessagePart;
}),
};
});
@@ -0,0 +1,21 @@
import { QueryRunner } from 'typeorm';
import { RegisteredInstanceCommand } from 'src/engine/core-modules/upgrade/decorators/registered-instance-command.decorator';
import { FastInstanceCommand } from 'src/engine/core-modules/upgrade/interfaces/fast-instance-command.interface';
@RegisteredInstanceCommand('2.19.0', 1811000000000)
export class AddPendingQuestionMessageIdToAgentChatThreadFastInstanceCommand
implements FastInstanceCommand
{
public async up(queryRunner: QueryRunner): Promise<void> {
await queryRunner.query(
`ALTER TABLE "core"."agentChatThread" ADD COLUMN IF NOT EXISTS "pendingQuestionMessageId" uuid`,
);
}
public async down(queryRunner: QueryRunner): Promise<void> {
await queryRunner.query(
`ALTER TABLE "core"."agentChatThread" DROP COLUMN IF EXISTS "pendingQuestionMessageId"`,
);
}
}
@@ -0,0 +1,4 @@
// Referenced by @WasIntroducedInUpgrade on the "pendingQuestionMessageId"
// column so pre-2.19 upgrade steps don't SELECT it before this command adds it.
export const ADD_PENDING_QUESTION_MESSAGE_ID_TO_AGENT_CHAT_THREAD_UPGRADE_COMMAND_NAME =
'2.19.0_AddPendingQuestionMessageIdToAgentChatThreadFastInstanceCommand_1811000000000';
@@ -87,6 +87,7 @@ import { EncryptNonSecretApplicationVariableSlowInstanceCommand } from 'src/data
import { MigrateAiModelPreferencesSlowInstanceCommand } from 'src/database/commands/upgrade-version-command/2-9/2-9-instance-command-slow-1799000010000-migrate-ai-model-preferences';
import { AddFolderImportToMessageFolderPendingSyncActionFastInstanceCommand } from './2-15/2-15-instance-command-fast-1781714499016-add-folder-import-to-message-folder-pending-sync-action';
import { AddViewKanbanColumnWidthFastInstanceCommand } from './2-15/2-15-instance-command-fast-1781900000000-add-view-kanban-column-width';
import { AddPendingQuestionMessageIdToAgentChatThreadFastInstanceCommand } from 'src/database/commands/upgrade-version-command/2-19/2-19-instance-command-fast-1811000000000-add-pending-question-to-agent-chat-thread';
import { DropMetadataStandardOverridesColumnFastInstanceCommand } from './2-20/2-20-instance-command-fast-1825000000000-drop-metadata-standard-overrides-column';
export const INSTANCE_COMMANDS = [
@@ -174,6 +175,7 @@ export const INSTANCE_COMMANDS = [
CreateApplicationTranslationCoreTableFastInstanceCommand,
AddTsVectorFieldMetadataIdToSearchFieldMetadataFastInstanceCommand,
BackfillTsVectorFieldMetadataIdOnSearchFieldMetadataSlowInstanceCommand,
AddPendingQuestionMessageIdToAgentChatThreadFastInstanceCommand,
AddMetadataOverridesColumnFastInstanceCommand,
BackfillMetadataOverridesSlowInstanceCommand,
AddLastStreamErrorToAgentChatThreadFastInstanceCommand,
@@ -59,6 +59,12 @@ Building or editing dashboards through the AI is not available yet — it is a c
- **Favorites are navigation menu items.** Twenty has no separate "Favorites" concept. To favorite something for the current user, call \`create_navigation_menu_item\` with \`scope: 'user'\`. Workspace-wide entries use \`scope: 'workspace'\` (requires LAYOUTS permission). Both are the same primitive — do not look for a separate favorites tool.
- **A default OBJECT navigation menu item is auto-created with \`create_object_metadata\`.** Don't immediately create another OBJECT item for the new object — only add a follow-up navigation item when the user is asking to pin a *different* view, folder, link, record, or page layout.
## Asking the user questions
- When a decision is genuinely ambiguous or consequential and you cannot infer it from the request or context, call \`ask_questions\` to ask the user one or more multiple-choice questions instead of guessing. The conversation pauses until they answer.
- Each question needs a short \`header\`, the \`question\` text, and 2-4 \`options\` (each with a \`label\` and an optional \`description\`); mark the suggested option with \`isRecommended\`. The user can always type a free-form answer instead of picking an option.
- Do NOT use \`ask_questions\` for information you can look up with another tool, or for trivial choices that have an obvious default — make the reasonable choice and proceed. Ask at most a few focused questions at once.
`,
// Browsing context hint
@@ -0,0 +1,13 @@
import { Field, InputType, Int } from '@nestjs/graphql';
@InputType()
export class AgentChatQuestionAnswerInput {
@Field(() => Int)
questionIndex: number;
@Field(() => [Int])
selectedOptionIndices: number[];
@Field(() => String, { nullable: true })
freeText?: string;
}
@@ -11,6 +11,7 @@ import {
} from 'typeorm';
import { ADD_LAST_STREAM_ERROR_TO_AGENT_CHAT_THREAD_UPGRADE_COMMAND_NAME } from 'src/database/commands/upgrade-version-command/2-19/add-last-stream-error-to-agent-chat-thread-upgrade-command-name.constant';
import { ADD_PENDING_QUESTION_MESSAGE_ID_TO_AGENT_CHAT_THREAD_UPGRADE_COMMAND_NAME } from 'src/database/commands/upgrade-version-command/2-19/add-pending-question-message-id-to-agent-chat-thread-upgrade-command-name.constant';
import { WasIntroducedInUpgrade } from 'src/engine/core-modules/upgrade/decorators/was-introduced-in-upgrade.decorator';
import { UserWorkspaceEntity } from 'src/engine/core-modules/user-workspace/user-workspace.entity';
import { AgentMessageEntity } from 'src/engine/metadata-modules/ai/ai-agent-execution/entities/agent-message.entity';
@@ -73,6 +74,13 @@ export class AgentChatThreadEntity {
@Column({ type: 'varchar', nullable: true })
activeStreamId: string | null;
@WasIntroducedInUpgrade({
upgradeCommandName:
ADD_PENDING_QUESTION_MESSAGE_ID_TO_AGENT_CHAT_THREAD_UPGRADE_COMMAND_NAME,
})
@Column({ type: 'uuid', nullable: true })
pendingQuestionMessageId: string | null;
@WasIntroducedInUpgrade({
upgradeCommandName:
ADD_LAST_STREAM_ERROR_TO_AGENT_CHAT_THREAD_UPGRADE_COMMAND_NAME,
@@ -18,4 +18,5 @@ export type StreamAgentChatJobData = {
hasTitle: boolean;
existingTurnId?: string;
conversationSizeTokens: number;
isResume?: boolean;
};
@@ -10,7 +10,7 @@ import type {
} from 'twenty-shared/ai';
import { isDefined } from 'twenty-shared/utils';
import { Repository } from 'typeorm';
import { v4 } from 'uuid';
import { v5 as uuidv5 } from 'uuid';
import { Process } from 'src/engine/core-modules/message-queue/decorators/process.decorator';
import { Processor } from 'src/engine/core-modules/message-queue/decorators/processor.decorator';
@@ -27,6 +27,7 @@ import { AgentChatEventPublisherService } from 'src/engine/metadata-modules/ai/a
import { AgentChatStreamingService } from 'src/engine/metadata-modules/ai/ai-chat/services/agent-chat-streaming.service';
import { AgentChatService } from 'src/engine/metadata-modules/ai/ai-chat/services/agent-chat.service';
import { ChatExecutionService } from 'src/engine/metadata-modules/ai/ai-chat/services/chat-execution.service';
import { findPendingQuestionPart } from 'src/engine/metadata-modules/ai/ai-chat/utils/find-pending-question-part.util';
import { getCancelChannel } from 'src/engine/metadata-modules/ai/ai-chat/utils/get-cancel-channel.util';
import { mapErrorToStreamError } from 'src/engine/metadata-modules/ai/ai-chat/utils/map-error-to-stream-error.util';
import type { AiModelConfig } from 'src/engine/metadata-modules/ai/ai-models/types/ai-model-config.type';
@@ -38,6 +39,11 @@ import { type StreamAgentChatJobData } from './stream-agent-chat-job.types';
export { STREAM_AGENT_CHAT_JOB_NAME, type StreamAgentChatJobData };
// Derive assistantMessageId deterministically from streamId so assistant-message
// persistence is idempotent per stream: a retried job for the stream is skipped,
// while each distinct resume in a turn persists its own message.
const ASSISTANT_MESSAGE_ID_NAMESPACE = '0b9c2a3d-4e5f-4a1b-8c2d-3e4f5a6b7c8d';
@Processor({ queueName: MessageQueue.aiStreamQueue, scope: Scope.REQUEST })
export class StreamAgentChatJob {
private readonly logger = new Logger(StreamAgentChatJob.name);
@@ -204,7 +210,10 @@ export class StreamAgentChatJob {
abortSignal: AbortSignal;
}): Promise<void> {
return new Promise<void>((resolve, reject) => {
const assistantMessageId = v4();
const assistantMessageId = uuidv5(
data.streamId,
ASSISTANT_MESSAGE_ID_NAMESPACE,
);
let streamUsage = {
inputTokens: 0,
@@ -516,7 +525,9 @@ export class StreamAgentChatJob {
(part) => part.type === 'text' && isNonEmptyString(part.text),
);
if (isAborted || !hasText) {
const pendingQuestionPart = findPendingQuestionPart(responseMessage.parts);
if ((isAborted || !hasText) && !isDefined(pendingQuestionPart)) {
this.logAssistantTurnWithoutText({
responseMessage,
isAborted,
@@ -544,13 +555,16 @@ export class StreamAgentChatJob {
const userMessage = await userMessagePromise;
if (
isDefined(userMessage.turnId) &&
(await this.agentChatService.hasAssistantMessageForTurn({
turnId: userMessage.turnId,
// Idempotent per stream: assistantMessageId is derived from the streamId,
// so a retried job for this stream is skipped here while each distinct
// resume in the turn persists its own message.
const assistantMessageAlreadyPersisted =
await this.agentChatService.hasMessageById({
id: assistantMessageId,
workspaceId,
}))
) {
});
if (assistantMessageAlreadyPersisted) {
return;
}
@@ -580,6 +594,9 @@ export class StreamAgentChatJob {
`"totalCacheCreationTokens" + ${totalCacheCreationTokens}`,
contextWindowTokens: modelConfig.contextWindowTokens,
conversationSize: lastStepConversationSize,
pendingQuestionMessageId: isDefined(pendingQuestionPart)
? assistantMessageId
: null,
lastStreamError: null,
},
);
@@ -8,6 +8,7 @@ import {
ResolveField,
} from '@nestjs/graphql';
import { generateId } from 'ai';
import GraphQLJSON from 'graphql-type-json';
import { PermissionFlagType } from 'twenty-shared/constants';
import { isDefined } from 'twenty-shared/utils';
@@ -24,6 +25,7 @@ import { SettingsPermissionGuard } from 'src/engine/guards/settings-permission.g
import { WorkspaceAuthGuard } from 'src/engine/guards/workspace-auth.guard';
import { AgentMessageDTO } from 'src/engine/metadata-modules/ai/ai-agent-execution/dtos/agent-message.dto';
import { type BrowsingContextType } from 'src/engine/metadata-modules/ai/ai-agent/types/browsingContext.type';
import { AgentChatQuestionAnswerInput } from 'src/engine/metadata-modules/ai/ai-chat/dtos/agent-chat-question-answer.input';
import { AgentChatThreadDTO } from 'src/engine/metadata-modules/ai/ai-chat/dtos/agent-chat-thread.dto';
import { FileAttachmentInput } from 'src/engine/metadata-modules/ai/ai-chat/dtos/file-attachment.input';
import { AiSystemPromptPreviewDTO } from 'src/engine/metadata-modules/ai/ai-chat/dtos/ai-system-prompt-preview.dto';
@@ -186,7 +188,10 @@ export class AgentChatResolver {
});
}
if (isDefined(thread.activeStreamId)) {
if (
isDefined(thread.activeStreamId) ||
isDefined(thread.pendingQuestionMessageId)
) {
const queuedMessage = await this.agentChatService.queueMessage({
threadId,
text,
@@ -259,6 +264,78 @@ export class AgentChatResolver {
};
}
@Mutation(() => SendChatMessageResultDTO)
async answerAgentChatQuestion(
@Args('threadId', { type: () => UUIDScalarType }) threadId: string,
@Args('messageId', { type: () => UUIDScalarType }) messageId: string,
@Args('answers', { type: () => [AgentChatQuestionAnswerInput] })
answers: AgentChatQuestionAnswerInput[],
@Args('modelId', { type: () => String, nullable: true })
modelId: string | undefined,
@AuthUserWorkspaceId() userWorkspaceId: string,
@AuthWorkspace() workspace: WorkspaceEntity,
): Promise<SendChatMessageResultDTO> {
if (this.aiModelRegistryService.getAvailableModels().length === 0) {
throw new AiException(
'No AI models are available. Configure at least one AI provider.',
AiExceptionCode.API_KEY_NOT_CONFIGURED,
);
}
const resolvedModelId = modelId ?? workspace.smartModel;
this.aiModelRegistryService.validateModelAvailability(
resolvedModelId,
workspace,
);
await this.billingUsageService.hasAvailableCreditsOrThrow(workspace.id);
const thread = await this.threadRepository.findOne(workspace.id, {
where: { id: threadId, userWorkspaceId },
});
if (!isDefined(thread)) {
throw new AiException(
'Thread not found',
AiExceptionCode.THREAD_NOT_FOUND,
);
}
const streamId = generateId();
const { turnId } = await this.agentChatService.resolvePendingQuestion({
threadId,
messageId,
answers,
streamId,
workspaceId: workspace.id,
});
try {
await this.agentChatStreamingService.enqueueResumeStream({
threadId,
userWorkspaceId,
workspace,
turnId,
streamId,
modelId,
});
} catch (error) {
// Roll back the streaming claim so the thread isn't stuck "streaming".
await this.threadRepository
.update(
workspace.id,
{ id: threadId, activeStreamId: streamId },
{ activeStreamId: null },
)
.catch(() => {});
throw error;
}
return { messageId, queued: false, streamId };
}
@Mutation(() => Boolean)
async stopAgentChatStream(
@Args('threadId', { type: () => UUIDScalarType }) threadId: string,
@@ -242,6 +242,51 @@ export class AgentChatStreamingService {
return { streamId, messageId: lastUserMessage.id };
}
async enqueueResumeStream({
threadId,
userWorkspaceId,
workspace,
turnId,
streamId,
modelId,
}: {
threadId: string;
userWorkspaceId: string;
workspace: WorkspaceEntity;
turnId: string | null;
streamId: string;
modelId?: string;
}): Promise<void> {
const thread = await this.threadRepository.findOneOrFail(workspace.id, {
where: { id: threadId },
});
const messages = await this.loadMessagesFromDB(
threadId,
userWorkspaceId,
workspace.id,
);
await this.messageQueueService.add<StreamAgentChatJobData>(
STREAM_AGENT_CHAT_JOB_NAME,
{
threadId,
streamId,
userWorkspaceId,
workspaceId: workspace.id,
messages,
browsingContext: null,
modelId,
lastUserMessageText: '',
lastUserMessageParts: [],
hasTitle: !!thread.title,
conversationSizeTokens: thread.conversationSize,
existingTurnId: turnId ?? undefined,
isResume: true,
},
);
}
async flushNextQueuedMessage(
threadId: string,
userWorkspaceId: string,
@@ -250,13 +295,17 @@ export class AgentChatStreamingService {
): Promise<void> {
const threadStatus = await this.threadRepository.findOne(workspaceId, {
where: { id: threadId },
select: ['id', 'deletedAt'],
select: ['id', 'deletedAt', 'pendingQuestionMessageId'],
});
if (!threadStatus || threadStatus.deletedAt) {
return;
}
if (isDefined(threadStatus.pendingQuestionMessageId)) {
return;
}
const queuedMessages = await this.agentChatService.getQueuedMessages({
threadId,
workspaceId,
@@ -1,6 +1,12 @@
import { Injectable, Logger } from '@nestjs/common';
import { ExtendedUIMessage } from 'twenty-shared/ai';
import {
ASK_QUESTIONS_TOOL_NAME,
type AskQuestionAnswer,
type AskQuestionItem,
type AskQuestionsToolResult,
ExtendedUIMessage,
} from 'twenty-shared/ai';
import { isDefined } from 'twenty-shared/utils';
import { In, IsNull, Not } from 'typeorm';
import type { QueryDeepPartialEntity } from 'typeorm/query-builder/QueryPartialEntity';
@@ -291,15 +297,15 @@ export class AgentChatService {
});
}
async hasAssistantMessageForTurn({
turnId,
async hasMessageById({
id,
workspaceId,
}: {
turnId: string;
id: string;
workspaceId: string;
}): Promise<boolean> {
const existingMessage = await this.messageRepository.findOne(workspaceId, {
where: { turnId, role: AgentMessageRole.ASSISTANT },
where: { id },
select: ['id'],
});
@@ -481,6 +487,136 @@ export class AgentChatService {
return savedTurnId;
}
async resolvePendingQuestion({
threadId,
messageId,
answers,
streamId,
workspaceId,
}: {
threadId: string;
messageId: string;
answers: AskQuestionAnswer[];
streamId: string;
workspaceId: string;
}): Promise<{ turnId: string | null }> {
const message = await this.messageRepository.findOne(workspaceId, {
where: { id: messageId, threadId },
relations: ['parts'],
});
if (!message) {
throw new AiException(
'Question message not found',
AiExceptionCode.MESSAGE_NOT_FOUND,
);
}
const pendingPart = (message.parts ?? []).find(
(part) =>
part.toolName === ASK_QUESTIONS_TOOL_NAME &&
(part.toolOutput as { result?: AskQuestionsToolResult } | null)?.result
?.status === 'pending',
);
if (!pendingPart) {
throw new AiException(
'No pending question to answer',
AiExceptionCode.QUESTION_NOT_PENDING,
);
}
const previousOutput =
(pendingPart.toolOutput as Record<string, unknown> | null) ?? {};
const previousResult = previousOutput.result as
| AskQuestionsToolResult
| undefined;
const questions = previousResult?.questions ?? [];
this.validateQuestionAnswers(answers, questions);
const claim = await this.threadRepository.update(
workspaceId,
{ id: threadId, pendingQuestionMessageId: messageId },
{ pendingQuestionMessageId: null, activeStreamId: streamId },
);
if ((claim.affected ?? 0) === 0) {
throw new AiException(
'No pending question to answer',
AiExceptionCode.QUESTION_NOT_PENDING,
);
}
try {
await this.messagePartRepository.update(
workspaceId,
{ id: pendingPart.id },
{
toolOutput: {
...previousOutput,
success: true,
message: 'User answered the questions.',
result: {
questions,
status: 'answered',
answers,
},
},
},
);
} catch (error) {
await this.threadRepository
.update(
workspaceId,
{ id: threadId, activeStreamId: streamId },
{ pendingQuestionMessageId: messageId, activeStreamId: null },
)
.catch(() => {});
throw error;
}
return { turnId: message.turnId };
}
private validateQuestionAnswers(
answers: AskQuestionAnswer[],
questions: AskQuestionItem[],
): void {
for (const answer of answers) {
const question = questions[answer.questionIndex];
if (!isDefined(question)) {
throw new AiException(
'Answer references an unknown question.',
AiExceptionCode.INVALID_QUESTION_ANSWER,
);
}
const hasInvalidOption = answer.selectedOptionIndices.some(
(optionIndex) =>
optionIndex < 0 || optionIndex >= question.options.length,
);
if (hasInvalidOption) {
throw new AiException(
'Answer references an unknown option.',
AiExceptionCode.INVALID_QUESTION_ANSWER,
);
}
if (
question.allowMultiSelect !== true &&
answer.selectedOptionIndices.length > 1
) {
throw new AiException(
'This question allows only one selection.',
AiExceptionCode.INVALID_QUESTION_ANSWER,
);
}
}
}
async updateThreadTitle({
threadId,
userWorkspaceId,
@@ -3,6 +3,7 @@ import { Injectable, Logger } from '@nestjs/common';
import { isNonEmptyString, isObject } from '@sniptt/guards';
import {
convertToModelMessages,
hasToolCall,
type LanguageModelUsage,
stepCountIs,
type StepResult,
@@ -53,6 +54,10 @@ import {
extractCacheCreationTokensFromSteps,
} from 'src/engine/metadata-modules/ai/ai-billing/utils/extract-cache-creation-tokens.util';
import { AI_CHAT_TOOL_NAMES_TO_PRELOAD } from 'src/engine/metadata-modules/ai/ai-chat/constants/ai-chat-tool-names-to-preload.const';
import {
ASK_QUESTIONS_TOOL_NAME,
createAskQuestionsTool,
} from 'src/engine/metadata-modules/ai/ai-chat/tools/ask-questions.tool';
import { MessagePruningService } from 'src/engine/metadata-modules/ai/ai-chat/services/message-pruning.service';
import { SystemPromptBuilderService } from 'src/engine/metadata-modules/ai/ai-chat/services/system-prompt-builder.service';
import { type ExtractedFile } from 'src/engine/metadata-modules/ai/ai-chat/types/extracted-file.type';
@@ -195,12 +200,14 @@ export class ChatExecutionService {
const preloadedToolNames = [
...Object.keys(preloadedTools),
...Object.keys(nativeTools),
ASK_QUESTIONS_TOOL_NAME,
];
// ToolSet is constant for the entire conversation — no mutation.
// learn_tools returns schemas as text; execute_tool dispatches via the registry.
const activeTools: ToolSet = {
...directTools,
[ASK_QUESTIONS_TOOL_NAME]: createAskQuestionsTool(),
[LEARN_TOOLS_TOOL_NAME]: createLearnToolsTool(
this.toolRegistry,
toolContext,
@@ -420,7 +427,9 @@ export class ChatExecutionService {
tools: activeTools,
abortSignal,
stopWhen: (step) =>
stepCountIs(AGENT_CONFIG.MAX_STEPS)(step) || hasNoMoreAvailableCredits,
stepCountIs(AGENT_CONFIG.MAX_STEPS)(step) ||
hasToolCall(ASK_QUESTIONS_TOOL_NAME)(step) ||
hasNoMoreAvailableCredits,
experimental_telemetry: AI_TELEMETRY_CONFIG,
providerOptions: getCallLevelProviderOptions({
sdkPackage: registeredModel.sdkPackage,
@@ -0,0 +1,119 @@
import {
ASK_QUESTIONS_TOOL_NAME,
askQuestionsInputSchema,
createAskQuestionsTool,
} from 'src/engine/metadata-modules/ai/ai-chat/tools/ask-questions.tool';
describe('ask_questions tool', () => {
it('is named ask_questions (plural)', () => {
expect(ASK_QUESTIONS_TOOL_NAME).toBe('ask_questions');
});
it('execute echoes the questions with a pending status', async () => {
const tool = createAskQuestionsTool();
const questions = [
{
header: 'Email type',
question: 'What type of email?',
options: [{ label: 'Welcome' }, { label: 'Offer' }],
},
];
const output = await tool.execute({ questions });
expect(output).toEqual({
success: true,
message: expect.any(String),
result: { questions, status: 'pending' },
});
});
it('rejects fewer than two options', () => {
const result = askQuestionsInputSchema.safeParse({
questions: [
{ header: 'h', question: 'q', options: [{ label: 'only one' }] },
],
});
expect(result.success).toBe(false);
});
it('rejects zero questions', () => {
const result = askQuestionsInputSchema.safeParse({ questions: [] });
expect(result.success).toBe(false);
});
it('rejects more than four questions', () => {
const question = {
header: 'h',
question: 'q',
options: [{ label: 'a' }, { label: 'b' }],
};
const result = askQuestionsInputSchema.safeParse({
questions: [question, question, question, question, question],
});
expect(result.success).toBe(false);
});
it('rejects more than four options', () => {
const result = askQuestionsInputSchema.safeParse({
questions: [
{
header: 'h',
question: 'q',
options: [
{ label: 'a' },
{ label: 'b' },
{ label: 'c' },
{ label: 'd' },
{ label: 'e' },
],
},
],
});
expect(result.success).toBe(false);
});
it('rejects more than one recommended option', () => {
const result = askQuestionsInputSchema.safeParse({
questions: [
{
header: 'h',
question: 'q',
options: [
{ label: 'a', isRecommended: true },
{ label: 'b', isRecommended: true },
],
},
],
});
expect(result.success).toBe(false);
});
it('accepts a valid multi-question payload', () => {
const result = askQuestionsInputSchema.safeParse({
questions: [
{
header: 'h1',
question: 'q1',
options: [{ label: 'a' }, { label: 'b' }],
},
{
header: 'h2',
question: 'q2',
options: [
{ label: 'c', description: 'desc', isRecommended: true },
{ label: 'd' },
],
allowMultiSelect: true,
},
],
});
expect(result.success).toBe(true);
});
});
@@ -0,0 +1,85 @@
import { z } from 'zod';
import {
ASK_QUESTIONS_TOOL_NAME,
type AskQuestionsToolInput,
type AskQuestionsToolResult,
} from 'twenty-shared/ai';
export { ASK_QUESTIONS_TOOL_NAME };
export const askQuestionsInputSchema = z.object({
questions: z
.array(
z.object({
header: z
.string()
.describe(
'Very short label/tag for the question (≤ ~32 chars), e.g. "Email type".',
),
question: z
.string()
.describe(
'The full question to ask the user. Be clear and specific.',
),
options: z
.array(
z.object({
label: z
.string()
.describe('Concise option the user can pick (1-5 words).'),
description: z
.string()
.optional()
.describe(
'Longer explanation shown when the user opens the option info icon.',
),
isRecommended: z
.boolean()
.optional()
.describe('Mark the single suggested option, if any.'),
}),
)
.min(2)
.max(4)
.refine(
(options) =>
options.filter((option) => option.isRecommended === true)
.length <= 1,
{ message: 'At most one option can be marked as recommended.' },
)
.describe('2-4 mutually exclusive options.'),
allowMultiSelect: z
.boolean()
.optional()
.describe('Allow the user to select more than one option.'),
}),
)
.min(1)
.max(4)
.describe('One to four questions to ask the user.'),
});
type AskQuestionsPendingOutput = {
success: true;
message: string;
result: AskQuestionsToolResult;
};
export const createAskQuestionsTool = () => ({
description:
'Ask the user one or more multiple-choice questions when you need a decision you cannot ' +
'infer from the request or context and that has no obvious default. The conversation ' +
'pauses until the user answers, then continues with their choice in mind. Prefer this ' +
'over guessing on consequential or ambiguous decisions. Do NOT use it for information you ' +
'could look up with another tool, or for trivial choices with an obvious default. The ' +
'user can always type a free-form answer instead of picking an option.',
inputSchema: askQuestionsInputSchema,
execute: async (
input: AskQuestionsToolInput,
): Promise<AskQuestionsPendingOutput> => ({
success: true,
message: 'Questions presented to the user; awaiting their answer.',
result: { questions: input.questions, status: 'pending' },
}),
});
@@ -0,0 +1,58 @@
import { type ExtendedUIMessagePart } from 'twenty-shared/ai';
import { findPendingQuestionPart } from 'src/engine/metadata-modules/ai/ai-chat/utils/find-pending-question-part.util';
const askQuestionsPart = (
status: 'pending' | 'answered',
): ExtendedUIMessagePart =>
({
type: 'tool-ask_questions',
toolCallId: 'call-1',
state: 'output-available',
input: { questions: [] },
output: {
success: true,
message: 'x',
result: {
questions: [{ header: 'h', question: 'q', options: [] }],
status,
},
},
}) as unknown as ExtendedUIMessagePart;
const textPart = (text: string): ExtendedUIMessagePart =>
({ type: 'text', text }) as ExtendedUIMessagePart;
describe('findPendingQuestionPart', () => {
it('returns the ask_questions part when status is pending', () => {
const part = findPendingQuestionPart([
textPart('hello'),
askQuestionsPart('pending'),
]);
expect(part).toBeDefined();
expect(part?.toolCallId).toBe('call-1');
});
it('returns undefined when the question has been answered', () => {
expect(
findPendingQuestionPart([askQuestionsPart('answered')]),
).toBeUndefined();
});
it('returns undefined when there is no ask_questions part', () => {
expect(findPendingQuestionPart([textPart('hello')])).toBeUndefined();
});
it('ignores other tool parts', () => {
const otherTool = {
type: 'tool-search_help_center',
toolCallId: 'call-2',
state: 'output-available',
input: {},
output: { success: true },
} as unknown as ExtendedUIMessagePart;
expect(findPendingQuestionPart([otherTool])).toBeUndefined();
});
});
@@ -0,0 +1,29 @@
import { getToolName, isToolUIPart } from 'ai';
import {
ASK_QUESTIONS_TOOL_NAME,
type AskQuestionsToolResult,
type ExtendedUIMessagePart,
} from 'twenty-shared/ai';
type ToolPartWithOutput = ExtendedUIMessagePart & {
toolCallId: string;
output?: { result?: AskQuestionsToolResult };
};
export const findPendingQuestionPart = (
parts: ExtendedUIMessagePart[],
): ToolPartWithOutput | undefined => {
for (const part of parts) {
if (!isToolUIPart(part) || getToolName(part) !== ASK_QUESTIONS_TOOL_NAME) {
continue;
}
const output = (part as ToolPartWithOutput).output;
if (output?.result?.status === 'pending') {
return part as ToolPartWithOutput;
}
}
return undefined;
};
@@ -13,6 +13,8 @@ export enum AiExceptionCode {
THREAD_NOT_FOUND = 'THREAD_NOT_FOUND',
INVALID_CHAT_THREAD_TITLE = 'INVALID_CHAT_THREAD_TITLE',
MESSAGE_NOT_FOUND = 'MESSAGE_NOT_FOUND',
QUESTION_NOT_PENDING = 'QUESTION_NOT_PENDING',
INVALID_QUESTION_ANSWER = 'INVALID_QUESTION_ANSWER',
API_KEY_NOT_CONFIGURED = 'API_KEY_NOT_CONFIGURED',
USER_WORKSPACE_ID_NOT_FOUND = 'USER_WORKSPACE_ID_NOT_FOUND',
ROLE_NOT_FOUND = 'ROLE_NOT_FOUND',
@@ -38,6 +40,10 @@ const getAiExceptionUserFriendlyMessage = (code: AiExceptionCode) => {
return msg`Chat thread title cannot be empty.`;
case AiExceptionCode.MESSAGE_NOT_FOUND:
return msg`Chat message not found.`;
case AiExceptionCode.QUESTION_NOT_PENDING:
return msg`This question has already been answered.`;
case AiExceptionCode.INVALID_QUESTION_ANSWER:
return msg`Invalid answer for this question.`;
case AiExceptionCode.API_KEY_NOT_CONFIGURED:
return msg`API key is not configured.`;
case AiExceptionCode.USER_WORKSPACE_ID_NOT_FOUND:
@@ -28,6 +28,8 @@ export const aiGraphqlApiExceptionHandler = (error: Error) => {
throw new NotFoundError(error);
case AiExceptionCode.INVALID_AGENT_INPUT:
case AiExceptionCode.INVALID_CHAT_THREAD_TITLE:
case AiExceptionCode.QUESTION_NOT_PENDING:
case AiExceptionCode.INVALID_QUESTION_ANSWER:
throw new UserInputError(error);
case AiExceptionCode.AGENT_ALREADY_EXISTS:
case AiExceptionCode.NO_FAILED_TURN_TO_RETRY:
@@ -0,0 +1 @@
export const ASK_QUESTIONS_TOOL_NAME = 'ask_questions';
+7
View File
@@ -10,6 +10,7 @@
export { AI_SDK_PACKAGE_LABELS } from './constants/ai-sdk-package-labels.const';
export type { AiSdkPackage } from './constants/ai-sdk-packages.const';
export { AI_SDK_PACKAGES } from './constants/ai-sdk-packages.const';
export { ASK_QUESTIONS_TOOL_NAME } from './constants/ask-questions-tool-name.const';
export type { DataResidency } from './constants/data-residency.const';
export { DATA_RESIDENCY_KEYS } from './constants/data-residency.const';
export type { DatabaseCrudOperation } from './constants/database-crud-operation.const';
@@ -28,6 +29,12 @@ export type {
AgentResponseSchema,
} from './types/agent-response-schema.type';
export type { AgentChatSubscriptionEvent } from './types/AgentChatSubscriptionEvent';
export type { AskQuestionAnswer } from './types/AskQuestionAnswer';
export type { AskQuestionItem } from './types/AskQuestionItem';
export type { AskQuestionOption } from './types/AskQuestionOption';
export type { AskQuestionsToolInput } from './types/AskQuestionsToolInput';
export type { AskQuestionsToolResult } from './types/AskQuestionsToolResult';
export type { AskQuestionsToolStatus } from './types/AskQuestionsToolStatus';
export type {
CodeExecutionFile,
ExtendedFileUIPart,
@@ -0,0 +1,5 @@
export type AskQuestionAnswer = {
questionIndex: number;
selectedOptionIndices: number[];
freeText?: string;
};
@@ -0,0 +1,8 @@
import { type AskQuestionOption } from '@/ai/types/AskQuestionOption';
export type AskQuestionItem = {
header: string;
question: string;
options: AskQuestionOption[];
allowMultiSelect?: boolean;
};
@@ -0,0 +1,5 @@
export type AskQuestionOption = {
label: string;
description?: string;
isRecommended?: boolean;
};
@@ -0,0 +1,5 @@
import { type AskQuestionItem } from '@/ai/types/AskQuestionItem';
export type AskQuestionsToolInput = {
questions: AskQuestionItem[];
};
@@ -0,0 +1,9 @@
import { type AskQuestionAnswer } from '@/ai/types/AskQuestionAnswer';
import { type AskQuestionItem } from '@/ai/types/AskQuestionItem';
import { type AskQuestionsToolStatus } from '@/ai/types/AskQuestionsToolStatus';
export type AskQuestionsToolResult = {
questions: AskQuestionItem[];
status: AskQuestionsToolStatus;
answers?: AskQuestionAnswer[];
};
@@ -0,0 +1 @@
export type AskQuestionsToolStatus = 'pending' | 'answered' | 'skipped';