Provide additional logsobservability to workflow runs (per node) (#21142)

Surfaces per-step "Logs" tabs in the workflow run side panel so users
can see what each step actually did (model + tokens + tool calls for AI,
console output for serverless functions, request/response for HTTP,
recipients/body for Email).

<img width="546" height="501" alt="ai_agent_without_websearch"
src="https://github.com/user-attachments/assets/c6ca3518-9489-4484-a570-3d0569ff3b03"
/>

## Storage

- New `stepLogs` JSONB column on the `workflowRun` workspace entity,
typed as `Record<string, WorkflowRunStepLog>` (keyed by step id).
- Schema lives in `twenty-shared`: `workflowRunStepLogSchema` with a
discriminated `details.type` union for `AI_AGENT | CODE | HTTP_REQUEST |
EMAIL` — frontends and backends consume the same Zod-inferred type.
- Field is added to existing workspaces via a workspace upgrade command
(`2-9 add-workflow-run-step-logs-field`); the standard-object metadata
declares it for new workspaces.
- Writes happen atomically per step in
`WorkflowRunStepLogWorkspaceService.setStepLog` using `jsonb_set`. That
lets concurrent steps in the same run write their own keys without
contending with the existing lock around `workflowRun.state`.
- Per-step payload is hard-capped at 256 KB; anything larger is dropped
with a `logger.warn`, so a pathological tool call can never bloat a row.
See below for more information.

## How logs are produced

**Aalmost everything was already being collected; this PR mostly
persists and renders it.**

- **AI agent** — `AgentAsyncExecutorService` already tracked token
usage, model id, native web-search count, and the AI SDK's `steps[]`. We
map those into the log via `mapAiStepsToToolCallLogs` (`searchVector`
stripped from record outputs, per-call input/output capped at 32/64 KB,
max 200 tool calls per step). The only new measurement is a wall-clock
`durationMs` taken around `executeAgent`, and we now fold native
web-search cost into the displayed `totalCostInDollars` (it was already
billed, just not shown).
- **Code / serverless function** — reuses the `console.log` output the
function runner already returns (`logsByLevel`);
`build-code-step-log.util` only repackages it.
- **HTTP request** — built from the action's existing input/output via
`build-http-request-step-log.util`. No new signals collected.
- **Email (send / draft)** — added `sanitizedHtmlBody` + `plainTextBody`
to the existing tool outputs (a small additive change), then
`build-email-step-log.util` consumes them.

No additional AI inference or external calls are made for logging — the
cost is a small CPU overhead per step plus the JSONB write.

## Security

The log surface intentionally shows whatever the workflow touched, which
made redaction and sanitization the main design concern.

- **HTTP — secrets in headers**: existing `SENSITIVE_HEADER_NAMES` set
(Authorization, Cookie, …) replaced with `[redacted]` in both request
and response.
- **HTTP — secrets in URLs**: `SENSITIVE_URL_PARAM_NAMES` (e.g.
`api_key`, `token`, `access_token`) replaced in the query string via
`URL`-based parsing.
- **HTTP — secrets in bodies**: `SENSITIVE_BODY_KEY_REGEX` deep-walks
JSON request/response bodies (object input or stringified JSON) and
redacts matching keys. Applied to the `error` field too, since
transport-layer errors sometimes embed structured payloads.
- **Email — XSS risk in body preview**: tool outputs now expose a
server-side `sanitizedHtmlBody`; the log builder prefers it over the raw
user-authored `input.body`, with `plainTextBody` as a second fallback.
The original raw body is only used if sanitization didn't happen (e.g.
tool failed before composing).
- **AI — internal/noisy data**: `searchVector` (Postgres tsvector
strings) is stripped from record outputs returned by Twenty tools to
avoid leaking internal full-text-search payloads.
- **DB bloat / runaway agents**: 256 KB per-step cap + 32 KB / 64 KB
per-tool-call input/output cap + 200 tool calls per step.

<img width="547" height="307" alt="logic_function"
src="https://github.com/user-attachments/assets/dd4a3d16-67f2-434b-95b3-bdcaf9ed053d"
/>

## More details on Log size & truncation

Logs are stored in `workflowRun.stepLogs` (JSONB), keyed by `stepId`.

### Per-step cap

Each step's log is hard-capped at **256 KB** (`MAX_STEP_LOG_BYTES` in
`WorkflowRunStepLogWorkspaceService.setStepLog`).

For ~99% of workflows this is roomy — typical real-world sizes:
- Code / serverless function: 1–20 KB
- HTTP request: 5–70 KB
- Email: 5–30 KB
- AI agent (a handful of tool calls): 5–50 KB

### Two layers of bounding

1. **Per-field truncation** in each builder (before writing):
   - **Code**: ≤ 500 entries, ≤ 4 KB per message, ≤ 8 KB stack trace
   - **HTTP**: ≤ 32 KB per body (request + response), UTF-8 byte-aware
   - **Email**: ≤ 8 KB body preview, UTF-8 byte-aware
- **AI agent**: ≤ 32 KB tool input, ≤ 64 KB tool output, ≤ 200 tool
calls/step

2. **Global per-step safety net** at write time: if the assembled
`stepLog` still exceeds 256 KB, the write is **dropped entirely** with a
`logger.warn`. The workflow itself keeps running unaffected.

### What this means in practice

- **Safe**: workflow execution, step results, downstream steps — never
blocked by log size.
- **Safe**: iterators (each iteration overwrites the previous log for
that `stepId`, so they can't accumulate).
- **Safe**: step retries (same `stepId` is overwritten, not appended).
- **Possible**: an AI agent step with many large tool outputs (e.g., 50+
heavy `web_search` calls) can exceed 256 KB → the **entire** step's log
is dropped, side panel shows "No logs were recorded for this step". The
user has no explicit signal that the log was dropped due to size (only
server-side warn).
- **Possible** (theoretical): a workflow with hundreds of distinct steps
could push the row toward Postgres's internal ~256 MB jsonb limit.
Beyond that, individual `jsonb_set` writes would error and be swallowed
by the action's try/catch — workflow still completes.

### Possible future hardening (not in this PR)

- Replace "drop entire log" with a stub that preserves the summary card
(cost, duration, status) and marks `truncated.reason = 'size_cap'`.
- Surface size-drops in the UI (similar to the existing
`<StyledTruncatedNotice>`).
- Emit a metric so dropped logs are observable in dashboards.
This commit is contained in:
Marie
2026-06-03 18:53:47 +02:00
committed by GitHub
parent 0671ff3de5
commit 4b15b949f3
74 changed files with 4834 additions and 445 deletions
@@ -1,7 +1,10 @@
import { registerEnumType } from '@nestjs/graphql';
import { type ActorMetadata, FieldMetadataType } from 'twenty-shared/types';
import { type WorkflowRunStepInfos } from 'twenty-shared/workflow';
import {
type WorkflowRunStepInfos,
type WorkflowRunStepLogs,
} from 'twenty-shared/workflow';
import { BaseWorkspaceEntity } from 'src/engine/twenty-orm/base.workspace-entity';
import { type FieldTypeAndNameMetadata } from 'src/engine/workspace-manager/utils/get-ts-vector-column-expression.util';
@@ -66,6 +69,7 @@ export class WorkflowRunWorkspaceEntity extends BaseWorkspaceEntity {
createdBy: ActorMetadata;
updatedBy: ActorMetadata;
state: WorkflowRunState;
stepLogs: WorkflowRunStepLogs | null;
position: number;
searchVector: string;
workflowVersion: EntityRelation<WorkflowVersionWorkspaceEntity>;
@@ -54,7 +54,10 @@ export class WorkflowVersionStepHelpersWorkspaceService {
{ shouldBypassPermissionChecks: true },
);
const updateData: Partial<WorkflowVersionWorkspaceEntity> = {};
const updateData: Pick<
Partial<WorkflowVersionWorkspaceEntity>,
'steps' | 'trigger'
> = {};
if (steps !== undefined) {
updateData.steps = steps;
@@ -12,15 +12,17 @@ import { DelayWorkflowAction } from 'src/modules/workflow/workflow-executor/work
import { EmptyWorkflowAction } from 'src/modules/workflow/workflow-executor/workflow-actions/empty/empty.workflow-action';
import { FilterWorkflowAction } from 'src/modules/workflow/workflow-executor/workflow-actions/filter/filter.workflow-action';
import { FormWorkflowAction } from 'src/modules/workflow/workflow-executor/workflow-actions/form/form.workflow-action';
import { HttpRequestWorkflowAction } from 'src/modules/workflow/workflow-executor/workflow-actions/http-request/http-request.workflow-action';
import { IfElseWorkflowAction } from 'src/modules/workflow/workflow-executor/workflow-actions/if-else/if-else.workflow-action';
import { IteratorWorkflowAction } from 'src/modules/workflow/workflow-executor/workflow-actions/iterator/iterator.workflow-action';
import { LogicFunctionWorkflowAction } from 'src/modules/workflow/workflow-executor/workflow-actions/logic-function/logic-function.workflow-action';
import { DraftEmailWorkflowAction } from 'src/modules/workflow/workflow-executor/workflow-actions/mail-sender/draft-email.workflow-action';
import { SendEmailWorkflowAction } from 'src/modules/workflow/workflow-executor/workflow-actions/mail-sender/send-email.workflow-action';
import { CreateRecordWorkflowAction } from 'src/modules/workflow/workflow-executor/workflow-actions/record-crud/create-record.workflow-action';
import { DeleteRecordWorkflowAction } from 'src/modules/workflow/workflow-executor/workflow-actions/record-crud/delete-record.workflow-action';
import { FindRecordsWorkflowAction } from 'src/modules/workflow/workflow-executor/workflow-actions/record-crud/find-records.workflow-action';
import { UpdateRecordWorkflowAction } from 'src/modules/workflow/workflow-executor/workflow-actions/record-crud/update-record.workflow-action';
import { UpsertRecordWorkflowAction } from 'src/modules/workflow/workflow-executor/workflow-actions/record-crud/upsert-record.workflow-action';
import { ToolExecutorWorkflowAction } from 'src/modules/workflow/workflow-executor/workflow-actions/tool-executor-workflow-action';
import { WorkflowActionType } from 'src/modules/workflow/workflow-executor/workflow-actions/types/workflow-action.type';
@Injectable()
@@ -37,7 +39,9 @@ export class WorkflowActionFactory {
private readonly filterWorkflowAction: FilterWorkflowAction,
private readonly ifElseWorkflowAction: IfElseWorkflowAction,
private readonly iteratorWorkflowAction: IteratorWorkflowAction,
private readonly toolExecutorWorkflowAction: ToolExecutorWorkflowAction,
private readonly httpRequestWorkflowAction: HttpRequestWorkflowAction,
private readonly sendEmailWorkflowAction: SendEmailWorkflowAction,
private readonly draftEmailWorkflowAction: DraftEmailWorkflowAction,
private readonly aiAgentWorkflowAction: AiAgentWorkflowAction,
private readonly emptyWorkflowAction: EmptyWorkflowAction,
private readonly delayWorkflowAction: DelayWorkflowAction,
@@ -50,9 +54,9 @@ export class WorkflowActionFactory {
case WorkflowActionType.LOGIC_FUNCTION:
return this.logicFunctionWorkflowAction;
case WorkflowActionType.SEND_EMAIL:
return this.toolExecutorWorkflowAction;
return this.sendEmailWorkflowAction;
case WorkflowActionType.DRAFT_EMAIL:
return this.toolExecutorWorkflowAction;
return this.draftEmailWorkflowAction;
case WorkflowActionType.CREATE_RECORD:
return this.createRecordWorkflowAction;
case WorkflowActionType.UPSERT_RECORD:
@@ -72,7 +76,7 @@ export class WorkflowActionFactory {
case WorkflowActionType.ITERATOR:
return this.iteratorWorkflowAction;
case WorkflowActionType.HTTP_REQUEST:
return this.toolExecutorWorkflowAction;
return this.httpRequestWorkflowAction;
case WorkflowActionType.AI_AGENT:
return this.aiAgentWorkflowAction;
case WorkflowActionType.EMPTY:
@@ -1,4 +1,4 @@
import { Injectable } from '@nestjs/common';
import { Injectable, Logger } from '@nestjs/common';
import { resolveInput } from 'twenty-shared/utils';
@@ -6,6 +6,7 @@ import { type WorkflowAction } from 'src/modules/workflow/workflow-executor/inte
import { UsageOperationType } from 'src/engine/core-modules/usage/enums/usage-operation-type.enum';
import { AgentAsyncExecutorService } from 'src/engine/metadata-modules/ai/ai-agent-execution/services/agent-async-executor.service';
import { type AgentExecutionResult } from 'src/engine/metadata-modules/ai/ai-agent-execution/types/agent-execution-result.type';
import { AgentEntity } from 'src/engine/metadata-modules/ai/ai-agent/entities/agent.entity';
import { InjectWorkspaceScopedRepository } from 'src/engine/twenty-orm/workspace-scoped-repository/inject-workspace-scoped-repository.decorator';
import { WorkspaceScopedRepository } from 'src/engine/twenty-orm/workspace-scoped-repository/workspace-scoped-repository';
@@ -17,14 +18,19 @@ import { WorkflowExecutionContextService } from 'src/modules/workflow/workflow-e
import { type WorkflowActionInput } from 'src/modules/workflow/workflow-executor/types/workflow-action-input';
import { type WorkflowActionOutput } from 'src/modules/workflow/workflow-executor/types/workflow-action-output.type';
import { findStepOrThrow } from 'src/modules/workflow/workflow-executor/utils/find-step-or-throw.util';
import { buildAiAgentStepLog } from 'src/modules/workflow/workflow-executor/workflow-actions/ai-agent/utils/build-ai-agent-step-log.util';
import { WorkflowRunStepLogWorkspaceService } from 'src/modules/workflow/workflow-runner/workflow-run/workflow-run-step-log.workspace-service';
import { isWorkflowAiAgentAction } from './guards/is-workflow-ai-agent-action.guard';
@Injectable()
export class AiAgentWorkflowAction implements WorkflowAction {
private readonly logger = new Logger(AiAgentWorkflowAction.name);
constructor(
private readonly aiAgentExecutionService: AgentAsyncExecutorService,
private readonly workflowExecutionContextService: WorkflowExecutionContextService,
private readonly workflowRunStepLogService: WorkflowRunStepLogWorkspaceService,
@InjectWorkspaceScopedRepository(AgentEntity)
private readonly agentRepository: WorkspaceScopedRepository<AgentEntity>,
) {}
@@ -73,27 +79,73 @@ export class AiAgentWorkflowAction implements WorkflowAction {
? executionContext.authContext.userWorkspaceId
: null;
const { result, hasNoMoreAvailableCredits } =
await this.aiAgentExecutionService.executeAgent({
agent,
userPrompt: resolveInput(prompt, context) as string,
actorContext: executionContext.isActingOnBehalfOfUser
? executionContext.initiator
: undefined,
authContext: executionContext.authContext,
workspaceId,
userWorkspaceId,
operationType: UsageOperationType.AI_WORKFLOW_TOKEN,
});
const startedAtMs = Date.now();
if (hasNoMoreAvailableCredits) {
const executionResult = await this.aiAgentExecutionService.executeAgent({
agent,
userPrompt: resolveInput(prompt, context) as string,
actorContext: executionContext.isActingOnBehalfOfUser
? executionContext.initiator
: undefined,
authContext: executionContext.authContext,
workspaceId,
userWorkspaceId,
operationType: UsageOperationType.AI_WORKFLOW_TOKEN,
});
const durationMs = Date.now() - startedAtMs;
await this.persistStepLog({
workflowRunId: runInfo.workflowRunId,
workspaceId,
stepId: currentStepId,
executionResult,
durationMs,
});
if (executionResult.hasNoMoreAvailableCredits) {
return {
error: 'AI agent stopped: no more available credits.',
};
}
return {
result,
result: executionResult.result,
};
}
private async persistStepLog({
workflowRunId,
workspaceId,
stepId,
executionResult,
durationMs,
}: {
workflowRunId: string;
workspaceId: string;
stepId: string;
executionResult: AgentExecutionResult;
durationMs: number;
}): Promise<void> {
const stepLog = buildAiAgentStepLog({ executionResult, durationMs });
if (!stepLog) {
return;
}
try {
await this.workflowRunStepLogService.setStepLog({
workflowRunId,
workspaceId,
stepId,
stepLog,
});
} catch (error) {
this.logger.warn(
`Failed to persist step log for workflowRun=${workflowRunId} step=${stepId}: ${
error instanceof Error ? error.message : String(error)
}`,
);
}
}
}
@@ -0,0 +1,98 @@
import { type AgentExecutionResult } from 'src/engine/metadata-modules/ai/ai-agent-execution/types/agent-execution-result.type';
import { buildAiAgentStepLog } from 'src/modules/workflow/workflow-executor/workflow-actions/ai-agent/utils/build-ai-agent-step-log.util';
jest.mock(
'src/engine/metadata-modules/ai/ai-agent-execution/utils/map-ai-steps-to-tool-call-logs.util',
() => ({
mapAiStepsToToolCallLogs: jest.fn().mockReturnValue([
{
toolName: 'web_search',
toolCallId: 'call-1',
state: 'success',
},
]),
}),
);
const baseExecutionResult: AgentExecutionResult = {
result: { answer: 'hello' },
usage: {
inputTokens: 100,
outputTokens: 50,
totalTokens: 150,
inputTokenDetails: { cacheReadTokens: 20 },
outputTokenDetails: { reasoningTokens: 10 },
} as AgentExecutionResult['usage'],
cacheCreationTokens: 5,
nativeWebSearchCallCount: 2,
hasNoMoreAvailableCredits: false,
modelId: 'claude-sonnet-4',
totalCostInDollars: 0.012,
creditsUsedMicro: 12_000,
steps: [] as AgentExecutionResult['steps'],
};
describe('buildAiAgentStepLog', () => {
it('returns null when the execution never resolved a model', () => {
const stepLog = buildAiAgentStepLog({
executionResult: { ...baseExecutionResult, modelId: undefined },
durationMs: 1234,
});
expect(stepLog).toBeNull();
});
it('builds an AI_AGENT step log with usage, cost, and tool calls', () => {
const stepLog = buildAiAgentStepLog({
executionResult: baseExecutionResult,
durationMs: 1234,
});
if (stepLog === null || stepLog.details.type !== 'AI_AGENT') {
throw new Error('Expected AI_AGENT details');
}
expect(stepLog.details.modelId).toBe('claude-sonnet-4');
expect(stepLog.details.durationMs).toBe(1234);
expect(stepLog.details.usage).toEqual({
inputTokens: 100,
outputTokens: 50,
reasoningTokens: 10,
cacheReadTokens: 20,
cacheCreationTokens: 5,
totalTokens: 155,
});
expect(stepLog.details.cost).toEqual({
totalCostInDollars: 0.012,
creditsUsedMicro: 12_000,
});
expect(stepLog.details.nativeWebSearchCallCount).toBe(2);
expect(stepLog.details.toolCalls).toHaveLength(1);
expect(stepLog.entries).toEqual([]);
});
it('falls back to zero usage / cost when the agent did not report them', () => {
const stepLog = buildAiAgentStepLog({
executionResult: {
...baseExecutionResult,
usage: {} as AgentExecutionResult['usage'],
cacheCreationTokens: 0,
totalCostInDollars: undefined,
creditsUsedMicro: undefined,
steps: undefined,
},
durationMs: 100,
});
if (stepLog === null || stepLog.details.type !== 'AI_AGENT') {
throw new Error('Expected AI_AGENT details');
}
expect(stepLog.details.usage.inputTokens).toBe(0);
expect(stepLog.details.usage.outputTokens).toBe(0);
expect(stepLog.details.usage.totalTokens).toBe(0);
expect(stepLog.details.cost.totalCostInDollars).toBe(0);
expect(stepLog.details.cost.creditsUsedMicro).toBe(0);
expect(stepLog.details.toolCalls).toEqual([]);
});
});
@@ -0,0 +1,52 @@
import {
type AiAgentStepLogDetails,
type WorkflowRunStepLog,
} from 'twenty-shared/workflow';
import { type AgentExecutionResult } from 'src/engine/metadata-modules/ai/ai-agent-execution/types/agent-execution-result.type';
import { mapAiStepsToToolCallLogs } from 'src/engine/metadata-modules/ai/ai-agent-execution/utils/map-ai-steps-to-tool-call-logs.util';
export const buildAiAgentStepLog = ({
executionResult,
durationMs,
}: {
executionResult: AgentExecutionResult;
durationMs: number;
}): WorkflowRunStepLog | null => {
if (!executionResult.modelId) {
return null;
}
const toolCalls = executionResult.steps
? mapAiStepsToToolCallLogs(executionResult.steps)
: [];
const details: AiAgentStepLogDetails = {
type: 'AI_AGENT',
modelId: executionResult.modelId,
usage: {
inputTokens: executionResult.usage.inputTokens ?? 0,
outputTokens: executionResult.usage.outputTokens ?? 0,
reasoningTokens:
executionResult.usage.outputTokenDetails?.reasoningTokens,
cacheReadTokens: executionResult.usage.inputTokenDetails?.cacheReadTokens,
cacheCreationTokens: executionResult.cacheCreationTokens,
totalTokens:
(executionResult.usage.totalTokens ?? 0) +
executionResult.cacheCreationTokens,
},
cost: {
totalCostInDollars: executionResult.totalCostInDollars ?? 0,
creditsUsedMicro: executionResult.creditsUsedMicro ?? 0,
},
nativeWebSearchCallCount: executionResult.nativeWebSearchCallCount,
toolCalls,
durationMs,
};
return {
details,
entries: [],
sizeBytes: 0,
};
};
@@ -2,9 +2,10 @@ import { Module } from '@nestjs/common';
import { LogicFunctionModule } from 'src/engine/core-modules/logic-function/logic-function.module';
import { CodeWorkflowAction } from 'src/modules/workflow/workflow-executor/workflow-actions/code/code.workflow-action';
import { WorkflowRunModule } from 'src/modules/workflow/workflow-runner/workflow-run/workflow-run.module';
@Module({
imports: [LogicFunctionModule],
imports: [LogicFunctionModule, WorkflowRunModule],
providers: [CodeWorkflowAction],
exports: [CodeWorkflowAction],
})
@@ -1,9 +1,10 @@
import { Injectable } from '@nestjs/common';
import { Injectable, Logger } from '@nestjs/common';
import { resolveInput } from 'twenty-shared/utils';
import { type WorkflowAction } from 'src/modules/workflow/workflow-executor/interfaces/workflow-action.interface';
import { type LogicFunctionExecuteResult } from 'src/engine/core-modules/logic-function/logic-function-drivers/interfaces/logic-function-driver.interface';
import { LogicFunctionExecutorService } from 'src/engine/core-modules/logic-function/logic-function-executor/logic-function-executor.service';
import {
WorkflowStepExecutorException,
@@ -14,11 +15,16 @@ import { type WorkflowActionOutput } from 'src/modules/workflow/workflow-executo
import { findStepOrThrow } from 'src/modules/workflow/workflow-executor/utils/find-step-or-throw.util';
import { isWorkflowCodeAction } from 'src/modules/workflow/workflow-executor/workflow-actions/code/guards/is-workflow-code-action.guard';
import { type WorkflowCodeActionInput } from 'src/modules/workflow/workflow-executor/workflow-actions/code/types/workflow-code-action-input.type';
import { buildCodeStepLog } from 'src/modules/workflow/workflow-executor/workflow-actions/code/utils/build-code-step-log.util';
import { WorkflowRunStepLogWorkspaceService } from 'src/modules/workflow/workflow-runner/workflow-run/workflow-run-step-log.workspace-service';
@Injectable()
export class CodeWorkflowAction implements WorkflowAction {
private readonly logger = new Logger(CodeWorkflowAction.name);
constructor(
private readonly logicFunctionExecutorService: LogicFunctionExecutorService,
private readonly workflowRunStepLogService: WorkflowRunStepLogWorkspaceService,
) {}
async execute({
@@ -52,10 +58,44 @@ export class CodeWorkflowAction implements WorkflowAction {
payload: workflowActionInput.logicFunctionInput,
});
await this.persistStepLog({
workflowRunId: runInfo.workflowRunId,
workspaceId,
stepId: currentStepId,
result,
});
if (result.error) {
return { error: result.error.errorMessage };
}
return { result: result.data || {} };
}
private async persistStepLog({
workflowRunId,
workspaceId,
stepId,
result,
}: {
workflowRunId: string;
workspaceId: string;
stepId: string;
result: LogicFunctionExecuteResult;
}): Promise<void> {
try {
await this.workflowRunStepLogService.setStepLog({
workflowRunId,
workspaceId,
stepId,
stepLog: buildCodeStepLog(result),
});
} catch (error) {
this.logger.warn(
`Failed to persist step log for workflowRun=${workflowRunId} step=${stepId}: ${
error instanceof Error ? error.message : String(error)
}`,
);
}
}
}
@@ -0,0 +1,72 @@
import { type WorkflowRunStepLog } from 'twenty-shared/workflow';
import { parseApplicationLogLines } from 'src/engine/core-modules/application-logs/utils/parse-application-log-lines';
import {
type LogicFunctionExecuteError,
type LogicFunctionExecuteResult,
} from 'src/engine/core-modules/logic-function/logic-function-drivers/interfaces/logic-function-driver.interface';
const MAX_ENTRIES = 500;
const MAX_MESSAGE_LENGTH = 4_000;
const MAX_STACK_TRACE_LENGTH = 8_000;
const truncate = (value: string, max: number): string =>
value.length > max ? `${value.slice(0, max)}…[truncated]` : value;
type StepLogEntry = WorkflowRunStepLog['entries'][number];
type LogLevel = StepLogEntry['level'];
const LEVEL_BY_INPUT: Record<string, LogLevel> = {
DEBUG: 'debug',
INFO: 'info',
WARN: 'warn',
ERROR: 'error',
};
const normalizeLevel = (rawLevel: string): LogLevel =>
LEVEL_BY_INPUT[rawLevel.toUpperCase()] ?? 'info';
const flattenStackTrace = (
stackTrace: LogicFunctionExecuteError['stackTrace'],
): string =>
Array.isArray(stackTrace) ? stackTrace.join('\n') : (stackTrace ?? '');
export const buildCodeStepLog = (
result: LogicFunctionExecuteResult,
): WorkflowRunStepLog => {
const parsedLines = parseApplicationLogLines(result.logs ?? '');
const droppedEntries = Math.max(0, parsedLines.length - MAX_ENTRIES);
const entries: StepLogEntry[] = parsedLines
.slice(0, MAX_ENTRIES)
.map((line) => ({
timestamp: line.timestamp.toISOString(),
level: normalizeLevel(line.level),
message: truncate(line.message, MAX_MESSAGE_LENGTH),
}));
const error = result.error
? {
type: result.error.errorType,
message: truncate(result.error.errorMessage, MAX_MESSAGE_LENGTH),
stackTrace: truncate(
flattenStackTrace(result.error.stackTrace),
MAX_STACK_TRACE_LENGTH,
),
}
: null;
return {
details: {
type: 'CODE',
durationMs: result.duration,
status: result.error ? 'ERROR' : 'SUCCESS',
error,
},
entries,
truncated:
droppedEntries > 0 ? { droppedEntries, droppedBytes: 0 } : undefined,
sizeBytes: 0,
};
};
@@ -0,0 +1,129 @@
import { Test, type TestingModule } from '@nestjs/testing';
import { HttpTool } from 'src/engine/core-modules/tool/tools/http-tool/http-tool';
import { HttpRequestWorkflowAction } from 'src/modules/workflow/workflow-executor/workflow-actions/http-request/http-request.workflow-action';
import { type WorkflowActionSettings } from 'src/modules/workflow/workflow-executor/workflow-actions/types/workflow-action-settings.type';
import {
type WorkflowAction,
WorkflowActionType,
} from 'src/modules/workflow/workflow-executor/workflow-actions/types/workflow-action.type';
import { WorkflowRunStepLogWorkspaceService } from 'src/modules/workflow/workflow-runner/workflow-run/workflow-run-step-log.workspace-service';
const baseSettings: WorkflowActionSettings = {
outputSchema: {},
errorHandlingOptions: {
retryOnFailure: { value: false },
continueOnFailure: { value: false },
},
input: {},
};
const buildHttpRequestStep = (input: Record<string, unknown>): WorkflowAction =>
({
id: 'step-1',
type: WorkflowActionType.HTTP_REQUEST,
name: 'HTTP Request',
valid: true,
settings: { ...baseSettings, input },
}) as WorkflowAction;
describe('HttpRequestWorkflowAction', () => {
let action: HttpRequestWorkflowAction;
let mockHttpTool: jest.Mocked<Pick<HttpTool, 'execute'>>;
let mockSetStepLog: jest.Mock;
beforeEach(async () => {
jest.clearAllMocks();
mockHttpTool = {
execute: jest.fn().mockResolvedValue({
result: { ok: true },
error: undefined,
status: 200,
}),
};
mockSetStepLog = jest.fn();
const module: TestingModule = await Test.createTestingModule({
providers: [
HttpRequestWorkflowAction,
{ provide: HttpTool, useValue: mockHttpTool },
{
provide: WorkflowRunStepLogWorkspaceService,
useValue: { setStepLog: mockSetStepLog },
},
],
}).compile();
action = module.get(HttpRequestWorkflowAction);
});
it('resolves variables in the request input and forwards them to the HTTP tool', async () => {
await action.execute({
currentStepId: 'step-1',
steps: [
buildHttpRequestStep({
url: 'https://api.example.com/users/{{trigger.id}}',
method: 'GET',
}),
],
context: { trigger: { id: '42' } },
runInfo: { workspaceId: 'workspace-1', workflowRunId: 'run-1' },
});
expect(mockHttpTool.execute).toHaveBeenCalledWith(
expect.objectContaining({
url: 'https://api.example.com/users/42',
method: 'GET',
}),
expect.objectContaining({ workspaceId: 'workspace-1' }),
);
});
it('persists an HTTP_REQUEST step log', async () => {
await action.execute({
currentStepId: 'step-1',
steps: [
buildHttpRequestStep({
url: 'https://api.example.com/users',
method: 'POST',
body: { name: 'John' },
}),
],
context: {},
runInfo: { workspaceId: 'workspace-1', workflowRunId: 'run-1' },
});
expect(mockSetStepLog).toHaveBeenCalledWith(
expect.objectContaining({
workflowRunId: 'run-1',
workspaceId: 'workspace-1',
stepId: 'step-1',
stepLog: expect.objectContaining({
details: expect.objectContaining({ type: 'HTTP_REQUEST' }),
}),
}),
);
});
it('throws when the current step is not an HTTP request action', async () => {
await expect(
action.execute({
currentStepId: 'step-1',
steps: [
{
...buildHttpRequestStep({
url: 'https://example.com',
method: 'GET',
}),
type: WorkflowActionType.SEND_EMAIL,
} as WorkflowAction,
],
context: {},
runInfo: { workspaceId: 'workspace-1', workflowRunId: 'run-1' },
}),
).rejects.toThrow('Step is not an HTTP request action');
expect(mockHttpTool.execute).not.toHaveBeenCalled();
});
});
@@ -0,0 +1,12 @@
import { Module } from '@nestjs/common';
import { ToolModule } from 'src/engine/core-modules/tool/tool.module';
import { HttpRequestWorkflowAction } from 'src/modules/workflow/workflow-executor/workflow-actions/http-request/http-request.workflow-action';
import { WorkflowRunModule } from 'src/modules/workflow/workflow-runner/workflow-run/workflow-run.module';
@Module({
imports: [ToolModule, WorkflowRunModule],
providers: [HttpRequestWorkflowAction],
exports: [HttpRequestWorkflowAction],
})
export class HttpRequestActionModule {}
@@ -0,0 +1,52 @@
import { Injectable } from '@nestjs/common';
import { type WorkflowRunStepLog } from 'twenty-shared/workflow';
import { HttpTool } from 'src/engine/core-modules/tool/tools/http-tool/http-tool';
import { type ToolOutput } from 'src/engine/core-modules/tool/types/tool-output.type';
import { type Tool } from 'src/engine/core-modules/tool/types/tool.type';
import {
WorkflowStepExecutorException,
WorkflowStepExecutorExceptionCode,
} from 'src/modules/workflow/workflow-executor/exceptions/workflow-step-executor.exception';
import { isWorkflowHttpRequestAction } from 'src/modules/workflow/workflow-executor/workflow-actions/http-request/guards/is-workflow-http-request-action.guard';
import { type WorkflowHttpRequestActionInput } from 'src/modules/workflow/workflow-executor/workflow-actions/http-request/types/workflow-http-request-action-input.type';
import { buildHttpRequestStepLog } from 'src/modules/workflow/workflow-executor/workflow-actions/http-request/utils/build-http-request-step-log.util';
import { ToolBackedWorkflowAction } from 'src/modules/workflow/workflow-executor/workflow-actions/tool-backed/tool-backed.workflow-action';
import { type WorkflowAction } from 'src/modules/workflow/workflow-executor/workflow-actions/types/workflow-action.type';
import { WorkflowRunStepLogWorkspaceService } from 'src/modules/workflow/workflow-runner/workflow-run/workflow-run-step-log.workspace-service';
@Injectable()
export class HttpRequestWorkflowAction extends ToolBackedWorkflowAction<WorkflowHttpRequestActionInput> {
constructor(
private readonly httpTool: HttpTool,
workflowRunStepLogService: WorkflowRunStepLogWorkspaceService,
) {
super(HttpRequestWorkflowAction.name, workflowRunStepLogService);
}
protected getTool(): Tool {
return this.httpTool;
}
protected assertStep(step: WorkflowAction): void {
if (!isWorkflowHttpRequestAction(step)) {
throw new WorkflowStepExecutorException(
'Step is not an HTTP request action',
WorkflowStepExecutorExceptionCode.INVALID_STEP_TYPE,
);
}
}
protected buildStepLog({
input,
output,
durationMs,
}: {
input: WorkflowHttpRequestActionInput;
output: ToolOutput;
durationMs: number;
}): WorkflowRunStepLog {
return buildHttpRequestStepLog({ input, output, durationMs });
}
}
@@ -0,0 +1,363 @@
import { type ToolOutput } from 'src/engine/core-modules/tool/types/tool-output.type';
import { type WorkflowHttpRequestActionInput } from 'src/modules/workflow/workflow-executor/workflow-actions/http-request/types/workflow-http-request-action-input.type';
import { buildHttpRequestStepLog } from 'src/modules/workflow/workflow-executor/workflow-actions/http-request/utils/build-http-request-step-log.util';
const baseInput: WorkflowHttpRequestActionInput = {
url: 'https://api.example.com/widgets',
method: 'POST',
headers: {
'content-type': 'application/json',
Authorization: 'Bearer super-secret-token',
},
body: { hello: 'world' },
};
const baseOutput: ToolOutput = {
success: true,
message: 'OK',
result: { id: 'abc' },
status: 201,
statusText: 'Created',
headers: {
'content-type': 'application/json',
'set-cookie': 'sid=abc; HttpOnly',
},
};
describe('buildHttpRequestStepLog', () => {
it('redacts sensitive request and response headers', () => {
const stepLog = buildHttpRequestStepLog({
input: baseInput,
output: baseOutput,
durationMs: 42,
});
if (stepLog.details.type !== 'HTTP_REQUEST') {
throw new Error('Expected HTTP_REQUEST details');
}
expect(stepLog.details.request.headers.Authorization).toBe('[redacted]');
expect(stepLog.details.request.headers['content-type']).toBe(
'application/json',
);
expect(stepLog.details.response?.headers['set-cookie']).toBe('[redacted]');
expect(stepLog.details.response?.headers['content-type']).toBe(
'application/json',
);
});
it('stringifies and reports body byte size', () => {
const stepLog = buildHttpRequestStepLog({
input: baseInput,
output: baseOutput,
durationMs: 10,
});
if (stepLog.details.type !== 'HTTP_REQUEST') {
throw new Error('Expected HTTP_REQUEST details');
}
expect(stepLog.details.request.body).toBe('{"hello":"world"}');
expect(stepLog.details.request.bodyBytes).toBe(17);
expect(stepLog.details.request.bodyTruncated).toBe(false);
expect(stepLog.details.response?.body).toBe('{"id":"abc"}');
expect(stepLog.details.response?.bodyBytes).toBe(12);
});
it('truncates oversized request bodies and marks bodyTruncated', () => {
const longPayload = 'x'.repeat(100_000);
const stepLog = buildHttpRequestStepLog({
input: { ...baseInput, body: { payload: longPayload } },
output: baseOutput,
durationMs: 10,
});
if (stepLog.details.type !== 'HTTP_REQUEST') {
throw new Error('Expected HTTP_REQUEST details');
}
expect(stepLog.details.request.bodyTruncated).toBe(true);
expect(stepLog.details.request.body).toContain('truncated');
expect(stepLog.details.request.bodyBytes).toBeGreaterThan(100_000);
});
it('truncates non-ASCII bodies by UTF-8 bytes, not UTF-16 code units', () => {
// CJK characters take 3 UTF-8 bytes each but 1 UTF-16 code unit.
// Before the byte-aware fix, `redacted.slice(0, 32_000)` on this payload
// would emit ~96 KB of UTF-8 (three times the intended cap). After the
// fix the truncated payload stays within the cap, plus at most one
// U+FFFD replacement char (~3 bytes) for a multi-byte sequence cut at
// the boundary.
const longCjkPayload = '日'.repeat(40_000);
const stepLog = buildHttpRequestStepLog({
input: { ...baseInput, body: { payload: longCjkPayload } },
output: baseOutput,
durationMs: 10,
});
if (stepLog.details.type !== 'HTTP_REQUEST') {
throw new Error('Expected HTTP_REQUEST details');
}
expect(stepLog.details.request.bodyTruncated).toBe(true);
const truncatedBody = stepLog.details.request.body ?? '';
const truncatedByteLength = Buffer.byteLength(
truncatedBody.replace('…[truncated]', ''),
'utf8',
);
expect(truncatedByteLength).toBeLessThanOrEqual(32_000 + 3);
});
it('omits response when the request never received one (transport error)', () => {
const stepLog = buildHttpRequestStepLog({
input: baseInput,
output: {
success: false,
message: `HTTP POST request to ${baseInput.url} failed`,
error: 'ENOTFOUND api.example.com',
},
durationMs: 30,
});
if (stepLog.details.type !== 'HTTP_REQUEST') {
throw new Error('Expected HTTP_REQUEST details');
}
expect(stepLog.details.response).toBeUndefined();
expect(stepLog.details.error).toBe('ENOTFOUND api.example.com');
});
it('captures response details when an HTTP error response is returned', () => {
const stepLog = buildHttpRequestStepLog({
input: baseInput,
output: {
success: false,
message: 'failed',
error: '{"code":"invalid"}',
status: 422,
statusText: 'Unprocessable Entity',
headers: { 'content-type': 'application/json' },
result: { code: 'invalid' },
},
durationMs: 55,
});
if (stepLog.details.type !== 'HTTP_REQUEST') {
throw new Error('Expected HTTP_REQUEST details');
}
expect(stepLog.details.response?.status).toBe(422);
expect(stepLog.details.response?.statusText).toBe('Unprocessable Entity');
expect(stepLog.details.response?.body).toBe('{"code":"invalid"}');
expect(stepLog.details.error).toBe('{"code":"invalid"}');
});
it('redacts sensitive query-string parameters in the request URL', () => {
const stepLog = buildHttpRequestStepLog({
input: {
...baseInput,
url: 'https://api.example.com/data?page=1&api_key=AKIA-leaked&token=oauth-leaked&safe=ok',
},
output: baseOutput,
durationMs: 1,
});
if (stepLog.details.type !== 'HTTP_REQUEST') {
throw new Error('Expected HTTP_REQUEST details');
}
const url = new URL(stepLog.details.request.url);
expect(url.searchParams.get('api_key')).toBe('[redacted]');
expect(url.searchParams.get('token')).toBe('[redacted]');
expect(url.searchParams.get('page')).toBe('1');
expect(url.searchParams.get('safe')).toBe('ok');
});
it('leaves the URL untouched when there are no sensitive params', () => {
const stepLog = buildHttpRequestStepLog({
input: { ...baseInput, url: 'https://api.example.com/widgets?page=2' },
output: baseOutput,
durationMs: 1,
});
if (stepLog.details.type !== 'HTTP_REQUEST') {
throw new Error('Expected HTTP_REQUEST details');
}
expect(stepLog.details.request.url).toBe(
'https://api.example.com/widgets?page=2',
);
});
it('redacts sensitive query params even when the URL is unparseable (regression)', () => {
// Whitespace in the host trips up the WHATWG URL parser, so this URL
// is rejected by `new URL()`. Before the fallback was added, the catch
// branch returned the raw URL with secrets intact.
const unparseableUrl =
'https://api example.com/data?page=1&api_key=AKIA-leaked&token=oauth-leaked&safe=ok';
expect(() => new URL(unparseableUrl)).toThrow();
const stepLog = buildHttpRequestStepLog({
input: { ...baseInput, url: unparseableUrl },
output: baseOutput,
durationMs: 1,
});
if (stepLog.details.type !== 'HTTP_REQUEST') {
throw new Error('Expected HTTP_REQUEST details');
}
const redactedUrl = stepLog.details.request.url;
expect(redactedUrl).not.toContain('AKIA-leaked');
expect(redactedUrl).not.toContain('oauth-leaked');
expect(redactedUrl).toContain('api_key=[redacted]');
expect(redactedUrl).toContain('token=[redacted]');
expect(redactedUrl).toContain('page=1');
expect(redactedUrl).toContain('safe=ok');
});
it('redacts percent-encoded sensitive param names in unparseable URLs', () => {
// `api%5Fkey` decodes to `api_key` — the fallback must decode before
// matching against the sensitive-name set.
const unparseableUrl = 'https://api example.com/x?api%5Fkey=leaked';
expect(() => new URL(unparseableUrl)).toThrow();
const stepLog = buildHttpRequestStepLog({
input: { ...baseInput, url: unparseableUrl },
output: baseOutput,
durationMs: 1,
});
if (stepLog.details.type !== 'HTTP_REQUEST') {
throw new Error('Expected HTTP_REQUEST details');
}
expect(stepLog.details.request.url).not.toContain('leaked');
expect(stepLog.details.request.url).toContain('api%5Fkey=[redacted]');
});
it('redacts sensitive keys in JSON request bodies (deep)', () => {
const stepLog = buildHttpRequestStepLog({
input: {
...baseInput,
body: {
username: 'alice',
password: 'hunter2',
credentials: { client_secret: 'oauth-secret', clientId: 'public' },
tokens: [{ access_token: 'aaa', issuedAt: 1 }],
} as unknown as WorkflowHttpRequestActionInput['body'],
},
output: baseOutput,
durationMs: 1,
});
if (stepLog.details.type !== 'HTTP_REQUEST') {
throw new Error('Expected HTTP_REQUEST details');
}
const parsed = JSON.parse(stepLog.details.request.body ?? '{}');
expect(parsed.username).toBe('alice');
expect(parsed.password).toBe('[redacted]');
expect(parsed.credentials.client_secret).toBe('[redacted]');
expect(parsed.credentials.clientId).toBe('public');
expect(parsed.tokens[0].access_token).toBe('[redacted]');
expect(parsed.tokens[0].issuedAt).toBe(1);
});
it('redacts sensitive keys in JSON response bodies (OAuth token endpoint shape)', () => {
const stepLog = buildHttpRequestStepLog({
input: baseInput,
output: {
...baseOutput,
result: {
token_type: 'Bearer',
access_token: 'leaked-access',
refresh_token: 'leaked-refresh',
expires_in: 3600,
},
},
durationMs: 1,
});
if (stepLog.details.type !== 'HTTP_REQUEST') {
throw new Error('Expected HTTP_REQUEST details');
}
const parsed = JSON.parse(stepLog.details.response?.body ?? '{}');
expect(parsed.access_token).toBe('[redacted]');
expect(parsed.refresh_token).toBe('[redacted]');
expect(parsed.token_type).toBe('Bearer');
expect(parsed.expires_in).toBe(3600);
});
it('redacts sensitive keys when a request body arrives as a JSON string', () => {
const stepLog = buildHttpRequestStepLog({
input: {
...baseInput,
body: '{"username":"alice","password":"hunter2"}',
},
output: baseOutput,
durationMs: 1,
});
if (stepLog.details.type !== 'HTTP_REQUEST') {
throw new Error('Expected HTTP_REQUEST details');
}
const parsed = JSON.parse(stepLog.details.request.body ?? '{}');
expect(parsed.password).toBe('[redacted]');
expect(parsed.username).toBe('alice');
});
it('leaves non-JSON string bodies untouched (e.g. form-encoded)', () => {
const stepLog = buildHttpRequestStepLog({
input: { ...baseInput, body: 'name=alice&topic=hello' },
output: baseOutput,
durationMs: 1,
});
if (stepLog.details.type !== 'HTTP_REQUEST') {
throw new Error('Expected HTTP_REQUEST details');
}
expect(stepLog.details.request.body).toBe('name=alice&topic=hello');
});
it('redacts sensitive keys when output.error is a structured object', () => {
const stepLog = buildHttpRequestStepLog({
input: baseInput,
output: {
success: false,
message: 'failed',
error: {
code: 'invalid_client',
client_secret: 'leaked',
} as unknown as string,
status: 401,
},
durationMs: 1,
});
if (stepLog.details.type !== 'HTTP_REQUEST') {
throw new Error('Expected HTTP_REQUEST details');
}
const parsed = JSON.parse(stepLog.details.error ?? '{}');
expect(parsed.client_secret).toBe('[redacted]');
expect(parsed.code).toBe('invalid_client');
});
});
@@ -0,0 +1,276 @@
import { type WorkflowRunStepLog } from 'twenty-shared/workflow';
import { type ToolOutput } from 'src/engine/core-modules/tool/types/tool-output.type';
import { type WorkflowHttpRequestActionInput } from 'src/modules/workflow/workflow-executor/workflow-actions/http-request/types/workflow-http-request-action-input.type';
import { truncateStringToUtf8ByteBudget } from 'src/utils/truncate-string-to-utf8-byte-budget.util';
const MAX_BODY_BYTES = 32_000;
const REDACTION_SENTINEL = '[redacted]';
const SENSITIVE_HEADER_NAMES = new Set([
'authorization',
'proxy-authorization',
'cookie',
'set-cookie',
'x-api-key',
'x-auth-token',
'x-csrf-token',
'x-amz-security-token',
'x-goog-api-key',
'api-key',
]);
const SENSITIVE_URL_PARAM_NAMES = new Set([
'api_key',
'apikey',
'api-key',
'token',
'access_token',
'refresh_token',
'id_token',
'auth',
'auth_token',
'authentication',
'secret',
'client_secret',
'private_key',
'key',
'sig',
'signature',
'password',
'passwd',
'pwd',
]);
const SENSITIVE_BODY_KEY_REGEX =
/^(password|passwd|pwd|.*_?token|.*_?secret|authorization|api[_-]?key|private[_-]?key|client[_-]?secret|x-?api-?key|x-?auth-?token|access[_-]?key)$/i;
const isSensitiveHeader = (name: string): boolean =>
SENSITIVE_HEADER_NAMES.has(name.toLowerCase());
const isSensitiveUrlParam = (name: string): boolean =>
SENSITIVE_URL_PARAM_NAMES.has(name.toLowerCase());
const isSensitiveBodyKey = (name: string): boolean =>
SENSITIVE_BODY_KEY_REGEX.test(name);
const redactHeaders = (
headers: Record<string, unknown> | undefined,
): Record<string, string> => {
if (!headers) {
return {};
}
const redacted: Record<string, string> = {};
for (const [name, value] of Object.entries(headers)) {
if (isSensitiveHeader(name)) {
redacted[name] = REDACTION_SENTINEL;
continue;
}
redacted[name] =
typeof value === 'string' ? value : (JSON.stringify(value) ?? '');
}
return redacted;
};
const URL_QUERY_PARAM_REGEX = /([?&])([^=&#]+)=([^&#]*)/g;
const safeDecodeUriComponent = (value: string): string => {
try {
return decodeURIComponent(value);
} catch {
return value;
}
};
const redactUrlQueryStringFallback = (rawUrl: string): string =>
rawUrl.replace(URL_QUERY_PARAM_REGEX, (match, prefix, name) => {
if (isSensitiveUrlParam(safeDecodeUriComponent(name))) {
return `${prefix}${name}=${REDACTION_SENTINEL}`;
}
return match;
});
const redactUrl = (rawUrl: string): string => {
let parsed: URL;
try {
parsed = new URL(rawUrl);
} catch {
return redactUrlQueryStringFallback(rawUrl);
}
let didRedact = false;
for (const paramName of [...parsed.searchParams.keys()]) {
if (isSensitiveUrlParam(paramName)) {
parsed.searchParams.set(paramName, REDACTION_SENTINEL);
didRedact = true;
}
}
return didRedact ? parsed.toString() : rawUrl;
};
const redactSensitiveBodyKeysDeep = (value: unknown): unknown => {
if (Array.isArray(value)) {
return value.map(redactSensitiveBodyKeysDeep);
}
if (value !== null && typeof value === 'object') {
const redacted: Record<string, unknown> = {};
for (const [key, nested] of Object.entries(value)) {
if (isSensitiveBodyKey(key)) {
redacted[key] = REDACTION_SENTINEL;
continue;
}
redacted[key] = redactSensitiveBodyKeysDeep(nested);
}
return redacted;
}
return value;
};
const redactJsonBodyString = (body: string): string => {
if (body.length === 0) {
return body;
}
let parsed: unknown;
try {
parsed = JSON.parse(body);
} catch {
return body;
}
if (parsed === null || typeof parsed !== 'object') {
return body;
}
const redactedTree = redactSensitiveBodyKeysDeep(parsed);
try {
return JSON.stringify(redactedTree);
} catch {
return body;
}
};
const redactErrorValue = (error: unknown): string | undefined => {
if (error === undefined || error === null) {
return undefined;
}
if (typeof error === 'string') {
return redactJsonBodyString(error);
}
try {
return JSON.stringify(redactSensitiveBodyKeysDeep(error));
} catch {
return String(error);
}
};
const stringifyBody = (body: unknown): string | undefined => {
if (body === undefined || body === null) {
return undefined;
}
if (typeof body === 'string') {
return body;
}
if (typeof body === 'object') {
try {
const redactedTree = redactSensitiveBodyKeysDeep(body);
return JSON.stringify(redactedTree);
} catch {
return String(body);
}
}
return String(body);
};
type SerializedBody = {
body: string | undefined;
bodyBytes: number | undefined;
bodyTruncated: boolean;
};
const serializeBody = (body: unknown): SerializedBody => {
const serialized = stringifyBody(body);
if (serialized === undefined) {
return { body: undefined, bodyBytes: undefined, bodyTruncated: false };
}
const redacted =
typeof body === 'string' ? redactJsonBodyString(serialized) : serialized;
const { value, originalBytes, truncated } = truncateStringToUtf8ByteBudget(
redacted,
MAX_BODY_BYTES,
);
return { body: value, bodyBytes: originalBytes, bodyTruncated: truncated };
};
export const buildHttpRequestStepLog = ({
input,
output,
durationMs,
}: {
input: WorkflowHttpRequestActionInput;
output: ToolOutput;
durationMs: number;
}): WorkflowRunStepLog => {
const requestBody = serializeBody(input.body);
const responseBody = serializeBody(output.result);
const hasResponseMetadata =
typeof output.status === 'number' ||
(output.headers !== undefined && Object.keys(output.headers).length > 0) ||
responseBody.body !== undefined;
const response = hasResponseMetadata
? {
status: output.status ?? 0,
statusText: output.statusText,
headers: redactHeaders(output.headers),
body: responseBody.body,
bodyBytes: responseBody.bodyBytes,
bodyTruncated: responseBody.bodyTruncated,
}
: undefined;
return {
details: {
type: 'HTTP_REQUEST',
request: {
method: input.method,
url: redactUrl(input.url),
headers: redactHeaders(input.headers),
body: requestBody.body,
bodyBytes: requestBody.bodyBytes,
bodyTruncated: requestBody.bodyTruncated,
},
response,
error: redactErrorValue(output.error),
durationMs,
},
entries: [],
sizeBytes: 0,
};
};
@@ -0,0 +1,139 @@
import { Test, type TestingModule } from '@nestjs/testing';
import { DraftEmailTool } from 'src/engine/core-modules/tool/tools/email-tool/draft-email-tool';
import { DraftEmailWorkflowAction } from 'src/modules/workflow/workflow-executor/workflow-actions/mail-sender/draft-email.workflow-action';
import { type WorkflowActionSettings } from 'src/modules/workflow/workflow-executor/workflow-actions/types/workflow-action-settings.type';
import {
type WorkflowAction,
WorkflowActionType,
} from 'src/modules/workflow/workflow-executor/workflow-actions/types/workflow-action.type';
import { WorkflowRunStepLogWorkspaceService } from 'src/modules/workflow/workflow-runner/workflow-run/workflow-run-step-log.workspace-service';
jest.mock(
'src/engine/core-modules/tool/tools/email-tool/utils/render-rich-text-to-html.util',
() => ({
renderRichTextToHtml: jest.fn().mockResolvedValue('<p>rendered html</p>'),
}),
);
const baseSettings: WorkflowActionSettings = {
outputSchema: {},
errorHandlingOptions: {
retryOnFailure: { value: false },
continueOnFailure: { value: false },
},
input: {},
};
const buildDraftEmailStep = (input: Record<string, unknown>): WorkflowAction =>
({
id: 'step-1',
type: WorkflowActionType.DRAFT_EMAIL,
name: 'Draft Email',
valid: true,
settings: { ...baseSettings, input },
}) as WorkflowAction;
describe('DraftEmailWorkflowAction', () => {
let action: DraftEmailWorkflowAction;
let mockDraftEmailTool: jest.Mocked<Pick<DraftEmailTool, 'execute'>>;
let mockSetStepLog: jest.Mock;
beforeEach(async () => {
jest.clearAllMocks();
mockDraftEmailTool = {
execute: jest.fn().mockResolvedValue({
result: { success: true },
error: undefined,
}),
};
mockSetStepLog = jest.fn();
const module: TestingModule = await Test.createTestingModule({
providers: [
DraftEmailWorkflowAction,
{ provide: DraftEmailTool, useValue: mockDraftEmailTool },
{
provide: WorkflowRunStepLogWorkspaceService,
useValue: { setStepLog: mockSetStepLog },
},
],
}).compile();
action = module.get(DraftEmailWorkflowAction);
});
it('runs the draft email tool and resolves variables in the body', async () => {
await action.execute({
currentStepId: 'step-1',
steps: [
buildDraftEmailStep({
connectedAccountId: 'account-1',
recipients: { to: 'test@example.com' },
subject: 'Draft Test',
body: '{{trigger.name}}',
}),
],
context: { trigger: { name: 'John' } },
runInfo: { workspaceId: 'workspace-1', workflowRunId: 'run-1' },
});
expect(mockDraftEmailTool.execute).toHaveBeenCalledWith(
expect.objectContaining({ body: 'John' }),
expect.objectContaining({ workspaceId: 'workspace-1' }),
);
});
it('persists a step log tagged with the DRAFT mode', async () => {
await action.execute({
currentStepId: 'step-1',
steps: [
buildDraftEmailStep({
connectedAccountId: 'account-1',
recipients: { to: 'test@example.com' },
subject: 'Draft Test',
body: 'hello',
}),
],
context: {},
runInfo: { workspaceId: 'workspace-1', workflowRunId: 'run-1' },
});
expect(mockSetStepLog).toHaveBeenCalledWith(
expect.objectContaining({
workflowRunId: 'run-1',
workspaceId: 'workspace-1',
stepId: 'step-1',
stepLog: expect.objectContaining({
details: expect.objectContaining({
type: 'EMAIL',
mode: 'DRAFT',
}),
}),
}),
);
});
it('throws when the current step is not a draft-email action', async () => {
await expect(
action.execute({
currentStepId: 'step-1',
steps: [
{
...buildDraftEmailStep({
connectedAccountId: 'account-1',
recipients: { to: 'test@example.com' },
subject: 'Wrong type',
}),
type: WorkflowActionType.SEND_EMAIL,
} as WorkflowAction,
],
context: {},
runInfo: { workspaceId: 'workspace-1', workflowRunId: 'run-1' },
}),
).rejects.toThrow('Step is not a draft-email action');
expect(mockDraftEmailTool.execute).not.toHaveBeenCalled();
});
});
@@ -1,14 +1,13 @@
import { Test, type TestingModule } from '@nestjs/testing';
import { DraftEmailTool } from 'src/engine/core-modules/tool/tools/email-tool/draft-email-tool';
import { SendEmailTool } from 'src/engine/core-modules/tool/tools/email-tool/send-email-tool';
import { HttpTool } from 'src/engine/core-modules/tool/tools/http-tool/http-tool';
import { ToolExecutorWorkflowAction } from 'src/modules/workflow/workflow-executor/workflow-actions/tool-executor-workflow-action';
import { SendEmailWorkflowAction } from 'src/modules/workflow/workflow-executor/workflow-actions/mail-sender/send-email.workflow-action';
import { type WorkflowActionSettings } from 'src/modules/workflow/workflow-executor/workflow-actions/types/workflow-action-settings.type';
import {
type WorkflowAction,
WorkflowActionType,
} from 'src/modules/workflow/workflow-executor/workflow-actions/types/workflow-action.type';
import { WorkflowRunStepLogWorkspaceService } from 'src/modules/workflow/workflow-runner/workflow-run/workflow-run-step-log.workspace-service';
jest.mock(
'src/engine/core-modules/tool/tools/email-tool/utils/render-rich-text-to-html.util',
@@ -36,53 +35,47 @@ const emailInput = {
subject: 'Test',
};
const buildEmailStep = (
type: 'SEND_EMAIL' | 'DRAFT_EMAIL',
input: Record<string, unknown>,
): WorkflowAction =>
const buildSendEmailStep = (input: Record<string, unknown>): WorkflowAction =>
({
id: 'step-1',
type: WorkflowActionType[type],
name: type === 'SEND_EMAIL' ? 'Send Email' : 'Draft Email',
type: WorkflowActionType.SEND_EMAIL,
name: 'Send Email',
valid: true,
settings: { ...baseSettings, input },
}) as WorkflowAction;
describe('ToolExecutorWorkflowAction', () => {
let action: ToolExecutorWorkflowAction;
describe('SendEmailWorkflowAction', () => {
let action: SendEmailWorkflowAction;
let mockSendEmailTool: jest.Mocked<Pick<SendEmailTool, 'execute'>>;
let mockDraftEmailTool: jest.Mocked<Pick<DraftEmailTool, 'execute'>>;
beforeEach(async () => {
jest.clearAllMocks();
const toolResult = {
result: { success: true },
error: undefined,
mockSendEmailTool = {
execute: jest.fn().mockResolvedValue({
result: { success: true },
error: undefined,
}),
};
mockSendEmailTool = { execute: jest.fn().mockResolvedValue(toolResult) };
mockDraftEmailTool = { execute: jest.fn().mockResolvedValue(toolResult) };
const module: TestingModule = await Test.createTestingModule({
providers: [
ToolExecutorWorkflowAction,
{ provide: HttpTool, useValue: { execute: jest.fn() } },
SendEmailWorkflowAction,
{ provide: SendEmailTool, useValue: mockSendEmailTool },
{ provide: DraftEmailTool, useValue: mockDraftEmailTool },
{
provide: WorkflowRunStepLogWorkspaceService,
useValue: { setStepLog: jest.fn() },
},
],
}).compile();
action = module.get(ToolExecutorWorkflowAction);
action = module.get(SendEmailWorkflowAction);
});
const executeWithBody = (
body: string | undefined,
type: 'SEND_EMAIL' | 'DRAFT_EMAIL' = 'SEND_EMAIL',
) =>
const executeWithBody = (body: string | undefined) =>
action.execute({
currentStepId: 'step-1',
steps: [buildEmailStep(type, { ...emailInput, body })],
steps: [buildSendEmailStep({ ...emailInput, body })],
context: {
trigger: {
name: 'John',
@@ -178,15 +171,25 @@ describe('ToolExecutorWorkflowAction', () => {
expect(renderRichTextToHtml).not.toHaveBeenCalled();
expect(mockSendEmailTool.execute).toHaveBeenCalled();
});
});
it('should apply the same body handling for DRAFT_EMAIL', async () => {
await executeWithBody('{{trigger.name}}', 'DRAFT_EMAIL');
describe('step type guard', () => {
it('throws when the current step is not a send-email action', async () => {
await expect(
action.execute({
currentStepId: 'step-1',
steps: [
{
...buildSendEmailStep({ ...emailInput, body: 'hi' }),
type: WorkflowActionType.DRAFT_EMAIL,
} as WorkflowAction,
],
context: {},
runInfo: { workspaceId: 'workspace-1', workflowRunId: 'run-1' },
}),
).rejects.toThrow('Step is not a send-email action');
expect(renderRichTextToHtml).not.toHaveBeenCalled();
expect(mockDraftEmailTool.execute).toHaveBeenCalledWith(
expect.objectContaining({ body: 'John' }),
expect.any(Object),
);
expect(mockSendEmailTool.execute).not.toHaveBeenCalled();
});
});
});
@@ -0,0 +1,40 @@
import { Injectable } from '@nestjs/common';
import { DraftEmailTool } from 'src/engine/core-modules/tool/tools/email-tool/draft-email-tool';
import { type Tool } from 'src/engine/core-modules/tool/types/tool.type';
import {
WorkflowStepExecutorException,
WorkflowStepExecutorExceptionCode,
} from 'src/modules/workflow/workflow-executor/exceptions/workflow-step-executor.exception';
import { EmailWorkflowActionBase } from 'src/modules/workflow/workflow-executor/workflow-actions/mail-sender/email-workflow-action.base';
import { isWorkflowDraftEmailAction } from 'src/modules/workflow/workflow-executor/workflow-actions/mail-sender/guards/is-workflow-draft-email-action.guard';
import { type EmailStepLogMode } from 'src/modules/workflow/workflow-executor/workflow-actions/mail-sender/utils/build-email-step-log.util';
import { type WorkflowAction } from 'src/modules/workflow/workflow-executor/workflow-actions/types/workflow-action.type';
import { WorkflowRunStepLogWorkspaceService } from 'src/modules/workflow/workflow-runner/workflow-run/workflow-run-step-log.workspace-service';
@Injectable()
export class DraftEmailWorkflowAction extends EmailWorkflowActionBase {
constructor(
private readonly draftEmailTool: DraftEmailTool,
workflowRunStepLogService: WorkflowRunStepLogWorkspaceService,
) {
super(DraftEmailWorkflowAction.name, workflowRunStepLogService);
}
protected getTool(): Tool {
return this.draftEmailTool;
}
protected getMode(): EmailStepLogMode {
return 'DRAFT';
}
protected assertStep(step: WorkflowAction): void {
if (!isWorkflowDraftEmailAction(step)) {
throw new WorkflowStepExecutorException(
'Step is not a draft-email action',
WorkflowStepExecutorExceptionCode.INVALID_STEP_TYPE,
);
}
}
}
@@ -0,0 +1,46 @@
import { type WorkflowRunStepLog } from 'twenty-shared/workflow';
import { isDefined } from 'twenty-shared/utils';
import { type ToolOutput } from 'src/engine/core-modules/tool/types/tool-output.type';
import { type WorkflowSendEmailActionInput } from 'src/modules/workflow/workflow-executor/workflow-actions/mail-sender/types/workflow-send-email-action-input.type';
import {
buildEmailStepLog,
type EmailStepLogMode,
} from 'src/modules/workflow/workflow-executor/workflow-actions/mail-sender/utils/build-email-step-log.util';
import { resolveEmailBody } from 'src/modules/workflow/workflow-executor/workflow-actions/mail-sender/utils/resolve-email-body.util';
import { ToolBackedWorkflowAction } from 'src/modules/workflow/workflow-executor/workflow-actions/tool-backed/tool-backed.workflow-action';
export abstract class EmailWorkflowActionBase extends ToolBackedWorkflowAction<WorkflowSendEmailActionInput> {
protected abstract getMode(): EmailStepLogMode;
protected override async preprocessInput(
rawInput: WorkflowSendEmailActionInput,
context: Record<string, unknown>,
): Promise<WorkflowSendEmailActionInput> {
if (!isDefined(rawInput.body)) {
return rawInput;
}
const renderedBody = await resolveEmailBody(rawInput.body, context);
return { ...rawInput, body: renderedBody };
}
protected buildStepLog({
input,
output,
durationMs,
}: {
input: WorkflowSendEmailActionInput;
output: ToolOutput;
durationMs: number;
}): WorkflowRunStepLog {
return buildEmailStepLog({
mode: this.getMode(),
input,
output,
durationMs,
});
}
}
@@ -0,0 +1,13 @@
import { Module } from '@nestjs/common';
import { ToolModule } from 'src/engine/core-modules/tool/tool.module';
import { DraftEmailWorkflowAction } from 'src/modules/workflow/workflow-executor/workflow-actions/mail-sender/draft-email.workflow-action';
import { SendEmailWorkflowAction } from 'src/modules/workflow/workflow-executor/workflow-actions/mail-sender/send-email.workflow-action';
import { WorkflowRunModule } from 'src/modules/workflow/workflow-runner/workflow-run/workflow-run.module';
@Module({
imports: [ToolModule, WorkflowRunModule],
providers: [SendEmailWorkflowAction, DraftEmailWorkflowAction],
exports: [SendEmailWorkflowAction, DraftEmailWorkflowAction],
})
export class MailSenderActionModule {}
@@ -0,0 +1,40 @@
import { Injectable } from '@nestjs/common';
import { SendEmailTool } from 'src/engine/core-modules/tool/tools/email-tool/send-email-tool';
import { type Tool } from 'src/engine/core-modules/tool/types/tool.type';
import {
WorkflowStepExecutorException,
WorkflowStepExecutorExceptionCode,
} from 'src/modules/workflow/workflow-executor/exceptions/workflow-step-executor.exception';
import { EmailWorkflowActionBase } from 'src/modules/workflow/workflow-executor/workflow-actions/mail-sender/email-workflow-action.base';
import { isWorkflowSendEmailAction } from 'src/modules/workflow/workflow-executor/workflow-actions/mail-sender/guards/is-workflow-send-email-action.guard';
import { type EmailStepLogMode } from 'src/modules/workflow/workflow-executor/workflow-actions/mail-sender/utils/build-email-step-log.util';
import { type WorkflowAction } from 'src/modules/workflow/workflow-executor/workflow-actions/types/workflow-action.type';
import { WorkflowRunStepLogWorkspaceService } from 'src/modules/workflow/workflow-runner/workflow-run/workflow-run-step-log.workspace-service';
@Injectable()
export class SendEmailWorkflowAction extends EmailWorkflowActionBase {
constructor(
private readonly sendEmailTool: SendEmailTool,
workflowRunStepLogService: WorkflowRunStepLogWorkspaceService,
) {
super(SendEmailWorkflowAction.name, workflowRunStepLogService);
}
protected getTool(): Tool {
return this.sendEmailTool;
}
protected getMode(): EmailStepLogMode {
return 'SEND';
}
protected assertStep(step: WorkflowAction): void {
if (!isWorkflowSendEmailAction(step)) {
throw new WorkflowStepExecutorException(
'Step is not a send-email action',
WorkflowStepExecutorExceptionCode.INVALID_STEP_TYPE,
);
}
}
}
@@ -0,0 +1,162 @@
import { type ToolOutput } from 'src/engine/core-modules/tool/types/tool-output.type';
import { type WorkflowSendEmailActionInput } from 'src/modules/workflow/workflow-executor/workflow-actions/mail-sender/types/workflow-send-email-action-input.type';
import { buildEmailStepLog } from 'src/modules/workflow/workflow-executor/workflow-actions/mail-sender/utils/build-email-step-log.util';
const baseInput: WorkflowSendEmailActionInput = {
connectedAccountId: 'account-1',
recipients: { to: 'alice@example.com, bob@example.com' },
subject: 'Welcome',
body: '<p>Hello</p>',
};
const successOutput: ToolOutput = {
success: true,
message: 'Email sent successfully to Alice',
result: {
recipients: ['alice@example.com', 'bob@example.com'],
ccRecipients: [],
bccRecipients: [],
subject: 'Welcome',
connectedAccountId: 'account-1',
attachmentCount: 0,
},
};
describe('buildEmailStepLog', () => {
it('builds a SUCCESS email log preferring parsed recipients from the tool output', () => {
const stepLog = buildEmailStepLog({
mode: 'SEND',
input: baseInput,
output: successOutput,
durationMs: 120,
});
if (stepLog.details.type !== 'EMAIL') {
throw new Error('Expected EMAIL details');
}
expect(stepLog.details.mode).toBe('SEND');
expect(stepLog.details.status).toBe('SUCCESS');
expect(stepLog.details.recipients.to).toEqual([
'alice@example.com',
'bob@example.com',
]);
expect(stepLog.details.recipients.cc).toBeUndefined();
expect(stepLog.details.subject).toBe('Welcome');
expect(stepLog.details.attachmentCount).toBe(0);
expect(stepLog.details.durationMs).toBe(120);
});
it('falls back to splitting the comma-separated input when the tool output has no parsed recipients', () => {
const stepLog = buildEmailStepLog({
mode: 'DRAFT',
input: {
...baseInput,
recipients: {
to: 'alice@example.com,bob@example.com ; carol@example.com',
cc: 'dan@example.com',
},
},
output: {
success: false,
message: 'Failed to create draft',
error: 'Connected account expired',
},
durationMs: 5,
});
if (stepLog.details.type !== 'EMAIL') {
throw new Error('Expected EMAIL details');
}
expect(stepLog.details.mode).toBe('DRAFT');
expect(stepLog.details.status).toBe('ERROR');
expect(stepLog.details.recipients.to).toEqual([
'alice@example.com',
'bob@example.com',
'carol@example.com',
]);
expect(stepLog.details.recipients.cc).toEqual(['dan@example.com']);
expect(stepLog.details.error).toBe('Connected account expired');
});
it('truncates oversized body previews and reports original byte size', () => {
const longBody = `<p>${'x'.repeat(20_000)}</p>`;
const stepLog = buildEmailStepLog({
mode: 'SEND',
input: { ...baseInput, body: longBody },
output: successOutput,
durationMs: 10,
});
if (stepLog.details.type !== 'EMAIL') {
throw new Error('Expected EMAIL details');
}
expect(stepLog.details.bodyTruncated).toBe(true);
expect(stepLog.details.bodyPreview).toContain('truncated');
expect(stepLog.details.bodyBytes).toBeGreaterThan(20_000);
});
it('prefers the sanitized HTML body from the tool output over the raw input body', () => {
const stepLog = buildEmailStepLog({
mode: 'SEND',
input: {
...baseInput,
body: '<script>alert("xss")</script><p>Hello</p>',
},
output: {
...successOutput,
result: {
...(successOutput.result as object),
sanitizedHtmlBody: '<p>Hello</p>',
plainTextBody: 'Hello',
},
},
durationMs: 10,
});
if (stepLog.details.type !== 'EMAIL') {
throw new Error('Expected EMAIL details');
}
expect(stepLog.details.bodyPreview).toBe('<p>Hello</p>');
expect(stepLog.details.bodyPreview).not.toContain('<script>');
});
it('falls back to the raw input body when the tool failed before composing', () => {
const stepLog = buildEmailStepLog({
mode: 'SEND',
input: { ...baseInput, body: '<p>Hello</p>' },
output: {
success: false,
message: 'Failed to send',
error: 'Auth expired',
},
durationMs: 5,
});
if (stepLog.details.type !== 'EMAIL') {
throw new Error('Expected EMAIL details');
}
expect(stepLog.details.bodyPreview).toBe('<p>Hello</p>');
});
it('omits cc/bcc when neither input nor output provides them', () => {
const stepLog = buildEmailStepLog({
mode: 'SEND',
input: { ...baseInput, recipients: { to: 'alice@example.com' } },
output: successOutput,
durationMs: 10,
});
if (stepLog.details.type !== 'EMAIL') {
throw new Error('Expected EMAIL details');
}
expect(stepLog.details.recipients.cc).toBeUndefined();
expect(stepLog.details.recipients.bcc).toBeUndefined();
});
});
@@ -0,0 +1,143 @@
import { type WorkflowRunStepLog } from 'twenty-shared/workflow';
import { type ToolOutput } from 'src/engine/core-modules/tool/types/tool-output.type';
import { type WorkflowSendEmailActionInput } from 'src/modules/workflow/workflow-executor/workflow-actions/mail-sender/types/workflow-send-email-action-input.type';
import { truncateStringToUtf8ByteBudget } from 'src/utils/truncate-string-to-utf8-byte-budget.util';
const MAX_BODY_PREVIEW_BYTES = 8_000;
export type EmailStepLogMode = 'SEND' | 'DRAFT';
const splitRecipients = (raw: string | undefined): string[] => {
if (raw === undefined || raw === null) {
return [];
}
return raw
.split(/[,;]/)
.map((entry) => entry.trim())
.filter((entry) => entry.length > 0);
};
const isStringArray = (value: unknown): value is string[] =>
Array.isArray(value) && value.every((item) => typeof item === 'string');
const pickRecipients = (
inputValue: string | undefined,
outputValue: unknown,
): string[] => {
if (isStringArray(outputValue)) {
return outputValue;
}
return splitRecipients(inputValue);
};
const truncateBody = (body: string | undefined) => {
if (body === undefined || body === null || body.length === 0) {
return {
bodyPreview: undefined,
bodyBytes: undefined,
bodyTruncated: false,
};
}
const { value, originalBytes, truncated } = truncateStringToUtf8ByteBudget(
body,
MAX_BODY_PREVIEW_BYTES,
);
return {
bodyPreview: value,
bodyBytes: originalBytes,
bodyTruncated: truncated,
};
};
const extractString = (output: ToolOutput, key: string): string | undefined => {
if (!output.result || typeof output.result !== 'object') {
return undefined;
}
const value = (output.result as Record<string, unknown>)[key];
return typeof value === 'string' ? value : undefined;
};
const extractNumber = (output: ToolOutput, key: string): number | undefined => {
if (!output.result || typeof output.result !== 'object') {
return undefined;
}
const value = (output.result as Record<string, unknown>)[key];
return typeof value === 'number' ? value : undefined;
};
const extractRecipientsField = (output: ToolOutput, key: string): unknown => {
if (!output.result || typeof output.result !== 'object') {
return undefined;
}
return (output.result as Record<string, unknown>)[key];
};
export const buildEmailStepLog = ({
mode,
input,
output,
durationMs,
}: {
mode: EmailStepLogMode;
input: WorkflowSendEmailActionInput;
output: ToolOutput;
durationMs: number;
}): WorkflowRunStepLog => {
const to = pickRecipients(
input.recipients?.to,
extractRecipientsField(output, 'recipients'),
);
const cc = pickRecipients(
input.recipients?.cc,
extractRecipientsField(output, 'ccRecipients'),
);
const bcc = pickRecipients(
input.recipients?.bcc,
extractRecipientsField(output, 'bccRecipients'),
);
const subject = extractString(output, 'subject') ?? input.subject;
const connectedAccountId =
extractString(output, 'connectedAccountId') ?? input.connectedAccountId;
const attachmentCount = extractNumber(output, 'attachmentCount');
const bodyForLog =
extractString(output, 'sanitizedHtmlBody') ??
extractString(output, 'plainTextBody') ??
input.body;
const body = truncateBody(bodyForLog);
return {
details: {
type: 'EMAIL',
mode,
status: output.success ? 'SUCCESS' : 'ERROR',
recipients: {
to,
cc: cc.length > 0 ? cc : undefined,
bcc: bcc.length > 0 ? bcc : undefined,
},
subject,
bodyPreview: body.bodyPreview,
bodyBytes: body.bodyBytes,
bodyTruncated: body.bodyTruncated,
connectedAccountId,
attachmentCount,
inReplyTo: input.inReplyTo,
error: output.error,
durationMs,
},
entries: [],
sizeBytes: 0,
};
};
@@ -0,0 +1,25 @@
import type { JSONContent } from '@tiptap/core';
import {
isDefined,
parseJson,
resolveRichTextVariables,
} from 'twenty-shared/utils';
import { renderRichTextToHtml } from 'src/engine/core-modules/tool/tools/email-tool/utils/render-rich-text-to-html.util';
export const resolveEmailBody = async (
body: string,
context: Record<string, unknown>,
): Promise<string> => {
const bodyWithResolvedVariables = resolveRichTextVariables(body, context);
const tipTapDocument = isDefined(bodyWithResolvedVariables)
? parseJson<JSONContent>(bodyWithResolvedVariables)
: null;
if (isDefined(tipTapDocument) && tipTapDocument.type === 'doc') {
return renderRichTextToHtml(tipTapDocument);
}
return bodyWithResolvedVariables ?? body;
};
@@ -0,0 +1,114 @@
import { Logger } from '@nestjs/common';
import { resolveInput } from 'twenty-shared/utils';
import { type WorkflowRunStepLog } from 'twenty-shared/workflow';
import { type ToolInput } from 'src/engine/core-modules/tool/types/tool-input.type';
import { type ToolOutput } from 'src/engine/core-modules/tool/types/tool-output.type';
import { type Tool } from 'src/engine/core-modules/tool/types/tool.type';
import { type WorkflowAction as WorkflowActionContract } from 'src/modules/workflow/workflow-executor/interfaces/workflow-action.interface';
import { type WorkflowActionInput } from 'src/modules/workflow/workflow-executor/types/workflow-action-input';
import { type WorkflowActionOutput } from 'src/modules/workflow/workflow-executor/types/workflow-action-output.type';
import { findStepOrThrow } from 'src/modules/workflow/workflow-executor/utils/find-step-or-throw.util';
import { type WorkflowAction } from 'src/modules/workflow/workflow-executor/workflow-actions/types/workflow-action.type';
import { WorkflowRunStepLogWorkspaceService } from 'src/modules/workflow/workflow-runner/workflow-run/workflow-run-step-log.workspace-service';
type BuildStepLogArgs<TInput> = {
input: TInput;
output: ToolOutput;
durationMs: number;
};
export abstract class ToolBackedWorkflowAction<
TInput extends ToolInput,
> implements WorkflowActionContract {
protected readonly logger: Logger;
protected constructor(
loggerName: string,
private readonly workflowRunStepLogService: WorkflowRunStepLogWorkspaceService,
) {
this.logger = new Logger(loggerName);
}
protected abstract getTool(): Tool;
protected abstract assertStep(step: WorkflowAction): void;
protected async preprocessInput(
rawInput: TInput,
_context: Record<string, unknown>,
): Promise<TInput> {
return rawInput;
}
protected abstract buildStepLog(
args: BuildStepLogArgs<TInput>,
): WorkflowRunStepLog;
async execute({
currentStepId,
steps,
context,
runInfo,
}: WorkflowActionInput): Promise<WorkflowActionOutput> {
const step = findStepOrThrow({ stepId: currentStepId, steps });
this.assertStep(step);
const rawInput = step.settings.input as TInput;
const preprocessed = await this.preprocessInput(rawInput, context);
const resolvedInput = resolveInput(preprocessed, context) as TInput;
const startedAt = Date.now();
const toolOutput = await this.getTool().execute(resolvedInput, {
workspaceId: runInfo.workspaceId,
});
const durationMs = Date.now() - startedAt;
await this.persistStepLog({
workflowRunId: runInfo.workflowRunId,
workspaceId: runInfo.workspaceId,
stepId: currentStepId,
input: resolvedInput,
output: toolOutput,
durationMs,
});
return {
result: toolOutput.result as object,
error: toolOutput.error,
};
}
private async persistStepLog({
workflowRunId,
workspaceId,
stepId,
input,
output,
durationMs,
}: {
workflowRunId: string;
workspaceId: string;
stepId: string;
input: TInput;
output: ToolOutput;
durationMs: number;
}): Promise<void> {
try {
await this.workflowRunStepLogService.setStepLog({
workflowRunId,
workspaceId,
stepId,
stepLog: this.buildStepLog({ input, output, durationMs }),
});
} catch (error) {
this.logger.warn(
`Failed to persist step log for workflowRun=${workflowRunId} step=${stepId}: ${
error instanceof Error ? error.message : String(error)
}`,
);
}
}
}
@@ -1,112 +0,0 @@
import { Injectable } from '@nestjs/common';
import type { JSONContent } from '@tiptap/core';
import {
isDefined,
parseJson,
resolveInput,
resolveRichTextVariables,
} from 'twenty-shared/utils';
import { type WorkflowAction } from 'src/modules/workflow/workflow-executor/interfaces/workflow-action.interface';
import { DraftEmailTool } from 'src/engine/core-modules/tool/tools/email-tool/draft-email-tool';
import { renderRichTextToHtml } from 'src/engine/core-modules/tool/tools/email-tool/utils/render-rich-text-to-html.util';
import { HttpTool } from 'src/engine/core-modules/tool/tools/http-tool/http-tool';
import { SendEmailTool } from 'src/engine/core-modules/tool/tools/email-tool/send-email-tool';
import { type ToolInput } from 'src/engine/core-modules/tool/types/tool-input.type';
import { type Tool } from 'src/engine/core-modules/tool/types/tool.type';
import {
WorkflowStepExecutorException,
WorkflowStepExecutorExceptionCode,
} from 'src/modules/workflow/workflow-executor/exceptions/workflow-step-executor.exception';
import { type WorkflowActionInput } from 'src/modules/workflow/workflow-executor/types/workflow-action-input';
import { type WorkflowActionOutput } from 'src/modules/workflow/workflow-executor/types/workflow-action-output.type';
import { type WorkflowSendEmailActionInput } from 'src/modules/workflow/workflow-executor/workflow-actions/mail-sender/types/workflow-send-email-action-input.type';
import { WorkflowActionType } from 'src/modules/workflow/workflow-executor/workflow-actions/types/workflow-action.type';
@Injectable()
export class ToolExecutorWorkflowAction implements WorkflowAction {
private readonly toolsByActionType: Map<WorkflowActionType, Tool>;
constructor(
private readonly httpTool: HttpTool,
private readonly sendEmailTool: SendEmailTool,
private readonly draftEmailTool: DraftEmailTool,
) {
this.toolsByActionType = new Map<WorkflowActionType, Tool>([
[WorkflowActionType.HTTP_REQUEST, this.httpTool],
[WorkflowActionType.SEND_EMAIL, this.sendEmailTool],
[WorkflowActionType.DRAFT_EMAIL, this.draftEmailTool],
]);
}
private async resolveEmailBody(
body: string,
context: Record<string, unknown>,
): Promise<string> {
const bodyWithResolvedVariables = resolveRichTextVariables(body, context);
const tipTapDocument = isDefined(bodyWithResolvedVariables)
? parseJson<JSONContent>(bodyWithResolvedVariables)
: null;
if (isDefined(tipTapDocument) && tipTapDocument.type === 'doc') {
return renderRichTextToHtml(tipTapDocument);
}
return bodyWithResolvedVariables ?? body;
}
async execute({
currentStepId,
steps,
context,
runInfo,
}: WorkflowActionInput): Promise<WorkflowActionOutput> {
const step = steps.find((step) => step.id === currentStepId);
if (!step) {
throw new WorkflowStepExecutorException(
'Step not found',
WorkflowStepExecutorExceptionCode.STEP_NOT_FOUND,
);
}
const tool = this.toolsByActionType.get(step.type);
if (!tool) {
throw new WorkflowStepExecutorException(
`No tool found for workflow action type: ${step.type}`,
WorkflowStepExecutorExceptionCode.INVALID_STEP_TYPE,
);
}
let toolInput = step.settings.input;
if (
step.type === WorkflowActionType.SEND_EMAIL ||
step.type === WorkflowActionType.DRAFT_EMAIL
) {
const emailInput = toolInput as WorkflowSendEmailActionInput;
if (isDefined(emailInput.body)) {
const emailBody = await this.resolveEmailBody(emailInput.body, context);
toolInput = {
...emailInput,
body: emailBody,
};
}
}
toolInput = resolveInput(toolInput, context) as ToolInput;
const toolOutput = await tool.execute(toolInput, {
workspaceId: runInfo.workspaceId,
});
return {
result: toolOutput.result as object,
error: toolOutput.error,
};
}
}
@@ -4,7 +4,6 @@ import { BillingModule } from 'src/engine/core-modules/billing/billing.module';
import { WorkspaceCacheModule } from 'src/engine/workspace-cache/workspace-cache.module';
import { FeatureFlagModule } from 'src/engine/core-modules/feature-flag/feature-flag.module';
import { MetricsModule } from 'src/engine/core-modules/metrics/metrics.module';
import { ToolModule } from 'src/engine/core-modules/tool/tool.module';
import { WorkflowCommonModule } from 'src/modules/workflow/common/workflow-common.module';
import { WorkflowActionFactory } from 'src/modules/workflow/workflow-executor/factories/workflow-action.factory';
import { AiAgentActionModule } from 'src/modules/workflow/workflow-executor/workflow-actions/ai-agent/ai-agent-action.module';
@@ -13,11 +12,12 @@ import { DelayActionModule } from 'src/modules/workflow/workflow-executor/workfl
import { EmptyActionModule } from 'src/modules/workflow/workflow-executor/workflow-actions/empty/empty-action.module';
import { FilterActionModule } from 'src/modules/workflow/workflow-executor/workflow-actions/filter/filter-action.module';
import { FormActionModule } from 'src/modules/workflow/workflow-executor/workflow-actions/form/form-action.module';
import { HttpRequestActionModule } from 'src/modules/workflow/workflow-executor/workflow-actions/http-request/http-request-action.module';
import { IfElseActionModule } from 'src/modules/workflow/workflow-executor/workflow-actions/if-else/if-else-action.module';
import { IteratorActionModule } from 'src/modules/workflow/workflow-executor/workflow-actions/iterator/iterator-action.module';
import { LogicFunctionActionModule } from 'src/modules/workflow/workflow-executor/workflow-actions/logic-function/logic-function-action.module';
import { MailSenderActionModule } from 'src/modules/workflow/workflow-executor/workflow-actions/mail-sender/mail-sender-action.module';
import { RecordCRUDActionModule } from 'src/modules/workflow/workflow-executor/workflow-actions/record-crud/record-crud-action.module';
import { ToolExecutorWorkflowAction } from 'src/modules/workflow/workflow-executor/workflow-actions/tool-executor-workflow-action';
import { WorkflowExecutorWorkspaceService } from 'src/modules/workflow/workflow-executor/workspace-services/workflow-executor.workspace-service';
import { WorkflowRunModule } from 'src/modules/workflow/workflow-runner/workflow-run/workflow-run.module';
@@ -38,14 +38,11 @@ import { WorkflowRunModule } from 'src/modules/workflow/workflow-runner/workflow
AiAgentActionModule,
EmptyActionModule,
FeatureFlagModule,
ToolModule,
HttpRequestActionModule,
MailSenderActionModule,
MetricsModule,
],
providers: [
WorkflowExecutorWorkspaceService,
WorkflowActionFactory,
ToolExecutorWorkflowAction,
],
providers: [WorkflowExecutorWorkspaceService, WorkflowActionFactory],
exports: [WorkflowExecutorWorkspaceService],
})
export class WorkflowExecutorModule {}
@@ -0,0 +1,89 @@
import { Injectable, Logger } from '@nestjs/common';
import { FeatureFlagKey } from 'twenty-shared/types';
import { type WorkflowRunStepLog } from 'twenty-shared/workflow';
import { FeatureFlagService } from 'src/engine/core-modules/feature-flag/services/feature-flag.service';
import { GlobalWorkspaceOrmManager } from 'src/engine/twenty-orm/global-workspace-datasource/global-workspace-orm.manager';
import { buildSystemAuthContext } from 'src/engine/twenty-orm/utils/build-system-auth-context.util';
import { type WorkflowRunWorkspaceEntity } from 'src/modules/workflow/common/standard-objects/workflow-run.workspace-entity';
const MAX_STEP_LOG_BYTES = 256_000;
const computeSizeBytes = (value: unknown): number => {
try {
return Buffer.byteLength(JSON.stringify(value) ?? '', 'utf8');
} catch {
return 0;
}
};
@Injectable()
export class WorkflowRunStepLogWorkspaceService {
private readonly logger = new Logger(WorkflowRunStepLogWorkspaceService.name);
constructor(
private readonly globalWorkspaceOrmManager: GlobalWorkspaceOrmManager,
private readonly featureFlagService: FeatureFlagService,
) {}
async setStepLog({
workflowRunId,
workspaceId,
stepId,
stepLog,
}: {
workflowRunId: string;
workspaceId: string;
stepId: string;
stepLog: WorkflowRunStepLog;
}): Promise<void> {
const isStepLogsEnabled = await this.featureFlagService.isFeatureEnabled(
FeatureFlagKey.IS_WORKFLOW_RUN_STEP_LOGS_ENABLED,
workspaceId,
);
if (!isStepLogsEnabled) {
return;
}
const sizeBytes = computeSizeBytes(stepLog);
if (sizeBytes > MAX_STEP_LOG_BYTES) {
this.logger.warn(
`Step log for workflowRun=${workflowRunId} step=${stepId} exceeds cap (${sizeBytes}b > ${MAX_STEP_LOG_BYTES}b) and will be dropped`,
);
return;
}
const stepLogWithSize: WorkflowRunStepLog = {
...stepLog,
sizeBytes,
};
const authContext = buildSystemAuthContext(workspaceId);
await this.globalWorkspaceOrmManager.executeInWorkspaceContext(async () => {
const workflowRunRepository =
await this.globalWorkspaceOrmManager.getRepository<WorkflowRunWorkspaceEntity>(
workspaceId,
'workflowRun',
{ shouldBypassPermissionChecks: true },
);
await workflowRunRepository
.createQueryBuilder()
.update()
.set({
stepLogs: () =>
`jsonb_set(COALESCE("stepLogs", '{}'::jsonb), ARRAY[:stepId]::text[], :stepLog::jsonb, true)`,
})
.where('id = :workflowRunId', { workflowRunId })
.setParameters({
stepId,
stepLog: JSON.stringify(stepLogWithSize),
})
.execute();
}, authContext);
}
}
@@ -4,12 +4,14 @@ import { NestjsQueryTypeOrmModule } from '@ptc-org/nestjs-query-typeorm';
import { WorkspaceIteratorModule } from 'src/database/commands/command-runners/workspace-iterator.module';
import { CacheLockModule } from 'src/engine/core-modules/cache-lock/cache-lock.module';
import { FeatureFlagModule } from 'src/engine/core-modules/feature-flag/feature-flag.module';
import { MetricsModule } from 'src/engine/core-modules/metrics/metrics.module';
import { RecordPositionModule } from 'src/engine/core-modules/record-position/record-position.module';
import { WorkspaceEntity } from 'src/engine/core-modules/workspace/workspace.entity';
import { ObjectMetadataEntity } from 'src/engine/metadata-modules/object-metadata/object-metadata.entity';
import { WorkflowCommonModule } from 'src/modules/workflow/common/workflow-common.module';
import { DeleteWorkflowRunsCommand } from 'src/modules/workflow/workflow-runner/workflow-run/command/delete-workflow-runs.command';
import { WorkflowRunStepLogWorkspaceService } from 'src/modules/workflow/workflow-runner/workflow-run/workflow-run-step-log.workspace-service';
import { WorkflowRunWorkspaceService } from 'src/modules/workflow/workflow-runner/workflow-run/workflow-run.workspace-service';
@Module({
@@ -23,8 +25,17 @@ import { WorkflowRunWorkspaceService } from 'src/modules/workflow/workflow-runne
CacheLockModule,
MetricsModule,
WorkspaceIteratorModule,
FeatureFlagModule,
],
providers: [
WorkflowRunWorkspaceService,
WorkflowRunStepLogWorkspaceService,
DeleteWorkflowRunsCommand,
],
exports: [
WorkflowRunWorkspaceService,
WorkflowRunStepLogWorkspaceService,
DeleteWorkflowRunsCommand,
],
providers: [WorkflowRunWorkspaceService, DeleteWorkflowRunsCommand],
exports: [WorkflowRunWorkspaceService, DeleteWorkflowRunsCommand],
})
export class WorkflowRunModule {}