Provide additional logsobservability to workflow runs (per node) (#21142)
Surfaces per-step "Logs" tabs in the workflow run side panel so users can see what each step actually did (model + tokens + tool calls for AI, console output for serverless functions, request/response for HTTP, recipients/body for Email). <img width="546" height="501" alt="ai_agent_without_websearch" src="https://github.com/user-attachments/assets/c6ca3518-9489-4484-a570-3d0569ff3b03" /> ## Storage - New `stepLogs` JSONB column on the `workflowRun` workspace entity, typed as `Record<string, WorkflowRunStepLog>` (keyed by step id). - Schema lives in `twenty-shared`: `workflowRunStepLogSchema` with a discriminated `details.type` union for `AI_AGENT | CODE | HTTP_REQUEST | EMAIL` — frontends and backends consume the same Zod-inferred type. - Field is added to existing workspaces via a workspace upgrade command (`2-9 add-workflow-run-step-logs-field`); the standard-object metadata declares it for new workspaces. - Writes happen atomically per step in `WorkflowRunStepLogWorkspaceService.setStepLog` using `jsonb_set`. That lets concurrent steps in the same run write their own keys without contending with the existing lock around `workflowRun.state`. - Per-step payload is hard-capped at 256 KB; anything larger is dropped with a `logger.warn`, so a pathological tool call can never bloat a row. See below for more information. ## How logs are produced **Aalmost everything was already being collected; this PR mostly persists and renders it.** - **AI agent** — `AgentAsyncExecutorService` already tracked token usage, model id, native web-search count, and the AI SDK's `steps[]`. We map those into the log via `mapAiStepsToToolCallLogs` (`searchVector` stripped from record outputs, per-call input/output capped at 32/64 KB, max 200 tool calls per step). The only new measurement is a wall-clock `durationMs` taken around `executeAgent`, and we now fold native web-search cost into the displayed `totalCostInDollars` (it was already billed, just not shown). - **Code / serverless function** — reuses the `console.log` output the function runner already returns (`logsByLevel`); `build-code-step-log.util` only repackages it. - **HTTP request** — built from the action's existing input/output via `build-http-request-step-log.util`. No new signals collected. - **Email (send / draft)** — added `sanitizedHtmlBody` + `plainTextBody` to the existing tool outputs (a small additive change), then `build-email-step-log.util` consumes them. No additional AI inference or external calls are made for logging — the cost is a small CPU overhead per step plus the JSONB write. ## Security The log surface intentionally shows whatever the workflow touched, which made redaction and sanitization the main design concern. - **HTTP — secrets in headers**: existing `SENSITIVE_HEADER_NAMES` set (Authorization, Cookie, …) replaced with `[redacted]` in both request and response. - **HTTP — secrets in URLs**: `SENSITIVE_URL_PARAM_NAMES` (e.g. `api_key`, `token`, `access_token`) replaced in the query string via `URL`-based parsing. - **HTTP — secrets in bodies**: `SENSITIVE_BODY_KEY_REGEX` deep-walks JSON request/response bodies (object input or stringified JSON) and redacts matching keys. Applied to the `error` field too, since transport-layer errors sometimes embed structured payloads. - **Email — XSS risk in body preview**: tool outputs now expose a server-side `sanitizedHtmlBody`; the log builder prefers it over the raw user-authored `input.body`, with `plainTextBody` as a second fallback. The original raw body is only used if sanitization didn't happen (e.g. tool failed before composing). - **AI — internal/noisy data**: `searchVector` (Postgres tsvector strings) is stripped from record outputs returned by Twenty tools to avoid leaking internal full-text-search payloads. - **DB bloat / runaway agents**: 256 KB per-step cap + 32 KB / 64 KB per-tool-call input/output cap + 200 tool calls per step. <img width="547" height="307" alt="logic_function" src="https://github.com/user-attachments/assets/dd4a3d16-67f2-434b-95b3-bdcaf9ed053d" /> ## More details on Log size & truncation Logs are stored in `workflowRun.stepLogs` (JSONB), keyed by `stepId`. ### Per-step cap Each step's log is hard-capped at **256 KB** (`MAX_STEP_LOG_BYTES` in `WorkflowRunStepLogWorkspaceService.setStepLog`). For ~99% of workflows this is roomy — typical real-world sizes: - Code / serverless function: 1–20 KB - HTTP request: 5–70 KB - Email: 5–30 KB - AI agent (a handful of tool calls): 5–50 KB ### Two layers of bounding 1. **Per-field truncation** in each builder (before writing): - **Code**: ≤ 500 entries, ≤ 4 KB per message, ≤ 8 KB stack trace - **HTTP**: ≤ 32 KB per body (request + response), UTF-8 byte-aware - **Email**: ≤ 8 KB body preview, UTF-8 byte-aware - **AI agent**: ≤ 32 KB tool input, ≤ 64 KB tool output, ≤ 200 tool calls/step 2. **Global per-step safety net** at write time: if the assembled `stepLog` still exceeds 256 KB, the write is **dropped entirely** with a `logger.warn`. The workflow itself keeps running unaffected. ### What this means in practice - **Safe**: workflow execution, step results, downstream steps — never blocked by log size. - **Safe**: iterators (each iteration overwrites the previous log for that `stepId`, so they can't accumulate). - **Safe**: step retries (same `stepId` is overwritten, not appended). - **Possible**: an AI agent step with many large tool outputs (e.g., 50+ heavy `web_search` calls) can exceed 256 KB → the **entire** step's log is dropped, side panel shows "No logs were recorded for this step". The user has no explicit signal that the log was dropped due to size (only server-side warn). - **Possible** (theoretical): a workflow with hundreds of distinct steps could push the row toward Postgres's internal ~256 MB jsonb limit. Beyond that, individual `jsonb_set` writes would error and be swallowed by the action's try/catch — workflow still completes. ### Possible future hardening (not in this PR) - Replace "drop entire log" with a stub that preserves the summary card (cost, duration, status) and marks `truncated.reason = 'size_cap'`. - Surface size-drops in the UI (similar to the existing `<StyledTruncatedNotice>`). - Emit a metric so dropped logs are observable in dashboards.
This commit is contained in:
+5
-1
@@ -1,7 +1,10 @@
|
||||
import { registerEnumType } from '@nestjs/graphql';
|
||||
|
||||
import { type ActorMetadata, FieldMetadataType } from 'twenty-shared/types';
|
||||
import { type WorkflowRunStepInfos } from 'twenty-shared/workflow';
|
||||
import {
|
||||
type WorkflowRunStepInfos,
|
||||
type WorkflowRunStepLogs,
|
||||
} from 'twenty-shared/workflow';
|
||||
|
||||
import { BaseWorkspaceEntity } from 'src/engine/twenty-orm/base.workspace-entity';
|
||||
import { type FieldTypeAndNameMetadata } from 'src/engine/workspace-manager/utils/get-ts-vector-column-expression.util';
|
||||
@@ -66,6 +69,7 @@ export class WorkflowRunWorkspaceEntity extends BaseWorkspaceEntity {
|
||||
createdBy: ActorMetadata;
|
||||
updatedBy: ActorMetadata;
|
||||
state: WorkflowRunState;
|
||||
stepLogs: WorkflowRunStepLogs | null;
|
||||
position: number;
|
||||
searchVector: string;
|
||||
workflowVersion: EntityRelation<WorkflowVersionWorkspaceEntity>;
|
||||
|
||||
+4
-1
@@ -54,7 +54,10 @@ export class WorkflowVersionStepHelpersWorkspaceService {
|
||||
{ shouldBypassPermissionChecks: true },
|
||||
);
|
||||
|
||||
const updateData: Partial<WorkflowVersionWorkspaceEntity> = {};
|
||||
const updateData: Pick<
|
||||
Partial<WorkflowVersionWorkspaceEntity>,
|
||||
'steps' | 'trigger'
|
||||
> = {};
|
||||
|
||||
if (steps !== undefined) {
|
||||
updateData.steps = steps;
|
||||
|
||||
+9
-5
@@ -12,15 +12,17 @@ import { DelayWorkflowAction } from 'src/modules/workflow/workflow-executor/work
|
||||
import { EmptyWorkflowAction } from 'src/modules/workflow/workflow-executor/workflow-actions/empty/empty.workflow-action';
|
||||
import { FilterWorkflowAction } from 'src/modules/workflow/workflow-executor/workflow-actions/filter/filter.workflow-action';
|
||||
import { FormWorkflowAction } from 'src/modules/workflow/workflow-executor/workflow-actions/form/form.workflow-action';
|
||||
import { HttpRequestWorkflowAction } from 'src/modules/workflow/workflow-executor/workflow-actions/http-request/http-request.workflow-action';
|
||||
import { IfElseWorkflowAction } from 'src/modules/workflow/workflow-executor/workflow-actions/if-else/if-else.workflow-action';
|
||||
import { IteratorWorkflowAction } from 'src/modules/workflow/workflow-executor/workflow-actions/iterator/iterator.workflow-action';
|
||||
import { LogicFunctionWorkflowAction } from 'src/modules/workflow/workflow-executor/workflow-actions/logic-function/logic-function.workflow-action';
|
||||
import { DraftEmailWorkflowAction } from 'src/modules/workflow/workflow-executor/workflow-actions/mail-sender/draft-email.workflow-action';
|
||||
import { SendEmailWorkflowAction } from 'src/modules/workflow/workflow-executor/workflow-actions/mail-sender/send-email.workflow-action';
|
||||
import { CreateRecordWorkflowAction } from 'src/modules/workflow/workflow-executor/workflow-actions/record-crud/create-record.workflow-action';
|
||||
import { DeleteRecordWorkflowAction } from 'src/modules/workflow/workflow-executor/workflow-actions/record-crud/delete-record.workflow-action';
|
||||
import { FindRecordsWorkflowAction } from 'src/modules/workflow/workflow-executor/workflow-actions/record-crud/find-records.workflow-action';
|
||||
import { UpdateRecordWorkflowAction } from 'src/modules/workflow/workflow-executor/workflow-actions/record-crud/update-record.workflow-action';
|
||||
import { UpsertRecordWorkflowAction } from 'src/modules/workflow/workflow-executor/workflow-actions/record-crud/upsert-record.workflow-action';
|
||||
import { ToolExecutorWorkflowAction } from 'src/modules/workflow/workflow-executor/workflow-actions/tool-executor-workflow-action';
|
||||
import { WorkflowActionType } from 'src/modules/workflow/workflow-executor/workflow-actions/types/workflow-action.type';
|
||||
|
||||
@Injectable()
|
||||
@@ -37,7 +39,9 @@ export class WorkflowActionFactory {
|
||||
private readonly filterWorkflowAction: FilterWorkflowAction,
|
||||
private readonly ifElseWorkflowAction: IfElseWorkflowAction,
|
||||
private readonly iteratorWorkflowAction: IteratorWorkflowAction,
|
||||
private readonly toolExecutorWorkflowAction: ToolExecutorWorkflowAction,
|
||||
private readonly httpRequestWorkflowAction: HttpRequestWorkflowAction,
|
||||
private readonly sendEmailWorkflowAction: SendEmailWorkflowAction,
|
||||
private readonly draftEmailWorkflowAction: DraftEmailWorkflowAction,
|
||||
private readonly aiAgentWorkflowAction: AiAgentWorkflowAction,
|
||||
private readonly emptyWorkflowAction: EmptyWorkflowAction,
|
||||
private readonly delayWorkflowAction: DelayWorkflowAction,
|
||||
@@ -50,9 +54,9 @@ export class WorkflowActionFactory {
|
||||
case WorkflowActionType.LOGIC_FUNCTION:
|
||||
return this.logicFunctionWorkflowAction;
|
||||
case WorkflowActionType.SEND_EMAIL:
|
||||
return this.toolExecutorWorkflowAction;
|
||||
return this.sendEmailWorkflowAction;
|
||||
case WorkflowActionType.DRAFT_EMAIL:
|
||||
return this.toolExecutorWorkflowAction;
|
||||
return this.draftEmailWorkflowAction;
|
||||
case WorkflowActionType.CREATE_RECORD:
|
||||
return this.createRecordWorkflowAction;
|
||||
case WorkflowActionType.UPSERT_RECORD:
|
||||
@@ -72,7 +76,7 @@ export class WorkflowActionFactory {
|
||||
case WorkflowActionType.ITERATOR:
|
||||
return this.iteratorWorkflowAction;
|
||||
case WorkflowActionType.HTTP_REQUEST:
|
||||
return this.toolExecutorWorkflowAction;
|
||||
return this.httpRequestWorkflowAction;
|
||||
case WorkflowActionType.AI_AGENT:
|
||||
return this.aiAgentWorkflowAction;
|
||||
case WorkflowActionType.EMPTY:
|
||||
|
||||
+67
-15
@@ -1,4 +1,4 @@
|
||||
import { Injectable } from '@nestjs/common';
|
||||
import { Injectable, Logger } from '@nestjs/common';
|
||||
|
||||
import { resolveInput } from 'twenty-shared/utils';
|
||||
|
||||
@@ -6,6 +6,7 @@ import { type WorkflowAction } from 'src/modules/workflow/workflow-executor/inte
|
||||
|
||||
import { UsageOperationType } from 'src/engine/core-modules/usage/enums/usage-operation-type.enum';
|
||||
import { AgentAsyncExecutorService } from 'src/engine/metadata-modules/ai/ai-agent-execution/services/agent-async-executor.service';
|
||||
import { type AgentExecutionResult } from 'src/engine/metadata-modules/ai/ai-agent-execution/types/agent-execution-result.type';
|
||||
import { AgentEntity } from 'src/engine/metadata-modules/ai/ai-agent/entities/agent.entity';
|
||||
import { InjectWorkspaceScopedRepository } from 'src/engine/twenty-orm/workspace-scoped-repository/inject-workspace-scoped-repository.decorator';
|
||||
import { WorkspaceScopedRepository } from 'src/engine/twenty-orm/workspace-scoped-repository/workspace-scoped-repository';
|
||||
@@ -17,14 +18,19 @@ import { WorkflowExecutionContextService } from 'src/modules/workflow/workflow-e
|
||||
import { type WorkflowActionInput } from 'src/modules/workflow/workflow-executor/types/workflow-action-input';
|
||||
import { type WorkflowActionOutput } from 'src/modules/workflow/workflow-executor/types/workflow-action-output.type';
|
||||
import { findStepOrThrow } from 'src/modules/workflow/workflow-executor/utils/find-step-or-throw.util';
|
||||
import { buildAiAgentStepLog } from 'src/modules/workflow/workflow-executor/workflow-actions/ai-agent/utils/build-ai-agent-step-log.util';
|
||||
import { WorkflowRunStepLogWorkspaceService } from 'src/modules/workflow/workflow-runner/workflow-run/workflow-run-step-log.workspace-service';
|
||||
|
||||
import { isWorkflowAiAgentAction } from './guards/is-workflow-ai-agent-action.guard';
|
||||
|
||||
@Injectable()
|
||||
export class AiAgentWorkflowAction implements WorkflowAction {
|
||||
private readonly logger = new Logger(AiAgentWorkflowAction.name);
|
||||
|
||||
constructor(
|
||||
private readonly aiAgentExecutionService: AgentAsyncExecutorService,
|
||||
private readonly workflowExecutionContextService: WorkflowExecutionContextService,
|
||||
private readonly workflowRunStepLogService: WorkflowRunStepLogWorkspaceService,
|
||||
@InjectWorkspaceScopedRepository(AgentEntity)
|
||||
private readonly agentRepository: WorkspaceScopedRepository<AgentEntity>,
|
||||
) {}
|
||||
@@ -73,27 +79,73 @@ export class AiAgentWorkflowAction implements WorkflowAction {
|
||||
? executionContext.authContext.userWorkspaceId
|
||||
: null;
|
||||
|
||||
const { result, hasNoMoreAvailableCredits } =
|
||||
await this.aiAgentExecutionService.executeAgent({
|
||||
agent,
|
||||
userPrompt: resolveInput(prompt, context) as string,
|
||||
actorContext: executionContext.isActingOnBehalfOfUser
|
||||
? executionContext.initiator
|
||||
: undefined,
|
||||
authContext: executionContext.authContext,
|
||||
workspaceId,
|
||||
userWorkspaceId,
|
||||
operationType: UsageOperationType.AI_WORKFLOW_TOKEN,
|
||||
});
|
||||
const startedAtMs = Date.now();
|
||||
|
||||
if (hasNoMoreAvailableCredits) {
|
||||
const executionResult = await this.aiAgentExecutionService.executeAgent({
|
||||
agent,
|
||||
userPrompt: resolveInput(prompt, context) as string,
|
||||
actorContext: executionContext.isActingOnBehalfOfUser
|
||||
? executionContext.initiator
|
||||
: undefined,
|
||||
authContext: executionContext.authContext,
|
||||
workspaceId,
|
||||
userWorkspaceId,
|
||||
operationType: UsageOperationType.AI_WORKFLOW_TOKEN,
|
||||
});
|
||||
|
||||
const durationMs = Date.now() - startedAtMs;
|
||||
|
||||
await this.persistStepLog({
|
||||
workflowRunId: runInfo.workflowRunId,
|
||||
workspaceId,
|
||||
stepId: currentStepId,
|
||||
executionResult,
|
||||
durationMs,
|
||||
});
|
||||
|
||||
if (executionResult.hasNoMoreAvailableCredits) {
|
||||
return {
|
||||
error: 'AI agent stopped: no more available credits.',
|
||||
};
|
||||
}
|
||||
|
||||
return {
|
||||
result,
|
||||
result: executionResult.result,
|
||||
};
|
||||
}
|
||||
|
||||
private async persistStepLog({
|
||||
workflowRunId,
|
||||
workspaceId,
|
||||
stepId,
|
||||
executionResult,
|
||||
durationMs,
|
||||
}: {
|
||||
workflowRunId: string;
|
||||
workspaceId: string;
|
||||
stepId: string;
|
||||
executionResult: AgentExecutionResult;
|
||||
durationMs: number;
|
||||
}): Promise<void> {
|
||||
const stepLog = buildAiAgentStepLog({ executionResult, durationMs });
|
||||
|
||||
if (!stepLog) {
|
||||
return;
|
||||
}
|
||||
|
||||
try {
|
||||
await this.workflowRunStepLogService.setStepLog({
|
||||
workflowRunId,
|
||||
workspaceId,
|
||||
stepId,
|
||||
stepLog,
|
||||
});
|
||||
} catch (error) {
|
||||
this.logger.warn(
|
||||
`Failed to persist step log for workflowRun=${workflowRunId} step=${stepId}: ${
|
||||
error instanceof Error ? error.message : String(error)
|
||||
}`,
|
||||
);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
+98
@@ -0,0 +1,98 @@
|
||||
import { type AgentExecutionResult } from 'src/engine/metadata-modules/ai/ai-agent-execution/types/agent-execution-result.type';
|
||||
import { buildAiAgentStepLog } from 'src/modules/workflow/workflow-executor/workflow-actions/ai-agent/utils/build-ai-agent-step-log.util';
|
||||
|
||||
jest.mock(
|
||||
'src/engine/metadata-modules/ai/ai-agent-execution/utils/map-ai-steps-to-tool-call-logs.util',
|
||||
() => ({
|
||||
mapAiStepsToToolCallLogs: jest.fn().mockReturnValue([
|
||||
{
|
||||
toolName: 'web_search',
|
||||
toolCallId: 'call-1',
|
||||
state: 'success',
|
||||
},
|
||||
]),
|
||||
}),
|
||||
);
|
||||
|
||||
const baseExecutionResult: AgentExecutionResult = {
|
||||
result: { answer: 'hello' },
|
||||
usage: {
|
||||
inputTokens: 100,
|
||||
outputTokens: 50,
|
||||
totalTokens: 150,
|
||||
inputTokenDetails: { cacheReadTokens: 20 },
|
||||
outputTokenDetails: { reasoningTokens: 10 },
|
||||
} as AgentExecutionResult['usage'],
|
||||
cacheCreationTokens: 5,
|
||||
nativeWebSearchCallCount: 2,
|
||||
hasNoMoreAvailableCredits: false,
|
||||
modelId: 'claude-sonnet-4',
|
||||
totalCostInDollars: 0.012,
|
||||
creditsUsedMicro: 12_000,
|
||||
steps: [] as AgentExecutionResult['steps'],
|
||||
};
|
||||
|
||||
describe('buildAiAgentStepLog', () => {
|
||||
it('returns null when the execution never resolved a model', () => {
|
||||
const stepLog = buildAiAgentStepLog({
|
||||
executionResult: { ...baseExecutionResult, modelId: undefined },
|
||||
durationMs: 1234,
|
||||
});
|
||||
|
||||
expect(stepLog).toBeNull();
|
||||
});
|
||||
|
||||
it('builds an AI_AGENT step log with usage, cost, and tool calls', () => {
|
||||
const stepLog = buildAiAgentStepLog({
|
||||
executionResult: baseExecutionResult,
|
||||
durationMs: 1234,
|
||||
});
|
||||
|
||||
if (stepLog === null || stepLog.details.type !== 'AI_AGENT') {
|
||||
throw new Error('Expected AI_AGENT details');
|
||||
}
|
||||
|
||||
expect(stepLog.details.modelId).toBe('claude-sonnet-4');
|
||||
expect(stepLog.details.durationMs).toBe(1234);
|
||||
expect(stepLog.details.usage).toEqual({
|
||||
inputTokens: 100,
|
||||
outputTokens: 50,
|
||||
reasoningTokens: 10,
|
||||
cacheReadTokens: 20,
|
||||
cacheCreationTokens: 5,
|
||||
totalTokens: 155,
|
||||
});
|
||||
expect(stepLog.details.cost).toEqual({
|
||||
totalCostInDollars: 0.012,
|
||||
creditsUsedMicro: 12_000,
|
||||
});
|
||||
expect(stepLog.details.nativeWebSearchCallCount).toBe(2);
|
||||
expect(stepLog.details.toolCalls).toHaveLength(1);
|
||||
expect(stepLog.entries).toEqual([]);
|
||||
});
|
||||
|
||||
it('falls back to zero usage / cost when the agent did not report them', () => {
|
||||
const stepLog = buildAiAgentStepLog({
|
||||
executionResult: {
|
||||
...baseExecutionResult,
|
||||
usage: {} as AgentExecutionResult['usage'],
|
||||
cacheCreationTokens: 0,
|
||||
totalCostInDollars: undefined,
|
||||
creditsUsedMicro: undefined,
|
||||
steps: undefined,
|
||||
},
|
||||
durationMs: 100,
|
||||
});
|
||||
|
||||
if (stepLog === null || stepLog.details.type !== 'AI_AGENT') {
|
||||
throw new Error('Expected AI_AGENT details');
|
||||
}
|
||||
|
||||
expect(stepLog.details.usage.inputTokens).toBe(0);
|
||||
expect(stepLog.details.usage.outputTokens).toBe(0);
|
||||
expect(stepLog.details.usage.totalTokens).toBe(0);
|
||||
expect(stepLog.details.cost.totalCostInDollars).toBe(0);
|
||||
expect(stepLog.details.cost.creditsUsedMicro).toBe(0);
|
||||
expect(stepLog.details.toolCalls).toEqual([]);
|
||||
});
|
||||
});
|
||||
+52
@@ -0,0 +1,52 @@
|
||||
import {
|
||||
type AiAgentStepLogDetails,
|
||||
type WorkflowRunStepLog,
|
||||
} from 'twenty-shared/workflow';
|
||||
|
||||
import { type AgentExecutionResult } from 'src/engine/metadata-modules/ai/ai-agent-execution/types/agent-execution-result.type';
|
||||
import { mapAiStepsToToolCallLogs } from 'src/engine/metadata-modules/ai/ai-agent-execution/utils/map-ai-steps-to-tool-call-logs.util';
|
||||
|
||||
export const buildAiAgentStepLog = ({
|
||||
executionResult,
|
||||
durationMs,
|
||||
}: {
|
||||
executionResult: AgentExecutionResult;
|
||||
durationMs: number;
|
||||
}): WorkflowRunStepLog | null => {
|
||||
if (!executionResult.modelId) {
|
||||
return null;
|
||||
}
|
||||
|
||||
const toolCalls = executionResult.steps
|
||||
? mapAiStepsToToolCallLogs(executionResult.steps)
|
||||
: [];
|
||||
|
||||
const details: AiAgentStepLogDetails = {
|
||||
type: 'AI_AGENT',
|
||||
modelId: executionResult.modelId,
|
||||
usage: {
|
||||
inputTokens: executionResult.usage.inputTokens ?? 0,
|
||||
outputTokens: executionResult.usage.outputTokens ?? 0,
|
||||
reasoningTokens:
|
||||
executionResult.usage.outputTokenDetails?.reasoningTokens,
|
||||
cacheReadTokens: executionResult.usage.inputTokenDetails?.cacheReadTokens,
|
||||
cacheCreationTokens: executionResult.cacheCreationTokens,
|
||||
totalTokens:
|
||||
(executionResult.usage.totalTokens ?? 0) +
|
||||
executionResult.cacheCreationTokens,
|
||||
},
|
||||
cost: {
|
||||
totalCostInDollars: executionResult.totalCostInDollars ?? 0,
|
||||
creditsUsedMicro: executionResult.creditsUsedMicro ?? 0,
|
||||
},
|
||||
nativeWebSearchCallCount: executionResult.nativeWebSearchCallCount,
|
||||
toolCalls,
|
||||
durationMs,
|
||||
};
|
||||
|
||||
return {
|
||||
details,
|
||||
entries: [],
|
||||
sizeBytes: 0,
|
||||
};
|
||||
};
|
||||
+2
-1
@@ -2,9 +2,10 @@ import { Module } from '@nestjs/common';
|
||||
|
||||
import { LogicFunctionModule } from 'src/engine/core-modules/logic-function/logic-function.module';
|
||||
import { CodeWorkflowAction } from 'src/modules/workflow/workflow-executor/workflow-actions/code/code.workflow-action';
|
||||
import { WorkflowRunModule } from 'src/modules/workflow/workflow-runner/workflow-run/workflow-run.module';
|
||||
|
||||
@Module({
|
||||
imports: [LogicFunctionModule],
|
||||
imports: [LogicFunctionModule, WorkflowRunModule],
|
||||
providers: [CodeWorkflowAction],
|
||||
exports: [CodeWorkflowAction],
|
||||
})
|
||||
|
||||
+41
-1
@@ -1,9 +1,10 @@
|
||||
import { Injectable } from '@nestjs/common';
|
||||
import { Injectable, Logger } from '@nestjs/common';
|
||||
|
||||
import { resolveInput } from 'twenty-shared/utils';
|
||||
|
||||
import { type WorkflowAction } from 'src/modules/workflow/workflow-executor/interfaces/workflow-action.interface';
|
||||
|
||||
import { type LogicFunctionExecuteResult } from 'src/engine/core-modules/logic-function/logic-function-drivers/interfaces/logic-function-driver.interface';
|
||||
import { LogicFunctionExecutorService } from 'src/engine/core-modules/logic-function/logic-function-executor/logic-function-executor.service';
|
||||
import {
|
||||
WorkflowStepExecutorException,
|
||||
@@ -14,11 +15,16 @@ import { type WorkflowActionOutput } from 'src/modules/workflow/workflow-executo
|
||||
import { findStepOrThrow } from 'src/modules/workflow/workflow-executor/utils/find-step-or-throw.util';
|
||||
import { isWorkflowCodeAction } from 'src/modules/workflow/workflow-executor/workflow-actions/code/guards/is-workflow-code-action.guard';
|
||||
import { type WorkflowCodeActionInput } from 'src/modules/workflow/workflow-executor/workflow-actions/code/types/workflow-code-action-input.type';
|
||||
import { buildCodeStepLog } from 'src/modules/workflow/workflow-executor/workflow-actions/code/utils/build-code-step-log.util';
|
||||
import { WorkflowRunStepLogWorkspaceService } from 'src/modules/workflow/workflow-runner/workflow-run/workflow-run-step-log.workspace-service';
|
||||
|
||||
@Injectable()
|
||||
export class CodeWorkflowAction implements WorkflowAction {
|
||||
private readonly logger = new Logger(CodeWorkflowAction.name);
|
||||
|
||||
constructor(
|
||||
private readonly logicFunctionExecutorService: LogicFunctionExecutorService,
|
||||
private readonly workflowRunStepLogService: WorkflowRunStepLogWorkspaceService,
|
||||
) {}
|
||||
|
||||
async execute({
|
||||
@@ -52,10 +58,44 @@ export class CodeWorkflowAction implements WorkflowAction {
|
||||
payload: workflowActionInput.logicFunctionInput,
|
||||
});
|
||||
|
||||
await this.persistStepLog({
|
||||
workflowRunId: runInfo.workflowRunId,
|
||||
workspaceId,
|
||||
stepId: currentStepId,
|
||||
result,
|
||||
});
|
||||
|
||||
if (result.error) {
|
||||
return { error: result.error.errorMessage };
|
||||
}
|
||||
|
||||
return { result: result.data || {} };
|
||||
}
|
||||
|
||||
private async persistStepLog({
|
||||
workflowRunId,
|
||||
workspaceId,
|
||||
stepId,
|
||||
result,
|
||||
}: {
|
||||
workflowRunId: string;
|
||||
workspaceId: string;
|
||||
stepId: string;
|
||||
result: LogicFunctionExecuteResult;
|
||||
}): Promise<void> {
|
||||
try {
|
||||
await this.workflowRunStepLogService.setStepLog({
|
||||
workflowRunId,
|
||||
workspaceId,
|
||||
stepId,
|
||||
stepLog: buildCodeStepLog(result),
|
||||
});
|
||||
} catch (error) {
|
||||
this.logger.warn(
|
||||
`Failed to persist step log for workflowRun=${workflowRunId} step=${stepId}: ${
|
||||
error instanceof Error ? error.message : String(error)
|
||||
}`,
|
||||
);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
+72
@@ -0,0 +1,72 @@
|
||||
import { type WorkflowRunStepLog } from 'twenty-shared/workflow';
|
||||
|
||||
import { parseApplicationLogLines } from 'src/engine/core-modules/application-logs/utils/parse-application-log-lines';
|
||||
import {
|
||||
type LogicFunctionExecuteError,
|
||||
type LogicFunctionExecuteResult,
|
||||
} from 'src/engine/core-modules/logic-function/logic-function-drivers/interfaces/logic-function-driver.interface';
|
||||
|
||||
const MAX_ENTRIES = 500;
|
||||
const MAX_MESSAGE_LENGTH = 4_000;
|
||||
const MAX_STACK_TRACE_LENGTH = 8_000;
|
||||
|
||||
const truncate = (value: string, max: number): string =>
|
||||
value.length > max ? `${value.slice(0, max)}…[truncated]` : value;
|
||||
|
||||
type StepLogEntry = WorkflowRunStepLog['entries'][number];
|
||||
|
||||
type LogLevel = StepLogEntry['level'];
|
||||
|
||||
const LEVEL_BY_INPUT: Record<string, LogLevel> = {
|
||||
DEBUG: 'debug',
|
||||
INFO: 'info',
|
||||
WARN: 'warn',
|
||||
ERROR: 'error',
|
||||
};
|
||||
|
||||
const normalizeLevel = (rawLevel: string): LogLevel =>
|
||||
LEVEL_BY_INPUT[rawLevel.toUpperCase()] ?? 'info';
|
||||
|
||||
const flattenStackTrace = (
|
||||
stackTrace: LogicFunctionExecuteError['stackTrace'],
|
||||
): string =>
|
||||
Array.isArray(stackTrace) ? stackTrace.join('\n') : (stackTrace ?? '');
|
||||
|
||||
export const buildCodeStepLog = (
|
||||
result: LogicFunctionExecuteResult,
|
||||
): WorkflowRunStepLog => {
|
||||
const parsedLines = parseApplicationLogLines(result.logs ?? '');
|
||||
const droppedEntries = Math.max(0, parsedLines.length - MAX_ENTRIES);
|
||||
|
||||
const entries: StepLogEntry[] = parsedLines
|
||||
.slice(0, MAX_ENTRIES)
|
||||
.map((line) => ({
|
||||
timestamp: line.timestamp.toISOString(),
|
||||
level: normalizeLevel(line.level),
|
||||
message: truncate(line.message, MAX_MESSAGE_LENGTH),
|
||||
}));
|
||||
|
||||
const error = result.error
|
||||
? {
|
||||
type: result.error.errorType,
|
||||
message: truncate(result.error.errorMessage, MAX_MESSAGE_LENGTH),
|
||||
stackTrace: truncate(
|
||||
flattenStackTrace(result.error.stackTrace),
|
||||
MAX_STACK_TRACE_LENGTH,
|
||||
),
|
||||
}
|
||||
: null;
|
||||
|
||||
return {
|
||||
details: {
|
||||
type: 'CODE',
|
||||
durationMs: result.duration,
|
||||
status: result.error ? 'ERROR' : 'SUCCESS',
|
||||
error,
|
||||
},
|
||||
entries,
|
||||
truncated:
|
||||
droppedEntries > 0 ? { droppedEntries, droppedBytes: 0 } : undefined,
|
||||
sizeBytes: 0,
|
||||
};
|
||||
};
|
||||
+129
@@ -0,0 +1,129 @@
|
||||
import { Test, type TestingModule } from '@nestjs/testing';
|
||||
|
||||
import { HttpTool } from 'src/engine/core-modules/tool/tools/http-tool/http-tool';
|
||||
import { HttpRequestWorkflowAction } from 'src/modules/workflow/workflow-executor/workflow-actions/http-request/http-request.workflow-action';
|
||||
import { type WorkflowActionSettings } from 'src/modules/workflow/workflow-executor/workflow-actions/types/workflow-action-settings.type';
|
||||
import {
|
||||
type WorkflowAction,
|
||||
WorkflowActionType,
|
||||
} from 'src/modules/workflow/workflow-executor/workflow-actions/types/workflow-action.type';
|
||||
import { WorkflowRunStepLogWorkspaceService } from 'src/modules/workflow/workflow-runner/workflow-run/workflow-run-step-log.workspace-service';
|
||||
|
||||
const baseSettings: WorkflowActionSettings = {
|
||||
outputSchema: {},
|
||||
errorHandlingOptions: {
|
||||
retryOnFailure: { value: false },
|
||||
continueOnFailure: { value: false },
|
||||
},
|
||||
input: {},
|
||||
};
|
||||
|
||||
const buildHttpRequestStep = (input: Record<string, unknown>): WorkflowAction =>
|
||||
({
|
||||
id: 'step-1',
|
||||
type: WorkflowActionType.HTTP_REQUEST,
|
||||
name: 'HTTP Request',
|
||||
valid: true,
|
||||
settings: { ...baseSettings, input },
|
||||
}) as WorkflowAction;
|
||||
|
||||
describe('HttpRequestWorkflowAction', () => {
|
||||
let action: HttpRequestWorkflowAction;
|
||||
let mockHttpTool: jest.Mocked<Pick<HttpTool, 'execute'>>;
|
||||
let mockSetStepLog: jest.Mock;
|
||||
|
||||
beforeEach(async () => {
|
||||
jest.clearAllMocks();
|
||||
|
||||
mockHttpTool = {
|
||||
execute: jest.fn().mockResolvedValue({
|
||||
result: { ok: true },
|
||||
error: undefined,
|
||||
status: 200,
|
||||
}),
|
||||
};
|
||||
mockSetStepLog = jest.fn();
|
||||
|
||||
const module: TestingModule = await Test.createTestingModule({
|
||||
providers: [
|
||||
HttpRequestWorkflowAction,
|
||||
{ provide: HttpTool, useValue: mockHttpTool },
|
||||
{
|
||||
provide: WorkflowRunStepLogWorkspaceService,
|
||||
useValue: { setStepLog: mockSetStepLog },
|
||||
},
|
||||
],
|
||||
}).compile();
|
||||
|
||||
action = module.get(HttpRequestWorkflowAction);
|
||||
});
|
||||
|
||||
it('resolves variables in the request input and forwards them to the HTTP tool', async () => {
|
||||
await action.execute({
|
||||
currentStepId: 'step-1',
|
||||
steps: [
|
||||
buildHttpRequestStep({
|
||||
url: 'https://api.example.com/users/{{trigger.id}}',
|
||||
method: 'GET',
|
||||
}),
|
||||
],
|
||||
context: { trigger: { id: '42' } },
|
||||
runInfo: { workspaceId: 'workspace-1', workflowRunId: 'run-1' },
|
||||
});
|
||||
|
||||
expect(mockHttpTool.execute).toHaveBeenCalledWith(
|
||||
expect.objectContaining({
|
||||
url: 'https://api.example.com/users/42',
|
||||
method: 'GET',
|
||||
}),
|
||||
expect.objectContaining({ workspaceId: 'workspace-1' }),
|
||||
);
|
||||
});
|
||||
|
||||
it('persists an HTTP_REQUEST step log', async () => {
|
||||
await action.execute({
|
||||
currentStepId: 'step-1',
|
||||
steps: [
|
||||
buildHttpRequestStep({
|
||||
url: 'https://api.example.com/users',
|
||||
method: 'POST',
|
||||
body: { name: 'John' },
|
||||
}),
|
||||
],
|
||||
context: {},
|
||||
runInfo: { workspaceId: 'workspace-1', workflowRunId: 'run-1' },
|
||||
});
|
||||
|
||||
expect(mockSetStepLog).toHaveBeenCalledWith(
|
||||
expect.objectContaining({
|
||||
workflowRunId: 'run-1',
|
||||
workspaceId: 'workspace-1',
|
||||
stepId: 'step-1',
|
||||
stepLog: expect.objectContaining({
|
||||
details: expect.objectContaining({ type: 'HTTP_REQUEST' }),
|
||||
}),
|
||||
}),
|
||||
);
|
||||
});
|
||||
|
||||
it('throws when the current step is not an HTTP request action', async () => {
|
||||
await expect(
|
||||
action.execute({
|
||||
currentStepId: 'step-1',
|
||||
steps: [
|
||||
{
|
||||
...buildHttpRequestStep({
|
||||
url: 'https://example.com',
|
||||
method: 'GET',
|
||||
}),
|
||||
type: WorkflowActionType.SEND_EMAIL,
|
||||
} as WorkflowAction,
|
||||
],
|
||||
context: {},
|
||||
runInfo: { workspaceId: 'workspace-1', workflowRunId: 'run-1' },
|
||||
}),
|
||||
).rejects.toThrow('Step is not an HTTP request action');
|
||||
|
||||
expect(mockHttpTool.execute).not.toHaveBeenCalled();
|
||||
});
|
||||
});
|
||||
+12
@@ -0,0 +1,12 @@
|
||||
import { Module } from '@nestjs/common';
|
||||
|
||||
import { ToolModule } from 'src/engine/core-modules/tool/tool.module';
|
||||
import { HttpRequestWorkflowAction } from 'src/modules/workflow/workflow-executor/workflow-actions/http-request/http-request.workflow-action';
|
||||
import { WorkflowRunModule } from 'src/modules/workflow/workflow-runner/workflow-run/workflow-run.module';
|
||||
|
||||
@Module({
|
||||
imports: [ToolModule, WorkflowRunModule],
|
||||
providers: [HttpRequestWorkflowAction],
|
||||
exports: [HttpRequestWorkflowAction],
|
||||
})
|
||||
export class HttpRequestActionModule {}
|
||||
+52
@@ -0,0 +1,52 @@
|
||||
import { Injectable } from '@nestjs/common';
|
||||
|
||||
import { type WorkflowRunStepLog } from 'twenty-shared/workflow';
|
||||
|
||||
import { HttpTool } from 'src/engine/core-modules/tool/tools/http-tool/http-tool';
|
||||
import { type ToolOutput } from 'src/engine/core-modules/tool/types/tool-output.type';
|
||||
import { type Tool } from 'src/engine/core-modules/tool/types/tool.type';
|
||||
import {
|
||||
WorkflowStepExecutorException,
|
||||
WorkflowStepExecutorExceptionCode,
|
||||
} from 'src/modules/workflow/workflow-executor/exceptions/workflow-step-executor.exception';
|
||||
import { isWorkflowHttpRequestAction } from 'src/modules/workflow/workflow-executor/workflow-actions/http-request/guards/is-workflow-http-request-action.guard';
|
||||
import { type WorkflowHttpRequestActionInput } from 'src/modules/workflow/workflow-executor/workflow-actions/http-request/types/workflow-http-request-action-input.type';
|
||||
import { buildHttpRequestStepLog } from 'src/modules/workflow/workflow-executor/workflow-actions/http-request/utils/build-http-request-step-log.util';
|
||||
import { ToolBackedWorkflowAction } from 'src/modules/workflow/workflow-executor/workflow-actions/tool-backed/tool-backed.workflow-action';
|
||||
import { type WorkflowAction } from 'src/modules/workflow/workflow-executor/workflow-actions/types/workflow-action.type';
|
||||
import { WorkflowRunStepLogWorkspaceService } from 'src/modules/workflow/workflow-runner/workflow-run/workflow-run-step-log.workspace-service';
|
||||
|
||||
@Injectable()
|
||||
export class HttpRequestWorkflowAction extends ToolBackedWorkflowAction<WorkflowHttpRequestActionInput> {
|
||||
constructor(
|
||||
private readonly httpTool: HttpTool,
|
||||
workflowRunStepLogService: WorkflowRunStepLogWorkspaceService,
|
||||
) {
|
||||
super(HttpRequestWorkflowAction.name, workflowRunStepLogService);
|
||||
}
|
||||
|
||||
protected getTool(): Tool {
|
||||
return this.httpTool;
|
||||
}
|
||||
|
||||
protected assertStep(step: WorkflowAction): void {
|
||||
if (!isWorkflowHttpRequestAction(step)) {
|
||||
throw new WorkflowStepExecutorException(
|
||||
'Step is not an HTTP request action',
|
||||
WorkflowStepExecutorExceptionCode.INVALID_STEP_TYPE,
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
protected buildStepLog({
|
||||
input,
|
||||
output,
|
||||
durationMs,
|
||||
}: {
|
||||
input: WorkflowHttpRequestActionInput;
|
||||
output: ToolOutput;
|
||||
durationMs: number;
|
||||
}): WorkflowRunStepLog {
|
||||
return buildHttpRequestStepLog({ input, output, durationMs });
|
||||
}
|
||||
}
|
||||
+363
@@ -0,0 +1,363 @@
|
||||
import { type ToolOutput } from 'src/engine/core-modules/tool/types/tool-output.type';
|
||||
import { type WorkflowHttpRequestActionInput } from 'src/modules/workflow/workflow-executor/workflow-actions/http-request/types/workflow-http-request-action-input.type';
|
||||
import { buildHttpRequestStepLog } from 'src/modules/workflow/workflow-executor/workflow-actions/http-request/utils/build-http-request-step-log.util';
|
||||
|
||||
const baseInput: WorkflowHttpRequestActionInput = {
|
||||
url: 'https://api.example.com/widgets',
|
||||
method: 'POST',
|
||||
headers: {
|
||||
'content-type': 'application/json',
|
||||
Authorization: 'Bearer super-secret-token',
|
||||
},
|
||||
body: { hello: 'world' },
|
||||
};
|
||||
|
||||
const baseOutput: ToolOutput = {
|
||||
success: true,
|
||||
message: 'OK',
|
||||
result: { id: 'abc' },
|
||||
status: 201,
|
||||
statusText: 'Created',
|
||||
headers: {
|
||||
'content-type': 'application/json',
|
||||
'set-cookie': 'sid=abc; HttpOnly',
|
||||
},
|
||||
};
|
||||
|
||||
describe('buildHttpRequestStepLog', () => {
|
||||
it('redacts sensitive request and response headers', () => {
|
||||
const stepLog = buildHttpRequestStepLog({
|
||||
input: baseInput,
|
||||
output: baseOutput,
|
||||
durationMs: 42,
|
||||
});
|
||||
|
||||
if (stepLog.details.type !== 'HTTP_REQUEST') {
|
||||
throw new Error('Expected HTTP_REQUEST details');
|
||||
}
|
||||
|
||||
expect(stepLog.details.request.headers.Authorization).toBe('[redacted]');
|
||||
expect(stepLog.details.request.headers['content-type']).toBe(
|
||||
'application/json',
|
||||
);
|
||||
|
||||
expect(stepLog.details.response?.headers['set-cookie']).toBe('[redacted]');
|
||||
expect(stepLog.details.response?.headers['content-type']).toBe(
|
||||
'application/json',
|
||||
);
|
||||
});
|
||||
|
||||
it('stringifies and reports body byte size', () => {
|
||||
const stepLog = buildHttpRequestStepLog({
|
||||
input: baseInput,
|
||||
output: baseOutput,
|
||||
durationMs: 10,
|
||||
});
|
||||
|
||||
if (stepLog.details.type !== 'HTTP_REQUEST') {
|
||||
throw new Error('Expected HTTP_REQUEST details');
|
||||
}
|
||||
|
||||
expect(stepLog.details.request.body).toBe('{"hello":"world"}');
|
||||
expect(stepLog.details.request.bodyBytes).toBe(17);
|
||||
expect(stepLog.details.request.bodyTruncated).toBe(false);
|
||||
|
||||
expect(stepLog.details.response?.body).toBe('{"id":"abc"}');
|
||||
expect(stepLog.details.response?.bodyBytes).toBe(12);
|
||||
});
|
||||
|
||||
it('truncates oversized request bodies and marks bodyTruncated', () => {
|
||||
const longPayload = 'x'.repeat(100_000);
|
||||
|
||||
const stepLog = buildHttpRequestStepLog({
|
||||
input: { ...baseInput, body: { payload: longPayload } },
|
||||
output: baseOutput,
|
||||
durationMs: 10,
|
||||
});
|
||||
|
||||
if (stepLog.details.type !== 'HTTP_REQUEST') {
|
||||
throw new Error('Expected HTTP_REQUEST details');
|
||||
}
|
||||
|
||||
expect(stepLog.details.request.bodyTruncated).toBe(true);
|
||||
expect(stepLog.details.request.body).toContain('truncated');
|
||||
expect(stepLog.details.request.bodyBytes).toBeGreaterThan(100_000);
|
||||
});
|
||||
|
||||
it('truncates non-ASCII bodies by UTF-8 bytes, not UTF-16 code units', () => {
|
||||
// CJK characters take 3 UTF-8 bytes each but 1 UTF-16 code unit.
|
||||
// Before the byte-aware fix, `redacted.slice(0, 32_000)` on this payload
|
||||
// would emit ~96 KB of UTF-8 (three times the intended cap). After the
|
||||
// fix the truncated payload stays within the cap, plus at most one
|
||||
// U+FFFD replacement char (~3 bytes) for a multi-byte sequence cut at
|
||||
// the boundary.
|
||||
const longCjkPayload = '日'.repeat(40_000);
|
||||
|
||||
const stepLog = buildHttpRequestStepLog({
|
||||
input: { ...baseInput, body: { payload: longCjkPayload } },
|
||||
output: baseOutput,
|
||||
durationMs: 10,
|
||||
});
|
||||
|
||||
if (stepLog.details.type !== 'HTTP_REQUEST') {
|
||||
throw new Error('Expected HTTP_REQUEST details');
|
||||
}
|
||||
|
||||
expect(stepLog.details.request.bodyTruncated).toBe(true);
|
||||
|
||||
const truncatedBody = stepLog.details.request.body ?? '';
|
||||
const truncatedByteLength = Buffer.byteLength(
|
||||
truncatedBody.replace('…[truncated]', ''),
|
||||
'utf8',
|
||||
);
|
||||
|
||||
expect(truncatedByteLength).toBeLessThanOrEqual(32_000 + 3);
|
||||
});
|
||||
|
||||
it('omits response when the request never received one (transport error)', () => {
|
||||
const stepLog = buildHttpRequestStepLog({
|
||||
input: baseInput,
|
||||
output: {
|
||||
success: false,
|
||||
message: `HTTP POST request to ${baseInput.url} failed`,
|
||||
error: 'ENOTFOUND api.example.com',
|
||||
},
|
||||
durationMs: 30,
|
||||
});
|
||||
|
||||
if (stepLog.details.type !== 'HTTP_REQUEST') {
|
||||
throw new Error('Expected HTTP_REQUEST details');
|
||||
}
|
||||
|
||||
expect(stepLog.details.response).toBeUndefined();
|
||||
expect(stepLog.details.error).toBe('ENOTFOUND api.example.com');
|
||||
});
|
||||
|
||||
it('captures response details when an HTTP error response is returned', () => {
|
||||
const stepLog = buildHttpRequestStepLog({
|
||||
input: baseInput,
|
||||
output: {
|
||||
success: false,
|
||||
message: 'failed',
|
||||
error: '{"code":"invalid"}',
|
||||
status: 422,
|
||||
statusText: 'Unprocessable Entity',
|
||||
headers: { 'content-type': 'application/json' },
|
||||
result: { code: 'invalid' },
|
||||
},
|
||||
durationMs: 55,
|
||||
});
|
||||
|
||||
if (stepLog.details.type !== 'HTTP_REQUEST') {
|
||||
throw new Error('Expected HTTP_REQUEST details');
|
||||
}
|
||||
|
||||
expect(stepLog.details.response?.status).toBe(422);
|
||||
expect(stepLog.details.response?.statusText).toBe('Unprocessable Entity');
|
||||
expect(stepLog.details.response?.body).toBe('{"code":"invalid"}');
|
||||
expect(stepLog.details.error).toBe('{"code":"invalid"}');
|
||||
});
|
||||
|
||||
it('redacts sensitive query-string parameters in the request URL', () => {
|
||||
const stepLog = buildHttpRequestStepLog({
|
||||
input: {
|
||||
...baseInput,
|
||||
url: 'https://api.example.com/data?page=1&api_key=AKIA-leaked&token=oauth-leaked&safe=ok',
|
||||
},
|
||||
output: baseOutput,
|
||||
durationMs: 1,
|
||||
});
|
||||
|
||||
if (stepLog.details.type !== 'HTTP_REQUEST') {
|
||||
throw new Error('Expected HTTP_REQUEST details');
|
||||
}
|
||||
|
||||
const url = new URL(stepLog.details.request.url);
|
||||
|
||||
expect(url.searchParams.get('api_key')).toBe('[redacted]');
|
||||
expect(url.searchParams.get('token')).toBe('[redacted]');
|
||||
expect(url.searchParams.get('page')).toBe('1');
|
||||
expect(url.searchParams.get('safe')).toBe('ok');
|
||||
});
|
||||
|
||||
it('leaves the URL untouched when there are no sensitive params', () => {
|
||||
const stepLog = buildHttpRequestStepLog({
|
||||
input: { ...baseInput, url: 'https://api.example.com/widgets?page=2' },
|
||||
output: baseOutput,
|
||||
durationMs: 1,
|
||||
});
|
||||
|
||||
if (stepLog.details.type !== 'HTTP_REQUEST') {
|
||||
throw new Error('Expected HTTP_REQUEST details');
|
||||
}
|
||||
|
||||
expect(stepLog.details.request.url).toBe(
|
||||
'https://api.example.com/widgets?page=2',
|
||||
);
|
||||
});
|
||||
|
||||
it('redacts sensitive query params even when the URL is unparseable (regression)', () => {
|
||||
// Whitespace in the host trips up the WHATWG URL parser, so this URL
|
||||
// is rejected by `new URL()`. Before the fallback was added, the catch
|
||||
// branch returned the raw URL with secrets intact.
|
||||
const unparseableUrl =
|
||||
'https://api example.com/data?page=1&api_key=AKIA-leaked&token=oauth-leaked&safe=ok';
|
||||
|
||||
expect(() => new URL(unparseableUrl)).toThrow();
|
||||
|
||||
const stepLog = buildHttpRequestStepLog({
|
||||
input: { ...baseInput, url: unparseableUrl },
|
||||
output: baseOutput,
|
||||
durationMs: 1,
|
||||
});
|
||||
|
||||
if (stepLog.details.type !== 'HTTP_REQUEST') {
|
||||
throw new Error('Expected HTTP_REQUEST details');
|
||||
}
|
||||
|
||||
const redactedUrl = stepLog.details.request.url;
|
||||
|
||||
expect(redactedUrl).not.toContain('AKIA-leaked');
|
||||
expect(redactedUrl).not.toContain('oauth-leaked');
|
||||
expect(redactedUrl).toContain('api_key=[redacted]');
|
||||
expect(redactedUrl).toContain('token=[redacted]');
|
||||
expect(redactedUrl).toContain('page=1');
|
||||
expect(redactedUrl).toContain('safe=ok');
|
||||
});
|
||||
|
||||
it('redacts percent-encoded sensitive param names in unparseable URLs', () => {
|
||||
// `api%5Fkey` decodes to `api_key` — the fallback must decode before
|
||||
// matching against the sensitive-name set.
|
||||
const unparseableUrl = 'https://api example.com/x?api%5Fkey=leaked';
|
||||
|
||||
expect(() => new URL(unparseableUrl)).toThrow();
|
||||
|
||||
const stepLog = buildHttpRequestStepLog({
|
||||
input: { ...baseInput, url: unparseableUrl },
|
||||
output: baseOutput,
|
||||
durationMs: 1,
|
||||
});
|
||||
|
||||
if (stepLog.details.type !== 'HTTP_REQUEST') {
|
||||
throw new Error('Expected HTTP_REQUEST details');
|
||||
}
|
||||
|
||||
expect(stepLog.details.request.url).not.toContain('leaked');
|
||||
expect(stepLog.details.request.url).toContain('api%5Fkey=[redacted]');
|
||||
});
|
||||
|
||||
it('redacts sensitive keys in JSON request bodies (deep)', () => {
|
||||
const stepLog = buildHttpRequestStepLog({
|
||||
input: {
|
||||
...baseInput,
|
||||
body: {
|
||||
username: 'alice',
|
||||
password: 'hunter2',
|
||||
credentials: { client_secret: 'oauth-secret', clientId: 'public' },
|
||||
tokens: [{ access_token: 'aaa', issuedAt: 1 }],
|
||||
} as unknown as WorkflowHttpRequestActionInput['body'],
|
||||
},
|
||||
output: baseOutput,
|
||||
durationMs: 1,
|
||||
});
|
||||
|
||||
if (stepLog.details.type !== 'HTTP_REQUEST') {
|
||||
throw new Error('Expected HTTP_REQUEST details');
|
||||
}
|
||||
|
||||
const parsed = JSON.parse(stepLog.details.request.body ?? '{}');
|
||||
|
||||
expect(parsed.username).toBe('alice');
|
||||
expect(parsed.password).toBe('[redacted]');
|
||||
expect(parsed.credentials.client_secret).toBe('[redacted]');
|
||||
expect(parsed.credentials.clientId).toBe('public');
|
||||
expect(parsed.tokens[0].access_token).toBe('[redacted]');
|
||||
expect(parsed.tokens[0].issuedAt).toBe(1);
|
||||
});
|
||||
|
||||
it('redacts sensitive keys in JSON response bodies (OAuth token endpoint shape)', () => {
|
||||
const stepLog = buildHttpRequestStepLog({
|
||||
input: baseInput,
|
||||
output: {
|
||||
...baseOutput,
|
||||
result: {
|
||||
token_type: 'Bearer',
|
||||
access_token: 'leaked-access',
|
||||
refresh_token: 'leaked-refresh',
|
||||
expires_in: 3600,
|
||||
},
|
||||
},
|
||||
durationMs: 1,
|
||||
});
|
||||
|
||||
if (stepLog.details.type !== 'HTTP_REQUEST') {
|
||||
throw new Error('Expected HTTP_REQUEST details');
|
||||
}
|
||||
|
||||
const parsed = JSON.parse(stepLog.details.response?.body ?? '{}');
|
||||
|
||||
expect(parsed.access_token).toBe('[redacted]');
|
||||
expect(parsed.refresh_token).toBe('[redacted]');
|
||||
expect(parsed.token_type).toBe('Bearer');
|
||||
expect(parsed.expires_in).toBe(3600);
|
||||
});
|
||||
|
||||
it('redacts sensitive keys when a request body arrives as a JSON string', () => {
|
||||
const stepLog = buildHttpRequestStepLog({
|
||||
input: {
|
||||
...baseInput,
|
||||
body: '{"username":"alice","password":"hunter2"}',
|
||||
},
|
||||
output: baseOutput,
|
||||
durationMs: 1,
|
||||
});
|
||||
|
||||
if (stepLog.details.type !== 'HTTP_REQUEST') {
|
||||
throw new Error('Expected HTTP_REQUEST details');
|
||||
}
|
||||
|
||||
const parsed = JSON.parse(stepLog.details.request.body ?? '{}');
|
||||
|
||||
expect(parsed.password).toBe('[redacted]');
|
||||
expect(parsed.username).toBe('alice');
|
||||
});
|
||||
|
||||
it('leaves non-JSON string bodies untouched (e.g. form-encoded)', () => {
|
||||
const stepLog = buildHttpRequestStepLog({
|
||||
input: { ...baseInput, body: 'name=alice&topic=hello' },
|
||||
output: baseOutput,
|
||||
durationMs: 1,
|
||||
});
|
||||
|
||||
if (stepLog.details.type !== 'HTTP_REQUEST') {
|
||||
throw new Error('Expected HTTP_REQUEST details');
|
||||
}
|
||||
|
||||
expect(stepLog.details.request.body).toBe('name=alice&topic=hello');
|
||||
});
|
||||
|
||||
it('redacts sensitive keys when output.error is a structured object', () => {
|
||||
const stepLog = buildHttpRequestStepLog({
|
||||
input: baseInput,
|
||||
output: {
|
||||
success: false,
|
||||
message: 'failed',
|
||||
error: {
|
||||
code: 'invalid_client',
|
||||
client_secret: 'leaked',
|
||||
} as unknown as string,
|
||||
status: 401,
|
||||
},
|
||||
durationMs: 1,
|
||||
});
|
||||
|
||||
if (stepLog.details.type !== 'HTTP_REQUEST') {
|
||||
throw new Error('Expected HTTP_REQUEST details');
|
||||
}
|
||||
|
||||
const parsed = JSON.parse(stepLog.details.error ?? '{}');
|
||||
|
||||
expect(parsed.client_secret).toBe('[redacted]');
|
||||
expect(parsed.code).toBe('invalid_client');
|
||||
});
|
||||
});
|
||||
+276
@@ -0,0 +1,276 @@
|
||||
import { type WorkflowRunStepLog } from 'twenty-shared/workflow';
|
||||
|
||||
import { type ToolOutput } from 'src/engine/core-modules/tool/types/tool-output.type';
|
||||
import { type WorkflowHttpRequestActionInput } from 'src/modules/workflow/workflow-executor/workflow-actions/http-request/types/workflow-http-request-action-input.type';
|
||||
import { truncateStringToUtf8ByteBudget } from 'src/utils/truncate-string-to-utf8-byte-budget.util';
|
||||
|
||||
const MAX_BODY_BYTES = 32_000;
|
||||
const REDACTION_SENTINEL = '[redacted]';
|
||||
|
||||
const SENSITIVE_HEADER_NAMES = new Set([
|
||||
'authorization',
|
||||
'proxy-authorization',
|
||||
'cookie',
|
||||
'set-cookie',
|
||||
'x-api-key',
|
||||
'x-auth-token',
|
||||
'x-csrf-token',
|
||||
'x-amz-security-token',
|
||||
'x-goog-api-key',
|
||||
'api-key',
|
||||
]);
|
||||
|
||||
const SENSITIVE_URL_PARAM_NAMES = new Set([
|
||||
'api_key',
|
||||
'apikey',
|
||||
'api-key',
|
||||
'token',
|
||||
'access_token',
|
||||
'refresh_token',
|
||||
'id_token',
|
||||
'auth',
|
||||
'auth_token',
|
||||
'authentication',
|
||||
'secret',
|
||||
'client_secret',
|
||||
'private_key',
|
||||
'key',
|
||||
'sig',
|
||||
'signature',
|
||||
'password',
|
||||
'passwd',
|
||||
'pwd',
|
||||
]);
|
||||
|
||||
const SENSITIVE_BODY_KEY_REGEX =
|
||||
/^(password|passwd|pwd|.*_?token|.*_?secret|authorization|api[_-]?key|private[_-]?key|client[_-]?secret|x-?api-?key|x-?auth-?token|access[_-]?key)$/i;
|
||||
|
||||
const isSensitiveHeader = (name: string): boolean =>
|
||||
SENSITIVE_HEADER_NAMES.has(name.toLowerCase());
|
||||
|
||||
const isSensitiveUrlParam = (name: string): boolean =>
|
||||
SENSITIVE_URL_PARAM_NAMES.has(name.toLowerCase());
|
||||
|
||||
const isSensitiveBodyKey = (name: string): boolean =>
|
||||
SENSITIVE_BODY_KEY_REGEX.test(name);
|
||||
|
||||
const redactHeaders = (
|
||||
headers: Record<string, unknown> | undefined,
|
||||
): Record<string, string> => {
|
||||
if (!headers) {
|
||||
return {};
|
||||
}
|
||||
|
||||
const redacted: Record<string, string> = {};
|
||||
|
||||
for (const [name, value] of Object.entries(headers)) {
|
||||
if (isSensitiveHeader(name)) {
|
||||
redacted[name] = REDACTION_SENTINEL;
|
||||
continue;
|
||||
}
|
||||
|
||||
redacted[name] =
|
||||
typeof value === 'string' ? value : (JSON.stringify(value) ?? '');
|
||||
}
|
||||
|
||||
return redacted;
|
||||
};
|
||||
|
||||
const URL_QUERY_PARAM_REGEX = /([?&])([^=&#]+)=([^&#]*)/g;
|
||||
|
||||
const safeDecodeUriComponent = (value: string): string => {
|
||||
try {
|
||||
return decodeURIComponent(value);
|
||||
} catch {
|
||||
return value;
|
||||
}
|
||||
};
|
||||
|
||||
const redactUrlQueryStringFallback = (rawUrl: string): string =>
|
||||
rawUrl.replace(URL_QUERY_PARAM_REGEX, (match, prefix, name) => {
|
||||
if (isSensitiveUrlParam(safeDecodeUriComponent(name))) {
|
||||
return `${prefix}${name}=${REDACTION_SENTINEL}`;
|
||||
}
|
||||
|
||||
return match;
|
||||
});
|
||||
|
||||
const redactUrl = (rawUrl: string): string => {
|
||||
let parsed: URL;
|
||||
|
||||
try {
|
||||
parsed = new URL(rawUrl);
|
||||
} catch {
|
||||
return redactUrlQueryStringFallback(rawUrl);
|
||||
}
|
||||
|
||||
let didRedact = false;
|
||||
|
||||
for (const paramName of [...parsed.searchParams.keys()]) {
|
||||
if (isSensitiveUrlParam(paramName)) {
|
||||
parsed.searchParams.set(paramName, REDACTION_SENTINEL);
|
||||
didRedact = true;
|
||||
}
|
||||
}
|
||||
|
||||
return didRedact ? parsed.toString() : rawUrl;
|
||||
};
|
||||
|
||||
const redactSensitiveBodyKeysDeep = (value: unknown): unknown => {
|
||||
if (Array.isArray(value)) {
|
||||
return value.map(redactSensitiveBodyKeysDeep);
|
||||
}
|
||||
|
||||
if (value !== null && typeof value === 'object') {
|
||||
const redacted: Record<string, unknown> = {};
|
||||
|
||||
for (const [key, nested] of Object.entries(value)) {
|
||||
if (isSensitiveBodyKey(key)) {
|
||||
redacted[key] = REDACTION_SENTINEL;
|
||||
continue;
|
||||
}
|
||||
|
||||
redacted[key] = redactSensitiveBodyKeysDeep(nested);
|
||||
}
|
||||
|
||||
return redacted;
|
||||
}
|
||||
|
||||
return value;
|
||||
};
|
||||
|
||||
const redactJsonBodyString = (body: string): string => {
|
||||
if (body.length === 0) {
|
||||
return body;
|
||||
}
|
||||
|
||||
let parsed: unknown;
|
||||
|
||||
try {
|
||||
parsed = JSON.parse(body);
|
||||
} catch {
|
||||
return body;
|
||||
}
|
||||
|
||||
if (parsed === null || typeof parsed !== 'object') {
|
||||
return body;
|
||||
}
|
||||
|
||||
const redactedTree = redactSensitiveBodyKeysDeep(parsed);
|
||||
|
||||
try {
|
||||
return JSON.stringify(redactedTree);
|
||||
} catch {
|
||||
return body;
|
||||
}
|
||||
};
|
||||
|
||||
const redactErrorValue = (error: unknown): string | undefined => {
|
||||
if (error === undefined || error === null) {
|
||||
return undefined;
|
||||
}
|
||||
|
||||
if (typeof error === 'string') {
|
||||
return redactJsonBodyString(error);
|
||||
}
|
||||
|
||||
try {
|
||||
return JSON.stringify(redactSensitiveBodyKeysDeep(error));
|
||||
} catch {
|
||||
return String(error);
|
||||
}
|
||||
};
|
||||
|
||||
const stringifyBody = (body: unknown): string | undefined => {
|
||||
if (body === undefined || body === null) {
|
||||
return undefined;
|
||||
}
|
||||
|
||||
if (typeof body === 'string') {
|
||||
return body;
|
||||
}
|
||||
|
||||
if (typeof body === 'object') {
|
||||
try {
|
||||
const redactedTree = redactSensitiveBodyKeysDeep(body);
|
||||
|
||||
return JSON.stringify(redactedTree);
|
||||
} catch {
|
||||
return String(body);
|
||||
}
|
||||
}
|
||||
|
||||
return String(body);
|
||||
};
|
||||
|
||||
type SerializedBody = {
|
||||
body: string | undefined;
|
||||
bodyBytes: number | undefined;
|
||||
bodyTruncated: boolean;
|
||||
};
|
||||
|
||||
const serializeBody = (body: unknown): SerializedBody => {
|
||||
const serialized = stringifyBody(body);
|
||||
|
||||
if (serialized === undefined) {
|
||||
return { body: undefined, bodyBytes: undefined, bodyTruncated: false };
|
||||
}
|
||||
|
||||
const redacted =
|
||||
typeof body === 'string' ? redactJsonBodyString(serialized) : serialized;
|
||||
|
||||
const { value, originalBytes, truncated } = truncateStringToUtf8ByteBudget(
|
||||
redacted,
|
||||
MAX_BODY_BYTES,
|
||||
);
|
||||
|
||||
return { body: value, bodyBytes: originalBytes, bodyTruncated: truncated };
|
||||
};
|
||||
|
||||
export const buildHttpRequestStepLog = ({
|
||||
input,
|
||||
output,
|
||||
durationMs,
|
||||
}: {
|
||||
input: WorkflowHttpRequestActionInput;
|
||||
output: ToolOutput;
|
||||
durationMs: number;
|
||||
}): WorkflowRunStepLog => {
|
||||
const requestBody = serializeBody(input.body);
|
||||
const responseBody = serializeBody(output.result);
|
||||
|
||||
const hasResponseMetadata =
|
||||
typeof output.status === 'number' ||
|
||||
(output.headers !== undefined && Object.keys(output.headers).length > 0) ||
|
||||
responseBody.body !== undefined;
|
||||
|
||||
const response = hasResponseMetadata
|
||||
? {
|
||||
status: output.status ?? 0,
|
||||
statusText: output.statusText,
|
||||
headers: redactHeaders(output.headers),
|
||||
body: responseBody.body,
|
||||
bodyBytes: responseBody.bodyBytes,
|
||||
bodyTruncated: responseBody.bodyTruncated,
|
||||
}
|
||||
: undefined;
|
||||
|
||||
return {
|
||||
details: {
|
||||
type: 'HTTP_REQUEST',
|
||||
request: {
|
||||
method: input.method,
|
||||
url: redactUrl(input.url),
|
||||
headers: redactHeaders(input.headers),
|
||||
body: requestBody.body,
|
||||
bodyBytes: requestBody.bodyBytes,
|
||||
bodyTruncated: requestBody.bodyTruncated,
|
||||
},
|
||||
response,
|
||||
error: redactErrorValue(output.error),
|
||||
durationMs,
|
||||
},
|
||||
entries: [],
|
||||
sizeBytes: 0,
|
||||
};
|
||||
};
|
||||
+139
@@ -0,0 +1,139 @@
|
||||
import { Test, type TestingModule } from '@nestjs/testing';
|
||||
|
||||
import { DraftEmailTool } from 'src/engine/core-modules/tool/tools/email-tool/draft-email-tool';
|
||||
import { DraftEmailWorkflowAction } from 'src/modules/workflow/workflow-executor/workflow-actions/mail-sender/draft-email.workflow-action';
|
||||
import { type WorkflowActionSettings } from 'src/modules/workflow/workflow-executor/workflow-actions/types/workflow-action-settings.type';
|
||||
import {
|
||||
type WorkflowAction,
|
||||
WorkflowActionType,
|
||||
} from 'src/modules/workflow/workflow-executor/workflow-actions/types/workflow-action.type';
|
||||
import { WorkflowRunStepLogWorkspaceService } from 'src/modules/workflow/workflow-runner/workflow-run/workflow-run-step-log.workspace-service';
|
||||
|
||||
jest.mock(
|
||||
'src/engine/core-modules/tool/tools/email-tool/utils/render-rich-text-to-html.util',
|
||||
() => ({
|
||||
renderRichTextToHtml: jest.fn().mockResolvedValue('<p>rendered html</p>'),
|
||||
}),
|
||||
);
|
||||
|
||||
const baseSettings: WorkflowActionSettings = {
|
||||
outputSchema: {},
|
||||
errorHandlingOptions: {
|
||||
retryOnFailure: { value: false },
|
||||
continueOnFailure: { value: false },
|
||||
},
|
||||
input: {},
|
||||
};
|
||||
|
||||
const buildDraftEmailStep = (input: Record<string, unknown>): WorkflowAction =>
|
||||
({
|
||||
id: 'step-1',
|
||||
type: WorkflowActionType.DRAFT_EMAIL,
|
||||
name: 'Draft Email',
|
||||
valid: true,
|
||||
settings: { ...baseSettings, input },
|
||||
}) as WorkflowAction;
|
||||
|
||||
describe('DraftEmailWorkflowAction', () => {
|
||||
let action: DraftEmailWorkflowAction;
|
||||
let mockDraftEmailTool: jest.Mocked<Pick<DraftEmailTool, 'execute'>>;
|
||||
let mockSetStepLog: jest.Mock;
|
||||
|
||||
beforeEach(async () => {
|
||||
jest.clearAllMocks();
|
||||
|
||||
mockDraftEmailTool = {
|
||||
execute: jest.fn().mockResolvedValue({
|
||||
result: { success: true },
|
||||
error: undefined,
|
||||
}),
|
||||
};
|
||||
mockSetStepLog = jest.fn();
|
||||
|
||||
const module: TestingModule = await Test.createTestingModule({
|
||||
providers: [
|
||||
DraftEmailWorkflowAction,
|
||||
{ provide: DraftEmailTool, useValue: mockDraftEmailTool },
|
||||
{
|
||||
provide: WorkflowRunStepLogWorkspaceService,
|
||||
useValue: { setStepLog: mockSetStepLog },
|
||||
},
|
||||
],
|
||||
}).compile();
|
||||
|
||||
action = module.get(DraftEmailWorkflowAction);
|
||||
});
|
||||
|
||||
it('runs the draft email tool and resolves variables in the body', async () => {
|
||||
await action.execute({
|
||||
currentStepId: 'step-1',
|
||||
steps: [
|
||||
buildDraftEmailStep({
|
||||
connectedAccountId: 'account-1',
|
||||
recipients: { to: 'test@example.com' },
|
||||
subject: 'Draft Test',
|
||||
body: '{{trigger.name}}',
|
||||
}),
|
||||
],
|
||||
context: { trigger: { name: 'John' } },
|
||||
runInfo: { workspaceId: 'workspace-1', workflowRunId: 'run-1' },
|
||||
});
|
||||
|
||||
expect(mockDraftEmailTool.execute).toHaveBeenCalledWith(
|
||||
expect.objectContaining({ body: 'John' }),
|
||||
expect.objectContaining({ workspaceId: 'workspace-1' }),
|
||||
);
|
||||
});
|
||||
|
||||
it('persists a step log tagged with the DRAFT mode', async () => {
|
||||
await action.execute({
|
||||
currentStepId: 'step-1',
|
||||
steps: [
|
||||
buildDraftEmailStep({
|
||||
connectedAccountId: 'account-1',
|
||||
recipients: { to: 'test@example.com' },
|
||||
subject: 'Draft Test',
|
||||
body: 'hello',
|
||||
}),
|
||||
],
|
||||
context: {},
|
||||
runInfo: { workspaceId: 'workspace-1', workflowRunId: 'run-1' },
|
||||
});
|
||||
|
||||
expect(mockSetStepLog).toHaveBeenCalledWith(
|
||||
expect.objectContaining({
|
||||
workflowRunId: 'run-1',
|
||||
workspaceId: 'workspace-1',
|
||||
stepId: 'step-1',
|
||||
stepLog: expect.objectContaining({
|
||||
details: expect.objectContaining({
|
||||
type: 'EMAIL',
|
||||
mode: 'DRAFT',
|
||||
}),
|
||||
}),
|
||||
}),
|
||||
);
|
||||
});
|
||||
|
||||
it('throws when the current step is not a draft-email action', async () => {
|
||||
await expect(
|
||||
action.execute({
|
||||
currentStepId: 'step-1',
|
||||
steps: [
|
||||
{
|
||||
...buildDraftEmailStep({
|
||||
connectedAccountId: 'account-1',
|
||||
recipients: { to: 'test@example.com' },
|
||||
subject: 'Wrong type',
|
||||
}),
|
||||
type: WorkflowActionType.SEND_EMAIL,
|
||||
} as WorkflowAction,
|
||||
],
|
||||
context: {},
|
||||
runInfo: { workspaceId: 'workspace-1', workflowRunId: 'run-1' },
|
||||
}),
|
||||
).rejects.toThrow('Step is not a draft-email action');
|
||||
|
||||
expect(mockDraftEmailTool.execute).not.toHaveBeenCalled();
|
||||
});
|
||||
});
|
||||
+37
-34
@@ -1,14 +1,13 @@
|
||||
import { Test, type TestingModule } from '@nestjs/testing';
|
||||
|
||||
import { DraftEmailTool } from 'src/engine/core-modules/tool/tools/email-tool/draft-email-tool';
|
||||
import { SendEmailTool } from 'src/engine/core-modules/tool/tools/email-tool/send-email-tool';
|
||||
import { HttpTool } from 'src/engine/core-modules/tool/tools/http-tool/http-tool';
|
||||
import { ToolExecutorWorkflowAction } from 'src/modules/workflow/workflow-executor/workflow-actions/tool-executor-workflow-action';
|
||||
import { SendEmailWorkflowAction } from 'src/modules/workflow/workflow-executor/workflow-actions/mail-sender/send-email.workflow-action';
|
||||
import { type WorkflowActionSettings } from 'src/modules/workflow/workflow-executor/workflow-actions/types/workflow-action-settings.type';
|
||||
import {
|
||||
type WorkflowAction,
|
||||
WorkflowActionType,
|
||||
} from 'src/modules/workflow/workflow-executor/workflow-actions/types/workflow-action.type';
|
||||
import { WorkflowRunStepLogWorkspaceService } from 'src/modules/workflow/workflow-runner/workflow-run/workflow-run-step-log.workspace-service';
|
||||
|
||||
jest.mock(
|
||||
'src/engine/core-modules/tool/tools/email-tool/utils/render-rich-text-to-html.util',
|
||||
@@ -36,53 +35,47 @@ const emailInput = {
|
||||
subject: 'Test',
|
||||
};
|
||||
|
||||
const buildEmailStep = (
|
||||
type: 'SEND_EMAIL' | 'DRAFT_EMAIL',
|
||||
input: Record<string, unknown>,
|
||||
): WorkflowAction =>
|
||||
const buildSendEmailStep = (input: Record<string, unknown>): WorkflowAction =>
|
||||
({
|
||||
id: 'step-1',
|
||||
type: WorkflowActionType[type],
|
||||
name: type === 'SEND_EMAIL' ? 'Send Email' : 'Draft Email',
|
||||
type: WorkflowActionType.SEND_EMAIL,
|
||||
name: 'Send Email',
|
||||
valid: true,
|
||||
settings: { ...baseSettings, input },
|
||||
}) as WorkflowAction;
|
||||
|
||||
describe('ToolExecutorWorkflowAction', () => {
|
||||
let action: ToolExecutorWorkflowAction;
|
||||
describe('SendEmailWorkflowAction', () => {
|
||||
let action: SendEmailWorkflowAction;
|
||||
let mockSendEmailTool: jest.Mocked<Pick<SendEmailTool, 'execute'>>;
|
||||
let mockDraftEmailTool: jest.Mocked<Pick<DraftEmailTool, 'execute'>>;
|
||||
|
||||
beforeEach(async () => {
|
||||
jest.clearAllMocks();
|
||||
|
||||
const toolResult = {
|
||||
result: { success: true },
|
||||
error: undefined,
|
||||
mockSendEmailTool = {
|
||||
execute: jest.fn().mockResolvedValue({
|
||||
result: { success: true },
|
||||
error: undefined,
|
||||
}),
|
||||
};
|
||||
|
||||
mockSendEmailTool = { execute: jest.fn().mockResolvedValue(toolResult) };
|
||||
mockDraftEmailTool = { execute: jest.fn().mockResolvedValue(toolResult) };
|
||||
|
||||
const module: TestingModule = await Test.createTestingModule({
|
||||
providers: [
|
||||
ToolExecutorWorkflowAction,
|
||||
{ provide: HttpTool, useValue: { execute: jest.fn() } },
|
||||
SendEmailWorkflowAction,
|
||||
{ provide: SendEmailTool, useValue: mockSendEmailTool },
|
||||
{ provide: DraftEmailTool, useValue: mockDraftEmailTool },
|
||||
{
|
||||
provide: WorkflowRunStepLogWorkspaceService,
|
||||
useValue: { setStepLog: jest.fn() },
|
||||
},
|
||||
],
|
||||
}).compile();
|
||||
|
||||
action = module.get(ToolExecutorWorkflowAction);
|
||||
action = module.get(SendEmailWorkflowAction);
|
||||
});
|
||||
|
||||
const executeWithBody = (
|
||||
body: string | undefined,
|
||||
type: 'SEND_EMAIL' | 'DRAFT_EMAIL' = 'SEND_EMAIL',
|
||||
) =>
|
||||
const executeWithBody = (body: string | undefined) =>
|
||||
action.execute({
|
||||
currentStepId: 'step-1',
|
||||
steps: [buildEmailStep(type, { ...emailInput, body })],
|
||||
steps: [buildSendEmailStep({ ...emailInput, body })],
|
||||
context: {
|
||||
trigger: {
|
||||
name: 'John',
|
||||
@@ -178,15 +171,25 @@ describe('ToolExecutorWorkflowAction', () => {
|
||||
expect(renderRichTextToHtml).not.toHaveBeenCalled();
|
||||
expect(mockSendEmailTool.execute).toHaveBeenCalled();
|
||||
});
|
||||
});
|
||||
|
||||
it('should apply the same body handling for DRAFT_EMAIL', async () => {
|
||||
await executeWithBody('{{trigger.name}}', 'DRAFT_EMAIL');
|
||||
describe('step type guard', () => {
|
||||
it('throws when the current step is not a send-email action', async () => {
|
||||
await expect(
|
||||
action.execute({
|
||||
currentStepId: 'step-1',
|
||||
steps: [
|
||||
{
|
||||
...buildSendEmailStep({ ...emailInput, body: 'hi' }),
|
||||
type: WorkflowActionType.DRAFT_EMAIL,
|
||||
} as WorkflowAction,
|
||||
],
|
||||
context: {},
|
||||
runInfo: { workspaceId: 'workspace-1', workflowRunId: 'run-1' },
|
||||
}),
|
||||
).rejects.toThrow('Step is not a send-email action');
|
||||
|
||||
expect(renderRichTextToHtml).not.toHaveBeenCalled();
|
||||
expect(mockDraftEmailTool.execute).toHaveBeenCalledWith(
|
||||
expect.objectContaining({ body: 'John' }),
|
||||
expect.any(Object),
|
||||
);
|
||||
expect(mockSendEmailTool.execute).not.toHaveBeenCalled();
|
||||
});
|
||||
});
|
||||
});
|
||||
+40
@@ -0,0 +1,40 @@
|
||||
import { Injectable } from '@nestjs/common';
|
||||
|
||||
import { DraftEmailTool } from 'src/engine/core-modules/tool/tools/email-tool/draft-email-tool';
|
||||
import { type Tool } from 'src/engine/core-modules/tool/types/tool.type';
|
||||
import {
|
||||
WorkflowStepExecutorException,
|
||||
WorkflowStepExecutorExceptionCode,
|
||||
} from 'src/modules/workflow/workflow-executor/exceptions/workflow-step-executor.exception';
|
||||
import { EmailWorkflowActionBase } from 'src/modules/workflow/workflow-executor/workflow-actions/mail-sender/email-workflow-action.base';
|
||||
import { isWorkflowDraftEmailAction } from 'src/modules/workflow/workflow-executor/workflow-actions/mail-sender/guards/is-workflow-draft-email-action.guard';
|
||||
import { type EmailStepLogMode } from 'src/modules/workflow/workflow-executor/workflow-actions/mail-sender/utils/build-email-step-log.util';
|
||||
import { type WorkflowAction } from 'src/modules/workflow/workflow-executor/workflow-actions/types/workflow-action.type';
|
||||
import { WorkflowRunStepLogWorkspaceService } from 'src/modules/workflow/workflow-runner/workflow-run/workflow-run-step-log.workspace-service';
|
||||
|
||||
@Injectable()
|
||||
export class DraftEmailWorkflowAction extends EmailWorkflowActionBase {
|
||||
constructor(
|
||||
private readonly draftEmailTool: DraftEmailTool,
|
||||
workflowRunStepLogService: WorkflowRunStepLogWorkspaceService,
|
||||
) {
|
||||
super(DraftEmailWorkflowAction.name, workflowRunStepLogService);
|
||||
}
|
||||
|
||||
protected getTool(): Tool {
|
||||
return this.draftEmailTool;
|
||||
}
|
||||
|
||||
protected getMode(): EmailStepLogMode {
|
||||
return 'DRAFT';
|
||||
}
|
||||
|
||||
protected assertStep(step: WorkflowAction): void {
|
||||
if (!isWorkflowDraftEmailAction(step)) {
|
||||
throw new WorkflowStepExecutorException(
|
||||
'Step is not a draft-email action',
|
||||
WorkflowStepExecutorExceptionCode.INVALID_STEP_TYPE,
|
||||
);
|
||||
}
|
||||
}
|
||||
}
|
||||
+46
@@ -0,0 +1,46 @@
|
||||
import { type WorkflowRunStepLog } from 'twenty-shared/workflow';
|
||||
|
||||
import { isDefined } from 'twenty-shared/utils';
|
||||
|
||||
import { type ToolOutput } from 'src/engine/core-modules/tool/types/tool-output.type';
|
||||
import { type WorkflowSendEmailActionInput } from 'src/modules/workflow/workflow-executor/workflow-actions/mail-sender/types/workflow-send-email-action-input.type';
|
||||
import {
|
||||
buildEmailStepLog,
|
||||
type EmailStepLogMode,
|
||||
} from 'src/modules/workflow/workflow-executor/workflow-actions/mail-sender/utils/build-email-step-log.util';
|
||||
import { resolveEmailBody } from 'src/modules/workflow/workflow-executor/workflow-actions/mail-sender/utils/resolve-email-body.util';
|
||||
import { ToolBackedWorkflowAction } from 'src/modules/workflow/workflow-executor/workflow-actions/tool-backed/tool-backed.workflow-action';
|
||||
|
||||
export abstract class EmailWorkflowActionBase extends ToolBackedWorkflowAction<WorkflowSendEmailActionInput> {
|
||||
protected abstract getMode(): EmailStepLogMode;
|
||||
|
||||
protected override async preprocessInput(
|
||||
rawInput: WorkflowSendEmailActionInput,
|
||||
context: Record<string, unknown>,
|
||||
): Promise<WorkflowSendEmailActionInput> {
|
||||
if (!isDefined(rawInput.body)) {
|
||||
return rawInput;
|
||||
}
|
||||
|
||||
const renderedBody = await resolveEmailBody(rawInput.body, context);
|
||||
|
||||
return { ...rawInput, body: renderedBody };
|
||||
}
|
||||
|
||||
protected buildStepLog({
|
||||
input,
|
||||
output,
|
||||
durationMs,
|
||||
}: {
|
||||
input: WorkflowSendEmailActionInput;
|
||||
output: ToolOutput;
|
||||
durationMs: number;
|
||||
}): WorkflowRunStepLog {
|
||||
return buildEmailStepLog({
|
||||
mode: this.getMode(),
|
||||
input,
|
||||
output,
|
||||
durationMs,
|
||||
});
|
||||
}
|
||||
}
|
||||
+13
@@ -0,0 +1,13 @@
|
||||
import { Module } from '@nestjs/common';
|
||||
|
||||
import { ToolModule } from 'src/engine/core-modules/tool/tool.module';
|
||||
import { DraftEmailWorkflowAction } from 'src/modules/workflow/workflow-executor/workflow-actions/mail-sender/draft-email.workflow-action';
|
||||
import { SendEmailWorkflowAction } from 'src/modules/workflow/workflow-executor/workflow-actions/mail-sender/send-email.workflow-action';
|
||||
import { WorkflowRunModule } from 'src/modules/workflow/workflow-runner/workflow-run/workflow-run.module';
|
||||
|
||||
@Module({
|
||||
imports: [ToolModule, WorkflowRunModule],
|
||||
providers: [SendEmailWorkflowAction, DraftEmailWorkflowAction],
|
||||
exports: [SendEmailWorkflowAction, DraftEmailWorkflowAction],
|
||||
})
|
||||
export class MailSenderActionModule {}
|
||||
+40
@@ -0,0 +1,40 @@
|
||||
import { Injectable } from '@nestjs/common';
|
||||
|
||||
import { SendEmailTool } from 'src/engine/core-modules/tool/tools/email-tool/send-email-tool';
|
||||
import { type Tool } from 'src/engine/core-modules/tool/types/tool.type';
|
||||
import {
|
||||
WorkflowStepExecutorException,
|
||||
WorkflowStepExecutorExceptionCode,
|
||||
} from 'src/modules/workflow/workflow-executor/exceptions/workflow-step-executor.exception';
|
||||
import { EmailWorkflowActionBase } from 'src/modules/workflow/workflow-executor/workflow-actions/mail-sender/email-workflow-action.base';
|
||||
import { isWorkflowSendEmailAction } from 'src/modules/workflow/workflow-executor/workflow-actions/mail-sender/guards/is-workflow-send-email-action.guard';
|
||||
import { type EmailStepLogMode } from 'src/modules/workflow/workflow-executor/workflow-actions/mail-sender/utils/build-email-step-log.util';
|
||||
import { type WorkflowAction } from 'src/modules/workflow/workflow-executor/workflow-actions/types/workflow-action.type';
|
||||
import { WorkflowRunStepLogWorkspaceService } from 'src/modules/workflow/workflow-runner/workflow-run/workflow-run-step-log.workspace-service';
|
||||
|
||||
@Injectable()
|
||||
export class SendEmailWorkflowAction extends EmailWorkflowActionBase {
|
||||
constructor(
|
||||
private readonly sendEmailTool: SendEmailTool,
|
||||
workflowRunStepLogService: WorkflowRunStepLogWorkspaceService,
|
||||
) {
|
||||
super(SendEmailWorkflowAction.name, workflowRunStepLogService);
|
||||
}
|
||||
|
||||
protected getTool(): Tool {
|
||||
return this.sendEmailTool;
|
||||
}
|
||||
|
||||
protected getMode(): EmailStepLogMode {
|
||||
return 'SEND';
|
||||
}
|
||||
|
||||
protected assertStep(step: WorkflowAction): void {
|
||||
if (!isWorkflowSendEmailAction(step)) {
|
||||
throw new WorkflowStepExecutorException(
|
||||
'Step is not a send-email action',
|
||||
WorkflowStepExecutorExceptionCode.INVALID_STEP_TYPE,
|
||||
);
|
||||
}
|
||||
}
|
||||
}
|
||||
+162
@@ -0,0 +1,162 @@
|
||||
import { type ToolOutput } from 'src/engine/core-modules/tool/types/tool-output.type';
|
||||
import { type WorkflowSendEmailActionInput } from 'src/modules/workflow/workflow-executor/workflow-actions/mail-sender/types/workflow-send-email-action-input.type';
|
||||
import { buildEmailStepLog } from 'src/modules/workflow/workflow-executor/workflow-actions/mail-sender/utils/build-email-step-log.util';
|
||||
|
||||
const baseInput: WorkflowSendEmailActionInput = {
|
||||
connectedAccountId: 'account-1',
|
||||
recipients: { to: 'alice@example.com, bob@example.com' },
|
||||
subject: 'Welcome',
|
||||
body: '<p>Hello</p>',
|
||||
};
|
||||
|
||||
const successOutput: ToolOutput = {
|
||||
success: true,
|
||||
message: 'Email sent successfully to Alice',
|
||||
result: {
|
||||
recipients: ['alice@example.com', 'bob@example.com'],
|
||||
ccRecipients: [],
|
||||
bccRecipients: [],
|
||||
subject: 'Welcome',
|
||||
connectedAccountId: 'account-1',
|
||||
attachmentCount: 0,
|
||||
},
|
||||
};
|
||||
|
||||
describe('buildEmailStepLog', () => {
|
||||
it('builds a SUCCESS email log preferring parsed recipients from the tool output', () => {
|
||||
const stepLog = buildEmailStepLog({
|
||||
mode: 'SEND',
|
||||
input: baseInput,
|
||||
output: successOutput,
|
||||
durationMs: 120,
|
||||
});
|
||||
|
||||
if (stepLog.details.type !== 'EMAIL') {
|
||||
throw new Error('Expected EMAIL details');
|
||||
}
|
||||
|
||||
expect(stepLog.details.mode).toBe('SEND');
|
||||
expect(stepLog.details.status).toBe('SUCCESS');
|
||||
expect(stepLog.details.recipients.to).toEqual([
|
||||
'alice@example.com',
|
||||
'bob@example.com',
|
||||
]);
|
||||
expect(stepLog.details.recipients.cc).toBeUndefined();
|
||||
expect(stepLog.details.subject).toBe('Welcome');
|
||||
expect(stepLog.details.attachmentCount).toBe(0);
|
||||
expect(stepLog.details.durationMs).toBe(120);
|
||||
});
|
||||
|
||||
it('falls back to splitting the comma-separated input when the tool output has no parsed recipients', () => {
|
||||
const stepLog = buildEmailStepLog({
|
||||
mode: 'DRAFT',
|
||||
input: {
|
||||
...baseInput,
|
||||
recipients: {
|
||||
to: 'alice@example.com,bob@example.com ; carol@example.com',
|
||||
cc: 'dan@example.com',
|
||||
},
|
||||
},
|
||||
output: {
|
||||
success: false,
|
||||
message: 'Failed to create draft',
|
||||
error: 'Connected account expired',
|
||||
},
|
||||
durationMs: 5,
|
||||
});
|
||||
|
||||
if (stepLog.details.type !== 'EMAIL') {
|
||||
throw new Error('Expected EMAIL details');
|
||||
}
|
||||
|
||||
expect(stepLog.details.mode).toBe('DRAFT');
|
||||
expect(stepLog.details.status).toBe('ERROR');
|
||||
expect(stepLog.details.recipients.to).toEqual([
|
||||
'alice@example.com',
|
||||
'bob@example.com',
|
||||
'carol@example.com',
|
||||
]);
|
||||
expect(stepLog.details.recipients.cc).toEqual(['dan@example.com']);
|
||||
expect(stepLog.details.error).toBe('Connected account expired');
|
||||
});
|
||||
|
||||
it('truncates oversized body previews and reports original byte size', () => {
|
||||
const longBody = `<p>${'x'.repeat(20_000)}</p>`;
|
||||
|
||||
const stepLog = buildEmailStepLog({
|
||||
mode: 'SEND',
|
||||
input: { ...baseInput, body: longBody },
|
||||
output: successOutput,
|
||||
durationMs: 10,
|
||||
});
|
||||
|
||||
if (stepLog.details.type !== 'EMAIL') {
|
||||
throw new Error('Expected EMAIL details');
|
||||
}
|
||||
|
||||
expect(stepLog.details.bodyTruncated).toBe(true);
|
||||
expect(stepLog.details.bodyPreview).toContain('truncated');
|
||||
expect(stepLog.details.bodyBytes).toBeGreaterThan(20_000);
|
||||
});
|
||||
|
||||
it('prefers the sanitized HTML body from the tool output over the raw input body', () => {
|
||||
const stepLog = buildEmailStepLog({
|
||||
mode: 'SEND',
|
||||
input: {
|
||||
...baseInput,
|
||||
body: '<script>alert("xss")</script><p>Hello</p>',
|
||||
},
|
||||
output: {
|
||||
...successOutput,
|
||||
result: {
|
||||
...(successOutput.result as object),
|
||||
sanitizedHtmlBody: '<p>Hello</p>',
|
||||
plainTextBody: 'Hello',
|
||||
},
|
||||
},
|
||||
durationMs: 10,
|
||||
});
|
||||
|
||||
if (stepLog.details.type !== 'EMAIL') {
|
||||
throw new Error('Expected EMAIL details');
|
||||
}
|
||||
|
||||
expect(stepLog.details.bodyPreview).toBe('<p>Hello</p>');
|
||||
expect(stepLog.details.bodyPreview).not.toContain('<script>');
|
||||
});
|
||||
|
||||
it('falls back to the raw input body when the tool failed before composing', () => {
|
||||
const stepLog = buildEmailStepLog({
|
||||
mode: 'SEND',
|
||||
input: { ...baseInput, body: '<p>Hello</p>' },
|
||||
output: {
|
||||
success: false,
|
||||
message: 'Failed to send',
|
||||
error: 'Auth expired',
|
||||
},
|
||||
durationMs: 5,
|
||||
});
|
||||
|
||||
if (stepLog.details.type !== 'EMAIL') {
|
||||
throw new Error('Expected EMAIL details');
|
||||
}
|
||||
|
||||
expect(stepLog.details.bodyPreview).toBe('<p>Hello</p>');
|
||||
});
|
||||
|
||||
it('omits cc/bcc when neither input nor output provides them', () => {
|
||||
const stepLog = buildEmailStepLog({
|
||||
mode: 'SEND',
|
||||
input: { ...baseInput, recipients: { to: 'alice@example.com' } },
|
||||
output: successOutput,
|
||||
durationMs: 10,
|
||||
});
|
||||
|
||||
if (stepLog.details.type !== 'EMAIL') {
|
||||
throw new Error('Expected EMAIL details');
|
||||
}
|
||||
|
||||
expect(stepLog.details.recipients.cc).toBeUndefined();
|
||||
expect(stepLog.details.recipients.bcc).toBeUndefined();
|
||||
});
|
||||
});
|
||||
+143
@@ -0,0 +1,143 @@
|
||||
import { type WorkflowRunStepLog } from 'twenty-shared/workflow';
|
||||
|
||||
import { type ToolOutput } from 'src/engine/core-modules/tool/types/tool-output.type';
|
||||
import { type WorkflowSendEmailActionInput } from 'src/modules/workflow/workflow-executor/workflow-actions/mail-sender/types/workflow-send-email-action-input.type';
|
||||
import { truncateStringToUtf8ByteBudget } from 'src/utils/truncate-string-to-utf8-byte-budget.util';
|
||||
|
||||
const MAX_BODY_PREVIEW_BYTES = 8_000;
|
||||
|
||||
export type EmailStepLogMode = 'SEND' | 'DRAFT';
|
||||
|
||||
const splitRecipients = (raw: string | undefined): string[] => {
|
||||
if (raw === undefined || raw === null) {
|
||||
return [];
|
||||
}
|
||||
|
||||
return raw
|
||||
.split(/[,;]/)
|
||||
.map((entry) => entry.trim())
|
||||
.filter((entry) => entry.length > 0);
|
||||
};
|
||||
|
||||
const isStringArray = (value: unknown): value is string[] =>
|
||||
Array.isArray(value) && value.every((item) => typeof item === 'string');
|
||||
|
||||
const pickRecipients = (
|
||||
inputValue: string | undefined,
|
||||
outputValue: unknown,
|
||||
): string[] => {
|
||||
if (isStringArray(outputValue)) {
|
||||
return outputValue;
|
||||
}
|
||||
|
||||
return splitRecipients(inputValue);
|
||||
};
|
||||
|
||||
const truncateBody = (body: string | undefined) => {
|
||||
if (body === undefined || body === null || body.length === 0) {
|
||||
return {
|
||||
bodyPreview: undefined,
|
||||
bodyBytes: undefined,
|
||||
bodyTruncated: false,
|
||||
};
|
||||
}
|
||||
|
||||
const { value, originalBytes, truncated } = truncateStringToUtf8ByteBudget(
|
||||
body,
|
||||
MAX_BODY_PREVIEW_BYTES,
|
||||
);
|
||||
|
||||
return {
|
||||
bodyPreview: value,
|
||||
bodyBytes: originalBytes,
|
||||
bodyTruncated: truncated,
|
||||
};
|
||||
};
|
||||
|
||||
const extractString = (output: ToolOutput, key: string): string | undefined => {
|
||||
if (!output.result || typeof output.result !== 'object') {
|
||||
return undefined;
|
||||
}
|
||||
|
||||
const value = (output.result as Record<string, unknown>)[key];
|
||||
|
||||
return typeof value === 'string' ? value : undefined;
|
||||
};
|
||||
|
||||
const extractNumber = (output: ToolOutput, key: string): number | undefined => {
|
||||
if (!output.result || typeof output.result !== 'object') {
|
||||
return undefined;
|
||||
}
|
||||
|
||||
const value = (output.result as Record<string, unknown>)[key];
|
||||
|
||||
return typeof value === 'number' ? value : undefined;
|
||||
};
|
||||
|
||||
const extractRecipientsField = (output: ToolOutput, key: string): unknown => {
|
||||
if (!output.result || typeof output.result !== 'object') {
|
||||
return undefined;
|
||||
}
|
||||
|
||||
return (output.result as Record<string, unknown>)[key];
|
||||
};
|
||||
|
||||
export const buildEmailStepLog = ({
|
||||
mode,
|
||||
input,
|
||||
output,
|
||||
durationMs,
|
||||
}: {
|
||||
mode: EmailStepLogMode;
|
||||
input: WorkflowSendEmailActionInput;
|
||||
output: ToolOutput;
|
||||
durationMs: number;
|
||||
}): WorkflowRunStepLog => {
|
||||
const to = pickRecipients(
|
||||
input.recipients?.to,
|
||||
extractRecipientsField(output, 'recipients'),
|
||||
);
|
||||
const cc = pickRecipients(
|
||||
input.recipients?.cc,
|
||||
extractRecipientsField(output, 'ccRecipients'),
|
||||
);
|
||||
const bcc = pickRecipients(
|
||||
input.recipients?.bcc,
|
||||
extractRecipientsField(output, 'bccRecipients'),
|
||||
);
|
||||
|
||||
const subject = extractString(output, 'subject') ?? input.subject;
|
||||
const connectedAccountId =
|
||||
extractString(output, 'connectedAccountId') ?? input.connectedAccountId;
|
||||
const attachmentCount = extractNumber(output, 'attachmentCount');
|
||||
|
||||
const bodyForLog =
|
||||
extractString(output, 'sanitizedHtmlBody') ??
|
||||
extractString(output, 'plainTextBody') ??
|
||||
input.body;
|
||||
const body = truncateBody(bodyForLog);
|
||||
|
||||
return {
|
||||
details: {
|
||||
type: 'EMAIL',
|
||||
mode,
|
||||
status: output.success ? 'SUCCESS' : 'ERROR',
|
||||
recipients: {
|
||||
to,
|
||||
cc: cc.length > 0 ? cc : undefined,
|
||||
bcc: bcc.length > 0 ? bcc : undefined,
|
||||
},
|
||||
subject,
|
||||
bodyPreview: body.bodyPreview,
|
||||
bodyBytes: body.bodyBytes,
|
||||
bodyTruncated: body.bodyTruncated,
|
||||
connectedAccountId,
|
||||
attachmentCount,
|
||||
inReplyTo: input.inReplyTo,
|
||||
error: output.error,
|
||||
durationMs,
|
||||
},
|
||||
entries: [],
|
||||
sizeBytes: 0,
|
||||
};
|
||||
};
|
||||
+25
@@ -0,0 +1,25 @@
|
||||
import type { JSONContent } from '@tiptap/core';
|
||||
|
||||
import {
|
||||
isDefined,
|
||||
parseJson,
|
||||
resolveRichTextVariables,
|
||||
} from 'twenty-shared/utils';
|
||||
|
||||
import { renderRichTextToHtml } from 'src/engine/core-modules/tool/tools/email-tool/utils/render-rich-text-to-html.util';
|
||||
|
||||
export const resolveEmailBody = async (
|
||||
body: string,
|
||||
context: Record<string, unknown>,
|
||||
): Promise<string> => {
|
||||
const bodyWithResolvedVariables = resolveRichTextVariables(body, context);
|
||||
const tipTapDocument = isDefined(bodyWithResolvedVariables)
|
||||
? parseJson<JSONContent>(bodyWithResolvedVariables)
|
||||
: null;
|
||||
|
||||
if (isDefined(tipTapDocument) && tipTapDocument.type === 'doc') {
|
||||
return renderRichTextToHtml(tipTapDocument);
|
||||
}
|
||||
|
||||
return bodyWithResolvedVariables ?? body;
|
||||
};
|
||||
+114
@@ -0,0 +1,114 @@
|
||||
import { Logger } from '@nestjs/common';
|
||||
|
||||
import { resolveInput } from 'twenty-shared/utils';
|
||||
import { type WorkflowRunStepLog } from 'twenty-shared/workflow';
|
||||
|
||||
import { type ToolInput } from 'src/engine/core-modules/tool/types/tool-input.type';
|
||||
import { type ToolOutput } from 'src/engine/core-modules/tool/types/tool-output.type';
|
||||
import { type Tool } from 'src/engine/core-modules/tool/types/tool.type';
|
||||
import { type WorkflowAction as WorkflowActionContract } from 'src/modules/workflow/workflow-executor/interfaces/workflow-action.interface';
|
||||
import { type WorkflowActionInput } from 'src/modules/workflow/workflow-executor/types/workflow-action-input';
|
||||
import { type WorkflowActionOutput } from 'src/modules/workflow/workflow-executor/types/workflow-action-output.type';
|
||||
import { findStepOrThrow } from 'src/modules/workflow/workflow-executor/utils/find-step-or-throw.util';
|
||||
import { type WorkflowAction } from 'src/modules/workflow/workflow-executor/workflow-actions/types/workflow-action.type';
|
||||
import { WorkflowRunStepLogWorkspaceService } from 'src/modules/workflow/workflow-runner/workflow-run/workflow-run-step-log.workspace-service';
|
||||
|
||||
type BuildStepLogArgs<TInput> = {
|
||||
input: TInput;
|
||||
output: ToolOutput;
|
||||
durationMs: number;
|
||||
};
|
||||
|
||||
export abstract class ToolBackedWorkflowAction<
|
||||
TInput extends ToolInput,
|
||||
> implements WorkflowActionContract {
|
||||
protected readonly logger: Logger;
|
||||
|
||||
protected constructor(
|
||||
loggerName: string,
|
||||
private readonly workflowRunStepLogService: WorkflowRunStepLogWorkspaceService,
|
||||
) {
|
||||
this.logger = new Logger(loggerName);
|
||||
}
|
||||
|
||||
protected abstract getTool(): Tool;
|
||||
|
||||
protected abstract assertStep(step: WorkflowAction): void;
|
||||
|
||||
protected async preprocessInput(
|
||||
rawInput: TInput,
|
||||
_context: Record<string, unknown>,
|
||||
): Promise<TInput> {
|
||||
return rawInput;
|
||||
}
|
||||
|
||||
protected abstract buildStepLog(
|
||||
args: BuildStepLogArgs<TInput>,
|
||||
): WorkflowRunStepLog;
|
||||
|
||||
async execute({
|
||||
currentStepId,
|
||||
steps,
|
||||
context,
|
||||
runInfo,
|
||||
}: WorkflowActionInput): Promise<WorkflowActionOutput> {
|
||||
const step = findStepOrThrow({ stepId: currentStepId, steps });
|
||||
|
||||
this.assertStep(step);
|
||||
|
||||
const rawInput = step.settings.input as TInput;
|
||||
const preprocessed = await this.preprocessInput(rawInput, context);
|
||||
const resolvedInput = resolveInput(preprocessed, context) as TInput;
|
||||
|
||||
const startedAt = Date.now();
|
||||
const toolOutput = await this.getTool().execute(resolvedInput, {
|
||||
workspaceId: runInfo.workspaceId,
|
||||
});
|
||||
const durationMs = Date.now() - startedAt;
|
||||
|
||||
await this.persistStepLog({
|
||||
workflowRunId: runInfo.workflowRunId,
|
||||
workspaceId: runInfo.workspaceId,
|
||||
stepId: currentStepId,
|
||||
input: resolvedInput,
|
||||
output: toolOutput,
|
||||
durationMs,
|
||||
});
|
||||
|
||||
return {
|
||||
result: toolOutput.result as object,
|
||||
error: toolOutput.error,
|
||||
};
|
||||
}
|
||||
|
||||
private async persistStepLog({
|
||||
workflowRunId,
|
||||
workspaceId,
|
||||
stepId,
|
||||
input,
|
||||
output,
|
||||
durationMs,
|
||||
}: {
|
||||
workflowRunId: string;
|
||||
workspaceId: string;
|
||||
stepId: string;
|
||||
input: TInput;
|
||||
output: ToolOutput;
|
||||
durationMs: number;
|
||||
}): Promise<void> {
|
||||
try {
|
||||
await this.workflowRunStepLogService.setStepLog({
|
||||
workflowRunId,
|
||||
workspaceId,
|
||||
stepId,
|
||||
stepLog: this.buildStepLog({ input, output, durationMs }),
|
||||
});
|
||||
} catch (error) {
|
||||
this.logger.warn(
|
||||
`Failed to persist step log for workflowRun=${workflowRunId} step=${stepId}: ${
|
||||
error instanceof Error ? error.message : String(error)
|
||||
}`,
|
||||
);
|
||||
}
|
||||
}
|
||||
}
|
||||
-112
@@ -1,112 +0,0 @@
|
||||
import { Injectable } from '@nestjs/common';
|
||||
import type { JSONContent } from '@tiptap/core';
|
||||
|
||||
import {
|
||||
isDefined,
|
||||
parseJson,
|
||||
resolveInput,
|
||||
resolveRichTextVariables,
|
||||
} from 'twenty-shared/utils';
|
||||
|
||||
import { type WorkflowAction } from 'src/modules/workflow/workflow-executor/interfaces/workflow-action.interface';
|
||||
|
||||
import { DraftEmailTool } from 'src/engine/core-modules/tool/tools/email-tool/draft-email-tool';
|
||||
import { renderRichTextToHtml } from 'src/engine/core-modules/tool/tools/email-tool/utils/render-rich-text-to-html.util';
|
||||
import { HttpTool } from 'src/engine/core-modules/tool/tools/http-tool/http-tool';
|
||||
import { SendEmailTool } from 'src/engine/core-modules/tool/tools/email-tool/send-email-tool';
|
||||
import { type ToolInput } from 'src/engine/core-modules/tool/types/tool-input.type';
|
||||
import { type Tool } from 'src/engine/core-modules/tool/types/tool.type';
|
||||
import {
|
||||
WorkflowStepExecutorException,
|
||||
WorkflowStepExecutorExceptionCode,
|
||||
} from 'src/modules/workflow/workflow-executor/exceptions/workflow-step-executor.exception';
|
||||
import { type WorkflowActionInput } from 'src/modules/workflow/workflow-executor/types/workflow-action-input';
|
||||
import { type WorkflowActionOutput } from 'src/modules/workflow/workflow-executor/types/workflow-action-output.type';
|
||||
import { type WorkflowSendEmailActionInput } from 'src/modules/workflow/workflow-executor/workflow-actions/mail-sender/types/workflow-send-email-action-input.type';
|
||||
import { WorkflowActionType } from 'src/modules/workflow/workflow-executor/workflow-actions/types/workflow-action.type';
|
||||
|
||||
@Injectable()
|
||||
export class ToolExecutorWorkflowAction implements WorkflowAction {
|
||||
private readonly toolsByActionType: Map<WorkflowActionType, Tool>;
|
||||
|
||||
constructor(
|
||||
private readonly httpTool: HttpTool,
|
||||
private readonly sendEmailTool: SendEmailTool,
|
||||
private readonly draftEmailTool: DraftEmailTool,
|
||||
) {
|
||||
this.toolsByActionType = new Map<WorkflowActionType, Tool>([
|
||||
[WorkflowActionType.HTTP_REQUEST, this.httpTool],
|
||||
[WorkflowActionType.SEND_EMAIL, this.sendEmailTool],
|
||||
[WorkflowActionType.DRAFT_EMAIL, this.draftEmailTool],
|
||||
]);
|
||||
}
|
||||
|
||||
private async resolveEmailBody(
|
||||
body: string,
|
||||
context: Record<string, unknown>,
|
||||
): Promise<string> {
|
||||
const bodyWithResolvedVariables = resolveRichTextVariables(body, context);
|
||||
const tipTapDocument = isDefined(bodyWithResolvedVariables)
|
||||
? parseJson<JSONContent>(bodyWithResolvedVariables)
|
||||
: null;
|
||||
|
||||
if (isDefined(tipTapDocument) && tipTapDocument.type === 'doc') {
|
||||
return renderRichTextToHtml(tipTapDocument);
|
||||
}
|
||||
|
||||
return bodyWithResolvedVariables ?? body;
|
||||
}
|
||||
|
||||
async execute({
|
||||
currentStepId,
|
||||
steps,
|
||||
context,
|
||||
runInfo,
|
||||
}: WorkflowActionInput): Promise<WorkflowActionOutput> {
|
||||
const step = steps.find((step) => step.id === currentStepId);
|
||||
|
||||
if (!step) {
|
||||
throw new WorkflowStepExecutorException(
|
||||
'Step not found',
|
||||
WorkflowStepExecutorExceptionCode.STEP_NOT_FOUND,
|
||||
);
|
||||
}
|
||||
|
||||
const tool = this.toolsByActionType.get(step.type);
|
||||
|
||||
if (!tool) {
|
||||
throw new WorkflowStepExecutorException(
|
||||
`No tool found for workflow action type: ${step.type}`,
|
||||
WorkflowStepExecutorExceptionCode.INVALID_STEP_TYPE,
|
||||
);
|
||||
}
|
||||
|
||||
let toolInput = step.settings.input;
|
||||
|
||||
if (
|
||||
step.type === WorkflowActionType.SEND_EMAIL ||
|
||||
step.type === WorkflowActionType.DRAFT_EMAIL
|
||||
) {
|
||||
const emailInput = toolInput as WorkflowSendEmailActionInput;
|
||||
|
||||
if (isDefined(emailInput.body)) {
|
||||
const emailBody = await this.resolveEmailBody(emailInput.body, context);
|
||||
toolInput = {
|
||||
...emailInput,
|
||||
body: emailBody,
|
||||
};
|
||||
}
|
||||
}
|
||||
|
||||
toolInput = resolveInput(toolInput, context) as ToolInput;
|
||||
|
||||
const toolOutput = await tool.execute(toolInput, {
|
||||
workspaceId: runInfo.workspaceId,
|
||||
});
|
||||
|
||||
return {
|
||||
result: toolOutput.result as object,
|
||||
error: toolOutput.error,
|
||||
};
|
||||
}
|
||||
}
|
||||
+5
-8
@@ -4,7 +4,6 @@ import { BillingModule } from 'src/engine/core-modules/billing/billing.module';
|
||||
import { WorkspaceCacheModule } from 'src/engine/workspace-cache/workspace-cache.module';
|
||||
import { FeatureFlagModule } from 'src/engine/core-modules/feature-flag/feature-flag.module';
|
||||
import { MetricsModule } from 'src/engine/core-modules/metrics/metrics.module';
|
||||
import { ToolModule } from 'src/engine/core-modules/tool/tool.module';
|
||||
import { WorkflowCommonModule } from 'src/modules/workflow/common/workflow-common.module';
|
||||
import { WorkflowActionFactory } from 'src/modules/workflow/workflow-executor/factories/workflow-action.factory';
|
||||
import { AiAgentActionModule } from 'src/modules/workflow/workflow-executor/workflow-actions/ai-agent/ai-agent-action.module';
|
||||
@@ -13,11 +12,12 @@ import { DelayActionModule } from 'src/modules/workflow/workflow-executor/workfl
|
||||
import { EmptyActionModule } from 'src/modules/workflow/workflow-executor/workflow-actions/empty/empty-action.module';
|
||||
import { FilterActionModule } from 'src/modules/workflow/workflow-executor/workflow-actions/filter/filter-action.module';
|
||||
import { FormActionModule } from 'src/modules/workflow/workflow-executor/workflow-actions/form/form-action.module';
|
||||
import { HttpRequestActionModule } from 'src/modules/workflow/workflow-executor/workflow-actions/http-request/http-request-action.module';
|
||||
import { IfElseActionModule } from 'src/modules/workflow/workflow-executor/workflow-actions/if-else/if-else-action.module';
|
||||
import { IteratorActionModule } from 'src/modules/workflow/workflow-executor/workflow-actions/iterator/iterator-action.module';
|
||||
import { LogicFunctionActionModule } from 'src/modules/workflow/workflow-executor/workflow-actions/logic-function/logic-function-action.module';
|
||||
import { MailSenderActionModule } from 'src/modules/workflow/workflow-executor/workflow-actions/mail-sender/mail-sender-action.module';
|
||||
import { RecordCRUDActionModule } from 'src/modules/workflow/workflow-executor/workflow-actions/record-crud/record-crud-action.module';
|
||||
import { ToolExecutorWorkflowAction } from 'src/modules/workflow/workflow-executor/workflow-actions/tool-executor-workflow-action';
|
||||
import { WorkflowExecutorWorkspaceService } from 'src/modules/workflow/workflow-executor/workspace-services/workflow-executor.workspace-service';
|
||||
import { WorkflowRunModule } from 'src/modules/workflow/workflow-runner/workflow-run/workflow-run.module';
|
||||
|
||||
@@ -38,14 +38,11 @@ import { WorkflowRunModule } from 'src/modules/workflow/workflow-runner/workflow
|
||||
AiAgentActionModule,
|
||||
EmptyActionModule,
|
||||
FeatureFlagModule,
|
||||
ToolModule,
|
||||
HttpRequestActionModule,
|
||||
MailSenderActionModule,
|
||||
MetricsModule,
|
||||
],
|
||||
providers: [
|
||||
WorkflowExecutorWorkspaceService,
|
||||
WorkflowActionFactory,
|
||||
ToolExecutorWorkflowAction,
|
||||
],
|
||||
providers: [WorkflowExecutorWorkspaceService, WorkflowActionFactory],
|
||||
exports: [WorkflowExecutorWorkspaceService],
|
||||
})
|
||||
export class WorkflowExecutorModule {}
|
||||
|
||||
+89
@@ -0,0 +1,89 @@
|
||||
import { Injectable, Logger } from '@nestjs/common';
|
||||
|
||||
import { FeatureFlagKey } from 'twenty-shared/types';
|
||||
import { type WorkflowRunStepLog } from 'twenty-shared/workflow';
|
||||
|
||||
import { FeatureFlagService } from 'src/engine/core-modules/feature-flag/services/feature-flag.service';
|
||||
import { GlobalWorkspaceOrmManager } from 'src/engine/twenty-orm/global-workspace-datasource/global-workspace-orm.manager';
|
||||
import { buildSystemAuthContext } from 'src/engine/twenty-orm/utils/build-system-auth-context.util';
|
||||
import { type WorkflowRunWorkspaceEntity } from 'src/modules/workflow/common/standard-objects/workflow-run.workspace-entity';
|
||||
|
||||
const MAX_STEP_LOG_BYTES = 256_000;
|
||||
|
||||
const computeSizeBytes = (value: unknown): number => {
|
||||
try {
|
||||
return Buffer.byteLength(JSON.stringify(value) ?? '', 'utf8');
|
||||
} catch {
|
||||
return 0;
|
||||
}
|
||||
};
|
||||
|
||||
@Injectable()
|
||||
export class WorkflowRunStepLogWorkspaceService {
|
||||
private readonly logger = new Logger(WorkflowRunStepLogWorkspaceService.name);
|
||||
|
||||
constructor(
|
||||
private readonly globalWorkspaceOrmManager: GlobalWorkspaceOrmManager,
|
||||
private readonly featureFlagService: FeatureFlagService,
|
||||
) {}
|
||||
async setStepLog({
|
||||
workflowRunId,
|
||||
workspaceId,
|
||||
stepId,
|
||||
stepLog,
|
||||
}: {
|
||||
workflowRunId: string;
|
||||
workspaceId: string;
|
||||
stepId: string;
|
||||
stepLog: WorkflowRunStepLog;
|
||||
}): Promise<void> {
|
||||
const isStepLogsEnabled = await this.featureFlagService.isFeatureEnabled(
|
||||
FeatureFlagKey.IS_WORKFLOW_RUN_STEP_LOGS_ENABLED,
|
||||
workspaceId,
|
||||
);
|
||||
|
||||
if (!isStepLogsEnabled) {
|
||||
return;
|
||||
}
|
||||
|
||||
const sizeBytes = computeSizeBytes(stepLog);
|
||||
|
||||
if (sizeBytes > MAX_STEP_LOG_BYTES) {
|
||||
this.logger.warn(
|
||||
`Step log for workflowRun=${workflowRunId} step=${stepId} exceeds cap (${sizeBytes}b > ${MAX_STEP_LOG_BYTES}b) and will be dropped`,
|
||||
);
|
||||
|
||||
return;
|
||||
}
|
||||
|
||||
const stepLogWithSize: WorkflowRunStepLog = {
|
||||
...stepLog,
|
||||
sizeBytes,
|
||||
};
|
||||
|
||||
const authContext = buildSystemAuthContext(workspaceId);
|
||||
|
||||
await this.globalWorkspaceOrmManager.executeInWorkspaceContext(async () => {
|
||||
const workflowRunRepository =
|
||||
await this.globalWorkspaceOrmManager.getRepository<WorkflowRunWorkspaceEntity>(
|
||||
workspaceId,
|
||||
'workflowRun',
|
||||
{ shouldBypassPermissionChecks: true },
|
||||
);
|
||||
|
||||
await workflowRunRepository
|
||||
.createQueryBuilder()
|
||||
.update()
|
||||
.set({
|
||||
stepLogs: () =>
|
||||
`jsonb_set(COALESCE("stepLogs", '{}'::jsonb), ARRAY[:stepId]::text[], :stepLog::jsonb, true)`,
|
||||
})
|
||||
.where('id = :workflowRunId', { workflowRunId })
|
||||
.setParameters({
|
||||
stepId,
|
||||
stepLog: JSON.stringify(stepLogWithSize),
|
||||
})
|
||||
.execute();
|
||||
}, authContext);
|
||||
}
|
||||
}
|
||||
+13
-2
@@ -4,12 +4,14 @@ import { NestjsQueryTypeOrmModule } from '@ptc-org/nestjs-query-typeorm';
|
||||
|
||||
import { WorkspaceIteratorModule } from 'src/database/commands/command-runners/workspace-iterator.module';
|
||||
import { CacheLockModule } from 'src/engine/core-modules/cache-lock/cache-lock.module';
|
||||
import { FeatureFlagModule } from 'src/engine/core-modules/feature-flag/feature-flag.module';
|
||||
import { MetricsModule } from 'src/engine/core-modules/metrics/metrics.module';
|
||||
import { RecordPositionModule } from 'src/engine/core-modules/record-position/record-position.module';
|
||||
import { WorkspaceEntity } from 'src/engine/core-modules/workspace/workspace.entity';
|
||||
import { ObjectMetadataEntity } from 'src/engine/metadata-modules/object-metadata/object-metadata.entity';
|
||||
import { WorkflowCommonModule } from 'src/modules/workflow/common/workflow-common.module';
|
||||
import { DeleteWorkflowRunsCommand } from 'src/modules/workflow/workflow-runner/workflow-run/command/delete-workflow-runs.command';
|
||||
import { WorkflowRunStepLogWorkspaceService } from 'src/modules/workflow/workflow-runner/workflow-run/workflow-run-step-log.workspace-service';
|
||||
import { WorkflowRunWorkspaceService } from 'src/modules/workflow/workflow-runner/workflow-run/workflow-run.workspace-service';
|
||||
|
||||
@Module({
|
||||
@@ -23,8 +25,17 @@ import { WorkflowRunWorkspaceService } from 'src/modules/workflow/workflow-runne
|
||||
CacheLockModule,
|
||||
MetricsModule,
|
||||
WorkspaceIteratorModule,
|
||||
FeatureFlagModule,
|
||||
],
|
||||
providers: [
|
||||
WorkflowRunWorkspaceService,
|
||||
WorkflowRunStepLogWorkspaceService,
|
||||
DeleteWorkflowRunsCommand,
|
||||
],
|
||||
exports: [
|
||||
WorkflowRunWorkspaceService,
|
||||
WorkflowRunStepLogWorkspaceService,
|
||||
DeleteWorkflowRunsCommand,
|
||||
],
|
||||
providers: [WorkflowRunWorkspaceService, DeleteWorkflowRunsCommand],
|
||||
exports: [WorkflowRunWorkspaceService, DeleteWorkflowRunsCommand],
|
||||
})
|
||||
export class WorkflowRunModule {}
|
||||
|
||||
Reference in New Issue
Block a user