feat(code-interpreter): reuse a warm sandbox per conversation (E2B) (#21664)

## What

The E2B code-interpreter driver created a **fresh sandbox on every
execution** and killed it in `finally`, so every call in a conversation
paid full cold-start and started blank. This PR keeps **one warm sandbox
per conversation** and, on idle, **pauses** it rather than killing it.

## How

- **Discovery without a registry:** the sandbox is tagged with the chat
`threadId` (scoped `workspaceId:threadId`) via E2B **metadata**, found
with `Sandbox.list({ query: { state: ['running','paused'], metadata }
})` and resumed with `Sandbox.connect()` (which auto-resumes a paused
sandbox). E2B is the source of truth — no Redis/DB mapping.
- **Pause/resume (E2B 2.x):** session sandboxes are created with
`lifecycle: { onTimeout: 'pause', autoResume: true }`. When idle they
**pause** — compute billing stops, filesystem **and** kernel/memory
state are preserved — and resume in ~1s on the next call. This replaces
the earlier keepalive approach.
- **No premature pause mid-run:** the sandbox is kept alive for
`max(execution timeout, idle window)`, so a long execution is never
paused underneath itself.
- **Tenant isolation:** discovery filters by the `twentySessionId` tag
and **re-checks it client-side**, so a loose server-side match can never
hand one conversation's warm sandbox (with its files, kernel state,
token) to another.
- **Concurrency:** executions sharing a session are serialized
in-process (one active stream per thread, run as a single job — the chat
resolver queues concurrent messages), so parallel tool calls can't race
the shared kernel.
- **Output isolation:** `/home/user/output` is reset at the start of
each reused run, so a call only returns the artifacts it actually
produced; durable state lives elsewhere and persists.

## SDK upgrade

`@e2b/code-interpreter` **`^1.0.4` → `^2.6.0`** (pulls `e2b@2.x`). The
typed pause/resume API, `lifecycle`, and the `state`/`metadata` list
filter only exist in the 2.x line; 1.x exposed them only as untyped
OpenAPI internals. `Sandbox.list()` is now a paginator (handled).

## Config

| Var | Default | Purpose |
|---|---|---|
| `CODE_INTERPRETER_TIMEOUT_MS` | `300000` | Max single-execution
duration. |
| `CODE_INTERPRETER_IDLE_TIMEOUT_MS` | `300000` | Idle window before the
warm sandbox auto-pauses. |

Reuse is always-on when a session id is present (chat path). The
workflow-agent path and the dev-only `LocalDriver` are unaffected.

## ⚠️ Open item before merge: paused-sandbox GC

E2B retains paused sandboxes **indefinitely** (no TTL). Unlike the old
keepalive path (which auto-killed on idle), pause means a conversation's
sandbox persists after the chat ends — so without garbage collection,
paused sandboxes accumulate (≈ one per historical conversation) and
consume storage. A GC policy is required; the approach + retention
window are being decided (see PR discussion). Also: the E2B runtime path
can't run in CI, so this still needs a **live smoke test** (reuse hit,
idle→pause, resume) and confirmation of paused-storage pricing before
rollout.

## Tests / checks

- Resolver unit tests (`getOrCreateSessionSandbox`): reuse+extend,
create-when-absent, duplicate reaping, connect-failure fallback,
keep-first-connectable-when-earlier-dead, **ignore cross-tenant
metadata**, and **kill-on-timeout-refresh-failure**.
- `nx typecheck twenty-server` (against e2b 2.x), `oxlint --type-aware`,
`oxfmt --check` all clean.

---------

Co-authored-by: Claude <noreply@anthropic.com>
This commit is contained in:
Félix Malfait
2026-06-17 10:12:24 +02:00
committed by GitHub
parent 74b56ba66c
commit bb6da7b7d1
30 changed files with 986 additions and 58 deletions
@@ -64,7 +64,13 @@ export class CodeInterpreterDriverFactory extends DriverFactoryBase<CodeInterpre
);
}
return new E2BDriver({ apiKey, timeoutMs });
return new E2BDriver({
apiKey,
timeoutMs,
idleTimeoutMs: this.twentyConfigService.get(
'CODE_INTERPRETER_IDLE_TIMEOUT_MS',
),
});
}
default:
@@ -1,5 +1,7 @@
import { Injectable } from '@nestjs/common';
import { isDefined } from 'twenty-shared/utils';
import { CodeInterpreterDriverFactory } from 'src/engine/core-modules/code-interpreter/code-interpreter-driver.factory';
import { CodeInterpreterDriverType } from 'src/engine/core-modules/code-interpreter/code-interpreter.interface';
import {
@@ -13,6 +15,10 @@ import { TwentyConfigService } from 'src/engine/core-modules/twenty-config/twent
@Injectable()
export class CodeInterpreterService implements CodeInterpreterDriver {
// One active stream per thread (the chat resolver queues the rest), so
// in-process chaining is enough to serialize a session — no distributed lock.
private readonly sessionExecutionTails = new Map<string, Promise<void>>();
constructor(
private readonly codeInterpreterDriverFactory: CodeInterpreterDriverFactory,
private readonly twentyConfigService: TwentyConfigService,
@@ -25,14 +31,75 @@ export class CodeInterpreterService implements CodeInterpreterDriver {
);
}
async releaseThreadSandbox(
workspaceId: string,
threadId: string,
): Promise<void> {
await this.codeInterpreterDriverFactory
.getCurrentDriver()
.releaseSession?.(`${workspaceId}:${threadId}`);
}
async sweepExpiredSandboxes(): Promise<number> {
const maxAgeMs = this.twentyConfigService.get(
'CODE_INTERPRETER_SESSION_MAX_AGE_MS',
);
return (
(await this.codeInterpreterDriverFactory
.getCurrentDriver()
.sweepExpiredSessions?.(maxAgeMs)) ?? 0
);
}
execute(
code: string,
files?: InputFile[],
context?: ExecutionContext,
callbacks?: StreamCallbacks,
): Promise<CodeExecutionResult> {
const driver = this.codeInterpreterDriverFactory.getCurrentDriver();
const sessionId = context?.sessionId;
return driver.execute(code, files, context, callbacks);
if (isDefined(sessionId)) {
return this.runSerializedPerSession(sessionId, () =>
this.runOnDriver(code, files, context, callbacks),
);
}
return this.runOnDriver(code, files, context, callbacks);
}
private runOnDriver(
code: string,
files?: InputFile[],
context?: ExecutionContext,
callbacks?: StreamCallbacks,
): Promise<CodeExecutionResult> {
return this.codeInterpreterDriverFactory
.getCurrentDriver()
.execute(code, files, context, callbacks);
}
private async runSerializedPerSession<T>(
sessionId: string,
task: () => Promise<T>,
): Promise<T> {
const previous =
this.sessionExecutionTails.get(sessionId) ?? Promise.resolve();
const result = previous.then(task, task);
const tail = result.then(
() => undefined,
() => undefined,
);
this.sessionExecutionTails.set(sessionId, tail);
try {
return await result;
} finally {
if (this.sessionExecutionTails.get(sessionId) === tail) {
this.sessionExecutionTails.delete(sessionId);
}
}
}
}
@@ -0,0 +1 @@
export const CODE_INTERPRETER_SESSION_CLEANUP_CRON_PATTERN = '0 * * * *';
@@ -0,0 +1 @@
export const SESSION_SANDBOX_METADATA_KEY = 'twentySessionId';
@@ -0,0 +1,13 @@
import { Module } from '@nestjs/common';
import { CodeInterpreterSessionCleanupCronCommand } from 'src/engine/core-modules/code-interpreter/crons/commands/code-interpreter-session-cleanup.cron.command';
import { CodeInterpreterSessionCleanupCronJob } from 'src/engine/core-modules/code-interpreter/crons/jobs/code-interpreter-session-cleanup.cron.job';
@Module({
providers: [
CodeInterpreterSessionCleanupCronJob,
CodeInterpreterSessionCleanupCronCommand,
],
exports: [CodeInterpreterSessionCleanupCronCommand],
})
export class CodeInterpreterSessionCleanupModule {}
@@ -0,0 +1,33 @@
import { Command, CommandRunner } from 'nest-commander';
import { CODE_INTERPRETER_SESSION_CLEANUP_CRON_PATTERN } from 'src/engine/core-modules/code-interpreter/constants/code-interpreter-session-cleanup-cron-pattern.constant';
import { CodeInterpreterSessionCleanupCronJob } from 'src/engine/core-modules/code-interpreter/crons/jobs/code-interpreter-session-cleanup.cron.job';
import { InjectMessageQueue } from 'src/engine/core-modules/message-queue/decorators/message-queue.decorator';
import { MessageQueue } from 'src/engine/core-modules/message-queue/message-queue.constants';
import { MessageQueueService } from 'src/engine/core-modules/message-queue/services/message-queue.service';
@Command({
name: 'cron:code-interpreter:session-cleanup',
description:
'Starts a cron job to reclaim expired (abandoned) code interpreter sandboxes',
})
export class CodeInterpreterSessionCleanupCronCommand extends CommandRunner {
constructor(
@InjectMessageQueue(MessageQueue.cronQueue)
private readonly messageQueueService: MessageQueueService,
) {
super();
}
async run(): Promise<void> {
await this.messageQueueService.addCron<undefined>({
jobName: CodeInterpreterSessionCleanupCronJob.name,
data: undefined,
options: {
repeat: {
pattern: CODE_INTERPRETER_SESSION_CLEANUP_CRON_PATTERN,
},
},
});
}
}
@@ -0,0 +1,36 @@
import { Injectable, Logger } from '@nestjs/common';
import { CodeInterpreterService } from 'src/engine/core-modules/code-interpreter/code-interpreter.service';
import { CODE_INTERPRETER_SESSION_CLEANUP_CRON_PATTERN } from 'src/engine/core-modules/code-interpreter/constants/code-interpreter-session-cleanup-cron-pattern.constant';
import { SentryCronMonitor } from 'src/engine/core-modules/cron/sentry-cron-monitor.decorator';
import { Process } from 'src/engine/core-modules/message-queue/decorators/process.decorator';
import { Processor } from 'src/engine/core-modules/message-queue/decorators/processor.decorator';
import { MessageQueue } from 'src/engine/core-modules/message-queue/message-queue.constants';
@Injectable()
@Processor(MessageQueue.cronQueue)
export class CodeInterpreterSessionCleanupCronJob {
private readonly logger = new Logger(
CodeInterpreterSessionCleanupCronJob.name,
);
constructor(
private readonly codeInterpreterService: CodeInterpreterService,
) {}
@Process(CodeInterpreterSessionCleanupCronJob.name)
@SentryCronMonitor(
CodeInterpreterSessionCleanupCronJob.name,
CODE_INTERPRETER_SESSION_CLEANUP_CRON_PATTERN,
)
async handle(): Promise<void> {
const reclaimedCount =
await this.codeInterpreterService.sweepExpiredSandboxes();
if (reclaimedCount > 0) {
this.logger.log(
`Reclaimed ${reclaimedCount} expired code interpreter sandbox(es)`,
);
}
}
}
@@ -2,8 +2,14 @@ import { promises as fs } from 'fs';
import { join } from 'path';
import { Sandbox } from '@e2b/code-interpreter';
import { Logger } from '@nestjs/common';
import { isDefined } from 'twenty-shared/utils';
import { DEFAULT_CODE_INTERPRETER_TIMEOUT_MS } from 'src/engine/core-modules/code-interpreter/code-interpreter.constants';
import { getOrCreateSessionSandbox } from 'src/engine/core-modules/code-interpreter/drivers/utils/get-or-create-session-sandbox.util';
import { releaseSessionSandboxes } from 'src/engine/core-modules/code-interpreter/drivers/utils/release-session-sandboxes.util';
import { sweepExpiredSessionSandboxes } from 'src/engine/core-modules/code-interpreter/drivers/utils/sweep-expired-session-sandboxes.util';
import { getMimeType } from 'src/engine/core-modules/code-interpreter/utils/get-mime-type.util';
import {
@@ -18,12 +24,13 @@ import {
export type E2BDriverOptions = {
apiKey: string;
timeoutMs?: number;
idleTimeoutMs?: number;
};
const SANDBOX_SCRIPTS_PATH = join(__dirname, '..', 'sandbox-scripts');
async function uploadDirectoryToSandbox(
sbx: Sandbox,
sandbox: Sandbox,
localPath: string,
remotePath: string,
) {
@@ -34,17 +41,19 @@ async function uploadDirectoryToSandbox(
const remoteEntryPath = `${remotePath}/${entry.name}`;
if (entry.isDirectory()) {
await uploadDirectoryToSandbox(sbx, localEntryPath, remoteEntryPath);
await uploadDirectoryToSandbox(sandbox, localEntryPath, remoteEntryPath);
} else {
const content = await fs.readFile(localEntryPath);
const arrayBuffer = new Uint8Array(content).buffer;
await sbx.files.write(remoteEntryPath, arrayBuffer);
await sandbox.files.write(remoteEntryPath, arrayBuffer);
}
}
}
export class E2BDriver implements CodeInterpreterDriver {
private readonly logger = new Logger(E2BDriver.name);
constructor(private options: E2BDriverOptions) {}
async execute(
@@ -53,27 +62,52 @@ export class E2BDriver implements CodeInterpreterDriver {
context?: ExecutionContext,
callbacks?: StreamCallbacks,
): Promise<CodeExecutionResult> {
const sbx = await Sandbox.create({
apiKey: this.options.apiKey,
timeoutMs: this.options.timeoutMs ?? DEFAULT_CODE_INTERPRETER_TIMEOUT_MS,
});
const { apiKey } = this.options;
const sessionId = context?.sessionId;
const idleTimeoutMs =
this.options.idleTimeoutMs ?? DEFAULT_CODE_INTERPRETER_TIMEOUT_MS;
const timeoutMs =
this.options.timeoutMs ?? DEFAULT_CODE_INTERPRETER_TIMEOUT_MS;
let sandbox: Sandbox;
let isReused = false;
let keepWarm = false;
if (isDefined(sessionId)) {
keepWarm = true;
({ sandbox, isReused } = await getOrCreateSessionSandbox({
sandboxApi: Sandbox,
apiKey,
sessionId,
timeoutMs,
idleTimeoutMs,
}));
} else {
sandbox = await Sandbox.create({ apiKey, timeoutMs });
}
try {
// Upload pre-installed scripts to sandbox
try {
await uploadDirectoryToSandbox(
sbx,
SANDBOX_SCRIPTS_PATH,
'/home/user/scripts',
);
} catch {
// Scripts directory might not exist
// A reused sandbox already has the scripts from its first run.
if (!isReused) {
try {
await uploadDirectoryToSandbox(
sandbox,
SANDBOX_SCRIPTS_PATH,
'/home/user/scripts',
);
} catch {
// Scripts directory might not exist
}
}
if (isReused) {
await this.resetOutputDirectory(sandbox);
}
for (const file of files ?? []) {
const arrayBuffer = new Uint8Array(file.content).buffer;
await sbx.files.write(`/home/user/${file.filename}`, arrayBuffer);
await sandbox.files.write(`/home/user/${file.filename}`, arrayBuffer);
}
const envSetup = context?.env
@@ -93,7 +127,7 @@ export class E2BDriver implements CodeInterpreterDriver {
const outputFiles: OutputFile[] = [];
let chartCounter = 0;
const execution = await sbx.runCode(envSetup + code, {
const execution = await sandbox.runCode(envSetup + code, {
onStdout: (data) => callbacks?.onStdout?.(data.line),
onStderr: (data) => callbacks?.onStderr?.(data.line),
onResult: async (result) => {
@@ -111,12 +145,13 @@ export class E2BDriver implements CodeInterpreterDriver {
});
try {
const outputDir = await sbx.files.list('/home/user/output');
const outputDir = await sandbox.files.list('/home/user/output');
for (const file of outputDir) {
if (file.type === 'file') {
const content = await sbx.files.read(
const content = await sandbox.files.read(
`/home/user/output/${file.name}`,
{ format: 'bytes' },
);
const outputFile: OutputFile = {
@@ -141,7 +176,35 @@ export class E2BDriver implements CodeInterpreterDriver {
error: execution.error?.value,
};
} finally {
await sbx.kill();
if (!keepWarm) {
await sandbox.kill();
}
}
}
private async resetOutputDirectory(sandbox: Sandbox): Promise<void> {
const outputDirectory = '/home/user/output';
try {
if (await sandbox.files.exists(outputDirectory)) {
await sandbox.files.remove(outputDirectory);
}
await sandbox.files.makeDir(outputDirectory);
} catch (error) {
const reason = error instanceof Error ? error.message : String(error);
this.logger.warn(
`Failed to reset reused sandbox output directory; results may include stale files: ${reason}`,
);
}
}
async releaseSession(sessionId: string): Promise<void> {
await releaseSessionSandboxes(Sandbox, this.options.apiKey, sessionId);
}
async sweepExpiredSessions(maxAgeMs: number): Promise<number> {
return sweepExpiredSessionSandboxes(Sandbox, this.options.apiKey, maxAgeMs);
}
}
@@ -20,6 +20,7 @@ export type CodeExecutionResult = {
export type ExecutionContext = {
env?: Record<string, string>;
sessionId?: string;
};
export type StreamCallbacks = {
@@ -35,4 +36,6 @@ export interface CodeInterpreterDriver {
context?: ExecutionContext,
callbacks?: StreamCallbacks,
): Promise<CodeExecutionResult>;
releaseSession?(sessionId: string): Promise<void>;
sweepExpiredSessions?(maxAgeMs: number): Promise<number>;
}
@@ -0,0 +1,250 @@
import { type Sandbox } from '@e2b/code-interpreter';
import { SESSION_SANDBOX_METADATA_KEY } from 'src/engine/core-modules/code-interpreter/constants/session-sandbox-metadata-key.constant';
import { getOrCreateSessionSandbox } from 'src/engine/core-modules/code-interpreter/drivers/utils/get-or-create-session-sandbox.util';
type SandboxApiMock = {
list: jest.Mock;
connect: jest.Mock;
create: jest.Mock;
kill: jest.Mock;
};
type ListedSandbox = { sandboxId: string; metadata: Record<string, string> };
const apiKey = 'test-api-key';
const sessionId = 'workspace-1:thread-1';
const timeoutMs = 600_000;
const idleTimeoutMs = 300_000;
// The resolver keeps the sandbox alive for the larger of the two.
const aliveTimeoutMs = 600_000;
// Sandbox.list() returns a paginator; emit the tagged sandboxes as a single page.
const paginatorOf = (items: ListedSandbox[]) => {
let fetched = false;
return {
get hasNext() {
return !fetched;
},
nextItems: async () => {
fetched = true;
return items;
},
};
};
const buildSandboxApi = (
overrides: Partial<SandboxApiMock> = {},
): { api: typeof Sandbox; mock: SandboxApiMock } => {
const mock: SandboxApiMock = {
list: jest.fn(() => paginatorOf([])),
connect: jest.fn(),
create: jest.fn(),
kill: jest.fn().mockResolvedValue(true),
...overrides,
};
return { api: mock as unknown as typeof Sandbox, mock };
};
const buildFakeSandbox = (): Sandbox =>
({
setTimeout: jest.fn().mockResolvedValue(undefined),
kill: jest.fn().mockResolvedValue(undefined),
}) as unknown as Sandbox;
const listedSandbox = (
sandboxId: string,
tag: string = sessionId,
): ListedSandbox => ({
sandboxId,
metadata: { [SESSION_SANDBOX_METADATA_KEY]: tag },
});
describe('getOrCreateSessionSandbox', () => {
it('reuses and extends the existing sandbox for the session', async () => {
const sandbox = buildFakeSandbox();
const { api, mock } = buildSandboxApi({
list: jest.fn(() => paginatorOf([listedSandbox('sbx-1')])),
connect: jest.fn().mockResolvedValue(sandbox),
});
const result = await getOrCreateSessionSandbox({
sandboxApi: api,
apiKey,
sessionId,
timeoutMs,
idleTimeoutMs,
});
expect(result).toEqual({ sandbox, isReused: true });
expect(mock.list).toHaveBeenCalledWith({
apiKey,
query: {
state: ['running', 'paused'],
metadata: { [SESSION_SANDBOX_METADATA_KEY]: sessionId },
},
});
expect(mock.connect).toHaveBeenCalledWith('sbx-1', { apiKey });
expect(sandbox.setTimeout).toHaveBeenCalledWith(aliveTimeoutMs);
expect(mock.create).not.toHaveBeenCalled();
expect(mock.kill).not.toHaveBeenCalled();
});
it('creates a new tagged auto-pausing sandbox when none exists for the session', async () => {
const sandbox = buildFakeSandbox();
const { api, mock } = buildSandboxApi({
list: jest.fn(() => paginatorOf([])),
create: jest.fn().mockResolvedValue(sandbox),
});
const result = await getOrCreateSessionSandbox({
sandboxApi: api,
apiKey,
sessionId,
timeoutMs,
idleTimeoutMs,
});
expect(result).toEqual({ sandbox, isReused: false });
expect(mock.create).toHaveBeenCalledWith({
apiKey,
timeoutMs: aliveTimeoutMs,
lifecycle: { onTimeout: 'pause', autoResume: true },
metadata: { [SESSION_SANDBOX_METADATA_KEY]: sessionId },
});
expect(mock.connect).not.toHaveBeenCalled();
});
it('reaps duplicate sandboxes and reuses a single one', async () => {
const keep = buildFakeSandbox();
const { api, mock } = buildSandboxApi({
list: jest.fn(() =>
paginatorOf([
listedSandbox('sbx-keep'),
listedSandbox('sbx-dup-1'),
listedSandbox('sbx-dup-2'),
]),
),
connect: jest.fn().mockResolvedValue(keep),
});
const result = await getOrCreateSessionSandbox({
sandboxApi: api,
apiKey,
sessionId,
timeoutMs,
idleTimeoutMs,
});
expect(result).toEqual({ sandbox: keep, isReused: true });
expect(mock.connect).toHaveBeenCalledTimes(1);
expect(mock.connect).toHaveBeenCalledWith('sbx-keep', { apiKey });
expect(mock.kill).toHaveBeenCalledWith('sbx-dup-1', { apiKey });
expect(mock.kill).toHaveBeenCalledWith('sbx-dup-2', { apiKey });
expect(mock.kill).not.toHaveBeenCalledWith('sbx-keep', { apiKey });
expect(mock.create).not.toHaveBeenCalled();
});
it('creates a fresh sandbox when connecting to the existing one fails', async () => {
const created = buildFakeSandbox();
const { api, mock } = buildSandboxApi({
list: jest.fn(() => paginatorOf([listedSandbox('sbx-dead')])),
connect: jest.fn().mockRejectedValue(new Error('sandbox not found')),
create: jest.fn().mockResolvedValue(created),
});
const result = await getOrCreateSessionSandbox({
sandboxApi: api,
apiKey,
sessionId,
timeoutMs,
idleTimeoutMs,
});
expect(result).toEqual({ sandbox: created, isReused: false });
expect(mock.create).toHaveBeenCalledTimes(1);
});
it('keeps the first connectable sandbox instead of cold-creating when an earlier one is dead', async () => {
const live = buildFakeSandbox();
const { api, mock } = buildSandboxApi({
list: jest.fn(() =>
paginatorOf([listedSandbox('sbx-dead'), listedSandbox('sbx-live')]),
),
connect: jest.fn((sandboxId: string) =>
sandboxId === 'sbx-live'
? Promise.resolve(live)
: Promise.reject(new Error('sandbox not found')),
),
});
const result = await getOrCreateSessionSandbox({
sandboxApi: api,
apiKey,
sessionId,
timeoutMs,
idleTimeoutMs,
});
expect(result).toEqual({ sandbox: live, isReused: true });
expect(live.setTimeout).toHaveBeenCalledWith(aliveTimeoutMs);
expect(mock.kill).toHaveBeenCalledWith('sbx-dead', { apiKey });
expect(mock.create).not.toHaveBeenCalled();
});
it('never reuses a sandbox whose metadata does not exactly match the session', async () => {
const created = buildFakeSandbox();
const { api, mock } = buildSandboxApi({
// A sandbox from a different session/tenant that a loose server-side match
// could surface — it must be ignored, never connected to or reaped.
list: jest.fn(() =>
paginatorOf([listedSandbox('sbx-foreign', 'workspace-2:thread-9')]),
),
create: jest.fn().mockResolvedValue(created),
});
const result = await getOrCreateSessionSandbox({
sandboxApi: api,
apiKey,
sessionId,
timeoutMs,
idleTimeoutMs,
});
expect(result).toEqual({ sandbox: created, isReused: false });
expect(mock.connect).not.toHaveBeenCalled();
expect(mock.kill).not.toHaveBeenCalled();
expect(mock.create).toHaveBeenCalledTimes(1);
});
it('kills a connected sandbox and creates fresh when refreshing its timeout fails', async () => {
const stale = {
setTimeout: jest
.fn()
.mockRejectedValue(new Error('timeout refresh failed')),
kill: jest.fn().mockResolvedValue(undefined),
} as unknown as Sandbox;
const created = buildFakeSandbox();
const { api, mock } = buildSandboxApi({
list: jest.fn(() => paginatorOf([listedSandbox('sbx-stale')])),
connect: jest.fn().mockResolvedValue(stale),
create: jest.fn().mockResolvedValue(created),
});
const result = await getOrCreateSessionSandbox({
sandboxApi: api,
apiKey,
sessionId,
timeoutMs,
idleTimeoutMs,
});
expect(result).toEqual({ sandbox: created, isReused: false });
expect(stale.setTimeout).toHaveBeenCalledWith(aliveTimeoutMs);
expect(stale.kill).toHaveBeenCalledTimes(1);
expect(mock.create).toHaveBeenCalledTimes(1);
});
});
@@ -0,0 +1,75 @@
import { type Sandbox } from '@e2b/code-interpreter';
import { SESSION_SANDBOX_METADATA_KEY } from 'src/engine/core-modules/code-interpreter/constants/session-sandbox-metadata-key.constant';
import { releaseSessionSandboxes } from 'src/engine/core-modules/code-interpreter/drivers/utils/release-session-sandboxes.util';
const apiKey = 'test-api-key';
const sessionId = 'workspace-1:thread-1';
type ListedSandbox = {
sandboxId: string;
metadata: Record<string, string>;
startedAt: string;
};
const paginatorOf = (items: ListedSandbox[]) => {
let fetched = false;
return {
get hasNext() {
return !fetched;
},
nextItems: async () => {
fetched = true;
return items;
},
};
};
const buildApi = (items: ListedSandbox[]) => {
const kill = jest.fn().mockResolvedValue(true);
const list = jest.fn(() => paginatorOf(items));
return { api: { list, kill } as unknown as typeof Sandbox, kill, list };
};
const sandboxInfo = (
sandboxId: string,
{ tag = sessionId }: { tag?: string | null } = {},
): ListedSandbox => ({
sandboxId,
metadata: tag === null ? {} : { [SESSION_SANDBOX_METADATA_KEY]: tag },
startedAt: new Date().toISOString(),
});
describe('releaseSessionSandboxes', () => {
it('kills every sandbox tagged for the session', async () => {
const { api, kill, list } = buildApi([
sandboxInfo('sbx-1'),
sandboxInfo('sbx-2'),
]);
await releaseSessionSandboxes(api, apiKey, sessionId);
expect(list).toHaveBeenCalledWith({
apiKey,
query: {
state: ['running', 'paused'],
metadata: { [SESSION_SANDBOX_METADATA_KEY]: sessionId },
},
});
expect(kill).toHaveBeenCalledWith('sbx-1', { apiKey });
expect(kill).toHaveBeenCalledWith('sbx-2', { apiKey });
});
it('ignores a sandbox whose tag does not match the session', async () => {
const { api, kill } = buildApi([
sandboxInfo('sbx-other', { tag: 'workspace-2:thread-9' }),
]);
await releaseSessionSandboxes(api, apiKey, sessionId);
expect(kill).not.toHaveBeenCalled();
});
});
@@ -0,0 +1,73 @@
import { type Sandbox } from '@e2b/code-interpreter';
import { SESSION_SANDBOX_METADATA_KEY } from 'src/engine/core-modules/code-interpreter/constants/session-sandbox-metadata-key.constant';
import { sweepExpiredSessionSandboxes } from 'src/engine/core-modules/code-interpreter/drivers/utils/sweep-expired-session-sandboxes.util';
const apiKey = 'test-api-key';
const sessionId = 'workspace-1:thread-1';
const dayMs = 24 * 60 * 60 * 1000;
type ListedSandbox = {
sandboxId: string;
metadata: Record<string, string>;
startedAt: string;
};
const paginatorOf = (items: ListedSandbox[]) => {
let fetched = false;
return {
get hasNext() {
return !fetched;
},
nextItems: async () => {
fetched = true;
return items;
},
};
};
const buildApi = (items: ListedSandbox[]) => {
const kill = jest.fn().mockResolvedValue(true);
const list = jest.fn(() => paginatorOf(items));
return { api: { list, kill } as unknown as typeof Sandbox, kill, list };
};
const sandboxInfo = (
sandboxId: string,
{ tag = sessionId, ageMs = 0 }: { tag?: string | null; ageMs?: number } = {},
): ListedSandbox => ({
sandboxId,
metadata: tag === null ? {} : { [SESSION_SANDBOX_METADATA_KEY]: tag },
startedAt: new Date(Date.now() - ageMs).toISOString(),
});
describe('sweepExpiredSessionSandboxes', () => {
it('kills only session sandboxes older than the max age', async () => {
const { api, kill } = buildApi([
sandboxInfo('sbx-old', { ageMs: 2 * dayMs }),
sandboxInfo('sbx-fresh', { ageMs: 60 * 1000 }),
sandboxInfo('sbx-untagged', { tag: null, ageMs: 5 * dayMs }),
]);
const killed = await sweepExpiredSessionSandboxes(api, apiKey, dayMs);
expect(killed).toBe(1);
expect(kill).toHaveBeenCalledWith('sbx-old', { apiKey });
expect(kill).not.toHaveBeenCalledWith('sbx-fresh', { apiKey });
expect(kill).not.toHaveBeenCalledWith('sbx-untagged', { apiKey });
});
it('lists both running and paused sandboxes', async () => {
const { api, list } = buildApi([]);
await sweepExpiredSessionSandboxes(api, apiKey, 1000);
expect(list).toHaveBeenCalledWith({
apiKey,
query: { state: ['running', 'paused'] },
});
});
});
@@ -0,0 +1,144 @@
import { type Sandbox } from '@e2b/code-interpreter';
import { isDefined } from 'twenty-shared/utils';
import { SESSION_SANDBOX_METADATA_KEY } from 'src/engine/core-modules/code-interpreter/constants/session-sandbox-metadata-key.constant';
type SandboxApi = typeof Sandbox;
type GetOrCreateSessionSandboxArgs = {
sandboxApi: SandboxApi;
apiKey: string;
sessionId: string;
timeoutMs: number;
idleTimeoutMs: number;
};
const listSandboxesForSession = async (
sandboxApi: SandboxApi,
apiKey: string,
sessionId: string,
) => {
const paginator = sandboxApi.list({
apiKey,
query: {
state: ['running', 'paused'],
metadata: { [SESSION_SANDBOX_METADATA_KEY]: sessionId },
},
});
const sandboxes: Awaited<ReturnType<typeof paginator.nextItems>> = [];
while (paginator.hasNext) {
sandboxes.push(...(await paginator.nextItems()));
}
// Re-check the tag client-side so a loose server-side match can never reuse
// another tenant's sandbox.
return sandboxes.filter(
(sandbox) => sandbox.metadata?.[SESSION_SANDBOX_METADATA_KEY] === sessionId,
);
};
const connectAndKeepAlive = async (
sandboxApi: SandboxApi,
apiKey: string,
sandboxId: string,
timeoutMs: number,
): Promise<Sandbox | undefined> => {
const sandbox = await sandboxApi
.connect(sandboxId, { apiKey })
.catch(() => undefined);
if (!isDefined(sandbox)) {
return undefined;
}
try {
await sandbox.setTimeout(timeoutMs);
return sandbox;
} catch {
// Couldn't refresh the timeout — kill it instead of leaking a running sandbox.
await sandbox.kill().catch(() => undefined);
return undefined;
}
};
const killSandboxById = (
sandboxApi: SandboxApi,
apiKey: string,
sandboxId: string,
) => sandboxApi.kill(sandboxId, { apiKey }).catch(() => undefined);
const createSessionSandbox = (
sandboxApi: SandboxApi,
apiKey: string,
sessionId: string,
timeoutMs: number,
) =>
sandboxApi.create({
apiKey,
timeoutMs,
lifecycle: { onTimeout: 'pause', autoResume: true },
metadata: { [SESSION_SANDBOX_METADATA_KEY]: sessionId },
});
export const getOrCreateSessionSandbox = async ({
sandboxApi,
apiKey,
sessionId,
timeoutMs,
idleTimeoutMs,
}: GetOrCreateSessionSandboxArgs): Promise<{
sandbox: Sandbox;
isReused: boolean;
}> => {
// The sandbox must outlive a single execution and the idle window before it
// auto-pauses, so take the larger of the two.
const aliveTimeoutMs = Math.max(timeoutMs, idleTimeoutMs);
const sessionSandboxes = await listSandboxesForSession(
sandboxApi,
apiKey,
sessionId,
);
let reusedSandbox: Sandbox | undefined;
let reusedSandboxId: string | undefined;
for (const { sandboxId } of sessionSandboxes) {
const sandbox = await connectAndKeepAlive(
sandboxApi,
apiKey,
sandboxId,
aliveTimeoutMs,
);
if (isDefined(sandbox)) {
reusedSandbox = sandbox;
reusedSandboxId = sandboxId;
break;
}
}
if (isDefined(reusedSandbox)) {
await Promise.all(
sessionSandboxes
.filter(({ sandboxId }) => sandboxId !== reusedSandboxId)
.map(({ sandboxId }) => killSandboxById(sandboxApi, apiKey, sandboxId)),
);
return { sandbox: reusedSandbox, isReused: true };
}
const sandbox = await createSessionSandbox(
sandboxApi,
apiKey,
sessionId,
aliveTimeoutMs,
);
return { sandbox, isReused: false };
};
@@ -0,0 +1,37 @@
import { type Sandbox } from '@e2b/code-interpreter';
import { SESSION_SANDBOX_METADATA_KEY } from 'src/engine/core-modules/code-interpreter/constants/session-sandbox-metadata-key.constant';
type SandboxApi = typeof Sandbox;
// Kills every sandbox tagged for a session — used when a conversation is deleted.
export const releaseSessionSandboxes = async (
sandboxApi: SandboxApi,
apiKey: string,
sessionId: string,
): Promise<void> => {
const paginator = sandboxApi.list({
apiKey,
query: {
state: ['running', 'paused'],
metadata: { [SESSION_SANDBOX_METADATA_KEY]: sessionId },
},
});
const sandboxes: Awaited<ReturnType<typeof paginator.nextItems>> = [];
while (paginator.hasNext) {
sandboxes.push(...(await paginator.nextItems()));
}
await Promise.all(
sandboxes
.filter(
(sandbox) =>
sandbox.metadata?.[SESSION_SANDBOX_METADATA_KEY] === sessionId,
)
.map((sandbox) =>
sandboxApi.kill(sandbox.sandboxId, { apiKey }).catch(() => undefined),
),
);
};
@@ -0,0 +1,42 @@
import { type Sandbox } from '@e2b/code-interpreter';
import { isDefined } from 'twenty-shared/utils';
import { SESSION_SANDBOX_METADATA_KEY } from 'src/engine/core-modules/code-interpreter/constants/session-sandbox-metadata-key.constant';
type SandboxApi = typeof Sandbox;
// E2B retains paused sandboxes indefinitely, so abandoned sessions need an age bound.
export const sweepExpiredSessionSandboxes = async (
sandboxApi: SandboxApi,
apiKey: string,
maxAgeMs: number,
): Promise<number> => {
const paginator = sandboxApi.list({
apiKey,
query: { state: ['running', 'paused'] },
});
const sandboxes: Awaited<ReturnType<typeof paginator.nextItems>> = [];
while (paginator.hasNext) {
sandboxes.push(...(await paginator.nextItems()));
}
const expiredBefore = Date.now() - maxAgeMs;
const expiredSandboxes = sandboxes.filter(
(sandbox) =>
isDefined(sandbox.metadata?.[SESSION_SANDBOX_METADATA_KEY]) &&
// startedAt is typed as Date but comes back as an ISO string at runtime.
new Date(sandbox.startedAt).getTime() < expiredBefore,
);
await Promise.all(
expiredSandboxes.map((sandbox) =>
sandboxApi.kill(sandbox.sandboxId, { apiKey }).catch(() => undefined),
),
);
return expiredSandboxes.length;
};