Charles Bochet
f5a42cdbae
## Why The AWS SDK defaults `requestTimeout` to `0`, which means *no timeout* — see `DEFAULT_REQUEST_TIMEOUT` in `@smithy/node-http-handler`. None of our clients overrode it, so a request that never completes holds its socket forever. Once that happens to `maxSockets` requests (default 50), the client's connection pool is permanently exhausted and every subsequent call on that process queues indefinitely rather than failing. To the caller it is indistinguishable from a hang. This caused a production incident on 2026-08-06. A single `twenty-server` pod reached: ``` @smithy/node-http-handler:WARN - socket usage at capacity=50 and 1004 additional requests are enqueued. ``` and never recovered — the queue grew monotonically and the pod completed **zero** Lambda invocations over 12 hours while its six siblings completed dozens each. Egress was fine (a direct HTTPS call to the Lambda API returned in 53ms) and the pod was never OOM-killed or restarted, so nothing surfaced as an error anywhere. The user-visible effect was that workspace creation hung. Activation reached `synchronizeTwentyStandardApplicationOrThrow`, blocked on a Lambda call, and never returned or threw — so the `catch` in `activateWorkspace` that resets a workspace to `PENDING_CREATION` never ran, and the workspace was stranded in `ONGOING_CREATION`. Retrying didn't help because the pod was still poisoned. With one bad pod out of seven, roughly one signup in seven failed: | pod | activations started | completed | |---|---|---| | healthy × 5 | 13 | 13 | | poisoned | 3 | **0** | Nothing appeared in Sentry, because nothing ever threw. ## What this changes - **Every AWS SDK client now sets `connectionTimeout` and `requestTimeout`** via a shared `buildAwsRequestHandlerOptions()` helper. A saturated pool now surfaces as an ordinary error the caller can catch and retry instead of hanging forever. This is the fix that matters. - **The Lambda client gets a higher ceiling and a larger socket pool.** Synchronous invocations legitimately hold a socket for as long as the function runs, so its `requestTimeout` clears `EXECUTOR_LAMBDA_TIMEOUT_SECONDS` (900s) with a minute to spare, and `maxSockets` goes to 200 so long invocations cannot starve the control-plane calls (layer lookups, waiters) sharing that client. - **`S3Client` and `STSClient` in `LambdaAwsClientService` are now reused.** Both were constructed on every call and never destroyed, so each leaked its own agent and socket pool. They are invalidated alongside `lambdaClient` when assume-role credentials refresh. No new dependency: `requestHandler` already accepts a plain `NodeHttpHandlerOptions` object. ## Deliberately not in scope - **A timeout around `activateWorkspace` itself.** A hung activation still strands a workspace in `ONGOING_CREATION` until the 5-minute stale-lock reclaim, and that only fires if the user happens to retry. Worth fixing separately. - **Alerting on `socket usage at capacity`.** That warning was the only signal this was happening and nobody was watching it — an infra change rather than a code one. ## Testing - Unit tests for the helper, including that the timeout is always non-zero. - `tsc --noEmit` clean on the touched files; `oxlint` reports 0 warnings and 0 errors. - Not reproducible in a test environment — the leak needs a saturated pool — so the mechanism above is evidenced from production logs and the SDK's own defaults rather than from a regression test. <!-- This is an auto-generated description by cubic. --> <a href="https://cubic.dev/pr/twentyhq/twenty/pull/23857?utm_source=github" target="_blank" rel="noopener noreferrer" data-no-image-dialog="true"><picture><source media="(prefers-color-scheme: dark)" srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source media="(prefers-color-scheme: light)" srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img alt="Review in cubic" src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a> <!-- End of auto-generated description by cubic. -->
The #1 Open-Source CRM
Website · 
Documentation · 
Roadmap · 
Discord · 
Figma
Why Twenty
Twenty gives technical teams the building blocks for a custom CRM that meets complex business needs and quickly adapts as the business evolves. Twenty is the CRM you build, ship, and version like the rest of your stack.
Learn more about why we built Twenty
Installation
Cloud
The fastest way to get started. Sign up at twenty.com and spin up a workspace in under a minute, with no infrastructure to manage and always up to date.
Build an app
Scaffold a new app with the Twenty CLI:
npx create-twenty-app my-app
Define objects, fields, and views as code:
import { defineObject, FieldType } from 'twenty-sdk/define';
export default defineObject({
nameSingular: 'deal',
namePlural: 'deals',
labelSingular: 'Deal',
labelPlural: 'Deals',
fields: [
{ name: 'name', label: 'Name', type: FieldType.TEXT },
{ name: 'amount', label: 'Amount', type: FieldType.CURRENCY },
{ name: 'closeDate', label: 'Close Date', type: FieldType.DATE_TIME },
],
});
Then ship it to your workspace:
npx twenty app:publish --private
See the app development guide for objects, views, agents, and logic functions.
Self-hosting
Run Twenty on your own infrastructure with Docker Compose, or contribute locally via the local setup guide.
Everything you need
Twenty gives you the building blocks of a modern CRM (objects, views, workflows, and agents) and lets you extend them as code. Here's a tour of what's in the box.
Want to go deeper? Read the 
User Guide for product walkthroughs, or the Documentation for developer reference.
|
|
|
|
|
|
Learn more about version control in doc
Stack
TypeScript
Nx
NestJS, with BullMQ, 
PostgreSQL, 
Redis
React, with Jotai, Linaria and Lingui
Thanks
Thanks to these amazing services that we use and recommend for code review (Greptile), catching bugs (Sentry) and translating (Crowdin).
Join the Community
Star the repo · Discord · 
Feature requests · Releases · 
X · 
LinkedIn · 
Crowdin · 
Contribute