db9da3194fafc591fc53cd5dfb3b84865bfec91d
8 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Rayan Salhab
|
6f0ac88e20 |
fix: batch viewGroup mutations sequentially to prevent race conditions (#19027)
## Bug Description When reordering stages in the Kanban board, the frontend fires all viewGroup update mutations concurrently via Promise.all, causing race conditions in the workspace migration runner's cache invalidation, database contention, and a thundering herd effect that stalls the server. ## Changes Changed `usePerformViewGroupAPIPersist` to execute viewGroup update mutations sequentially instead of concurrently. The `Promise.all` pattern fired all N mutations simultaneously, each triggering a full workspace migration runner pipeline (transaction + cache invalidation). The sequential `for...of` loop ensures each mutation completes (including its cache invalidation) before the next begins, eliminating the race condition. ## Related Issue Fixes #18865 ## Testing This fix addresses the root cause identified in the Sonarly analysis on the issue. The concurrent mutation pattern was causing: - PostgreSQL row-level lock contention on viewGroup rows - Cache thundering herd from repeated invalidation/recomputation cycles - Server stalls requiring container restarts The sequential approach ensures proper ordering and prevents these race conditions. --------- Co-authored-by: Rayan <rayan@example.com> Co-authored-by: Charles Bochet <charles@twenty.com> |
||
|
Paul Rastoin
|
c7f6036a47 |
[SDK] twenty-ui/display selective re-export to avoid bloating icons (#19010)
<img width="1946" height="792" alt="image" src="https://github.com/user-attachments/assets/d0abf62b-85d4-4f5f-b6dc-f2cc1c691f7e" /> Avoid re-exporting twenty-ui icons bundle that are massive ~4MB As discussed with @charlesBochet the problem should rather be treated at twenty-ui level at some point, that's quite a quick workaround in order to avoid overloading the twenty-sdk build size When time comes, where twenty-ui is mature enough to get published we will work on its bundle size |
||
|
Paul Rastoin
|
160a80cbcb | Bump prelease version sdk, sdk-client, create-twenty-app (#19000) | ||
|
Paul Rastoin
|
052aecccc7 |
Refactor dependency graph for SDK, client-sdk and create-app (#18963)
## Summary
### Externalize `twenty-client-sdk` from `twenty-sdk`
Previously, `twenty-client-sdk` was listed as a `devDependency` of
`twenty-sdk`, which caused Vite to bundle it inline into the dist
output. This meant end-user apps had two copies of `twenty-client-sdk`:
one hidden inside `twenty-sdk`'s bundle, and one installed explicitly in
their `node_modules`. These copies could drift apart since they weren't
guaranteed to be the same version.
**Change:** Moved `twenty-client-sdk` from `devDependencies` to
`dependencies` in `twenty-sdk/package.json`. Vite's `external` function
now recognizes it and keeps it as an external `require`/`import` in the
dist output. End users get a single deduplicated copy resolved by their
package manager.
### Externalize `twenty-sdk` from `create-twenty-app`
Similarly, `create-twenty-app` had `twenty-sdk` as a `devDependency`
(bundled inline). After refactoring `create-twenty-app` to
programmatically import operations from `twenty-sdk` (instead of
shelling out via `execSync`), it became a proper runtime dependency.
**Change:** Moved `twenty-sdk` from `devDependencies` to `dependencies`
in `create-twenty-app/package.json`.
### Switch E2E CI to `yarn npm publish`
The `workspace:*` protocol in `dependencies` is a Yarn-specific feature.
`npm publish` publishes it as-is (which breaks for consumers), while
`yarn npm publish` automatically replaces `workspace:*` with the
resolved version at publish time (e.g., `workspace:*` becomes `=1.2.3`).
**Change:** Replaced `npm publish` with `yarn npm publish` in
`.github/workflows/ci-create-app-e2e.yaml`.
### Replace `execSync` with programmatic SDK calls in
`create-twenty-app`
`create-twenty-app` was shelling out to `yarn twenty remote add` and
`yarn twenty server start` via `execSync`, which assumed the `twenty`
binary was already installed in the scaffolded app. This was fragile and
created an implicit circular dependency.
**Changes:**
- Replaced `execSync('yarn twenty remote add ...')` with a direct call
to `authLoginOAuth()` from `twenty-sdk/cli`
- Replaced `execSync('yarn twenty server start')` with a direct call to
`serverStart()` from `twenty-sdk/cli`
- Deleted the duplicated `setup-local-instance.ts` from
`create-twenty-app`
### Centralize `serverStart` as a dedicated operation
The Docker server start logic was previously inline in the `server
start` CLI command handler (`server.ts`), and `setup-local-instance.ts`
was shelling out to `yarn twenty server start` to invoke it -- meaning
`twenty-sdk` was calling itself via a child process.
**Changes:**
- Extracted the Docker container management logic into a new
`serverStart` operation (`cli/operations/server-start.ts`)
- Merged the detect-or-start flow from `setup-local-instance.ts` into
`serverStart` (detect across multiple ports, start Docker if needed,
poll for health)
- Deleted `setup-local-instance.ts` from `twenty-sdk`
- Added `onProgress` callback (consistent with other operations like
`appBuild`) instead of direct `console.log` calls
- Both the `server start` CLI command and `create-twenty-app` now call
`serverStart()` programmatically
related to https://github.com/twentyhq/twenty-infra/pull/525
|
||
|
Lucas Bordeau
|
33a474c8e6 |
Add record table widget feature flag (#18960)
Introduce a new feature flag for the record table widget, enabling conditional rendering and state management based on its status. Update related components and tests accordingly. --------- Co-authored-by: Charles Bochet <charlesBochet@users.noreply.github.com> |
||
|
Félix Malfait
|
5f0d6553f6 |
Add object type filter dropdown to side panel record searches (#18912)
## Summary - Adds an object type filter dropdown to the side panel, allowing users to scope record search results to a specific object type (e.g. People, Companies, Opportunities) - The filter appears as a funnel icon next to the search input in both the global search (SearchRecords page) and the "Pick a record" sidebar flow - Replaces the AI sparkles button on the search records page with the filter icon - Uses colored icons matching the navigation menu style (NavigationMenuItemStyleIcon + getStandardObjectIconColor) ## Test plan - [ ] Open the side panel, type something to enter SearchRecords mode, verify the filter icon appears - [ ] Click the filter icon, verify the dropdown opens with "Object" header, search input, "All Objects" and individual object types with colored icons - [ ] Select an object type, verify only records of that type appear in search results - [ ] Select "All Objects", verify all record types appear again - [ ] Verify the filter icon turns blue when a filter is active - [ ] In layout customization mode, add a new sidebar item > Record, verify the filter dropdown also works there - [ ] Close and reopen the side panel, verify the filter resets to "All Objects" - [ ] Verify the AI sparkles button no longer appears on the command menu root page - [ ] Verify the AI edit icon still appears on Ask AI pages Made with [Cursor](https://cursor.com) |
||
|
Abdul Rahman
|
37640521d5 |
Batch create, update, and delete navigation menu items (#18882)
Co-authored-by: Charles Bochet <charles@twenty.com> |
||
|
Paul Rastoin
|
4ea2e32366 |
Refactor twenty client sdk provisioning for logic function and front-component (#18544)
## 1. The `twenty-client-sdk` Package (Source of Truth)
The monorepo package at `packages/twenty-client-sdk` ships with:
- A **pre-built metadata client** (static, generated from a fixed
schema)
- A **stub core client** that throws at runtime (`CoreApiClient was not
generated...`)
- Both ESM (`.mjs`) and CJS (`.cjs`) bundles in `dist/`
- A `package.json` with proper `exports` map for
`twenty-client-sdk/core`, `twenty-client-sdk/metadata`, and
`twenty-client-sdk/generate`
## 2. Generation & Upload (Server-Side, at Migration Time)
**When**: `WorkspaceMigrationRunnerService.run()` executes after a
metadata schema change.
**What happens in `SdkClientGenerationService.generateAndStore()`**:
1. Copies the stub `twenty-client-sdk` package from the server's assets
(resolved via `SDK_CLIENT_PACKAGE_DIRNAME` — from
`dist/assets/twenty-client-sdk/` in production, or from `node_modules`
in dev)
2. Filters out `node_modules/` and `src/` during copy — only
`package.json` + `dist/` are kept (like an npm publish)
3. Calls `replaceCoreClient()` which uses `@genql/cli` to introspect the
**application-scoped** GraphQL schema and generates a real
`CoreApiClient`, then compiles it to ESM+CJS and overwrites
`dist/core.mjs` and `dist/core.cjs`
4. Archives the **entire package** (with `package.json` + `dist/`) into
`twenty-client-sdk.zip`
5. Uploads the single archive to S3 under
`FileFolder.GeneratedSdkClient`
6. Sets `isSdkLayerStale = true` on the `ApplicationEntity` in the
database
## 3. Invalidation Signal
The `isSdkLayerStale` boolean column on `ApplicationEntity` is the
invalidation mechanism:
- **Set to `true`** by `generateAndStore()` after uploading a new client
archive
- **Checked** by both logic function drivers before execution — if
`true`, they rebuild their local layer
- **Set back to `false`** by `markSdkLayerFresh()` after the driver has
successfully consumed the new archive
Default is `false` so existing applications without a generated client
aren't affected.
## 4a. Logic Functions — Local Driver
**`ensureSdkLayer()`** is called before every execution:
1. Checks if the local SDK layer directory exists AND `isSdkLayerStale`
is `false` → early return
2. Otherwise, cleans the local layer directory
3. Calls `downloadAndExtractToPackage()` which streams the zip from S3
directly to disk and extracts the full package into
`<tmpdir>/sdk/<workspaceId>-<appId>/node_modules/twenty-client-sdk/`
4. Calls `markSdkLayerFresh()` to set `isSdkLayerStale = false`
**At execution time**, `assembleNodeModules()` symlinks everything from
the deps layer's `node_modules/` **except** `twenty-client-sdk`, which
is symlinked from the SDK layer instead. This ensures the logic
function's `import ... from 'twenty-client-sdk/core'` resolves to the
generated client.
## 4b. Logic Functions — Lambda Driver
**`ensureSdkLayer()`** is called during `build()`:
1. Checks if `isSdkLayerStale` is `false` and an existing Lambda layer
ARN exists → early return
2. Otherwise, deletes all existing layer versions for this SDK layer
name
3. Calls `downloadArchiveBuffer()` to get the raw zip from S3 (no disk
extraction)
4. Calls `reprefixZipEntries()` which streams the zip entries into a
**new zip** with the path prefix
`nodejs/node_modules/twenty-client-sdk/` — this is the Lambda layer
convention path. All done in memory, no disk round-trip
5. Publishes the re-prefixed zip as a new Lambda layer via
`publishLayer()`
6. Calls `markSdkLayerFresh()`
**At function creation**, the Lambda is created with **two layers**:
`[depsLayerArn, sdkLayerArn]`. The SDK layer is listed last so it
overwrites the stub `twenty-client-sdk` from the deps layer (later
layers take precedence in Lambda's `/opt` merge).
## 5. Front Components
Front components are built by `app:build` with `twenty-client-sdk/core`
and `twenty-client-sdk/metadata` as **esbuild externals**. The stored
`.mjs` in S3 has unresolved bare import specifiers like `import {
CoreApiClient } from 'twenty-client-sdk/core'`.
SDK import resolution is split between the **frontend host** (fetching &
caching SDK modules) and the **Web Worker** (rewriting imports):
**Server endpoints**:
- `GET /rest/front-components/:id` —
`FrontComponentService.getBuiltComponentStream()` returns the **raw
`.mjs`** directly from file storage. No bundling, no SDK injection.
- `GET /rest/sdk-client/:applicationId/:moduleName` —
`SdkClientController` reads a single file (e.g. `dist/core.mjs`) from
the generated SDK archive via
`SdkClientGenerationService.readFileFromArchive()` and serves it as
JavaScript.
**Frontend host** (`FrontComponentRenderer` in `twenty-front`):
1. Queries `FindOneFrontComponent` which returns `applicationId`,
`builtComponentChecksum`, `usesSdkClient`, and `applicationTokenPair`
2. If `usesSdkClient` is `true`, renders
`FrontComponentRendererWithSdkClient` which calls the
`useApplicationSdkClient` hook
3. `useApplicationSdkClient({ applicationId, accessToken })` checks the
Jotai atom family cache for existing blob URLs. On cache miss, fetches
both SDK modules from `GET /rest/sdk-client/:applicationId/core` and
`/metadata`, creates **blob URLs** for each, and stores them in the atom
family
4. Once the blob URLs are cached, passes them as `sdkClientUrls`
(already blob URLs, not server URLs) to `SharedFrontComponentRenderer` →
`FrontComponentWorkerEffect` → worker's `render()` call via
`HostToWorkerRenderContext`
**Worker** (`remote-worker.ts` in `twenty-sdk`):
1. Fetches the raw component `.mjs` source as text
2. If `sdkClientUrls` are provided and the source contains SDK import
specifiers (`twenty-client-sdk/core`, `twenty-client-sdk/metadata`),
**rewrites** the bare specifiers to the blob URLs received from the host
(e.g. `'twenty-client-sdk/core'` → `'blob:...'`)
3. Creates a blob URL for the rewritten source and `import()`s it
4. Revokes only the component blob URL after the module is loaded — the
SDK blob URLs are owned and managed by the host's Jotai cache
This approach eliminates server-side esbuild bundling on every request,
caches SDK modules per application in the frontend, and keeps the
worker's job to a simple string rewrite.
## Summary Diagram
```
app:build (SDK)
└─ twenty-client-sdk stub (metadata=real, core=stub)
│
▼
WorkspaceMigrationRunnerService.run()
└─ SdkClientGenerationService.generateAndStore()
├─ Copy stub package (package.json + dist/)
├─ replaceCoreClient() → regenerate core.mjs/core.cjs
├─ Zip entire package → upload to S3
└─ Set isSdkLayerStale = true
│
┌────────┴────────────────────┐
▼ ▼
Logic Functions Front Components
│ │
├─ Local Driver ├─ GET /rest/sdk-client/:appId/core
│ └─ downloadAndExtract │ → core.mjs from archive
│ → symlink into │
│ node_modules ├─ Host (useApplicationSdkClient)
│ │ ├─ Fetch SDK modules
└─ Lambda Driver │ ├─ Create blob URLs
└─ downloadArchiveBuffer │ └─ Cache in Jotai atom family
→ reprefixZipEntries │
→ publish as Lambda ├─ GET /rest/front-components/:id
layer │ → raw .mjs (no bundling)
│
└─ Worker (browser)
├─ Fetch component .mjs
├─ Rewrite imports → blob URLs
└─ import() rewritten source
```
## Next PR
- Estimate perf improvement by implementing a redis caching for front
component client storage ( we don't even cache front comp initially )
- Implem frontent blob invalidation sse event from server
---------
Co-authored-by: Charles Bochet <charlesBochet@users.noreply.github.com>
|