b768441c13aed1c978522f59d2cc326f3bd5a708
5 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Parship Chowdhury
|
4899f00bc7 |
fix: frontend build failing after code formatting library update (#21519)
Starting the frontend(`nx start twenty-front`) was failing because one of its dependencies - the client SDK - can not be built. A recent update made the sdk compatible with a newer version of Prettier. That update started using parts of prettier that the build setup did not recognize, so the build stopped with an error about a missing module. This PR updated the SDK’s build configuration, so it correctly handles those prettier imports. <!-- This is an auto-generated description by cubic. --> <a href="https://cubic.dev/pr/twentyhq/twenty/pull/21519?utm_source=github" target="_blank" rel="noopener noreferrer" data-no-image-dialog="true"><picture><source media="(prefers-color-scheme: dark)" srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source media="(prefers-color-scheme: light)" srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img alt="Review in cubic" src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a> <!-- End of auto-generated description by cubic. --> --------- Signed-off-by: Parship Chowdhury <parshipchowdhury@gmail.com> Co-authored-by: Charles Bochet <charles@twenty.com> |
||
|
Charles Bochet
|
4305a7dc84 |
fix(twenty-client-sdk): make genql codegen formatter prettier-3 compatible (fixes app-sync server crash) (#21354)
## Problem
Syncing a `twenty-sdk` app against a server (`twenty dev`) **crashes the
server process**. The metadata migration completes, then the server-side
`GqlTypeGenerator` regenerates typed clients via the vendored genql
codegen in `twenty-client-sdk`, which throws and exits node:
```
ConfigError: Couldn't find plugin for AST format "estree".
Plugins must be explicitly added to the standalone bundle.
at .../packages/twenty-client-sdk/dist/generate.cjs
Node.js v24.5.0 ← process exits
```
The CLI sees `ECONNRESET`; the app row still persists because the crash
happens after the metadata commit. Any app sync takes the server down.
## Root cause
The genql codegen formatter
[`prettify.ts`](https://github.com/twentyhq/twenty/blob/main/packages/twenty-client-sdk/src/generate/genql/helpers/prettify.ts)
was vendored (in #21339) targeting **prettier 2.8**: a synchronous
`format()` and `prettier/parser-typescript`, which in 2.8 bundles the
estree printer. `package.json` pins `prettier ^2.8.8`, but the monorepo
actually resolves/bundles **prettier 3.8.3** (`^2.8.8` is silently
unsatisfied — no 2.8.x is nested for this package, and the subpath
imports are bundled from the hoisted 3.8.3). Under prettier 3 the estree
printer must be added explicitly (`prettier/plugins/estree`) and
`format()` is async — so the codegen throws.
`prettier/parser-typescript` / `parser-graphql` don't even exist in
prettier 3 (only `prettier/plugins/*`), so the declared `^2.8.8` was
already inconsistent with what runs.
## Fix
- `prettify`: switch to the prettier-3 entrypoints
`prettier/plugins/{graphql,typescript,estree}`, `await` the async
`format()`, and fall back to the unformatted (still valid) code on any
failure so cosmetic formatting can never crash codegen again.
- `RenderContext.toCode` + `clientTasks`: propagate the now-async
`prettify` (await the four `toCode` call sites).
- Bump the declared `prettier` dependency `^2.8.8 → ^3.8.3` to match
what is actually used (only consumer; minimal lockfile diff).
## Verification (local, source server on :3000)
- `twenty dev --once` now completes: `Registering application → Syncing
manifest → Generating API client → ✓ Synced` with the **server staying
up**.
- The app integration test passes (full re-sync of 6 metadata objects +
`MetadataApiClient`/`CoreApiClient` CRUD through the generated genql
runtime).
- `nx build twenty-client-sdk` (incl. `tsgo` typecheck) passes.
Release note: this is a v2.11 blocker — without it, installing/syncing
any app crashes the server.
|
||
|
Charles Bochet
|
37b986aa4b |
security: vendor @genql/cli codegen to drop undici/native-fetch (#21339)
## What Vendors a narrowed copy of [`@genql/cli@3.0.5`](https://github.com/remorses/genql) (MIT) into `packages/twenty-client-sdk/src/generate/genql/` and repoints the two client generators at it, then removes `@genql/cli` from `twenty-client-sdk`, `twenty-sdk` and `create-twenty-app`. ## Why `@genql/cli` was used **only** to generate the typed GraphQL client from an SDL string. It is unmaintained and pulls in vulnerable/abandoned transitives — `undici@5` (**30 Dependabot alerts**), `native-fetch`, `listr`, `yargs`, etc. None of these were ever executed by Twenty: the sole consumer of `undici`/`native-fetch` is `@genql/cli`'s live-endpoint schema-introspection path, and Twenty always passes a schema string, never an endpoint. Removing the package eliminates the dependency at the source — for Twenty and for scaffolded end-user apps. ## What changed vs upstream The vendored copy (`genql/README.md` + `genql/LICENSE`) keeps the `render/` and `runtime/` trees verbatim and narrows the orchestration: - **Dropped the endpoint/introspection path** (`schema/fetchSchema.ts`) — the only `undici`/`native-fetch`/`qs` consumer. - **Dropped `listr`** — generation tasks run as plain sequential `async` functions (file contents unchanged). - **Replaced `fs-extra`/`mkdirp`/`rimraf`** with `node:fs`. - **Runtime templates are imported as `?raw`** and bundled, instead of read from `node_modules` at generation time. - **Kept `prettier@^2.8` and `@graphql-tools/*`** so the generated output is byte-for-byte identical. ## Verification - **Byte-identical output**: regenerating the metadata client from its committed schema produces a recursive-diff-clean result vs the previous `@genql/cli` output (including the copied `runtime/` folder). The core client generates and esbuild-bundles cleanly. - The public `twenty-client-sdk/generate` barrel API is unchanged (twenty-server / twenty-sdk consumers unaffected). - `undici@^5`, `native-fetch`, `@genql/cli`, `listr`, `yargs@^15` and `subscriptions-transport-ws@0.9` are gone from `yarn.lock` (net −364 lines). - `twenty-client-sdk` and `twenty-sdk` typecheck, lint and build; `twenty-client-sdk` tests pass (9/9). ## Notes - The vendored folder is excluded from `oxlint`/`oxfmt` (it is third-party code, with `@ts-nocheck` on the verbatim renderers, mirroring the generated output). - Stacks conceptually on #21334 (drops `@genql/runtime`); the two are independent and only overlap trivially in `yarn.lock`. `@genql/runtime` is intentionally left for that PR. |
||
|
martmull
|
27aea728df |
2474 add a utils to perform route trigger logic function requests in front components (#21330)
- exports `RestApiClient` from 'twenty-client-sdk/rest';` - documents the rest client |
||
|
Paul Rastoin
|
4ea2e32366 |
Refactor twenty client sdk provisioning for logic function and front-component (#18544)
## 1. The `twenty-client-sdk` Package (Source of Truth)
The monorepo package at `packages/twenty-client-sdk` ships with:
- A **pre-built metadata client** (static, generated from a fixed
schema)
- A **stub core client** that throws at runtime (`CoreApiClient was not
generated...`)
- Both ESM (`.mjs`) and CJS (`.cjs`) bundles in `dist/`
- A `package.json` with proper `exports` map for
`twenty-client-sdk/core`, `twenty-client-sdk/metadata`, and
`twenty-client-sdk/generate`
## 2. Generation & Upload (Server-Side, at Migration Time)
**When**: `WorkspaceMigrationRunnerService.run()` executes after a
metadata schema change.
**What happens in `SdkClientGenerationService.generateAndStore()`**:
1. Copies the stub `twenty-client-sdk` package from the server's assets
(resolved via `SDK_CLIENT_PACKAGE_DIRNAME` — from
`dist/assets/twenty-client-sdk/` in production, or from `node_modules`
in dev)
2. Filters out `node_modules/` and `src/` during copy — only
`package.json` + `dist/` are kept (like an npm publish)
3. Calls `replaceCoreClient()` which uses `@genql/cli` to introspect the
**application-scoped** GraphQL schema and generates a real
`CoreApiClient`, then compiles it to ESM+CJS and overwrites
`dist/core.mjs` and `dist/core.cjs`
4. Archives the **entire package** (with `package.json` + `dist/`) into
`twenty-client-sdk.zip`
5. Uploads the single archive to S3 under
`FileFolder.GeneratedSdkClient`
6. Sets `isSdkLayerStale = true` on the `ApplicationEntity` in the
database
## 3. Invalidation Signal
The `isSdkLayerStale` boolean column on `ApplicationEntity` is the
invalidation mechanism:
- **Set to `true`** by `generateAndStore()` after uploading a new client
archive
- **Checked** by both logic function drivers before execution — if
`true`, they rebuild their local layer
- **Set back to `false`** by `markSdkLayerFresh()` after the driver has
successfully consumed the new archive
Default is `false` so existing applications without a generated client
aren't affected.
## 4a. Logic Functions — Local Driver
**`ensureSdkLayer()`** is called before every execution:
1. Checks if the local SDK layer directory exists AND `isSdkLayerStale`
is `false` → early return
2. Otherwise, cleans the local layer directory
3. Calls `downloadAndExtractToPackage()` which streams the zip from S3
directly to disk and extracts the full package into
`<tmpdir>/sdk/<workspaceId>-<appId>/node_modules/twenty-client-sdk/`
4. Calls `markSdkLayerFresh()` to set `isSdkLayerStale = false`
**At execution time**, `assembleNodeModules()` symlinks everything from
the deps layer's `node_modules/` **except** `twenty-client-sdk`, which
is symlinked from the SDK layer instead. This ensures the logic
function's `import ... from 'twenty-client-sdk/core'` resolves to the
generated client.
## 4b. Logic Functions — Lambda Driver
**`ensureSdkLayer()`** is called during `build()`:
1. Checks if `isSdkLayerStale` is `false` and an existing Lambda layer
ARN exists → early return
2. Otherwise, deletes all existing layer versions for this SDK layer
name
3. Calls `downloadArchiveBuffer()` to get the raw zip from S3 (no disk
extraction)
4. Calls `reprefixZipEntries()` which streams the zip entries into a
**new zip** with the path prefix
`nodejs/node_modules/twenty-client-sdk/` — this is the Lambda layer
convention path. All done in memory, no disk round-trip
5. Publishes the re-prefixed zip as a new Lambda layer via
`publishLayer()`
6. Calls `markSdkLayerFresh()`
**At function creation**, the Lambda is created with **two layers**:
`[depsLayerArn, sdkLayerArn]`. The SDK layer is listed last so it
overwrites the stub `twenty-client-sdk` from the deps layer (later
layers take precedence in Lambda's `/opt` merge).
## 5. Front Components
Front components are built by `app:build` with `twenty-client-sdk/core`
and `twenty-client-sdk/metadata` as **esbuild externals**. The stored
`.mjs` in S3 has unresolved bare import specifiers like `import {
CoreApiClient } from 'twenty-client-sdk/core'`.
SDK import resolution is split between the **frontend host** (fetching &
caching SDK modules) and the **Web Worker** (rewriting imports):
**Server endpoints**:
- `GET /rest/front-components/:id` —
`FrontComponentService.getBuiltComponentStream()` returns the **raw
`.mjs`** directly from file storage. No bundling, no SDK injection.
- `GET /rest/sdk-client/:applicationId/:moduleName` —
`SdkClientController` reads a single file (e.g. `dist/core.mjs`) from
the generated SDK archive via
`SdkClientGenerationService.readFileFromArchive()` and serves it as
JavaScript.
**Frontend host** (`FrontComponentRenderer` in `twenty-front`):
1. Queries `FindOneFrontComponent` which returns `applicationId`,
`builtComponentChecksum`, `usesSdkClient`, and `applicationTokenPair`
2. If `usesSdkClient` is `true`, renders
`FrontComponentRendererWithSdkClient` which calls the
`useApplicationSdkClient` hook
3. `useApplicationSdkClient({ applicationId, accessToken })` checks the
Jotai atom family cache for existing blob URLs. On cache miss, fetches
both SDK modules from `GET /rest/sdk-client/:applicationId/core` and
`/metadata`, creates **blob URLs** for each, and stores them in the atom
family
4. Once the blob URLs are cached, passes them as `sdkClientUrls`
(already blob URLs, not server URLs) to `SharedFrontComponentRenderer` →
`FrontComponentWorkerEffect` → worker's `render()` call via
`HostToWorkerRenderContext`
**Worker** (`remote-worker.ts` in `twenty-sdk`):
1. Fetches the raw component `.mjs` source as text
2. If `sdkClientUrls` are provided and the source contains SDK import
specifiers (`twenty-client-sdk/core`, `twenty-client-sdk/metadata`),
**rewrites** the bare specifiers to the blob URLs received from the host
(e.g. `'twenty-client-sdk/core'` → `'blob:...'`)
3. Creates a blob URL for the rewritten source and `import()`s it
4. Revokes only the component blob URL after the module is loaded — the
SDK blob URLs are owned and managed by the host's Jotai cache
This approach eliminates server-side esbuild bundling on every request,
caches SDK modules per application in the frontend, and keeps the
worker's job to a simple string rewrite.
## Summary Diagram
```
app:build (SDK)
└─ twenty-client-sdk stub (metadata=real, core=stub)
│
▼
WorkspaceMigrationRunnerService.run()
└─ SdkClientGenerationService.generateAndStore()
├─ Copy stub package (package.json + dist/)
├─ replaceCoreClient() → regenerate core.mjs/core.cjs
├─ Zip entire package → upload to S3
└─ Set isSdkLayerStale = true
│
┌────────┴────────────────────┐
▼ ▼
Logic Functions Front Components
│ │
├─ Local Driver ├─ GET /rest/sdk-client/:appId/core
│ └─ downloadAndExtract │ → core.mjs from archive
│ → symlink into │
│ node_modules ├─ Host (useApplicationSdkClient)
│ │ ├─ Fetch SDK modules
└─ Lambda Driver │ ├─ Create blob URLs
└─ downloadArchiveBuffer │ └─ Cache in Jotai atom family
→ reprefixZipEntries │
→ publish as Lambda ├─ GET /rest/front-components/:id
layer │ → raw .mjs (no bundling)
│
└─ Worker (browser)
├─ Fetch component .mjs
├─ Rewrite imports → blob URLs
└─ import() rewritten source
```
## Next PR
- Estimate perf improvement by implementing a redis caching for front
component client storage ( we don't even cache front comp initially )
- Implem frontent blob invalidation sse event from server
---------
Co-authored-by: Charles Bochet <charlesBochet@users.noreply.github.com>
|
