d1b556f4a895c92f7af5d5eed336d6ec283f3f2a
2 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Paul Rastoin
|
71a1ff7ac8 |
Cache twenty-client-sdk modules host-side via content-addressed URLs (#22981)
## Context Front component sources are fetched host-side and integrity-verified by the SHA-256 checksum embedded in their URL, cached in Cache Storage — a layer that exists specifically because their download URLs are presigned and rotate. The `twenty-client-sdk` modules (`core` and `metadata`) were re-fetched on every render and could not be cached safely: their URLs carried no checksum and the server exposed no freshness signal. This PR makes the SDK module URLs **content-addressed** and relies on the **browser HTTP cache** for immutability, and it keys the checksums on their real owners: the **application** for `core`, the **instance** for `metadata`. The checksum does double duty: cache invalidation (regeneration changes the checksum → the URL changes → guaranteed cache miss) and a server-side cacheability guard (the server only grants `immutable` when the checksum in the URL matches the authoritative checksum it knows for that module — persisted at generation time for `core`, hashed once at bootstrap for `metadata` — so no per-request hashing of the served bytes). Note this is **not** an end-to-end integrity guarantee: there is no client-side hash verification, and on a fingerprint mismatch the server still serves the current bytes with `no-store` (self-healing for stale URLs) rather than failing. <img width="2412" height="926" alt="image" src="https://github.com/user-attachments/assets/d97935d2-0fdb-4c44-89ac-596b7ca8ca64" /> Closes twentyhq/core-team-issues#2688. ## Routes | Module | URL | Scope | | --- | --- | --- | | `core` | `/rest/sdk-client/{applicationId}/core[/{checksum}]` | Per application (generated bundle) | | `metadata` | `/rest/sdk-client/metadata[/{checksum}]` | **Instance-wide**: no application segment, so every application converges on one URL and the browser downloads the module once per release instead of once per application | The previous application-scoped metadata path (`/rest/sdk-client/{applicationId}/metadata[/{checksum}]`) is **kept for backward compatibility**, new clients just stop generating those URLs. The instance-wide route is declared before the parameterized route so `metadata/{checksum}` is not swallowed as `:applicationId/:moduleName`. ## Caching model | Request | `Cache-Control` | Effect | | --- | --- | --- | | Fingerprinted URL, checksum matches the known module checksum | `immutable` | Cached indefinitely by the browser HTTP cache; a new checksum is a new URL | | Fingerprinted URL, checksum does not match | `no-store` | Current bytes served uncached (self-healing for stale URLs) | | Bare URL (pre-generation fallback, `core` only in practice) | `no-store` | Never cached | - Both responses also set `X-Content-Type-Options: nosniff` and `Content-Type: application/javascript`. - SDK modules are intentionally **not** placed in Cache Storage. That layer stays reserved for the presigned/rotating component-source URLs; SDK modules are served directly and authenticated, so the browser HTTP cache (keyed by the content-addressed URL) is their single cache layer. ## Checksum provenance - **core** — per **application**, persisted on `application.sdkClientCoreChecksum` at generation time and read back from `flatApplicationMaps` (never re-hashed per request). - **metadata** — **instance-wide**, hashed once from the installed `twenty-client-sdk/dist/metadata.mjs` package (warmed at bootstrap, memoized per process) and served straight from that package, so it is fresh from the first request after a release with no archive dependency. ## Server (twenty-server) - Hash `dist/core.mjs` at SDK generation and persist `sdkClientCoreChecksum` via `applicationRepository.update`. Adds the nullable text column to `application.entity.ts` (mirroring `packageJsonChecksum`) plus a fast instance command with up/down; `FlatApplication` picks it up automatically. - New **application-scoped** query `applicationSdkClientChecksums(applicationId: UUID!): SdkClientChecksums` on `ApplicationResolver` (metadata schema, `WorkspaceAuthGuard` + `NoPermissionGuard`). `SdkClientChecksums.core` is **nullable** and stays `null` until the SDK has been generated at least once; `metadata` is **always present** (bootstrap-warmed), so the metadata module is cacheable from the very first render of any app. The query itself returns `null` only for unknown applications. - `SdkClientChecksumsDTO` now lives in the shared `core-modules/sdk-client/dtos/`. `FrontComponentDTO` and the `frontComponent` resolver no longer carry checksums (decoupled from the front-component row). - `sdk-client` controller: instance-wide `metadata[/:checksum]` route (no workspace-cache or application lookup, serves the memoized installed module) + application-scoped `:applicationId/:moduleName[/:checksum]` route (serves `core` from the per-application archive, `metadata` kept for back-compat). Cacheability compares the URL checksum against the **known** checksum — persisted `sdkClientCoreChecksum` for `core`, memoized package hash for `metadata` — instead of hashing the served bytes on every request: `immutable` on match, `no-store` otherwise (bare URL or stale fingerprint), plus `nosniff`. A persisted checksum out of sync with the archive only downgrades to `no-store` until the next regeneration. ## Front (twenty-front) - New metadata query `GetApplicationSdkClientChecksums`, keyed by `applicationId`; removed the `sdkClientChecksums` selection from `FindOneFrontComponent`. - `getSdkClientUrls` builds the two module URLs independently: `/sdk-client/{applicationId}/core/{checksum}` and the **instance-wide** `/sdk-client/metadata/{checksum}` (no application segment → one shared browser cache entry per release across all applications). Each falls back to its bare URL when its checksum is absent — since `core` is nullable, a never-generated app still gets a content-addressed metadata URL and only `core` falls back. The checksum type is sourced from the codegen `SdkClientChecksums` type rather than a hand-maintained duplicate. - `FrontComponentRenderer` is split into a gating outer component (runs `FindOneFrontComponent`, renders nothing while loading) and a content component that receives a guaranteed-non-null `frontComponent`. Following project conventions, the side effects live in dedicated effect components: `FrontComponentLoadErrorSnackBarEffect` (query error → snackbar) and `FrontComponentApplicationTokenPairEffect` (mirrors the query-derived token pair into component state unconditionally, `null` included, so revoked credentials can never be retained or refreshed). The content component fetches checksums via the application-keyed query and **gates the mount of SDK-using components on that query**, so the very first module fetch is always the content-addressed (`immutable`) URL instead of the bare `no-store` one. Non-SDK components skip the query and are never blocked. - **Live invalidation without reload:** SDK regeneration updates the application row, and the server broadcasts an `application` metadata event carrying the new core checksum. `useOnApplicationSdkClientChecksumsUpdated` / `useUpdateSdkClientChecksumsApolloCache` patch the application-keyed checksum query cache (core only; the instance-wide metadata is preserved), so every mounted component of that application picks up the new URL at once. This replaces the previous frontComponent-derived field and closes the earlier "known gap" (a mounted component staying on a session-old checksum until a full reload). The cache-patching callback is memoized (`useCallback`) so the window listener is registered once per application, and the listener is **skipped entirely** for non-SDK components (`useListenToMetadataOperationBrowserEvent` gained a `skip` option) — they register no listener and never refetch a query they don't consume. ## Renderer (twenty-front-component-renderer) - SDK sources are fetched through a dedicated plain authenticated fetch, `fetchJavaScriptModuleSourceText` (Bearer header, `credentials: 'omit'`), instead of the Cache Storage `fetchComponentSource` path; `fetchSdkClientSources` uses it. Execution stays exclusively in the opaque-origin worker via blob URLs; the host only fetches and forwards source strings (no hashing host-side). Staleness self-resolves through the checksum: new checksum → new URL → cache miss. <!-- This is an auto-generated description by cubic. --> <a href="https://cubic.dev/pr/twentyhq/twenty/pull/22981?utm_source=github" target="_blank" rel="noopener noreferrer" data-no-image-dialog="true"><picture><source media="(prefers-color-scheme: dark)" srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source media="(prefers-color-scheme: light)" srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img alt="Review in cubic" src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a> <!-- End of auto-generated description by cubic. --> |
||
|
Paul Rastoin
|
17d34a6fe3 |
[Front-comp-renderer] Host componentSource caching (#22958)
## Context The front component source cache introduced in the sandbox refactor was a silent no-op: it ran inside the sandboxed worker (opaque-origin `allow-scripts` iframe), where the `caches` global does not exist. Every render re-fetched the component JS from the network — nothing ever appeared in Cache Storage. ## Change <img width="1580" height="622" alt="image" src="https://github.com/user-attachments/assets/06903abf-b313-4d15-8db4-80950d4bf5ba" /> Moves component source resolution and caching from the worker to the host, where Cache Storage works: - `fetchComponentSource`, `fetchComponentSourceFromNetwork`, `frontComponentCacheStorageService` and `extractComponentChecksumFromUrl` relocated from `remote/worker/utils/` to `host/utils/` (`buildAuthorizationHeadersFromAccessToken` to shared `utils/`, still used by the worker for SDK module fetches) - `FrontComponentWorkerEffect` resolves the source before `thread.imports.render(...)` (with a cancellation guard) and passes `componentSource` in the render payload - `loadFrontComponentModule` no longer fetches: it keeps only sandbox-side work (SDK import rewrite, blob URL creation, `import()`) - New: stale-entry eviction — writing a new checksummed entry deletes older entries of the same `front-components/{id}/` prefix ## Security invariant The host only fetches, hashes and caches the source string — it never executes it. Execution stays exclusively in the opaque-origin worker via blob URL import. SHA-256 checksum verification is kept on both cache read (poisoned-entry guard: any same-origin code can write to Cache Storage) and cache write. ## Out of scope SDK client module caching — follow-up tracked in [twentyhq/core-team-issues#2688](https://github.com/twentyhq/core-team-issues/issues/2688), requires content-addressed URLs (server-side checksum at SDK generation time, exposed via GraphQL and embedded in the `/rest/sdk-client/...` URL), then reuses this same host-side cache path. <!-- This is an auto-generated description by cubic. --> <a href="https://cubic.dev/pr/twentyhq/twenty/pull/22958?utm_source=github" target="_blank" rel="noopener noreferrer" data-no-image-dialog="true"><picture><source media="(prefers-color-scheme: dark)" srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source media="(prefers-color-scheme: light)" srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img alt="Review in cubic" src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a> <!-- End of auto-generated description by cubic. --> |
