Cache twenty-client-sdk modules host-side via content-addressed URLs (#22981)

## Context Front component sources are fetched host-side and integrity-verified by the SHA-256 checksum embedded in their URL, cached in Cache Storage — a layer that exists specifically because their download URLs are presigned and rotate. The `twenty-client-sdk` modules (`core` and `metadata`) were re-fetched on every render and could not be cached safely: their URLs carried no checksum and the server exposed no freshness signal. This PR makes the SDK module URLs **content-addressed** and relies on the **browser HTTP cache** for immutability, and it keys the checksums on their real owners: the **application** for `core`, the **instance** for `metadata`. The checksum does double duty: cache invalidation (regeneration changes the checksum → the URL changes → guaranteed cache miss) and a server-side cacheability guard (the server only grants `immutable` when the checksum in the URL matches the authoritative checksum it knows for that module — persisted at generation time for `core`, hashed once at bootstrap for `metadata` — so no per-request hashing of the served bytes). Note this is **not** an end-to-end integrity guarantee: there is no client-side hash verification, and on a fingerprint mismatch the server still serves the current bytes with `no-store` (self-healing for stale URLs) rather than failing. <img width="2412" height="926" alt="image" src="https://github.com/user-attachments/assets/d97935d2-0fdb-4c44-89ac-596b7ca8ca64" /> Closes twentyhq/core-team-issues#2688. ## Routes | Module | URL | Scope | | --- | --- | --- | | `core` | `/rest/sdk-client/{applicationId}/core[/{checksum}]` | Per application (generated bundle) | | `metadata` | `/rest/sdk-client/metadata[/{checksum}]` | **Instance-wide**: no application segment, so every application converges on one URL and the browser downloads the module once per release instead of once per application | The previous application-scoped metadata path (`/rest/sdk-client/{applicationId}/metadata[/{checksum}]`) is **kept for backward compatibility**, new clients just stop generating those URLs. The instance-wide route is declared before the parameterized route so `metadata/{checksum}` is not swallowed as `:applicationId/:moduleName`. ## Caching model | Request | `Cache-Control` | Effect | | --- | --- | --- | | Fingerprinted URL, checksum matches the known module checksum | `immutable` | Cached indefinitely by the browser HTTP cache; a new checksum is a new URL | | Fingerprinted URL, checksum does not match | `no-store` | Current bytes served uncached (self-healing for stale URLs) | | Bare URL (pre-generation fallback, `core` only in practice) | `no-store` | Never cached | - Both responses also set `X-Content-Type-Options: nosniff` and `Content-Type: application/javascript`. - SDK modules are intentionally **not** placed in Cache Storage. That layer stays reserved for the presigned/rotating component-source URLs; SDK modules are served directly and authenticated, so the browser HTTP cache (keyed by the content-addressed URL) is their single cache layer. ## Checksum provenance - **core** — per **application**, persisted on `application.sdkClientCoreChecksum` at generation time and read back from `flatApplicationMaps` (never re-hashed per request). - **metadata** — **instance-wide**, hashed once from the installed `twenty-client-sdk/dist/metadata.mjs` package (warmed at bootstrap, memoized per process) and served straight from that package, so it is fresh from the first request after a release with no archive dependency. ## Server (twenty-server) - Hash `dist/core.mjs` at SDK generation and persist `sdkClientCoreChecksum` via `applicationRepository.update`. Adds the nullable text column to `application.entity.ts` (mirroring `packageJsonChecksum`) plus a fast instance command with up/down; `FlatApplication` picks it up automatically. - New **application-scoped** query `applicationSdkClientChecksums(applicationId: UUID!): SdkClientChecksums` on `ApplicationResolver` (metadata schema, `WorkspaceAuthGuard` + `NoPermissionGuard`). `SdkClientChecksums.core` is **nullable** and stays `null` until the SDK has been generated at least once; `metadata` is **always present** (bootstrap-warmed), so the metadata module is cacheable from the very first render of any app. The query itself returns `null` only for unknown applications. - `SdkClientChecksumsDTO` now lives in the shared `core-modules/sdk-client/dtos/`. `FrontComponentDTO` and the `frontComponent` resolver no longer carry checksums (decoupled from the front-component row). - `sdk-client` controller: instance-wide `metadata[/:checksum]` route (no workspace-cache or application lookup, serves the memoized installed module) + application-scoped `:applicationId/:moduleName[/:checksum]` route (serves `core` from the per-application archive, `metadata` kept for back-compat). Cacheability compares the URL checksum against the **known** checksum — persisted `sdkClientCoreChecksum` for `core`, memoized package hash for `metadata` — instead of hashing the served bytes on every request: `immutable` on match, `no-store` otherwise (bare URL or stale fingerprint), plus `nosniff`. A persisted checksum out of sync with the archive only downgrades to `no-store` until the next regeneration. ## Front (twenty-front) - New metadata query `GetApplicationSdkClientChecksums`, keyed by `applicationId`; removed the `sdkClientChecksums` selection from `FindOneFrontComponent`. - `getSdkClientUrls` builds the two module URLs independently: `/sdk-client/{applicationId}/core/{checksum}` and the **instance-wide** `/sdk-client/metadata/{checksum}` (no application segment → one shared browser cache entry per release across all applications). Each falls back to its bare URL when its checksum is absent — since `core` is nullable, a never-generated app still gets a content-addressed metadata URL and only `core` falls back. The checksum type is sourced from the codegen `SdkClientChecksums` type rather than a hand-maintained duplicate. - `FrontComponentRenderer` is split into a gating outer component (runs `FindOneFrontComponent`, renders nothing while loading) and a content component that receives a guaranteed-non-null `frontComponent`. Following project conventions, the side effects live in dedicated effect components: `FrontComponentLoadErrorSnackBarEffect` (query error → snackbar) and `FrontComponentApplicationTokenPairEffect` (mirrors the query-derived token pair into component state unconditionally, `null` included, so revoked credentials can never be retained or refreshed). The content component fetches checksums via the application-keyed query and **gates the mount of SDK-using components on that query**, so the very first module fetch is always the content-addressed (`immutable`) URL instead of the bare `no-store` one. Non-SDK components skip the query and are never blocked. - **Live invalidation without reload:** SDK regeneration updates the application row, and the server broadcasts an `application` metadata event carrying the new core checksum. `useOnApplicationSdkClientChecksumsUpdated` / `useUpdateSdkClientChecksumsApolloCache` patch the application-keyed checksum query cache (core only; the instance-wide metadata is preserved), so every mounted component of that application picks up the new URL at once. This replaces the previous frontComponent-derived field and closes the earlier "known gap" (a mounted component staying on a session-old checksum until a full reload). The cache-patching callback is memoized (`useCallback`) so the window listener is registered once per application, and the listener is **skipped entirely** for non-SDK components (`useListenToMetadataOperationBrowserEvent` gained a `skip` option) — they register no listener and never refetch a query they don't consume. ## Renderer (twenty-front-component-renderer) - SDK sources are fetched through a dedicated plain authenticated fetch, `fetchJavaScriptModuleSourceText` (Bearer header, `credentials: 'omit'`), instead of the Cache Storage `fetchComponentSource` path; `fetchSdkClientSources` uses it. Execution stays exclusively in the opaque-origin worker via blob URLs; the host only fetches and forwards source strings (no hashing host-side). Staleness self-resolves through the checksum: new checksum → new URL → cache miss.  <a href="https://cubic.dev/pr/twentyhq/twenty/pull/22981?utm_source=github" target="_blank" rel="noopener noreferrer" data-no-image-dialog="true"><picture><source media="(prefers-color-scheme: dark)" srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source media="(prefers-color-scheme: light)" srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img alt="Review in cubic" src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a>
2026-07-21 17:05:00 +02:00
parent 8f704e87e3
commit 71a1ff7ac8
58 changed files with 2137 additions and 1048 deletions
@@ -0,0 +1,64 @@
+import { fetchJavaScriptModuleSourceText } from '@/host/utils/fetchJavaScriptModuleSourceText';
+
+const originalFetch = globalThis.fetch;
+
+describe('fetchJavaScriptModuleSourceText', () => {
+  afterEach(() => {
+    globalThis.fetch = originalFetch;
+  });
+
+  it('should return the response text when the response is ok', async () => {
+    globalThis.fetch = jest.fn(async () => ({
+      ok: true,
+      text: async () => 'module source',
+    })) as unknown as typeof fetch;
+
+    await expect(
+      fetchJavaScriptModuleSourceText('https://api.twenty.test/core/abc.js'),
+    ).resolves.toBe('module source');
+  });
+
+  it('should forward headers and omit credentials', async () => {
+    const fetchSpy = jest.fn(async () => ({
+      ok: true,
+      text: async () => '',
+    }));
+    globalThis.fetch = fetchSpy as unknown as typeof fetch;
+
+    await fetchJavaScriptModuleSourceText(
+      'https://api.twenty.test/core/abc.js',
+      { Authorization: 'Bearer token' },
+    );
+
+    expect(fetchSpy).toHaveBeenCalledWith(
+      'https://api.twenty.test/core/abc.js',
+      { headers: { Authorization: 'Bearer token' }, credentials: 'omit' },
+    );
+  });
+
+  it('should reject with a coded error when the response is not ok', async () => {
+    globalThis.fetch = jest.fn(async () => ({
+      ok: false,
+      status: 404,
+      statusText: 'Not Found',
+    })) as unknown as typeof fetch;
+
+    await expect(
+      fetchJavaScriptModuleSourceText('https://api.twenty.test/core/abc.js'),
+    ).rejects.toMatchObject({ code: 'FRONT_COMPONENT_MODULE_FETCH_FAILED' });
+  });
+
+  it('should wrap fetch rejections in a coded error', async () => {
+    globalThis.fetch = jest.fn(async () => {
+      throw new TypeError('Failed to fetch');
+    }) as unknown as typeof fetch;
+
+    await expect(
+      fetchJavaScriptModuleSourceText('https://api.twenty.test/core/abc.js'),
+    ).rejects.toMatchObject({
+      code: 'FRONT_COMPONENT_MODULE_FETCH_FAILED',
+      message:
+        'Failed to fetch SDK client module https://api.twenty.test/core/abc.js: Failed to fetch',
+    });
+  });
+});
@@ -0,0 +1,33 @@
+import { CustomError } from 'twenty-shared/utils';
+
+// Plain fetch for a JavaScript module served directly from a content-addressed,
+// immutable URL. The browser HTTP cache handles both reuse and checksum-based
+// invalidation (a new checksum yields a new URL), so no host-side CacheStorage
+// layer is needed here — that layer exists only to cache front-component
+// sources across the rotating presigned URLs they are served through.
+export const fetchJavaScriptModuleSourceText = async (
+  url: string,
+  headers?: Record<string, string>,
+): Promise<string> => {
+  let response: Response;
+
+  try {
+    response = await fetch(url, { headers, credentials: 'omit' });
+  } catch (error) {
+    throw new CustomError(
+      `Failed to fetch SDK client module ${url}: ${
+        error instanceof Error ? error.message : String(error)
+      }`,
+      'FRONT_COMPONENT_MODULE_FETCH_FAILED',
+    );
+  }
+
+  if (!response.ok) {
+    throw new CustomError(
+      `Failed to fetch SDK client module ${url}: ${response.status} ${response.statusText}`,
+      'FRONT_COMPONENT_MODULE_FETCH_FAILED',
+    );
+  }
+
+  return response.text();
+};
@@ -0,0 +1,18 @@
+import { fetchJavaScriptModuleSourceText } from '@/host/utils/fetchJavaScriptModuleSourceText';
+import { type SdkClientSources } from '@/types/SdkClientSources';
+import { type SdkClientUrls } from '@/types/SdkClientUrls';
+
+export const fetchSdkClientSources = async ({
+  sdkClientUrls,
+  headers,
+}: {
+  sdkClientUrls: SdkClientUrls;
+  headers?: Record<string, string>;
+}): Promise<SdkClientSources> => {
+  const [core, metadata] = await Promise.all([
+    fetchJavaScriptModuleSourceText(sdkClientUrls.core, headers),
+    fetchJavaScriptModuleSourceText(sdkClientUrls.metadata, headers),
+  ]);
+
+  return { core, metadata };
+};