From 9e20e2222a6b867567f49ddc2c16ea07db54cca7 Mon Sep 17 00:00:00 2001 From: Paul Rastoin <45004772+prastoin@users.noreply.github.com> Date: Fri, 10 Jul 2026 12:12:58 +0200 Subject: [PATCH] Fix front-component serving on Safari, kill stale presigned caching, and cache built bundles client-side (#22672) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit ## Context Built front-component bundles are served via `GET /rest/front-components/:id/:cacheKey`. On S3-backed storage (Twenty Cloud) the endpoint used to 302-redirect the worker's authenticated fetch to a presigned S3 URL. That redirect caused two bugs, and fixing it removed the caching the redirect was accidentally providing — so this PR also adds a proper client-side cache. Closes twentyhq/core-team-issues#2653. ### Bug 1 — Safari 403 (Authorization header forwarded across redirect) The renderer worker fetches the bundle with `Authorization: Bearer`. The controller answered with a 302 to a presigned S3 URL. Per the Fetch spec, browsers must strip `Authorization` on a cross-origin redirect. Chrome/Firefox do, but Safari/WebKit forwards it, so S3 receives both a query-string signature and an `Authorization` header and rejects with `InvalidArgument: Only one auth mechanism allowed`. Result: front components never load in Safari on S3-backed storage. ### Bug 2 — 302 cached publicly (browser-independent) The redirect branch set no `Cache-Control`, so a CDN could cache it far beyond the presigned URL's TTL (`STORAGE_S3_PRESIGNED_URL_EXPIRES_IN`, 900s). Consequences: any client re-served the cached 302 after 15 min hits an expired signature (403, also affects Chrome), and the cached redirect containing a live presigned URL is served to unauthenticated requests (short-lived auth bypass). ### Regression this introduces — warm-load caching lost Marking the handoff `no-store` (Bug 2 fix) is correct, but it means the built bundle is no longer cached anywhere on the S3 path. The browser HTTP cache cannot compensate: the presigned URL that actually returns the bytes carries a fresh `X-Amz-Date`/`X-Amz-Signature` on every request, so each download is a brand-new cache key and never hits. Net effect without mitigation: every worker mount re-downloads the full bundle. ## What changed - **Front components return a 200 JSON body instead of a 302.** The controller now responds `200 { url }` with `Cache-Control: private, no-store`. The worker parses the JSON and issues a separate header-less `fetch(url)` to S3. No redirect means the `Authorization` header is never forwarded, making it browser-independent, and the handoff carrying the presigned URL is never cached. The stream path (local storage) is unchanged. - **Client-side bundle cache in the renderer (restores warm loads).** `fetchComponentSource` wraps the fetch chain in a `CacheStorage` layer keyed by the **content-addressed** `/front-components/:id/:checksum.js` URL. A hit returns the stored bundle and skips **both** the `no-store` handoff to Twenty and the S3 download — restoring cross-session warm loads without ever persisting a presigned credential. Because `CacheStorage` is writable by any same-origin code (including the untrusted component code this cache feeds), cached content is verified against the sha-256 checksum embedded in the URL on every read, and evicted on mismatch. Caching degrades to a plain fetch where `CacheStorage` or WebCrypto is unavailable. - **sha-256 checksums for built front components.** The SDK build and workspace prefill now fingerprint built front-component bundles with sha-256 (WebCrypto has no md5), enabling the integrity check above. Other file folders keep md5. Legacy md5-fingerprinted URLs (32-hex) simply bypass the cache — already-synced components keep working and start benefiting from caching on their next build/sync. - **WebKit e2e coverage.** Added a `webkit` project to the postcard example's Playwright config mirroring `chrome` (shared setup + storageState), plus iframe/worker diagnostics logging so front-component failures surface in the test log. `TZ` is pinned to `Europe/Paris` because WebKit on Linux ignores Playwright's `timezoneId` emulation and rejects the runner's legacy `CET` alias, which crashed the record page before the component could render. ### Why we hand off to S3 instead of streaming through Twenty On S3-backed storage we deliberately **do not** proxy/stream the bundle bytes through the API. The controller returns the presigned URL and the worker fetches the content directly from S3, for two reasons: - **Server CPU/bandwidth.** Streaming every bundle on every cold load would put the API server on the hot path for all front-component content. Handing off to S3 keeps that load off the server. - **Domain isolation.** Front-component content is fetched from the object-storage domain (e.g. `s3.domain.com`), a different origin than the API and the front app. Serving untrusted/app-authored bundle content from a separate domain than `twenty.com` keeps it off the app's origin. The stream path is kept only as the local-storage fallback (no S3/presign available), where these concerns don't apply. ## Examples ### The JSON handoff (S3 path) ```http GET /rest/front-components/d3b07384-.../a1b2c3d4.js HTTP/1.1 Host: twenty.com Authorization: Bearer ``` ```http HTTP/1.1 200 OK Content-Type: application/json Cache-Control: private, no-store {"url":"https://s3.domain.com/bucket/.../checkout-widget.mjs?X-Amz-Date=20260709T091500Z&X-Amz-Expires=900&...&X-Amz-Signature=AAAA1111..."} ``` The worker then fetches that presigned URL **without** headers (the Safari fix) and gets the bundle bytes. ### Why the browser HTTP cache can't reuse it | | Load 1 (09:15) | Load 2 (09:30) | Same key? | |---|---|---|---| | Twenty handoff URL | `.../a1b2c3d4.js` | `.../a1b2c3d4.js` | ✅ but response is `no-store` | | Presigned `X-Amz-Signature` | `AAAA1111...` | `ZZZZ9999...` | ❌ | | Effective S3 URL (the HTTP cache key) | `...&X-Amz-Signature=AAAA1111...` | `...&X-Amz-Signature=ZZZZ9999...` | ❌ new key → miss | ### What the CacheStorage layer stores ``` key = https://twenty.com/rest/front-components/d3b07384-.../a1b2c3d4.js (stable, chosen by us) value = (NOT the presigned URL) ``` Keying by the stable logical URL (not the volatile URL the bytes arrived from) is the one thing the native HTTP cache can't express. The presigned URL is used once and discarded. ### Invalidation No TTL and no explicit delete — invalidation is by key change. A rebuild changes the checksum → changes the URL → guaranteed miss on the new key. The old entry is orphaned and reclaimed by normal browser eviction (quota/LRU; Safari ITP after 7 idle days). Global invalidation lever: bump the cache name suffix (`front-component-source-v1`). ## Deploy note — front/server release window Old frontend bundles (already-open tabs) hitting the new server receive the JSON handoff where they expect raw JS and fail to render until the tab is reloaded. The other direction is safe: the new worker against an old server follows the 302 transparently (the content-type check falls through to `response.text()`). Accepted as a short deploy-window trade-off. ## Follow-ups (not in this PR) - The client-side cache is a bridge for the `no-store` presigned handoff. If built components are later served from a stable, non-signed, public-by-URL path (they are already content-addressed by checksum, so `immutable` is safe), the browser + CDN cache natively and this custom layer can be removed. - `GET /file/:fileFolder/:id` presigned 302s still carry no `Cache-Control`. An explicit policy there (bounded `private, max-age` below the presigned TTL) was prototyped in this PR and deliberately dropped to keep the scope on front components — the file path authenticates via a query-param token (part of any cache key), so its exposure differs and deserves its own PR. ## Non-goals Per the issue, file serving keeps its query-param token + 302 model. Native browser loads (``, downloads) cannot do a two-step fetch and already work on Safari. The public-asset redirect is left untouched since its caching is intentional. ## Test plan - Renderer: `fetchComponentSource.spec.ts` covers cache miss + write, verified cache hit (no network), poisoned-entry eviction, checksum-mismatch (never cached), non-fingerprinted and legacy-md5 URL bypass, and the no-`CacheStorage` / no-WebCrypto fallbacks. `fetchComponentSourceFromNetwork.spec.ts` covers the direct JS response, the JSON handoff follow-through (header-less presigned fetch), and error mapping. - e2e: the postcard front-component spec now runs on both Chromium and WebKit against prod-parity storage (S3 + Lambda). - `oxlint` + `oxfmt` clean; typecheck passes on changed packages. ### Reproduction proof — Safari was always broken (e2e probe) We ran the prod-parity postcard e2e suite (S3 storage + Lambda) with WebKit against **`main` without this fix**, via a throwaway probe PR: twentyhq/twenty#22717. Result — [ci-privileged run 29015624468](https://github.com/twentyhq/ci-privileged/actions/runs/29015624468): ``` 1 failed [webkit] › card-front-component.spec.ts:61 › renders the postcard name and status badge in the record preview 2 passed (1.4m) ``` `[webkit]` times out waiting for `getByTestId('postcard-card')` to become visible (*element(s) not found*) while the Chromium run of the same spec passes. This confirms the front component **never rendered in Safari** on S3-backed storage prior to this PR — it is a genuine, browser-specific bug, not a flake. The fix in this PR is expected to turn that same `[webkit]` assertion green. Note: running the WebKit tests in CI requires the WebKit browser binary and its system dependencies in the e2e job (now installed via `npx playwright install --with-deps chromium webkit`). --- .../postcard/e2e/card-front-component.spec.ts | 36 +++ .../examples/postcard/playwright.config.ts | 20 ++ .../src/remote/worker/remote-worker.ts | 24 +- .../__tests__/fetchComponentSource.spec.ts | 263 ++++++++++++++++++ .../fetchComponentSourceFromNetwork.spec.ts | 135 +++++++++ .../utils/extractComponentChecksumFromUrl.ts | 11 + .../worker/utils/fetchComponentSource.ts | 54 ++++ .../utils/fetchComponentSourceFromNetwork.ts | 41 +++ .../frontComponentCacheStorageService.ts | 79 ++++++ .../build/common/esbuild-result-processor.ts | 12 +- .../file/interfaces/file-folder.interface.ts | 5 + .../controllers/front-component.controller.ts | 5 +- .../front-component.service.ts | 4 - .../prefill-front-component.service.ts | 4 +- 14 files changed, 665 insertions(+), 28 deletions(-) create mode 100644 packages/twenty-front-component-renderer/src/remote/worker/utils/__tests__/fetchComponentSource.spec.ts create mode 100644 packages/twenty-front-component-renderer/src/remote/worker/utils/__tests__/fetchComponentSourceFromNetwork.spec.ts create mode 100644 packages/twenty-front-component-renderer/src/remote/worker/utils/extractComponentChecksumFromUrl.ts create mode 100644 packages/twenty-front-component-renderer/src/remote/worker/utils/fetchComponentSource.ts create mode 100644 packages/twenty-front-component-renderer/src/remote/worker/utils/fetchComponentSourceFromNetwork.ts create mode 100644 packages/twenty-front-component-renderer/src/remote/worker/utils/frontComponentCacheStorageService.ts diff --git a/packages/twenty-apps/examples/postcard/e2e/card-front-component.spec.ts b/packages/twenty-apps/examples/postcard/e2e/card-front-component.spec.ts index fae5d31d85..de05756711 100644 --- a/packages/twenty-apps/examples/postcard/e2e/card-front-component.spec.ts +++ b/packages/twenty-apps/examples/postcard/e2e/card-front-component.spec.ts @@ -58,6 +58,42 @@ test.describe('Postcard card front component', () => { } }); + // The front component renders in a separate iframe + Web Worker, so its + // failures never reach the assertion output. Surface browser console output, + // uncaught errors, failed requests, and 4xx/5xx responses (e.g. a presigned + // S3 403) in the test log so CI failures are diagnosable without the trace. + test.beforeEach(({ page }) => { + page.on('console', (message) => { + console.log(`[browser:${message.type()}] ${message.text()}`); + }); + + page.on('pageerror', (error) => { + console.log(`[pageerror] ${error.message}\n${error.stack ?? ''}`); + }); + + page.on('requestfailed', (request) => { + console.log( + `[requestfailed] ${request.method()} ${request.url()} — ${ + request.failure()?.errorText ?? 'unknown error' + }`, + ); + }); + + page.on('response', (response) => { + if (response.status() >= 400) { + console.log( + `[response ${response.status()}] ${response + .request() + .method()} ${response.url()}`, + ); + } + }); + + page.on('worker', (worker) => { + console.log(`[worker started] ${worker.url()}`); + }); + }); + test('renders the postcard name and status badge in the record preview', async ({ page, }) => { diff --git a/packages/twenty-apps/examples/postcard/playwright.config.ts b/packages/twenty-apps/examples/postcard/playwright.config.ts index e74fc22876..69d0fd8984 100644 --- a/packages/twenty-apps/examples/postcard/playwright.config.ts +++ b/packages/twenty-apps/examples/postcard/playwright.config.ts @@ -1,6 +1,14 @@ import { defineConfig, devices } from '@playwright/test'; import * as path from 'path'; +// WebKit on Linux ignores Playwright's `timezoneId` context emulation for +// Intl.resolvedOptions(), so it falls back to the runner's OS zone — the legacy +// `CET` alias, which WebKit's ICU rejects. `formatInTimeZone` then throws and +// crashes the record page before the front component can render. Pinning TZ +// here forces a WebKit-supported IANA zone at the OS/ICU level (the layer +// WebKit actually reads); the browser process Playwright spawns inherits it. +process.env.TZ = 'Europe/Paris'; + // Front-end base URL of the running Twenty instance under test. const FRONT_BASE_URL = process.env.FRONT_BASE_URL ?? 'http://localhost:3001'; @@ -19,6 +27,10 @@ export default defineConfig({ screenshot: 'only-on-failure', headless: true, testIdAttribute: 'data-testid', + // Browser-level timezone emulation. Honored by Chromium; WebKit/Linux + // ignores it for Intl.resolvedOptions() (see the TZ pin above, which is the + // actual fix there). Kept aligned with TZ for deterministic date rendering. + timezoneId: 'Europe/Paris', }, expect: { timeout: 15_000, @@ -36,5 +48,13 @@ export default defineConfig({ }, dependencies: ['setup'], }, + { + name: 'webkit', + use: { + ...devices['Desktop Safari'], + storageState: path.resolve(__dirname, 'e2e/.auth/user.json'), + }, + dependencies: ['setup'], + }, ], }); diff --git a/packages/twenty-front-component-renderer/src/remote/worker/remote-worker.ts b/packages/twenty-front-component-renderer/src/remote/worker/remote-worker.ts index 2e02fc71ce..1d2cc526ca 100644 --- a/packages/twenty-front-component-renderer/src/remote/worker/remote-worker.ts +++ b/packages/twenty-front-component-renderer/src/remote/worker/remote-worker.ts @@ -15,6 +15,7 @@ import { isDefined } from 'twenty-shared/utils'; import { installStyleBridge } from '@/polyfills/installStyleBridge'; import { installStylePropertyOnRemoteElements } from '@/remote/utils/installStylePropertyOnRemoteElements'; import { patchRemoteElementAttributes } from '@/remote/utils/patchRemoteElementAttributes'; +import { fetchComponentSource } from './utils/fetchComponentSource'; import { installErrorEventBridge } from './utils/installErrorEventBridge'; import { type FrontComponentExecutionContext } from 'twenty-sdk/front-component'; import { frontComponentHostCommunicationApi } from '@/constants/frontComponentHostCommunicationApi'; @@ -38,21 +39,6 @@ exposeGlobals({ __HTML_TAG_TO_CUSTOM_ELEMENT_TAG__: HTML_TAG_TO_CUSTOM_ELEMENT_TAG, }); -const fetchComponentSource = async ( - url: string, - headers?: Record, -): Promise => { - const response = await fetch(url, { headers }); - - if (!response.ok) { - throw new Error( - `Failed to fetch ${url}: ${response.status} ${response.statusText}`, - ); - } - - return response.text(); -}; - const SDK_IMPORT_SPECIFIERS = [ 'twenty-client-sdk/core', 'twenty-client-sdk/metadata', @@ -122,10 +108,10 @@ const render: WorkerExports['render'] = async ( ? { Authorization: `Bearer ${renderContext.applicationAccessToken}` } : undefined; - const componentSource = await fetchComponentSource( - renderContext.componentUrl, - authHeaders, - ); + const componentSource = await fetchComponentSource({ + url: renderContext.componentUrl, + headers: authHeaders, + }); const hasSdkImports = isDefined(renderContext.sdkClientUrls) && diff --git a/packages/twenty-front-component-renderer/src/remote/worker/utils/__tests__/fetchComponentSource.spec.ts b/packages/twenty-front-component-renderer/src/remote/worker/utils/__tests__/fetchComponentSource.spec.ts new file mode 100644 index 0000000000..ff3ca7e9c2 --- /dev/null +++ b/packages/twenty-front-component-renderer/src/remote/worker/utils/__tests__/fetchComponentSource.spec.ts @@ -0,0 +1,263 @@ +import { createHash, webcrypto } from 'node:crypto'; +import { TextEncoder as NodeTextEncoder } from 'node:util'; + +import { fetchComponentSource } from '@/remote/worker/utils/fetchComponentSource'; + +const COMPONENT_SOURCE = 'export default () => {};'; + +const computeSha256Hex = (content: string): string => + createHash('sha256').update(content).digest('hex'); + +const buildFingerprintedUrl = (checksum: string): string => + `https://api.twenty.com/rest/front-components/component-id/${checksum}.js`; + +const FINGERPRINTED_URL = buildFingerprintedUrl( + computeSha256Hex(COMPONENT_SOURCE), +); +const BARE_URL = 'https://api.twenty.com/rest/front-components/component-id'; +const LEGACY_MD5_URL = buildFingerprintedUrl( + createHash('md5').update(COMPONENT_SOURCE).digest('hex'), +); + +const createFakeJsResponse = (body: string) => ({ + ok: true, + status: 200, + statusText: 'OK', + headers: { + get: (name: string) => + name === 'content-type' ? 'application/javascript' : null, + }, + text: jest.fn(async () => body), + json: jest.fn(async () => undefined), +}); + +class FakeCache { + private readonly store = new Map(); + + match = jest.fn(async (key: string) => { + const cached = this.store.get(key); + + return isDefinedString(cached) ? { text: async () => cached } : undefined; + }); + + put = jest.fn( + async (key: string, response: { text: () => Promise }) => { + this.store.set(key, await response.text()); + }, + ); + + delete = jest.fn(async (key: string) => this.store.delete(key)); +} + +const isDefinedString = (value: string | undefined): value is string => + value !== undefined; + +const setupCaches = (cache: FakeCache) => { + (globalThis as unknown as { caches: unknown }).caches = { + open: jest.fn(async () => cache), + }; +}; + +describe('fetchComponentSource', () => { + const originalFetch = globalThis.fetch; + const originalCaches = (globalThis as unknown as { caches?: unknown }).caches; + const originalResponse = (globalThis as unknown as { Response?: unknown }) + .Response; + const originalCrypto = Object.getOwnPropertyDescriptor(globalThis, 'crypto'); + const originalTextEncoder = ( + globalThis as unknown as { TextEncoder?: unknown } + ).TextEncoder; + + beforeEach(() => { + (globalThis as unknown as { Response: unknown }).Response = class { + body: string; + constructor(body: string) { + this.body = body; + } + async text() { + return this.body; + } + }; + + Object.defineProperty(globalThis, 'crypto', { + value: webcrypto, + configurable: true, + }); + (globalThis as unknown as { TextEncoder: unknown }).TextEncoder = + NodeTextEncoder; + }); + + afterEach(() => { + globalThis.fetch = originalFetch; + (globalThis as unknown as { caches?: unknown }).caches = originalCaches; + (globalThis as unknown as { Response?: unknown }).Response = + originalResponse; + + if (originalCrypto !== undefined) { + Object.defineProperty(globalThis, 'crypto', originalCrypto); + } + + (globalThis as unknown as { TextEncoder?: unknown }).TextEncoder = + originalTextEncoder; + jest.clearAllMocks(); + }); + + it('fetches from the network and caches the result on a miss', async () => { + const cache = new FakeCache(); + + setupCaches(cache); + + const fetchMock = jest.fn(async () => + createFakeJsResponse(COMPONENT_SOURCE), + ); + + globalThis.fetch = fetchMock as unknown as typeof fetch; + + const source = await fetchComponentSource({ url: FINGERPRINTED_URL }); + + expect(source).toBe(COMPONENT_SOURCE); + expect(cache.put).toHaveBeenCalledTimes(1); + expect(cache.put.mock.calls[0][0]).toBe(FINGERPRINTED_URL); + }); + + it('serves a verified cache hit without hitting the network', async () => { + const cache = new FakeCache(); + + await cache.put(FINGERPRINTED_URL, { + text: async () => COMPONENT_SOURCE, + }); + + setupCaches(cache); + + const fetchMock = jest.fn(); + + globalThis.fetch = fetchMock as unknown as typeof fetch; + + const source = await fetchComponentSource({ url: FINGERPRINTED_URL }); + + expect(source).toBe(COMPONENT_SOURCE); + expect(fetchMock).not.toHaveBeenCalled(); + }); + + it('evicts a poisoned cache entry and refetches from the network', async () => { + const cache = new FakeCache(); + + await cache.put(FINGERPRINTED_URL, { + text: async () => 'globalThis.injectedByAnotherComponent = true;', + }); + + setupCaches(cache); + + const fetchMock = jest.fn(async () => + createFakeJsResponse(COMPONENT_SOURCE), + ); + + globalThis.fetch = fetchMock as unknown as typeof fetch; + + const source = await fetchComponentSource({ url: FINGERPRINTED_URL }); + + expect(source).toBe(COMPONENT_SOURCE); + expect(cache.delete).toHaveBeenCalledWith(FINGERPRINTED_URL); + expect(fetchMock).toHaveBeenCalledTimes(1); + }); + + it('does not cache a network response whose checksum does not match the URL', async () => { + const cache = new FakeCache(); + + setupCaches(cache); + + const staleUrl = buildFingerprintedUrl( + computeSha256Hex('some other build output'), + ); + + const fetchMock = jest.fn(async () => + createFakeJsResponse(COMPONENT_SOURCE), + ); + + globalThis.fetch = fetchMock as unknown as typeof fetch; + + const source = await fetchComponentSource({ url: staleUrl }); + + expect(source).toBe(COMPONENT_SOURCE); + expect(cache.put).not.toHaveBeenCalled(); + }); + + it('never touches the cache for a non-fingerprinted URL', async () => { + const cache = new FakeCache(); + + setupCaches(cache); + + const fetchMock = jest.fn(async () => + createFakeJsResponse('bare url source'), + ); + + globalThis.fetch = fetchMock as unknown as typeof fetch; + + const source = await fetchComponentSource({ url: BARE_URL }); + + expect(source).toBe('bare url source'); + expect(cache.match).not.toHaveBeenCalled(); + expect(cache.put).not.toHaveBeenCalled(); + }); + + it('never touches the cache for a legacy md5-fingerprinted URL', async () => { + const cache = new FakeCache(); + + setupCaches(cache); + + const fetchMock = jest.fn(async () => + createFakeJsResponse(COMPONENT_SOURCE), + ); + + globalThis.fetch = fetchMock as unknown as typeof fetch; + + const source = await fetchComponentSource({ url: LEGACY_MD5_URL }); + + expect(source).toBe(COMPONENT_SOURCE); + expect(cache.match).not.toHaveBeenCalled(); + expect(cache.put).not.toHaveBeenCalled(); + }); + + it('falls back to the network when CacheStorage is unavailable', async () => { + (globalThis as unknown as { caches?: unknown }).caches = undefined; + + const fetchMock = jest.fn(async () => + createFakeJsResponse(COMPONENT_SOURCE), + ); + + globalThis.fetch = fetchMock as unknown as typeof fetch; + + const source = await fetchComponentSource({ url: FINGERPRINTED_URL }); + + expect(source).toBe(COMPONENT_SOURCE); + expect(fetchMock).toHaveBeenCalledTimes(1); + }); + + it('does not trust the cache when WebCrypto is unavailable', async () => { + const cache = new FakeCache(); + + await cache.put(FINGERPRINTED_URL, { + text: async () => COMPONENT_SOURCE, + }); + cache.put.mockClear(); + + setupCaches(cache); + + Object.defineProperty(globalThis, 'crypto', { + value: undefined, + configurable: true, + }); + + const fetchMock = jest.fn(async () => + createFakeJsResponse(COMPONENT_SOURCE), + ); + + globalThis.fetch = fetchMock as unknown as typeof fetch; + + const source = await fetchComponentSource({ url: FINGERPRINTED_URL }); + + expect(source).toBe(COMPONENT_SOURCE); + expect(fetchMock).toHaveBeenCalledTimes(1); + expect(cache.put).not.toHaveBeenCalled(); + }); +}); diff --git a/packages/twenty-front-component-renderer/src/remote/worker/utils/__tests__/fetchComponentSourceFromNetwork.spec.ts b/packages/twenty-front-component-renderer/src/remote/worker/utils/__tests__/fetchComponentSourceFromNetwork.spec.ts new file mode 100644 index 0000000000..13f5012016 --- /dev/null +++ b/packages/twenty-front-component-renderer/src/remote/worker/utils/__tests__/fetchComponentSourceFromNetwork.spec.ts @@ -0,0 +1,135 @@ +import { fetchComponentSourceFromNetwork } from '@/remote/worker/utils/fetchComponentSourceFromNetwork'; + +const COMPONENT_URL = + 'https://api.twenty.com/rest/front-components/component-id/checksum-abc.js'; +const PRESIGNED_URL = 'https://s3.example.com/component.js?signed=abc'; + +type FakeResponseInit = { + ok?: boolean; + status?: number; + statusText?: string; + contentType?: string; + body?: string; + json?: unknown; +}; + +const createFakeResponse = ({ + ok = true, + status = 200, + statusText = 'OK', + contentType = 'application/javascript', + body = '', + json, +}: FakeResponseInit) => ({ + ok, + status, + statusText, + headers: { + get: (name: string) => (name === 'content-type' ? contentType : null), + }, + text: jest.fn(async () => body), + json: jest.fn(async () => json), +}); + +describe('fetchComponentSourceFromNetwork', () => { + const originalFetch = globalThis.fetch; + + afterEach(() => { + globalThis.fetch = originalFetch; + jest.clearAllMocks(); + }); + + it('returns the raw body and forwards headers for a direct JS response', async () => { + const fetchMock = jest.fn(async () => + createFakeResponse({ body: 'export default () => {};' }), + ); + + globalThis.fetch = fetchMock as unknown as typeof fetch; + + const source = await fetchComponentSourceFromNetwork({ + url: COMPONENT_URL, + headers: { Authorization: 'Bearer token' }, + }); + + expect(source).toBe('export default () => {};'); + expect(fetchMock).toHaveBeenCalledWith(COMPONENT_URL, { + headers: { Authorization: 'Bearer token' }, + }); + }); + + it('follows the JSON handoff and fetches the presigned URL header-less', async () => { + const fetchMock = jest + .fn() + .mockResolvedValueOnce( + createFakeResponse({ + contentType: 'application/json', + json: { url: PRESIGNED_URL }, + }), + ) + .mockResolvedValueOnce( + createFakeResponse({ body: 'presigned bundle source' }), + ); + + globalThis.fetch = fetchMock as unknown as typeof fetch; + + const source = await fetchComponentSourceFromNetwork({ + url: COMPONENT_URL, + headers: { Authorization: 'Bearer token' }, + }); + + expect(source).toBe('presigned bundle source'); + expect(fetchMock).toHaveBeenNthCalledWith(1, COMPONENT_URL, { + headers: { Authorization: 'Bearer token' }, + }); + expect(fetchMock).toHaveBeenNthCalledWith(2, PRESIGNED_URL); + }); + + it('throws when the initial response is not ok', async () => { + const fetchMock = jest.fn(async () => + createFakeResponse({ ok: false, status: 404, statusText: 'Not Found' }), + ); + + globalThis.fetch = fetchMock as unknown as typeof fetch; + + await expect( + fetchComponentSourceFromNetwork({ url: COMPONENT_URL }), + ).rejects.toThrow(`Failed to fetch ${COMPONENT_URL}: 404 Not Found`); + }); + + it('throws when the JSON handoff payload is invalid', async () => { + const fetchMock = jest.fn(async () => + createFakeResponse({ + contentType: 'application/json', + json: { notUrl: 'nope' }, + }), + ); + + globalThis.fetch = fetchMock as unknown as typeof fetch; + + await expect( + fetchComponentSourceFromNetwork({ url: COMPONENT_URL }), + ).rejects.toThrow( + `Invalid component source handoff response from ${COMPONENT_URL}`, + ); + }); + + it('throws when the presigned response is not ok', async () => { + const fetchMock = jest + .fn() + .mockResolvedValueOnce( + createFakeResponse({ + contentType: 'application/json', + json: { url: PRESIGNED_URL }, + }), + ) + .mockResolvedValueOnce( + createFakeResponse({ ok: false, status: 403, statusText: 'Forbidden' }), + ); + + globalThis.fetch = fetchMock as unknown as typeof fetch; + + await expect( + fetchComponentSourceFromNetwork({ url: COMPONENT_URL }), + ).rejects.toThrow('Failed to fetch presigned URL: 403 Forbidden'); + }); +}); diff --git a/packages/twenty-front-component-renderer/src/remote/worker/utils/extractComponentChecksumFromUrl.ts b/packages/twenty-front-component-renderer/src/remote/worker/utils/extractComponentChecksumFromUrl.ts new file mode 100644 index 0000000000..609ff88d75 --- /dev/null +++ b/packages/twenty-front-component-renderer/src/remote/worker/utils/extractComponentChecksumFromUrl.ts @@ -0,0 +1,11 @@ +const SHA256_FINGERPRINTED_URL_PATTERN = /\/([0-9a-f]{64})\.js$/; + +export const extractComponentChecksumFromUrl = ({ + url, +}: { + url: string; +}): string | undefined => { + const match = url.match(SHA256_FINGERPRINTED_URL_PATTERN); + + return match?.[1]; +}; diff --git a/packages/twenty-front-component-renderer/src/remote/worker/utils/fetchComponentSource.ts b/packages/twenty-front-component-renderer/src/remote/worker/utils/fetchComponentSource.ts new file mode 100644 index 0000000000..a2345c11d9 --- /dev/null +++ b/packages/twenty-front-component-renderer/src/remote/worker/utils/fetchComponentSource.ts @@ -0,0 +1,54 @@ +import { isDefined } from 'twenty-shared/utils'; + +import { extractComponentChecksumFromUrl } from '@/remote/worker/utils/extractComponentChecksumFromUrl'; +import { fetchComponentSourceFromNetwork } from '@/remote/worker/utils/fetchComponentSourceFromNetwork'; +import { frontComponentCacheStorageService } from '@/remote/worker/utils/frontComponentCacheStorageService'; + +export const fetchComponentSource = async ({ + url, + headers, +}: { + url: string; + headers?: Record; +}): Promise => { + const expectedChecksum = extractComponentChecksumFromUrl({ url }); + + const cache = isDefined(expectedChecksum) + ? await frontComponentCacheStorageService.open() + : undefined; + + if (isDefined(cache) && isDefined(expectedChecksum)) { + const cachedSource = await frontComponentCacheStorageService.read({ + cache, + url, + }); + + if (isDefined(cachedSource)) { + const cachedSourceChecksum = + await frontComponentCacheStorageService.computeChecksum({ + source: cachedSource, + }); + + if (cachedSourceChecksum === expectedChecksum) { + return cachedSource; + } + + // Poisoned or corrupt entry: any same-origin code can write to + // CacheStorage, so a mismatch means the content cannot be trusted. + frontComponentCacheStorageService.delete({ cache, url }); + } + } + + const source = await fetchComponentSourceFromNetwork({ url, headers }); + + if (isDefined(cache) && isDefined(expectedChecksum)) { + const sourceChecksum = + await frontComponentCacheStorageService.computeChecksum({ source }); + + if (sourceChecksum === expectedChecksum) { + frontComponentCacheStorageService.write({ cache, url, source }); + } + } + + return source; +}; diff --git a/packages/twenty-front-component-renderer/src/remote/worker/utils/fetchComponentSourceFromNetwork.ts b/packages/twenty-front-component-renderer/src/remote/worker/utils/fetchComponentSourceFromNetwork.ts new file mode 100644 index 0000000000..afd8ce74a0 --- /dev/null +++ b/packages/twenty-front-component-renderer/src/remote/worker/utils/fetchComponentSourceFromNetwork.ts @@ -0,0 +1,41 @@ +import { z } from 'zod'; + +const componentSourceHandoffSchema = z.object({ url: z.url() }); + +export const fetchComponentSourceFromNetwork = async ({ + url, + headers, +}: { + url: string; + headers?: Record; +}): Promise => { + const response = await fetch(url, { headers }); + + if (!response.ok) { + throw new Error( + `Failed to fetch ${url}: ${response.status} ${response.statusText}`, + ); + } + + const contentType = response.headers.get('content-type') ?? ''; + + if (!contentType.includes('application/json')) { + return response.text(); + } + + const handoff = componentSourceHandoffSchema.safeParse(await response.json()); + + if (!handoff.success) { + throw new Error(`Invalid component source handoff response from ${url}`); + } + + const presignedResponse = await fetch(handoff.data.url); + + if (!presignedResponse.ok) { + throw new Error( + `Failed to fetch presigned URL: ${presignedResponse.status} ${presignedResponse.statusText}`, + ); + } + + return presignedResponse.text(); +}; diff --git a/packages/twenty-front-component-renderer/src/remote/worker/utils/frontComponentCacheStorageService.ts b/packages/twenty-front-component-renderer/src/remote/worker/utils/frontComponentCacheStorageService.ts new file mode 100644 index 0000000000..8884cfb06d --- /dev/null +++ b/packages/twenty-front-component-renderer/src/remote/worker/utils/frontComponentCacheStorageService.ts @@ -0,0 +1,79 @@ +import { isDefined } from 'twenty-shared/utils'; + +const FRONT_COMPONENT_SOURCE_CACHE_NAME = 'front-component-source-v1'; + +export const frontComponentCacheStorageService = { + computeChecksum: async ({ + source, + }: { + source: string; + }): Promise => { + if (typeof crypto === 'undefined' || !isDefined(crypto.subtle)) { + return undefined; + } + + try { + const digest = await crypto.subtle.digest( + 'SHA-256', + new TextEncoder().encode(source), + ); + + return Array.from(new Uint8Array(digest)) + .map((byte) => byte.toString(16).padStart(2, '0')) + .join(''); + } catch { + return undefined; + } + }, + + open: async (): Promise => { + if (typeof caches === 'undefined') { + return undefined; + } + + try { + return await caches.open(FRONT_COMPONENT_SOURCE_CACHE_NAME); + } catch { + return undefined; + } + }, + + read: async ({ + cache, + url, + }: { + cache: Cache; + url: string; + }): Promise => { + try { + const cachedResponse = await cache.match(url); + + return isDefined(cachedResponse) ? cachedResponse.text() : undefined; + } catch { + return undefined; + } + }, + + write: ({ + cache, + url, + source, + }: { + cache: Cache; + url: string; + source: string; + }): void => { + cache + .put( + url, + new Response(source, { + headers: { 'Content-Type': 'application/javascript' }, + }), + ) + .catch(() => undefined); + }, + + delete: ({ cache, url }: { cache: Cache; url: string }): void => { + cache.delete(url).catch(() => undefined); + }, +}; diff --git a/packages/twenty-sdk/src/cli/utilities/build/common/esbuild-result-processor.ts b/packages/twenty-sdk/src/cli/utilities/build/common/esbuild-result-processor.ts index 447ca5a378..8bc8448606 100644 --- a/packages/twenty-sdk/src/cli/utilities/build/common/esbuild-result-processor.ts +++ b/packages/twenty-sdk/src/cli/utilities/build/common/esbuild-result-processor.ts @@ -1,9 +1,9 @@ +import { type OnFileBuiltCallback } from '@/cli/utilities/build/common/restartable-watcher-interface'; import crypto from 'crypto'; import type * as esbuild from 'esbuild'; import { readFile } from 'node:fs/promises'; import path from 'path'; -import { type OnFileBuiltCallback } from '@/cli/utilities/build/common/restartable-watcher-interface'; -import { type FileFolder } from 'twenty-shared/types'; +import { FileFolder } from 'twenty-shared/types'; const SDK_CLIENT_IMPORT_PREFIX = 'twenty-client-sdk'; @@ -34,7 +34,13 @@ export const processEsbuildResult = async ({ const relativeSourcePath = path.relative(appPath, absoluteSourcePath); const content = await readFile(absoluteBuiltFile); - const checksum = crypto.createHash('md5').update(content).digest('hex'); + + const checksumAlgorithm = + fileFolder === FileFolder.BuiltFrontComponent ? 'sha256' : 'md5'; + const checksum = crypto + .createHash(checksumAlgorithm) + .update(content) + .digest('hex'); const lastChecksum = lastChecksums.get(relativeBuiltPath); diff --git a/packages/twenty-server/src/engine/core-modules/file/interfaces/file-folder.interface.ts b/packages/twenty-server/src/engine/core-modules/file/interfaces/file-folder.interface.ts index 9df2b56095..4508ff343d 100644 --- a/packages/twenty-server/src/engine/core-modules/file/interfaces/file-folder.interface.ts +++ b/packages/twenty-server/src/engine/core-modules/file/interfaces/file-folder.interface.ts @@ -15,6 +15,11 @@ export const IMMUTABLE_FILE_CACHE_CONTROL = 'private, max-age=86400, immutable'; export const PUBLIC_ASSET_CACHE_CONTROL = 'public, max-age=3600'; +// Responses embedding a short-lived presigned URL must never be stored: +// a cached copy either 403s once the signature expires or leaks a live +// presigned URL to clients that never authenticated. +export const PRESIGNED_URL_NO_STORE_CACHE_CONTROL = 'private, no-store'; + export const fileFolderConfigs: Record = { [FileFolder.CorePicture]: { ignoreExpirationToken: true, diff --git a/packages/twenty-server/src/engine/metadata-modules/front-component/controllers/front-component.controller.ts b/packages/twenty-server/src/engine/metadata-modules/front-component/controllers/front-component.controller.ts index 430ef15c29..7b76f3473d 100644 --- a/packages/twenty-server/src/engine/metadata-modules/front-component/controllers/front-component.controller.ts +++ b/packages/twenty-server/src/engine/metadata-modules/front-component/controllers/front-component.controller.ts @@ -17,6 +17,7 @@ import { FileStorageException, FileStorageExceptionCode, } from 'src/engine/core-modules/file-storage/interfaces/file-storage-exception'; +import { PRESIGNED_URL_NO_STORE_CACHE_CONTROL } from 'src/engine/core-modules/file/interfaces/file-folder.interface'; import { setFileResponseHeaders } from 'src/engine/core-modules/file/utils/set-file-response-headers.utils'; import { WorkspaceEntity } from 'src/engine/core-modules/workspace/workspace.entity'; @@ -85,7 +86,9 @@ export class FrontComponentController { }); if (fileResponse.type === 'redirect') { - return res.redirect(fileResponse.presignedUrl); + res.setHeader('Cache-Control', PRESIGNED_URL_NO_STORE_CACHE_CONTROL); + + return res.json({ url: fileResponse.presignedUrl }); } setFileResponseHeaders( diff --git a/packages/twenty-server/src/engine/metadata-modules/front-component/front-component.service.ts b/packages/twenty-server/src/engine/metadata-modules/front-component/front-component.service.ts index be125dbc2d..f71937fb20 100644 --- a/packages/twenty-server/src/engine/metadata-modules/front-component/front-component.service.ts +++ b/packages/twenty-server/src/engine/metadata-modules/front-component/front-component.service.ts @@ -6,7 +6,6 @@ import { isDefined } from 'twenty-shared/utils'; import { ApplicationService } from 'src/engine/core-modules/application/application.service'; import { type FlatApplication } from 'src/engine/core-modules/application/types/flat-application.type'; import { FileStorageService } from 'src/engine/core-modules/file-storage/services/file-storage.service'; -import { fileFolderConfigs } from 'src/engine/core-modules/file/interfaces/file-folder.interface'; import { type FileResponse } from 'src/engine/core-modules/file/types/file-response.type'; import { getContentDisposition } from 'src/engine/core-modules/file/utils/get-content-disposition.utils'; import { TwentyConfigService } from 'src/engine/core-modules/twenty-config/twenty-config.service'; @@ -333,9 +332,6 @@ export class FrontComponentService { ), responseContentType: mimeType, responseContentDisposition: getContentDisposition(mimeType), - responseCacheControl: - fileFolderConfigs[FileFolder.BuiltFrontComponent].cacheControl ?? - undefined, }); if (presignedUrl) { diff --git a/packages/twenty-server/src/engine/workspace-manager/standard-objects-prefill-data/services/prefill-front-component.service.ts b/packages/twenty-server/src/engine/workspace-manager/standard-objects-prefill-data/services/prefill-front-component.service.ts index b3e7fce203..2f37d69a51 100644 --- a/packages/twenty-server/src/engine/workspace-manager/standard-objects-prefill-data/services/prefill-front-component.service.ts +++ b/packages/twenty-server/src/engine/workspace-manager/standard-objects-prefill-data/services/prefill-front-component.service.ts @@ -102,8 +102,10 @@ export class PrefillFrontComponentService { }, }); + // sha-256 (not md5) so the renderer can verify cached bundles against + // the URL checksum with WebCrypto, which has no md5 support const checksum = crypto - .createHash('md5') + .createHash('sha256') .update(builtFile.content) .digest('hex');