houseassassin/twenty
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
14,001 Commits 1 Branch 0 Tags
ea7863dc4e20c91ae69bb3eca3ce67a17e0220ea
Commit Graph

3 Commits

Author SHA1 Message Date
Paul Rastoin 9e20e2222a Fix front-component serving on Safari, kill stale presigned caching, and cache built bundles client-side (#22672) 
## Context

Built front-component bundles are served via `GET
/rest/front-components/:id/:cacheKey`. On S3-backed storage (Twenty
Cloud) the endpoint used to 302-redirect the worker's authenticated
fetch to a presigned S3 URL. That redirect caused two bugs, and fixing
it removed the caching the redirect was accidentally providing — so this
PR also adds a proper client-side cache.

Closes twentyhq/core-team-issues#2653.

### Bug 1 — Safari 403 (Authorization header forwarded across redirect)

The renderer worker fetches the bundle with `Authorization: Bearer`. The
controller answered with a 302 to a presigned S3 URL. Per the Fetch
spec, browsers must strip `Authorization` on a cross-origin redirect.
Chrome/Firefox do, but Safari/WebKit forwards it, so S3 receives both a
query-string signature and an `Authorization` header and rejects with
`InvalidArgument: Only one auth mechanism allowed`. Result: front
components never load in Safari on S3-backed storage.

### Bug 2 — 302 cached publicly (browser-independent)

The redirect branch set no `Cache-Control`, so a CDN could cache it far
beyond the presigned URL's TTL (`STORAGE_S3_PRESIGNED_URL_EXPIRES_IN`,
900s). Consequences: any client re-served the cached 302 after 15 min
hits an expired signature (403, also affects Chrome), and the cached
redirect containing a live presigned URL is served to unauthenticated
requests (short-lived auth bypass).

### Regression this introduces — warm-load caching lost

Marking the handoff `no-store` (Bug 2 fix) is correct, but it means the
built bundle is no longer cached anywhere on the S3 path. The browser
HTTP cache cannot compensate: the presigned URL that actually returns
the bytes carries a fresh `X-Amz-Date`/`X-Amz-Signature` on every
request, so each download is a brand-new cache key and never hits. Net
effect without mitigation: every worker mount re-downloads the full
bundle.

## What changed

- **Front components return a 200 JSON body instead of a 302.** The
controller now responds `200 { url }` with `Cache-Control: private,
no-store`. The worker parses the JSON and issues a separate header-less
`fetch(url)` to S3. No redirect means the `Authorization` header is
never forwarded, making it browser-independent, and the handoff carrying
the presigned URL is never cached. The stream path (local storage) is
unchanged.
- **Client-side bundle cache in the renderer (restores warm loads).**
`fetchComponentSource` wraps the fetch chain in a `CacheStorage` layer
keyed by the **content-addressed** `/front-components/:id/:checksum.js`
URL. A hit returns the stored bundle and skips **both** the `no-store`
handoff to Twenty and the S3 download — restoring cross-session warm
loads without ever persisting a presigned credential. Because
`CacheStorage` is writable by any same-origin code (including the
untrusted component code this cache feeds), cached content is verified
against the sha-256 checksum embedded in the URL on every read, and
evicted on mismatch. Caching degrades to a plain fetch where
`CacheStorage` or WebCrypto is unavailable.
- **sha-256 checksums for built front components.** The SDK build and
workspace prefill now fingerprint built front-component bundles with
sha-256 (WebCrypto has no md5), enabling the integrity check above.
Other file folders keep md5. Legacy md5-fingerprinted URLs (32-hex)
simply bypass the cache — already-synced components keep working and
start benefiting from caching on their next build/sync.
- **WebKit e2e coverage.** Added a `webkit` project to the postcard
example's Playwright config mirroring `chrome` (shared setup +
storageState), plus iframe/worker diagnostics logging so front-component
failures surface in the test log. `TZ` is pinned to `Europe/Paris`
because WebKit on Linux ignores Playwright's `timezoneId` emulation and
rejects the runner's legacy `CET` alias, which crashed the record page
before the component could render.

### Why we hand off to S3 instead of streaming through Twenty

On S3-backed storage we deliberately **do not** proxy/stream the bundle
bytes through the API. The controller returns the presigned URL and the
worker fetches the content directly from S3, for two reasons:

- **Server CPU/bandwidth.** Streaming every bundle on every cold load
would put the API server on the hot path for all front-component
content. Handing off to S3 keeps that load off the server.
- **Domain isolation.** Front-component content is fetched from the
object-storage domain (e.g. `s3.domain.com`), a different origin than
the API and the front app. Serving untrusted/app-authored bundle content
from a separate domain than `twenty.com` keeps it off the app's origin.

The stream path is kept only as the local-storage fallback (no
S3/presign available), where these concerns don't apply.

## Examples

### The JSON handoff (S3 path)

```http
GET /rest/front-components/d3b07384-.../a1b2c3d4.js HTTP/1.1
Host: twenty.com
Authorization: Bearer <worker-token>
```

```http
HTTP/1.1 200 OK
Content-Type: application/json
Cache-Control: private, no-store

{"url":"https://s3.domain.com/bucket/.../checkout-widget.mjs?X-Amz-Date=20260709T091500Z&X-Amz-Expires=900&...&X-Amz-Signature=AAAA1111..."}
```

The worker then fetches that presigned URL **without** headers (the
Safari fix) and gets the bundle bytes.

### Why the browser HTTP cache can't reuse it

| | Load 1 (09:15) | Load 2 (09:30) | Same key? |
|---|---|---|---|
| Twenty handoff URL | `.../a1b2c3d4.js` | `.../a1b2c3d4.js` | ✅ but
response is `no-store` |
| Presigned `X-Amz-Signature` | `AAAA1111...` | `ZZZZ9999...` | ❌ |
| Effective S3 URL (the HTTP cache key) |
`...&X-Amz-Signature=AAAA1111...` | `...&X-Amz-Signature=ZZZZ9999...` |
❌ new key → miss |

### What the CacheStorage layer stores

```
key   = https://twenty.com/rest/front-components/d3b07384-.../a1b2c3d4.js   (stable, chosen by us)
value = <bundle JS bytes>                                                   (NOT the presigned URL)
```

Keying by the stable logical URL (not the volatile URL the bytes arrived
from) is the one thing the native HTTP cache can't express. The
presigned URL is used once and discarded.

### Invalidation

No TTL and no explicit delete — invalidation is by key change. A rebuild
changes the checksum → changes the URL → guaranteed miss on the new key.
The old entry is orphaned and reclaimed by normal browser eviction
(quota/LRU; Safari ITP after 7 idle days). Global invalidation lever:
bump the cache name suffix (`front-component-source-v1`).

## Deploy note — front/server release window

Old frontend bundles (already-open tabs) hitting the new server receive
the JSON handoff where they expect raw JS and fail to render until the
tab is reloaded. The other direction is safe: the new worker against an
old server follows the 302 transparently (the content-type check falls
through to `response.text()`). Accepted as a short deploy-window
trade-off.

## Follow-ups (not in this PR)

- The client-side cache is a bridge for the `no-store` presigned
handoff. If built components are later served from a stable, non-signed,
public-by-URL path (they are already content-addressed by checksum, so
`immutable` is safe), the browser + CDN cache natively and this custom
layer can be removed.
- `GET /file/:fileFolder/:id` presigned 302s still carry no
`Cache-Control`. An explicit policy there (bounded `private, max-age`
below the presigned TTL) was prototyped in this PR and deliberately
dropped to keep the scope on front components — the file path
authenticates via a query-param token (part of any cache key), so its
exposure differs and deserves its own PR.

## Non-goals

Per the issue, file serving keeps its query-param token + 302 model.
Native browser loads (`<img>`, downloads) cannot do a two-step fetch and
already work on Safari. The public-asset redirect is left untouched
since its caching is intentional.

## Test plan

- Renderer: `fetchComponentSource.spec.ts` covers cache miss + write,
verified cache hit (no network), poisoned-entry eviction,
checksum-mismatch (never cached), non-fingerprinted and legacy-md5 URL
bypass, and the no-`CacheStorage` / no-WebCrypto fallbacks.
`fetchComponentSourceFromNetwork.spec.ts` covers the direct JS response,
the JSON handoff follow-through (header-less presigned fetch), and error
mapping.
- e2e: the postcard front-component spec now runs on both Chromium and
WebKit against prod-parity storage (S3 + Lambda).
- `oxlint` + `oxfmt` clean; typecheck passes on changed packages.

### Reproduction proof — Safari was always broken (e2e probe)

We ran the prod-parity postcard e2e suite (S3 storage + Lambda) with
WebKit against **`main` without this fix**, via a throwaway probe PR:
twentyhq/twenty#22717.

Result — [ci-privileged run
29015624468](https://github.com/twentyhq/ci-privileged/actions/runs/29015624468):

```
1 failed
  [webkit] › card-front-component.spec.ts:61 › renders the postcard name and status badge in the record preview
2 passed (1.4m)
```

`[webkit]` times out waiting for `getByTestId('postcard-card')` to
become visible (*element(s) not found*) while the Chromium run of the
same spec passes. This confirms the front component **never rendered in
Safari** on S3-backed storage prior to this PR — it is a genuine,
browser-specific bug, not a flake. The fix in this PR is expected to
turn that same `[webkit]` assertion green.

Note: running the WebKit tests in CI requires the WebKit browser binary
and its system dependencies in the e2e job (now installed via `npx
playwright install --with-deps chromium webkit`).
 2026-07-10 10:12:58 +00:00
Paul Rastoin f04db9751f fix(client-sdk): bundle metadata client into a single self-contained file (#22085) 
## Problem

A front component that imports `MetadataApiClient` from
`twenty-client-sdk/metadata` crashes at render time:

```
FrontComponent error: Failed to resolve module specifier "./chunk-Dqa2HsxW.mjs".
Invalid relative url or base scheme isn't hierarchical.
```

(hash differs per build). The equivalent component using `CoreApiClient`
from `twenty-client-sdk/core` works fine.

## Root cause

The front-component renderer loads each SDK client as a **single
in-memory blob-URL module** and only rewrites the two bare specifiers it
knows (`twenty-client-sdk/core`, `twenty-client-sdk/metadata`). A
blob-URL module cannot resolve a **relative** `import … from
"./chunk-*.mjs"` (blob URLs aren't hierarchical), and that chunk isn't
served anyway.

Only two entrypoints are externalized by the front-component build
(`FRONT_COMPONENT_EXTERNAL_MODULES`) and thus served as blob modules:
`core` and `metadata`. Everything else (`rest`, `generate`) is bundled
into the component and is unaffected. Of those two:

| client | how `dist/*.mjs` is produced | self-contained? |
|---|---|---|
| **core** | esbuild single-file bundle (`compileGeneratedClient`),
re-run per workspace at server runtime by `replaceCoreClient` | ✅ |
| **metadata** | the shared multi-entry Vite build, which hoists shared
code into a relative `chunk-*.mjs` | ❌ |

The metadata client is built once at package-build time (it is not
workspace-specific) and was shipped straight from the multi-entry Vite
output, keeping the unresolvable relative chunk import.

## Regression trace

This was **not** broken on arrival — it regressed via a transitive
bundler swap:

| Date | Commit | Event |
|---|---|---|
| 2026-05-20 | `a26fe3bb65` | Metadata-client-in-front-components
shipped; `twenty-client-sdk` on **Vite 7 (Rollup)** |
| 2026-06-08 | `d2e7dc0e74` (#21309, *"security: bump vulnerable direct
dependencies"*) | Bumped **Vite 7 → 8**, introducing **Rolldown 1.0.3**
(no rolldown entries in the lockfile before this commit) |

Vite 7 is Rollup-based; Vite 8 uses Rolldown. The breaking artifact is
literally a `\0rolldown/runtime.js` shared chunk — a Rolldown construct
that could not have existed before the bump. So the metadata
front-component path worked from 2026-05-20 until the 2026-06-08
security dependency bump silently changed the bundler and split out the
shared runtime chunk.

## Fix

Build the metadata client as its **own single-entry Vite library**
(`vite.metadata.config.ts`) so its output is a single self-contained
file with no shared chunk. `core` / `rest` / `generate` stay in the main
multi-entry build (`vite.config.ts`); shared config (`isExternal`,
`entryFileNames`) is factored into `vite.shared.ts`. The build pipeline
runs `vite build && vite build -c vite.metadata.config.ts`.

The server picks this up automatically: `SdkClientGenerationService`
ships the pre-built package `dist/` and only regenerates the **core**
client; it never regenerates metadata. No server-side change required.

## Regression guard (e2e)

The postcard example's `card.front-component.tsx` previously used
`CoreApiClient` only, so this metadata-only regression had no e2e
coverage. It now loads and round-trips all three SDK clients (`Core`,
`Metadata`, `Rest`) via an SDK health panel, and the e2e asserts the
blob-served `core` + `metadata` probes reach `ok` — which only happens
if those bundles resolve and function. A future chunk-import regression
in either blob module would crash the component on load and fail the
test.

## Verification

- `npx nx build twenty-client-sdk` succeeds.
- `dist/metadata.mjs` / `dist/metadata.cjs`: **0** `chunk-*` imports,
**0** relative imports; both load and export `MetadataApiClient` +
`MetadataSchema`.
- `dist/metadata/index.d.ts` types still emitted.
- `npx nx typecheck` + `npx nx lint twenty-client-sdk` pass; postcard
app typecheck + lint pass.

## Notes

- `dist/` is not committed (CI builds it); a running server must rebuild
`twenty-client-sdk` for the fix to take effect.
- The e2e was validated statically (typecheck + lint); running it
end-to-end requires a live stack with a seeded postcard record.
 2026-06-25 10:19:18 +02:00
Paul Rastoin d1ba63d4a4 Postcard app e2e front component rendering test (#21600) 
# Introduction
Creating a playwright test, quite granular and verbose that will verify
that the post card preview front component is rendered as expected on
the tested twenty instance

This covers everything e2e from twenty front, front comp renderer,
assets cdn rendered redirection etc
Style bridge etc

## Note

The playwright test setup assumes the application has already been
installed once, it's mainly used by the merge queue as a high level
front component and logic function ( will be in the same ci ) regression
bottleneck
The goal isn't for this test to be run locally
 2026-06-15 13:02:19 +00:00