Cache twenty-client-sdk modules host-side via content-addressed URLs (#22981)

## Context

Front component sources are fetched host-side and integrity-verified by
the SHA-256 checksum embedded in their URL, cached in Cache Storage — a
layer that exists specifically because their download URLs are presigned
and rotate. The `twenty-client-sdk` modules (`core` and `metadata`) were
re-fetched on every render and could not be cached safely: their URLs
carried no checksum and the server exposed no freshness signal.

This PR makes the SDK module URLs **content-addressed** and relies on
the **browser HTTP cache** for immutability, and it keys the checksums
on their real owners: the **application** for `core`, the **instance**
for `metadata`. The checksum does double duty: cache invalidation
(regeneration changes the checksum → the URL changes → guaranteed cache
miss) and a server-side cacheability guard (the server only grants
`immutable` when the checksum in the URL matches the authoritative
checksum it knows for that module — persisted at generation time for
`core`, hashed once at bootstrap for `metadata` — so no per-request
hashing of the served bytes). Note this is **not** an end-to-end
integrity guarantee: there is no client-side hash verification, and on a
fingerprint mismatch the server still serves the current bytes with
`no-store` (self-healing for stale URLs) rather than failing.

<img width="2412" height="926" alt="image"
src="https://github.com/user-attachments/assets/d97935d2-0fdb-4c44-89ac-596b7ca8ca64"
/>

Closes twentyhq/core-team-issues#2688.

## Routes

| Module | URL | Scope |
| --- | --- | --- |
| `core` | `/rest/sdk-client/{applicationId}/core[/{checksum}]` | Per
application (generated bundle) |
| `metadata` | `/rest/sdk-client/metadata[/{checksum}]` |
**Instance-wide**: no application segment, so every application
converges on one URL and the browser downloads the module once per
release instead of once per application |

The previous application-scoped metadata path
(`/rest/sdk-client/{applicationId}/metadata[/{checksum}]`) is **kept for
backward compatibility**, new clients just stop generating those URLs.
The instance-wide route is declared before the parameterized route so
`metadata/{checksum}` is not swallowed as `:applicationId/:moduleName`.

## Caching model

| Request | `Cache-Control` | Effect |
| --- | --- | --- |
| Fingerprinted URL, checksum matches the known module checksum |
`immutable` | Cached indefinitely by the browser HTTP cache; a new
checksum is a new URL |
| Fingerprinted URL, checksum does not match | `no-store` | Current
bytes served uncached (self-healing for stale URLs) |
| Bare URL (pre-generation fallback, `core` only in practice) |
`no-store` | Never cached |

- Both responses also set `X-Content-Type-Options: nosniff` and
`Content-Type: application/javascript`.
- SDK modules are intentionally **not** placed in Cache Storage. That
layer stays reserved for the presigned/rotating component-source URLs;
SDK modules are served directly and authenticated, so the browser HTTP
cache (keyed by the content-addressed URL) is their single cache layer.

## Checksum provenance

- **core** — per **application**, persisted on
`application.sdkClientCoreChecksum` at generation time and read back
from `flatApplicationMaps` (never re-hashed per request).
- **metadata** — **instance-wide**, hashed once from the installed
`twenty-client-sdk/dist/metadata.mjs` package (warmed at bootstrap,
memoized per process) and served straight from that package, so it is
fresh from the first request after a release with no archive dependency.

## Server (twenty-server)

- Hash `dist/core.mjs` at SDK generation and persist
`sdkClientCoreChecksum` via `applicationRepository.update`. Adds the
nullable text column to `application.entity.ts` (mirroring
`packageJsonChecksum`) plus a fast instance command with up/down;
`FlatApplication` picks it up automatically.
- New **application-scoped** query
`applicationSdkClientChecksums(applicationId: UUID!):
SdkClientChecksums` on `ApplicationResolver` (metadata schema,
`WorkspaceAuthGuard` + `NoPermissionGuard`). `SdkClientChecksums.core`
is **nullable** and stays `null` until the SDK has been generated at
least once; `metadata` is **always present** (bootstrap-warmed), so the
metadata module is cacheable from the very first render of any app. The
query itself returns `null` only for unknown applications.
- `SdkClientChecksumsDTO` now lives in the shared
`core-modules/sdk-client/dtos/`. `FrontComponentDTO` and the
`frontComponent` resolver no longer carry checksums (decoupled from the
front-component row).
- `sdk-client` controller: instance-wide `metadata[/:checksum]` route
(no workspace-cache or application lookup, serves the memoized installed
module) + application-scoped `:applicationId/:moduleName[/:checksum]`
route (serves `core` from the per-application archive, `metadata` kept
for back-compat). Cacheability compares the URL checksum against the
**known** checksum — persisted `sdkClientCoreChecksum` for `core`,
memoized package hash for `metadata` — instead of hashing the served
bytes on every request: `immutable` on match, `no-store` otherwise (bare
URL or stale fingerprint), plus `nosniff`. A persisted checksum out of
sync with the archive only downgrades to `no-store` until the next
regeneration.

## Front (twenty-front)

- New metadata query `GetApplicationSdkClientChecksums`, keyed by
`applicationId`; removed the `sdkClientChecksums` selection from
`FindOneFrontComponent`.
- `getSdkClientUrls` builds the two module URLs independently:
`/sdk-client/{applicationId}/core/{checksum}` and the **instance-wide**
`/sdk-client/metadata/{checksum}` (no application segment → one shared
browser cache entry per release across all applications). Each falls
back to its bare URL when its checksum is absent — since `core` is
nullable, a never-generated app still gets a content-addressed metadata
URL and only `core` falls back. The checksum type is sourced from the
codegen `SdkClientChecksums` type rather than a hand-maintained
duplicate.
- `FrontComponentRenderer` is split into a gating outer component (runs
`FindOneFrontComponent`, renders nothing while loading) and a content
component that receives a guaranteed-non-null `frontComponent`.
Following project conventions, the side effects live in dedicated effect
components: `FrontComponentLoadErrorSnackBarEffect` (query error →
snackbar) and `FrontComponentApplicationTokenPairEffect` (mirrors the
query-derived token pair into component state unconditionally, `null`
included, so revoked credentials can never be retained or refreshed).
The content component fetches checksums via the application-keyed query
and **gates the mount of SDK-using components on that query**, so the
very first module fetch is always the content-addressed (`immutable`)
URL instead of the bare `no-store` one. Non-SDK components skip the
query and are never blocked.
- **Live invalidation without reload:** SDK regeneration updates the
application row, and the server broadcasts an `application` metadata
event carrying the new core checksum.
`useOnApplicationSdkClientChecksumsUpdated` /
`useUpdateSdkClientChecksumsApolloCache` patch the application-keyed
checksum query cache (core only; the instance-wide metadata is
preserved), so every mounted component of that application picks up the
new URL at once. This replaces the previous frontComponent-derived field
and closes the earlier "known gap" (a mounted component staying on a
session-old checksum until a full reload). The cache-patching callback
is memoized (`useCallback`) so the window listener is registered once
per application, and the listener is **skipped entirely** for non-SDK
components (`useListenToMetadataOperationBrowserEvent` gained a `skip`
option) — they register no listener and never refetch a query they don't
consume.

## Renderer (twenty-front-component-renderer)

- SDK sources are fetched through a dedicated plain authenticated fetch,
`fetchJavaScriptModuleSourceText` (Bearer header, `credentials:
'omit'`), instead of the Cache Storage `fetchComponentSource` path;
`fetchSdkClientSources` uses it. Execution stays exclusively in the
opaque-origin worker via blob URLs; the host only fetches and forwards
source strings (no hashing host-side). Staleness self-resolves through
the checksum: new checksum → new URL → cache miss.

<!-- This is an auto-generated description by cubic. -->
<a
href="https://cubic.dev/pr/twentyhq/twenty/pull/22981?utm_source=github"
target="_blank" rel="noopener noreferrer"
data-no-image-dialog="true"><picture><source
media="(prefers-color-scheme: dark)"
srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source
media="(prefers-color-scheme: light)"
srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img
alt="Review in cubic"
src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a>
<!-- End of auto-generated description by cubic. -->
This commit is contained in:
Paul Rastoin
2026-07-21 17:05:00 +02:00
committed by GitHub
parent 8f704e87e3
commit 71a1ff7ac8
58 changed files with 2137 additions and 1048 deletions
@@ -982,6 +982,11 @@ The `JSONObject` scalar type represents JSON objects as specified by [ECMA-404](
"""
scalar JSONObject
type SdkClientChecksums {
core: String
metadata: String!
}
type RatioAggregateConfig {
fieldMetadataId: UUID!
optionValue: String!
@@ -3048,6 +3053,7 @@ type MinimalMetadata {
type Query {
navigationMenuItems: [NavigationMenuItem!]!
navigationMenuItem(id: UUID!): NavigationMenuItem
applicationSdkClientChecksums(applicationId: UUID!): SdkClientChecksums
enterprisePortalSession(returnUrlPath: String): String
enterpriseCheckoutSession(billingInterval: String): String
enterpriseSubscriptionStatus: EnterpriseSubscriptionStatusDTO
@@ -704,6 +704,12 @@ export interface User {
/** Onboarding status */
export type OnboardingStatus = 'PLAN_REQUIRED' | 'WORKSPACE_ACTIVATION' | 'PROFILE_CREATION' | 'SYNC_EMAIL' | 'APPS_INSTALLATION' | 'INVITE_TEAM' | 'COMPLETED'
export interface SdkClientChecksums {
core?: Scalars['String']
metadata: Scalars['String']
__typename: 'SdkClientChecksums'
}
export interface RatioAggregateConfig {
fieldMetadataId: Scalars['UUID']
optionValue: Scalars['String']
@@ -2704,6 +2710,7 @@ export interface MinimalMetadata {
export interface Query {
navigationMenuItems: NavigationMenuItem[]
navigationMenuItem?: NavigationMenuItem
applicationSdkClientChecksums?: SdkClientChecksums
enterprisePortalSession?: Scalars['String']
enterpriseCheckoutSession?: Scalars['String']
enterpriseSubscriptionStatus?: EnterpriseSubscriptionStatusDTO
@@ -3770,6 +3777,13 @@ export interface UserGenqlSelection{
__scalar?: boolean | number
}
export interface SdkClientChecksumsGenqlSelection{
core?: boolean | number
metadata?: boolean | number
__typename?: boolean | number
__scalar?: boolean | number
}
export interface RatioAggregateConfigGenqlSelection{
fieldMetadataId?: boolean | number
optionValue?: boolean | number
@@ -5898,6 +5912,7 @@ export interface MinimalMetadataGenqlSelection{
export interface QueryGenqlSelection{
navigationMenuItems?: NavigationMenuItemGenqlSelection
navigationMenuItem?: (NavigationMenuItemGenqlSelection & { __args: {id: Scalars['UUID']} })
applicationSdkClientChecksums?: (SdkClientChecksumsGenqlSelection & { __args: {applicationId: Scalars['UUID']} })
enterprisePortalSession?: { __args: {returnUrlPath?: (Scalars['String'] | null)} } | boolean | number
enterpriseCheckoutSession?: { __args: {billingInterval?: (Scalars['String'] | null)} } | boolean | number
enterpriseSubscriptionStatus?: EnterpriseSubscriptionStatusDTOGenqlSelection
@@ -6995,6 +7010,14 @@ export interface LogicFunctionLogsInput {applicationId?: (Scalars['UUID'] | null
const SdkClientChecksums_possibleTypes: string[] = ['SdkClientChecksums']
export const isSdkClientChecksums = (obj?: { __typename?: any } | null): obj is SdkClientChecksums => {
if (!obj?.__typename) throw new Error('__typename is missing in "isSdkClientChecksums"')
return SdkClientChecksums_possibleTypes.includes(obj.__typename)
}
const RatioAggregateConfig_possibleTypes: string[] = ['RatioAggregateConfig']
export const isRatioAggregateConfig = (obj?: { __typename?: any } | null): obj is RatioAggregateConfig => {
if (!obj?.__typename) throw new Error('__typename is missing in "isRatioAggregateConfig"')
File diff suppressed because it is too large Load Diff
@@ -1,4 +1,4 @@
import { fetchJavaScriptModuleSourceText } from '../fetchJavaScriptModuleSourceText';
import { fetchJavaScriptModuleSourceText } from '@/host/utils/fetchJavaScriptModuleSourceText';
const originalFetch = globalThis.fetch;
@@ -14,11 +14,11 @@ describe('fetchJavaScriptModuleSourceText', () => {
})) as unknown as typeof fetch;
await expect(
fetchJavaScriptModuleSourceText('https://api.twenty.test/component.js'),
fetchJavaScriptModuleSourceText('https://api.twenty.test/core/abc.js'),
).resolves.toBe('module source');
});
it('should forward headers to fetch', async () => {
it('should forward headers and omit credentials', async () => {
const fetchSpy = jest.fn(async () => ({
ok: true,
text: async () => '',
@@ -26,15 +26,13 @@ describe('fetchJavaScriptModuleSourceText', () => {
globalThis.fetch = fetchSpy as unknown as typeof fetch;
await fetchJavaScriptModuleSourceText(
'https://api.twenty.test/component.js',
{
Authorization: 'Bearer token',
},
'https://api.twenty.test/core/abc.js',
{ Authorization: 'Bearer token' },
);
expect(fetchSpy).toHaveBeenCalledWith(
'https://api.twenty.test/component.js',
{ headers: { Authorization: 'Bearer token' } },
'https://api.twenty.test/core/abc.js',
{ headers: { Authorization: 'Bearer token' }, credentials: 'omit' },
);
});
@@ -46,7 +44,7 @@ describe('fetchJavaScriptModuleSourceText', () => {
})) as unknown as typeof fetch;
await expect(
fetchJavaScriptModuleSourceText('https://api.twenty.test/component.js'),
fetchJavaScriptModuleSourceText('https://api.twenty.test/core/abc.js'),
).rejects.toMatchObject({ code: 'FRONT_COMPONENT_MODULE_FETCH_FAILED' });
});
@@ -56,25 +54,11 @@ describe('fetchJavaScriptModuleSourceText', () => {
}) as unknown as typeof fetch;
await expect(
fetchJavaScriptModuleSourceText('https://api.twenty.test/component.js'),
fetchJavaScriptModuleSourceText('https://api.twenty.test/core/abc.js'),
).rejects.toMatchObject({
code: 'FRONT_COMPONENT_MODULE_FETCH_FAILED',
message:
'Failed to fetch front component module https://api.twenty.test/component.js: Failed to fetch',
'Failed to fetch SDK client module https://api.twenty.test/core/abc.js: Failed to fetch',
});
});
it('should include the url and status in the error message', async () => {
globalThis.fetch = jest.fn(async () => ({
ok: false,
status: 403,
statusText: 'Forbidden',
})) as unknown as typeof fetch;
await expect(
fetchJavaScriptModuleSourceText('https://api.twenty.test/component.js'),
).rejects.toThrow(
'Failed to fetch front component module https://api.twenty.test/component.js: 403 Forbidden',
);
});
});
@@ -0,0 +1,33 @@
import { CustomError } from 'twenty-shared/utils';
// Plain fetch for a JavaScript module served directly from a content-addressed,
// immutable URL. The browser HTTP cache handles both reuse and checksum-based
// invalidation (a new checksum yields a new URL), so no host-side CacheStorage
// layer is needed here — that layer exists only to cache front-component
// sources across the rotating presigned URLs they are served through.
export const fetchJavaScriptModuleSourceText = async (
url: string,
headers?: Record<string, string>,
): Promise<string> => {
let response: Response;
try {
response = await fetch(url, { headers, credentials: 'omit' });
} catch (error) {
throw new CustomError(
`Failed to fetch SDK client module ${url}: ${
error instanceof Error ? error.message : String(error)
}`,
'FRONT_COMPONENT_MODULE_FETCH_FAILED',
);
}
if (!response.ok) {
throw new CustomError(
`Failed to fetch SDK client module ${url}: ${response.status} ${response.statusText}`,
'FRONT_COMPONENT_MODULE_FETCH_FAILED',
);
}
return response.text();
};
@@ -0,0 +1,18 @@
import { fetchJavaScriptModuleSourceText } from '@/host/utils/fetchJavaScriptModuleSourceText';
import { type SdkClientSources } from '@/types/SdkClientSources';
import { type SdkClientUrls } from '@/types/SdkClientUrls';
export const fetchSdkClientSources = async ({
sdkClientUrls,
headers,
}: {
sdkClientUrls: SdkClientUrls;
headers?: Record<string, string>;
}): Promise<SdkClientSources> => {
const [core, metadata] = await Promise.all([
fetchJavaScriptModuleSourceText(sdkClientUrls.core, headers),
fetchJavaScriptModuleSourceText(sdkClientUrls.metadata, headers),
]);
return { core, metadata };
};
@@ -1,17 +1,20 @@
import { release, retain } from '@quilted/threads';
import { RemoteReceiver } from '@remote-dom/core/receivers';
import { useEffect, useRef } from 'react';
import { isDefined } from 'twenty-shared/utils';
import { buildHostFetchPolicyFromFrontComponentUrls } from '@/host/utils/buildHostFetchPolicyFromFrontComponentUrls';
import { createFrontComponentHostThread } from '@/host/utils/createFrontComponentHostThread';
import { createHostFetchEnforcingPolicy } from '@/host/utils/createHostFetchEnforcingPolicy';
import { fetchComponentSource } from '@/host/utils/fetchComponentSource';
import { fetchSdkClientSources } from '@/host/utils/fetchSdkClientSources';
import { FRONT_COMPONENT_SANDBOX_DOCUMENT } from '@/remote/sandbox/generated/frontComponentSandboxDocument';
import { createFrontComponentSandboxIframe } from '@/remote/sandbox/utils/createFrontComponentSandboxIframe';
import { createFrontComponentSandboxMessageHandler } from '@/remote/sandbox/utils/createFrontComponentSandboxMessageHandler';
import { type FrontComponentThread } from '@/types/FrontComponentThread';
import { type SdkClientUrls } from '@/types/SdkClientUrls';
import { buildAuthorizationHeadersFromAccessToken } from '@/utils/buildAuthorizationHeadersFromAccessToken';
import { containsSdkClientImportSpecifier } from '@/utils/containsSdkClientImportSpecifier';
type FrontComponentWorkerEffectProps = {
componentUrl: string;
@@ -77,24 +80,39 @@ export const FrontComponentWorkerEffect = ({
const resolveComponentSourceAndRender = async () => {
try {
const authorizationHeaders = buildAuthorizationHeadersFromAccessToken(
applicationAccessToken,
);
const componentSource = await fetchComponentSource({
url: componentUrl,
headers: buildAuthorizationHeadersFromAccessToken(
applicationAccessToken,
),
headers: authorizationHeaders,
});
if (isCancelled) {
return;
}
const sdkClientSources =
isDefined(sdkClientUrls) &&
containsSdkClientImportSpecifier(componentSource)
? await fetchSdkClientSources({
sdkClientUrls,
headers: authorizationHeaders,
})
: undefined;
if (isCancelled) {
return;
}
await thread.imports.render(newReceiver.connection, {
componentUrl,
componentSource,
applicationAccessToken,
apiUrl,
functionsBaseUrl,
sdkClientUrls,
sdkClientSources,
hostFetchOrigins: hostFetchPolicy.allowedOrigins,
applicationVariables,
});
@@ -0,0 +1,26 @@
import { createSdkClientModuleBlobUrls } from '../createSdkClientModuleBlobUrls';
describe('createSdkClientModuleBlobUrls', () => {
const createObjectURLMock = jest.fn();
beforeEach(() => {
jest.clearAllMocks();
createObjectURLMock.mockImplementation(
(blob: Blob) => `blob:mock/${(blob as Blob).size}`,
);
global.URL.createObjectURL =
createObjectURLMock as unknown as typeof URL.createObjectURL;
});
it('mints a blob url for each sdk client module source', () => {
const blobUrls = createSdkClientModuleBlobUrls({
core: 'export const core = 1;',
metadata: 'export const metadata = 2;',
});
expect(createObjectURLMock).toHaveBeenCalledTimes(2);
expect(blobUrls.core).toMatch(/^blob:mock\//);
expect(blobUrls.metadata).toMatch(/^blob:mock\//);
expect(blobUrls.core).not.toEqual(blobUrls.metadata);
});
});
@@ -1,99 +0,0 @@
import { fetchSdkClientModulesAsBlobUrls } from '../fetchSdkClientModulesAsBlobUrls';
const originalFetch = globalThis.fetch;
const originalCreateObjectUrl = URL.createObjectURL;
const originalRevokeObjectUrl = URL.revokeObjectURL;
const sdkClientUrls = {
core: 'https://api.twenty.test/sdk-client/application-id/core',
metadata: 'https://api.twenty.test/sdk-client/application-id/metadata',
};
describe('fetchSdkClientModulesAsBlobUrls', () => {
afterEach(() => {
globalThis.fetch = originalFetch;
URL.createObjectURL = originalCreateObjectUrl;
URL.revokeObjectURL = originalRevokeObjectUrl;
});
it('should fetch both sdk modules and return their blob urls', async () => {
const fetchSpy = jest.fn(async (url: string) => ({
ok: true,
text: async () => `source of ${url}`,
}));
globalThis.fetch = fetchSpy as unknown as typeof fetch;
URL.createObjectURL = jest
.fn()
.mockReturnValueOnce('blob:core-url')
.mockReturnValueOnce('blob:metadata-url');
await expect(
fetchSdkClientModulesAsBlobUrls(sdkClientUrls),
).resolves.toEqual({
core: 'blob:core-url',
metadata: 'blob:metadata-url',
});
expect(fetchSpy).toHaveBeenCalledWith(sdkClientUrls.core, {
headers: undefined,
});
expect(fetchSpy).toHaveBeenCalledWith(sdkClientUrls.metadata, {
headers: undefined,
});
});
it('should forward headers to both module fetches', async () => {
const fetchSpy = jest.fn(async () => ({
ok: true,
text: async () => '',
}));
globalThis.fetch = fetchSpy as unknown as typeof fetch;
URL.createObjectURL = jest.fn(() => 'blob:mock-url');
await fetchSdkClientModulesAsBlobUrls(sdkClientUrls, {
Authorization: 'Bearer token',
});
expect(fetchSpy).toHaveBeenCalledWith(sdkClientUrls.core, {
headers: { Authorization: 'Bearer token' },
});
expect(fetchSpy).toHaveBeenCalledWith(sdkClientUrls.metadata, {
headers: { Authorization: 'Bearer token' },
});
});
it('should propagate the fetch error when one module fails to load', async () => {
globalThis.fetch = jest.fn(async (url: string) => ({
ok: url !== sdkClientUrls.metadata,
status: 500,
statusText: 'Internal Server Error',
text: async () => '',
})) as unknown as typeof fetch;
URL.createObjectURL = jest.fn(() => 'blob:mock-url');
URL.revokeObjectURL = jest.fn();
await expect(
fetchSdkClientModulesAsBlobUrls(sdkClientUrls),
).rejects.toMatchObject({
code: 'FRONT_COMPONENT_MODULE_FETCH_FAILED',
});
});
it('should revoke the created blob url when the other module fails to load', async () => {
globalThis.fetch = jest.fn(async (url: string) => ({
ok: url !== sdkClientUrls.metadata,
status: 500,
statusText: 'Internal Server Error',
text: async () => '',
})) as unknown as typeof fetch;
URL.createObjectURL = jest.fn(() => 'blob:core-url');
const revokeObjectUrlSpy = jest.fn();
URL.revokeObjectURL = revokeObjectUrlSpy;
await expect(
fetchSdkClientModulesAsBlobUrls(sdkClientUrls),
).rejects.toMatchObject({
code: 'FRONT_COMPONENT_MODULE_FETCH_FAILED',
});
expect(revokeObjectUrlSpy).toHaveBeenCalledWith('blob:core-url');
});
});
@@ -1,4 +0,0 @@
import { SDK_CLIENT_IMPORT_SPECIFIERS } from '@/remote/worker/constants/SdkClientImportSpecifiers';
export const containsSdkClientImportSpecifier = (source: string): boolean =>
SDK_CLIENT_IMPORT_SPECIFIERS.some((specifier) => source.includes(specifier));
@@ -0,0 +1,10 @@
import { createJavaScriptModuleBlobUrl } from '@/remote/worker/utils/createJavaScriptModuleBlobUrl';
import { type SdkClientSources } from '@/types/SdkClientSources';
import { type SdkClientUrls } from '@/types/SdkClientUrls';
export const createSdkClientModuleBlobUrls = (
sdkClientSources: SdkClientSources,
): SdkClientUrls => ({
core: createJavaScriptModuleBlobUrl(sdkClientSources.core),
metadata: createJavaScriptModuleBlobUrl(sdkClientSources.metadata),
});
@@ -1,28 +0,0 @@
import { CustomError } from 'twenty-shared/utils';
export const fetchJavaScriptModuleSourceText = async (
url: string,
headers?: Record<string, string>,
): Promise<string> => {
let response: Response;
try {
response = await fetch(url, { headers });
} catch (error) {
throw new CustomError(
`Failed to fetch front component module ${url}: ${
error instanceof Error ? error.message : String(error)
}`,
'FRONT_COMPONENT_MODULE_FETCH_FAILED',
);
}
if (!response.ok) {
throw new CustomError(
`Failed to fetch front component module ${url}: ${response.status} ${response.statusText}`,
'FRONT_COMPONENT_MODULE_FETCH_FAILED',
);
}
return response.text();
};
@@ -1,34 +0,0 @@
import { createJavaScriptModuleBlobUrl } from '@/remote/worker/utils/createJavaScriptModuleBlobUrl';
import { fetchJavaScriptModuleSourceText } from '@/remote/worker/utils/fetchJavaScriptModuleSourceText';
import { type SdkClientUrls } from '@/types/SdkClientUrls';
export const fetchSdkClientModulesAsBlobUrls = async (
sdkClientUrls: SdkClientUrls,
headers?: Record<string, string>,
): Promise<SdkClientUrls> => {
const [coreResult, metadataResult] = await Promise.allSettled([
fetchJavaScriptModuleSourceText(sdkClientUrls.core, headers).then(
createJavaScriptModuleBlobUrl,
),
fetchJavaScriptModuleSourceText(sdkClientUrls.metadata, headers).then(
createJavaScriptModuleBlobUrl,
),
]);
if (
coreResult.status === 'rejected' ||
metadataResult.status === 'rejected'
) {
for (const result of [coreResult, metadataResult]) {
if (result.status === 'fulfilled') {
URL.revokeObjectURL(result.value);
}
}
throw coreResult.status === 'rejected'
? coreResult.reason
: (metadataResult as PromiseRejectedResult).reason;
}
return { core: coreResult.value, metadata: metadataResult.value };
};
@@ -1,17 +1,14 @@
import { isDefined } from 'twenty-shared/utils';
import { containsSdkClientImportSpecifier } from '@/remote/worker/utils/containsSdkClientImportSpecifier';
import { createJavaScriptModuleBlobUrl } from '@/remote/worker/utils/createJavaScriptModuleBlobUrl';
import { fetchSdkClientModulesAsBlobUrls } from '@/remote/worker/utils/fetchSdkClientModulesAsBlobUrls';
import { createSdkClientModuleBlobUrls } from '@/remote/worker/utils/createSdkClientModuleBlobUrls';
import { revokeSdkClientModuleBlobUrls } from '@/remote/worker/utils/revokeSdkClientModuleBlobUrls';
import { rewriteSdkClientImportsToBlobUrls } from '@/remote/worker/utils/rewriteSdkClientImportsToBlobUrls';
import { type SdkClientUrls } from '@/types/SdkClientUrls';
import { buildAuthorizationHeadersFromAccessToken } from '@/utils/buildAuthorizationHeadersFromAccessToken';
import { type SdkClientSources } from '@/types/SdkClientSources';
type LoadFrontComponentModuleInput = {
componentSource: string;
sdkClientUrls?: SdkClientUrls;
applicationAccessToken?: string;
sdkClientSources?: SdkClientSources;
};
type FrontComponentModule = {
@@ -20,21 +17,11 @@ type FrontComponentModule = {
export const loadFrontComponentModule = async ({
componentSource,
sdkClientUrls,
applicationAccessToken,
sdkClientSources,
}: LoadFrontComponentModuleInput): Promise<FrontComponentModule> => {
const authorizationHeaders = buildAuthorizationHeadersFromAccessToken(
applicationAccessToken,
);
const sdkModuleBlobUrls =
isDefined(sdkClientUrls) &&
containsSdkClientImportSpecifier(componentSource)
? await fetchSdkClientModulesAsBlobUrls(
sdkClientUrls,
authorizationHeaders,
)
: null;
const sdkModuleBlobUrls = isDefined(sdkClientSources)
? createSdkClientModuleBlobUrls(sdkClientSources)
: null;
const componentModuleSource = isDefined(sdkModuleBlobUrls)
? rewriteSdkClientImportsToBlobUrls(componentSource, sdkModuleBlobUrls)
@@ -34,8 +34,7 @@ export const renderFrontComponent = async ({
const componentModule = await loadFrontComponentModule({
componentSource: renderContext.componentSource,
sdkClientUrls: renderContext.sdkClientUrls,
applicationAccessToken: renderContext.applicationAccessToken,
sdkClientSources: renderContext.sdkClientSources,
});
componentModule.default(renderContainer);
@@ -1,14 +1,6 @@
import { type SDK_CLIENT_IMPORT_SPECIFIERS } from '@/remote/worker/constants/SdkClientImportSpecifiers';
import { type SDK_CLIENT_IMPORT_SPECIFIERS } from '@/constants/SdkClientImportSpecifiers';
import { type SdkClientUrls } from '@/types/SdkClientUrls';
const escapeRegExpToken = (value: string): string =>
value.replace(/[.*+?^${}()|[\]\\]/g, '\\$&');
const buildImportContextPattern = (specifier: string): RegExp =>
new RegExp(
`(\\bfrom\\s*|\\bimport\\s*\\(\\s*|\\bimport\\s*)(["'])${escapeRegExpToken(specifier)}\\2`,
'g',
);
import { buildSdkClientImportContextPattern } from '@/utils/buildSdkClientImportContextPattern';
export const rewriteSdkClientImportsToBlobUrls = (
source: string,
@@ -26,7 +18,7 @@ export const rewriteSdkClientImportsToBlobUrls = (
for (const [specifier, blobUrl] of Object.entries(specifierToBlobUrl)) {
rewrittenSource = rewrittenSource.replace(
buildImportContextPattern(specifier),
buildSdkClientImportContextPattern(specifier),
(_fullMatch, importContext: string, quote: string) =>
`${importContext}${quote}${blobUrl}${quote}`,
);
@@ -1,4 +1,4 @@
import { type SdkClientUrls } from '@/types/SdkClientUrls';
import { type SdkClientSources } from '@/types/SdkClientSources';
export type HostToWorkerRenderContext = {
componentUrl: string;
@@ -6,7 +6,7 @@ export type HostToWorkerRenderContext = {
applicationAccessToken?: string;
apiUrl?: string;
functionsBaseUrl?: string;
sdkClientUrls?: SdkClientUrls;
sdkClientSources?: SdkClientSources;
hostFetchOrigins?: string[];
applicationVariables?: Record<string, string>;
};
@@ -0,0 +1,4 @@
export type SdkClientSources = {
core: string;
metadata: string;
};
@@ -0,0 +1,61 @@
import { containsSdkClientImportSpecifier } from '../containsSdkClientImportSpecifier';
describe('containsSdkClientImportSpecifier', () => {
it('returns true when the source imports an sdk client module', () => {
expect(
containsSdkClientImportSpecifier(
"import { getClient } from 'twenty-client-sdk/core';",
),
).toBe(true);
expect(
containsSdkClientImportSpecifier(
"import { getMetadata } from 'twenty-client-sdk/metadata';",
),
).toBe(true);
});
it('returns true for dynamic import and re-export module positions', () => {
expect(
containsSdkClientImportSpecifier(
"const client = await import('twenty-client-sdk/core');",
),
).toBe(true);
expect(
containsSdkClientImportSpecifier(
"export { getClient } from 'twenty-client-sdk/core';",
),
).toBe(true);
});
it('returns false when the source does not import an sdk client module', () => {
expect(
containsSdkClientImportSpecifier("import { useState } from 'react';"),
).toBe(false);
});
it('returns false for a sibling module sharing the specifier prefix', () => {
expect(
containsSdkClientImportSpecifier(
"import { thing } from 'twenty-client-sdk/core-extra';",
),
).toBe(false);
expect(
containsSdkClientImportSpecifier(
"import { thing } from 'twenty-client-sdk/metadata-utils';",
),
).toBe(false);
});
it('returns false when the specifier only appears outside an import position', () => {
expect(
containsSdkClientImportSpecifier(
"const docsUrl = 'twenty-client-sdk/core';",
),
).toBe(false);
expect(
containsSdkClientImportSpecifier(
'// see twenty-client-sdk/core for the generated client',
),
).toBe(false);
});
});
@@ -0,0 +1,8 @@
const escapeRegExpToken = (value: string): string =>
value.replace(/[.*+?^${}()|[\]\\]/g, '\\$&');
export const buildSdkClientImportContextPattern = (specifier: string): RegExp =>
new RegExp(
`(\\bfrom\\s*|\\bimport\\s*\\(\\s*|\\bimport\\s*)(["'])${escapeRegExpToken(specifier)}\\2`,
'g',
);
@@ -0,0 +1,7 @@
import { SDK_CLIENT_IMPORT_SPECIFIERS } from '@/constants/SdkClientImportSpecifiers';
import { buildSdkClientImportContextPattern } from '@/utils/buildSdkClientImportContextPattern';
export const containsSdkClientImportSpecifier = (source: string): boolean =>
SDK_CLIENT_IMPORT_SPECIFIERS.some((specifier) =>
buildSdkClientImportContextPattern(specifier).test(source),
);
@@ -4345,6 +4345,7 @@ export type Query = {
appConnections: Array<AppConnection>;
applicationConnectionProviders: Array<ApplicationConnectionProvider>;
applicationRegistrationTarballUrl?: Maybe<Scalars['String']['output']>;
applicationSdkClientChecksums?: Maybe<SdkClientChecksums>;
barChartData: BarChartData;
billingPortalSession: BillingSession;
chatMessages: Array<AgentMessage>;
@@ -4482,6 +4483,11 @@ export type QueryApplicationRegistrationTarballUrlArgs = {
};
export type QueryApplicationSdkClientChecksumsArgs = {
applicationId: Scalars['UUID']['input'];
};
export type QueryBarChartDataArgs = {
input: BarChartDataInput;
};
@@ -5052,6 +5058,12 @@ export enum SsoIdentityProviderStatus {
Inactive = 'Inactive'
}
export type SdkClientChecksums = {
__typename?: 'SdkClientChecksums';
core?: Maybe<Scalars['String']['output']>;
metadata: Scalars['String']['output'];
};
export type SearchField = {
__typename?: 'SearchField';
createdAt: Scalars['DateTime']['output'];
@@ -7202,6 +7214,13 @@ export type FindOneFrontComponentQueryVariables = Exact<{
export type FindOneFrontComponentQuery = { __typename?: 'Query', frontComponent?: { __typename?: 'FrontComponent', id: string, name: string, applicationId: string, builtComponentChecksum: string, isHeadless: boolean, usesSdkClient: boolean, applicationVariables?: any | null, applicationTokenPair?: { __typename?: 'ApplicationTokenPair', applicationAccessToken: { __typename?: 'AuthToken', token: string, expiresAt: string }, applicationRefreshToken: { __typename?: 'AuthToken', token: string, expiresAt: string } } | null } | null };
export type GetApplicationSdkClientChecksumsQueryVariables = Exact<{
applicationId: Scalars['UUID']['input'];
}>;
export type GetApplicationSdkClientChecksumsQuery = { __typename?: 'Query', applicationSdkClientChecksums?: { __typename?: 'SdkClientChecksums', core?: string | null, metadata: string } | null };
export type LogicFunctionFieldsFragment = { __typename?: 'LogicFunction', id: string, name: string, description?: string | null, runtime: string, timeoutSeconds: number, executionMode: LogicFunctionExecutionMode, sourceHandlerPath: string, handlerName: string, cronTriggerSettings?: any | null, databaseEventTriggerSettings?: any | null, httpRouteTriggerSettings?: any | null, toolTriggerSettings?: any | null, workflowActionTriggerSettings?: any | null, applicationId?: string | null, universalIdentifier?: string | null, createdAt: string, updatedAt: string };
export type CreateOneLogicFunctionMutationVariables = Exact<{
@@ -9001,6 +9020,7 @@ export const UploadWorkflowFileDocument = {"kind":"Document","definitions":[{"ki
export const RenewApplicationTokenDocument = {"kind":"Document","definitions":[{"kind":"OperationDefinition","operation":"mutation","name":{"kind":"Name","value":"RenewApplicationToken"},"variableDefinitions":[{"kind":"VariableDefinition","variable":{"kind":"Variable","name":{"kind":"Name","value":"applicationRefreshToken"}},"type":{"kind":"NonNullType","type":{"kind":"NamedType","name":{"kind":"Name","value":"String"}}}}],"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"renewApplicationToken"},"arguments":[{"kind":"Argument","name":{"kind":"Name","value":"applicationRefreshToken"},"value":{"kind":"Variable","name":{"kind":"Name","value":"applicationRefreshToken"}}}],"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"applicationAccessToken"},"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"token"}},{"kind":"Field","name":{"kind":"Name","value":"expiresAt"}}]}},{"kind":"Field","name":{"kind":"Name","value":"applicationRefreshToken"},"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"token"}},{"kind":"Field","name":{"kind":"Name","value":"expiresAt"}}]}}]}}]}}]} as unknown as DocumentNode<RenewApplicationTokenMutation, RenewApplicationTokenMutationVariables>;
export const FindManyFrontComponentsDocument = {"kind":"Document","definitions":[{"kind":"OperationDefinition","operation":"query","name":{"kind":"Name","value":"FindManyFrontComponents"},"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"frontComponents"},"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"id"}},{"kind":"Field","name":{"kind":"Name","value":"name"}},{"kind":"Field","name":{"kind":"Name","value":"applicationId"}},{"kind":"Field","name":{"kind":"Name","value":"builtComponentChecksum"}},{"kind":"Field","name":{"kind":"Name","value":"builtComponentPath"}},{"kind":"Field","name":{"kind":"Name","value":"componentName"}},{"kind":"Field","name":{"kind":"Name","value":"createdAt"}},{"kind":"Field","name":{"kind":"Name","value":"description"}},{"kind":"Field","name":{"kind":"Name","value":"isHeadless"}},{"kind":"Field","name":{"kind":"Name","value":"sourceComponentPath"}},{"kind":"Field","name":{"kind":"Name","value":"universalIdentifier"}},{"kind":"Field","name":{"kind":"Name","value":"updatedAt"}},{"kind":"Field","name":{"kind":"Name","value":"usesSdkClient"}}]}}]}}]} as unknown as DocumentNode<FindManyFrontComponentsQuery, FindManyFrontComponentsQueryVariables>;
export const FindOneFrontComponentDocument = {"kind":"Document","definitions":[{"kind":"OperationDefinition","operation":"query","name":{"kind":"Name","value":"FindOneFrontComponent"},"variableDefinitions":[{"kind":"VariableDefinition","variable":{"kind":"Variable","name":{"kind":"Name","value":"id"}},"type":{"kind":"NonNullType","type":{"kind":"NamedType","name":{"kind":"Name","value":"UUID"}}}}],"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"frontComponent"},"arguments":[{"kind":"Argument","name":{"kind":"Name","value":"id"},"value":{"kind":"Variable","name":{"kind":"Name","value":"id"}}}],"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"id"}},{"kind":"Field","name":{"kind":"Name","value":"name"}},{"kind":"Field","name":{"kind":"Name","value":"applicationId"}},{"kind":"Field","name":{"kind":"Name","value":"builtComponentChecksum"}},{"kind":"Field","name":{"kind":"Name","value":"isHeadless"}},{"kind":"Field","name":{"kind":"Name","value":"usesSdkClient"}},{"kind":"Field","name":{"kind":"Name","value":"applicationVariables"}},{"kind":"Field","name":{"kind":"Name","value":"applicationTokenPair"},"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"applicationAccessToken"},"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"token"}},{"kind":"Field","name":{"kind":"Name","value":"expiresAt"}}]}},{"kind":"Field","name":{"kind":"Name","value":"applicationRefreshToken"},"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"token"}},{"kind":"Field","name":{"kind":"Name","value":"expiresAt"}}]}}]}}]}}]}}]} as unknown as DocumentNode<FindOneFrontComponentQuery, FindOneFrontComponentQueryVariables>;
export const GetApplicationSdkClientChecksumsDocument = {"kind":"Document","definitions":[{"kind":"OperationDefinition","operation":"query","name":{"kind":"Name","value":"GetApplicationSdkClientChecksums"},"variableDefinitions":[{"kind":"VariableDefinition","variable":{"kind":"Variable","name":{"kind":"Name","value":"applicationId"}},"type":{"kind":"NonNullType","type":{"kind":"NamedType","name":{"kind":"Name","value":"UUID"}}}}],"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"applicationSdkClientChecksums"},"arguments":[{"kind":"Argument","name":{"kind":"Name","value":"applicationId"},"value":{"kind":"Variable","name":{"kind":"Name","value":"applicationId"}}}],"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"core"}},{"kind":"Field","name":{"kind":"Name","value":"metadata"}}]}}]}}]} as unknown as DocumentNode<GetApplicationSdkClientChecksumsQuery, GetApplicationSdkClientChecksumsQueryVariables>;
export const CreateOneLogicFunctionDocument = {"kind":"Document","definitions":[{"kind":"OperationDefinition","operation":"mutation","name":{"kind":"Name","value":"CreateOneLogicFunction"},"variableDefinitions":[{"kind":"VariableDefinition","variable":{"kind":"Variable","name":{"kind":"Name","value":"input"}},"type":{"kind":"NonNullType","type":{"kind":"NamedType","name":{"kind":"Name","value":"CreateLogicFunctionFromSourceInput"}}}}],"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"createOneLogicFunction"},"arguments":[{"kind":"Argument","name":{"kind":"Name","value":"input"},"value":{"kind":"Variable","name":{"kind":"Name","value":"input"}}}],"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"FragmentSpread","name":{"kind":"Name","value":"LogicFunctionFields"}}]}}]}},{"kind":"FragmentDefinition","name":{"kind":"Name","value":"LogicFunctionFields"},"typeCondition":{"kind":"NamedType","name":{"kind":"Name","value":"LogicFunction"}},"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"id"}},{"kind":"Field","name":{"kind":"Name","value":"name"}},{"kind":"Field","name":{"kind":"Name","value":"description"}},{"kind":"Field","name":{"kind":"Name","value":"runtime"}},{"kind":"Field","name":{"kind":"Name","value":"timeoutSeconds"}},{"kind":"Field","name":{"kind":"Name","value":"executionMode"}},{"kind":"Field","name":{"kind":"Name","value":"sourceHandlerPath"}},{"kind":"Field","name":{"kind":"Name","value":"handlerName"}},{"kind":"Field","name":{"kind":"Name","value":"cronTriggerSettings"}},{"kind":"Field","name":{"kind":"Name","value":"databaseEventTriggerSettings"}},{"kind":"Field","name":{"kind":"Name","value":"httpRouteTriggerSettings"}},{"kind":"Field","name":{"kind":"Name","value":"toolTriggerSettings"}},{"kind":"Field","name":{"kind":"Name","value":"workflowActionTriggerSettings"}},{"kind":"Field","name":{"kind":"Name","value":"applicationId"}},{"kind":"Field","name":{"kind":"Name","value":"universalIdentifier"}},{"kind":"Field","name":{"kind":"Name","value":"createdAt"}},{"kind":"Field","name":{"kind":"Name","value":"updatedAt"}}]}}]} as unknown as DocumentNode<CreateOneLogicFunctionMutation, CreateOneLogicFunctionMutationVariables>;
export const DeleteOneLogicFunctionDocument = {"kind":"Document","definitions":[{"kind":"OperationDefinition","operation":"mutation","name":{"kind":"Name","value":"DeleteOneLogicFunction"},"variableDefinitions":[{"kind":"VariableDefinition","variable":{"kind":"Variable","name":{"kind":"Name","value":"input"}},"type":{"kind":"NonNullType","type":{"kind":"NamedType","name":{"kind":"Name","value":"LogicFunctionIdInput"}}}}],"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"deleteOneLogicFunction"},"arguments":[{"kind":"Argument","name":{"kind":"Name","value":"input"},"value":{"kind":"Variable","name":{"kind":"Name","value":"input"}}}],"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"FragmentSpread","name":{"kind":"Name","value":"LogicFunctionFields"}}]}}]}},{"kind":"FragmentDefinition","name":{"kind":"Name","value":"LogicFunctionFields"},"typeCondition":{"kind":"NamedType","name":{"kind":"Name","value":"LogicFunction"}},"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"id"}},{"kind":"Field","name":{"kind":"Name","value":"name"}},{"kind":"Field","name":{"kind":"Name","value":"description"}},{"kind":"Field","name":{"kind":"Name","value":"runtime"}},{"kind":"Field","name":{"kind":"Name","value":"timeoutSeconds"}},{"kind":"Field","name":{"kind":"Name","value":"executionMode"}},{"kind":"Field","name":{"kind":"Name","value":"sourceHandlerPath"}},{"kind":"Field","name":{"kind":"Name","value":"handlerName"}},{"kind":"Field","name":{"kind":"Name","value":"cronTriggerSettings"}},{"kind":"Field","name":{"kind":"Name","value":"databaseEventTriggerSettings"}},{"kind":"Field","name":{"kind":"Name","value":"httpRouteTriggerSettings"}},{"kind":"Field","name":{"kind":"Name","value":"toolTriggerSettings"}},{"kind":"Field","name":{"kind":"Name","value":"workflowActionTriggerSettings"}},{"kind":"Field","name":{"kind":"Name","value":"applicationId"}},{"kind":"Field","name":{"kind":"Name","value":"universalIdentifier"}},{"kind":"Field","name":{"kind":"Name","value":"createdAt"}},{"kind":"Field","name":{"kind":"Name","value":"updatedAt"}}]}}]} as unknown as DocumentNode<DeleteOneLogicFunctionMutation, DeleteOneLogicFunctionMutationVariables>;
export const ExecuteOneLogicFunctionDocument = {"kind":"Document","definitions":[{"kind":"OperationDefinition","operation":"mutation","name":{"kind":"Name","value":"ExecuteOneLogicFunction"},"variableDefinitions":[{"kind":"VariableDefinition","variable":{"kind":"Variable","name":{"kind":"Name","value":"input"}},"type":{"kind":"NonNullType","type":{"kind":"NamedType","name":{"kind":"Name","value":"ExecuteOneLogicFunctionInput"}}}}],"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"executeOneLogicFunction"},"arguments":[{"kind":"Argument","name":{"kind":"Name","value":"input"},"value":{"kind":"Variable","name":{"kind":"Name","value":"input"}}}],"selectionSet":{"kind":"SelectionSet","selections":[{"kind":"Field","name":{"kind":"Name","value":"data"}},{"kind":"Field","name":{"kind":"Name","value":"logs"}},{"kind":"Field","name":{"kind":"Name","value":"duration"}},{"kind":"Field","name":{"kind":"Name","value":"status"}},{"kind":"Field","name":{"kind":"Name","value":"error"}}]}}]}}]} as unknown as DocumentNode<ExecuteOneLogicFunctionMutation, ExecuteOneLogicFunctionMutationVariables>;
@@ -11,14 +11,20 @@ export const useListenToMetadataOperationBrowserEvent = <
onMetadataOperationBrowserEvent,
metadataName,
operationTypes,
skip = false,
}: {
onMetadataOperationBrowserEvent: (
detail: MetadataOperationBrowserEventDetail<T>,
) => void;
metadataName?: BroadcastEntityName;
operationTypes?: MetadataOperation<T>['type'][];
skip?: boolean;
}) => {
useEffect(() => {
if (skip) {
return;
}
const handleMetadataOperationEvent = (
event: CustomEvent<MetadataOperationBrowserEventDetail<T>>,
) => {
@@ -49,5 +55,5 @@ export const useListenToMetadataOperationBrowserEvent = <
handleMetadataOperationEvent as EventListener,
);
};
}, [metadataName, onMetadataOperationBrowserEvent, operationTypes]);
}, [metadataName, onMetadataOperationBrowserEvent, operationTypes, skip]);
};
@@ -5,6 +5,7 @@ import { type AllMetadataName } from 'twenty-shared/metadata';
// workspace migration / syncable entity system (ALL_METADATA_NAME).
const ALL_NON_SYNCABLE_BROADCAST_ENTITY_NAME = {
agentChatThread: 'agentChatThread',
application: 'application',
} as const;
type NonSyncableBroadcastEntityName =
@@ -0,0 +1,25 @@
import { frontComponentApplicationTokenPairComponentState } from '@/front-components/states/frontComponentApplicationTokenPairComponentState';
import { useSetAtomComponentState } from '@/ui/utilities/state/jotai/hooks/useSetAtomComponentState';
import { useEffect } from 'react';
import { type ApplicationTokenPair } from '~/generated-metadata/graphql';
type FrontComponentApplicationTokenPairEffectProps = {
frontComponentId: string;
applicationTokenPair: ApplicationTokenPair | null;
};
export const FrontComponentApplicationTokenPairEffect = ({
frontComponentId,
applicationTokenPair,
}: FrontComponentApplicationTokenPairEffectProps) => {
const setFrontComponentApplicationTokenPair = useSetAtomComponentState(
frontComponentApplicationTokenPairComponentState,
frontComponentId,
);
useEffect(() => {
setFrontComponentApplicationTokenPair(applicationTokenPair);
}, [applicationTokenPair, setFrontComponentApplicationTokenPair]);
return null;
};
@@ -0,0 +1,24 @@
import { useSnackBar } from '@/ui/feedback/snack-bar-manager/hooks/useSnackBar';
import { t } from '@lingui/core/macro';
import { useEffect } from 'react';
import { isDefined } from 'twenty-shared/utils';
type FrontComponentLoadErrorSnackBarEffectProps = {
errorMessage?: string;
};
export const FrontComponentLoadErrorSnackBarEffect = ({
errorMessage,
}: FrontComponentLoadErrorSnackBarEffectProps) => {
const { enqueueErrorSnackBar } = useSnackBar();
useEffect(() => {
if (isDefined(errorMessage)) {
enqueueErrorSnackBar({
message: t`Failed to load front component: ${errorMessage}`,
});
}
}, [errorMessage, enqueueErrorSnackBar]);
return null;
};
@@ -1,20 +1,25 @@
import { FrontComponentApplicationTokenPairEffect } from '@/front-components/components/FrontComponentApplicationTokenPairEffect';
import { FrontComponentLoadErrorSnackBarEffect } from '@/front-components/components/FrontComponentLoadErrorSnackBarEffect';
import { FrontComponentRendererProvider } from '@/front-components/components/FrontComponentRendererProvider';
import { useFrontComponentExecutionContext } from '@/front-components/hooks/useFrontComponentExecutionContext';
import { useOnApplicationSdkClientChecksumsUpdated } from '@/front-components/hooks/useOnApplicationSdkClientChecksumsUpdated';
import { useOnFrontComponentUpdated } from '@/front-components/hooks/useOnFrontComponentUpdated';
import { getFrontComponentUrl } from '@/front-components/utils/getFrontComponentUrl';
import { getSdkClientUrls } from '@/front-components/utils/getSdkClientUrls';
import { useGetLogicFunctionHttpUrl } from '@/settings/logic-functions/hooks/useGetLogicFunctionHttpUrl';
import { useFrontComponentExecutionContext } from '@/front-components/hooks/useFrontComponentExecutionContext';
import { useOnFrontComponentUpdated } from '@/front-components/hooks/useOnFrontComponentUpdated';
import { frontComponentApplicationTokenPairComponentState } from '@/front-components/states/frontComponentApplicationTokenPairComponentState';
import { getFrontComponentUrl } from '@/front-components/utils/getFrontComponentUrl';
import { useSnackBar } from '@/ui/feedback/snack-bar-manager/hooks/useSnackBar';
import { useSetAtomComponentState } from '@/ui/utilities/state/jotai/hooks/useSetAtomComponentState';
import { useQuery } from '@apollo/client/react';
import { t } from '@lingui/core/macro';
import { useCallback, useContext, useEffect, useMemo } from 'react';
import { useCallback, useContext, useMemo } from 'react';
import { FrontComponentRenderer as SharedFrontComponentRenderer } from 'twenty-front-component-renderer';
import { isDefined } from 'twenty-shared/utils';
import { ThemeContext } from 'twenty-ui/theme-constants';
import { REACT_APP_SERVER_BASE_URL } from '~/config';
import { useQuery } from '@apollo/client/react';
import { FindOneFrontComponentDocument } from '~/generated-metadata/graphql';
import {
FindOneFrontComponentDocument,
type FindOneFrontComponentQuery,
GetApplicationSdkClientChecksumsDocument,
} from '~/generated-metadata/graphql';
type FrontComponentRendererProps = {
frontComponentId: string;
@@ -22,20 +27,55 @@ type FrontComponentRendererProps = {
selectedRecordIds?: string[];
};
type ResolvedFrontComponent = NonNullable<
FindOneFrontComponentQuery['frontComponent']
>;
type FrontComponentRendererContentProps = {
frontComponent: ResolvedFrontComponent;
commandMenuItemId?: string;
selectedRecordIds?: string[];
};
export const FrontComponentRenderer = ({
frontComponentId,
commandMenuItemId,
selectedRecordIds,
}: FrontComponentRendererProps) => {
const { data, loading, error } = useQuery(FindOneFrontComponentDocument, {
variables: { id: frontComponentId },
});
useOnFrontComponentUpdated({
frontComponentId,
});
const frontComponent = data?.frontComponent;
return (
<>
<FrontComponentLoadErrorSnackBarEffect errorMessage={error?.message} />
{!loading && isDefined(frontComponent) && (
<FrontComponentRendererContent
frontComponent={frontComponent}
commandMenuItemId={commandMenuItemId}
selectedRecordIds={selectedRecordIds}
/>
)}
</>
);
};
const FrontComponentRendererContent = ({
frontComponent,
commandMenuItemId,
selectedRecordIds,
}: FrontComponentRendererContentProps) => {
const { colorScheme } = useContext(ThemeContext);
const { enqueueErrorSnackBar } = useSnackBar();
const { functionsBaseUrl } = useGetLogicFunctionHttpUrl();
const setFrontComponentApplicationTokenPair = useSetAtomComponentState(
frontComponentApplicationTokenPairComponentState,
frontComponentId,
);
const { id: frontComponentId, applicationId, usesSdkClient } = frontComponent;
const { executionContext, frontComponentHostCommunicationApi } =
useFrontComponentExecutionContext({
@@ -51,78 +91,69 @@ export const FrontComponentRenderer = ({
return;
}
const errorMessage = error.message;
enqueueErrorSnackBar({
message: t`Failed to load front component: ${errorMessage}`,
message: t`Failed to load front component: ${error.message}`,
});
},
[enqueueErrorSnackBar],
);
const { data, loading, error } = useQuery(FindOneFrontComponentDocument, {
variables: { id: frontComponentId },
const applicationTokenPair = frontComponent.applicationTokenPair ?? null;
const { data: sdkClientChecksumsData, loading: sdkClientChecksumsLoading } =
useQuery(GetApplicationSdkClientChecksumsDocument, {
variables: { applicationId },
skip: !usesSdkClient,
});
useOnApplicationSdkClientChecksumsUpdated({
applicationId,
skip: !usesSdkClient,
});
useEffect(() => {
if (error) {
handleError(error);
}
}, [error, handleError]);
const applicationTokenPair =
data?.frontComponent?.applicationTokenPair ?? null;
useEffect(() => {
if (isDefined(applicationTokenPair)) {
setFrontComponentApplicationTokenPair(applicationTokenPair);
}
}, [applicationTokenPair, setFrontComponentApplicationTokenPair]);
useOnFrontComponentUpdated({
frontComponentId,
});
const applicationId = data?.frontComponent?.applicationId;
const sdkClientChecksums =
sdkClientChecksumsData?.applicationSdkClientChecksums;
const sdkClientUrls = useMemo(
() =>
isDefined(applicationId) ? getSdkClientUrls(applicationId) : undefined,
[applicationId],
() => getSdkClientUrls(applicationId, sdkClientChecksums),
[applicationId, sdkClientChecksums],
);
if (
loading ||
!isDefined(data?.frontComponent) ||
!isDefined(applicationTokenPair)
) {
return null;
}
const componentUrl = getFrontComponentUrl({
frontComponentId,
checksum: data.frontComponent.builtComponentChecksum,
checksum: frontComponent.builtComponentChecksum,
});
const accessToken = applicationTokenPair.applicationAccessToken.token;
const applicationVariables = frontComponent.applicationVariables ?? undefined;
const applicationVariables =
data.frontComponent.applicationVariables ?? undefined;
const isSdkClientReady = !usesSdkClient || !sdkClientChecksumsLoading;
return (
<FrontComponentRendererProvider frontComponentId={frontComponentId}>
<SharedFrontComponentRenderer
colorScheme={colorScheme}
componentUrl={componentUrl}
applicationAccessToken={accessToken}
apiUrl={REACT_APP_SERVER_BASE_URL}
functionsBaseUrl={functionsBaseUrl}
sdkClientUrls={sdkClientUrls}
executionContext={executionContext}
frontComponentHostCommunicationApi={frontComponentHostCommunicationApi}
applicationVariables={applicationVariables}
onError={handleError}
<>
<FrontComponentApplicationTokenPairEffect
frontComponentId={frontComponentId}
applicationTokenPair={applicationTokenPair}
/>
</FrontComponentRendererProvider>
{isDefined(applicationTokenPair) && isSdkClientReady && (
<FrontComponentRendererProvider frontComponentId={frontComponentId}>
<SharedFrontComponentRenderer
colorScheme={colorScheme}
componentUrl={componentUrl}
applicationAccessToken={
applicationTokenPair.applicationAccessToken.token
}
apiUrl={REACT_APP_SERVER_BASE_URL}
functionsBaseUrl={functionsBaseUrl}
sdkClientUrls={sdkClientUrls}
executionContext={executionContext}
frontComponentHostCommunicationApi={
frontComponentHostCommunicationApi
}
applicationVariables={applicationVariables}
onError={handleError}
/>
</FrontComponentRendererProvider>
)}
</>
);
};
@@ -0,0 +1,10 @@
import gql from 'graphql-tag';
export const GET_APPLICATION_SDK_CLIENT_CHECKSUMS = gql`
query GetApplicationSdkClientChecksums($applicationId: UUID!) {
applicationSdkClientChecksums(applicationId: $applicationId) {
core
metadata
}
}
`;
@@ -0,0 +1,194 @@
import { renderHook } from '@testing-library/react';
import { type MetadataOperationBrowserEventDetail } from '@/browser-event/types/MetadataOperationBrowserEventDetail';
import { useUpdateSdkClientChecksumsApolloCache } from '@/front-components/hooks/useUpdateSdkClientChecksumsApolloCache';
import { type ApplicationSdkClientChecksumsBroadcastRecord } from '@/front-components/types/ApplicationSdkClientChecksumsBroadcastRecord';
const mockReadQuery = jest.fn();
const mockUpdateQuery = jest.fn();
const mockQuery = jest.fn().mockResolvedValue({ data: undefined });
const mockApolloClient = {
cache: { readQuery: mockReadQuery, updateQuery: mockUpdateQuery },
query: mockQuery,
};
jest.mock('@apollo/client/react', () => ({
...jest.requireActual('@apollo/client/react'),
useApolloClient: () => mockApolloClient,
}));
const APPLICATION_ID = 'app-test-id';
const CACHED_CHECKSUM_PAIR = {
applicationSdkClientChecksums: {
__typename: 'SdkClientChecksums' as const,
core: 'old-core-checksum',
metadata: 'old-metadata-checksum',
},
};
const buildApplicationRecord = (
overrides: Partial<ApplicationSdkClientChecksumsBroadcastRecord> = {},
): ApplicationSdkClientChecksumsBroadcastRecord => ({
id: APPLICATION_ID,
sdkClientCoreChecksum: 'a'.repeat(64),
...overrides,
});
const buildUpdateDetail = (
updatedRecord: ApplicationSdkClientChecksumsBroadcastRecord,
): MetadataOperationBrowserEventDetail<ApplicationSdkClientChecksumsBroadcastRecord> => ({
metadataName: 'application',
operation: {
type: 'update',
updatedRecord,
},
});
describe('useUpdateSdkClientChecksumsApolloCache', () => {
beforeEach(() => {
jest.clearAllMocks();
mockReadQuery.mockReturnValue(CACHED_CHECKSUM_PAIR);
});
it('should call cache.updateQuery when the application id matches', () => {
const { result } = renderHook(() =>
useUpdateSdkClientChecksumsApolloCache({
applicationId: APPLICATION_ID,
}),
);
result.current.updateSdkClientChecksumsApolloCache(
buildUpdateDetail(buildApplicationRecord()),
);
expect(mockUpdateQuery).toHaveBeenCalledTimes(1);
expect(mockQuery).not.toHaveBeenCalled();
});
it('should not call cache.updateQuery when the application id does not match', () => {
const { result } = renderHook(() =>
useUpdateSdkClientChecksumsApolloCache({
applicationId: APPLICATION_ID,
}),
);
result.current.updateSdkClientChecksumsApolloCache(
buildUpdateDetail(buildApplicationRecord({ id: 'other-app-id' })),
);
expect(mockUpdateQuery).not.toHaveBeenCalled();
expect(mockQuery).not.toHaveBeenCalled();
});
it('should not call cache.updateQuery when applicationId is undefined', () => {
const { result } = renderHook(() =>
useUpdateSdkClientChecksumsApolloCache({}),
);
result.current.updateSdkClientChecksumsApolloCache(
buildUpdateDetail(buildApplicationRecord()),
);
expect(mockUpdateQuery).not.toHaveBeenCalled();
expect(mockQuery).not.toHaveBeenCalled();
});
it('should not call cache.updateQuery when checksums are missing', () => {
const { result } = renderHook(() =>
useUpdateSdkClientChecksumsApolloCache({
applicationId: APPLICATION_ID,
}),
);
result.current.updateSdkClientChecksumsApolloCache(
buildUpdateDetail(
buildApplicationRecord({ sdkClientCoreChecksum: null }),
),
);
expect(mockUpdateQuery).not.toHaveBeenCalled();
expect(mockQuery).not.toHaveBeenCalled();
});
it('should not call cache.updateQuery for create operations', () => {
const { result } = renderHook(() =>
useUpdateSdkClientChecksumsApolloCache({
applicationId: APPLICATION_ID,
}),
);
result.current.updateSdkClientChecksumsApolloCache({
metadataName: 'application',
operation: {
type: 'create',
createdRecord: buildApplicationRecord(),
},
});
expect(mockUpdateQuery).not.toHaveBeenCalled();
expect(mockQuery).not.toHaveBeenCalled();
});
it('should update only the core checksum and preserve the cached metadata checksum', () => {
const { result } = renderHook(() =>
useUpdateSdkClientChecksumsApolloCache({
applicationId: APPLICATION_ID,
}),
);
result.current.updateSdkClientChecksumsApolloCache(
buildUpdateDetail(buildApplicationRecord()),
);
const updaterFn = mockUpdateQuery.mock.calls[0][1];
const updatedData = updaterFn(CACHED_CHECKSUM_PAIR);
expect(updatedData.applicationSdkClientChecksums).toEqual({
__typename: 'SdkClientChecksums',
core: 'a'.repeat(64),
metadata: 'old-metadata-checksum',
});
});
it('should refetch the checksum query when no checksum pair is cached yet', () => {
mockReadQuery.mockReturnValue({ applicationSdkClientChecksums: null });
const { result } = renderHook(() =>
useUpdateSdkClientChecksumsApolloCache({
applicationId: APPLICATION_ID,
}),
);
result.current.updateSdkClientChecksumsApolloCache(
buildUpdateDetail(buildApplicationRecord()),
);
expect(mockUpdateQuery).not.toHaveBeenCalled();
expect(mockQuery).toHaveBeenCalledTimes(1);
expect(mockQuery).toHaveBeenCalledWith(
expect.objectContaining({
variables: { applicationId: APPLICATION_ID },
fetchPolicy: 'network-only',
}),
);
});
it('should refetch the checksum query when the query result is absent from the cache', () => {
mockReadQuery.mockReturnValue(null);
const { result } = renderHook(() =>
useUpdateSdkClientChecksumsApolloCache({
applicationId: APPLICATION_ID,
}),
);
result.current.updateSdkClientChecksumsApolloCache(
buildUpdateDetail(buildApplicationRecord()),
);
expect(mockUpdateQuery).not.toHaveBeenCalled();
expect(mockQuery).toHaveBeenCalledTimes(1);
});
});
@@ -0,0 +1,26 @@
import { useListenToMetadataOperationBrowserEvent } from '@/browser-event/hooks/useListenToMetadataOperationBrowserEvent';
import { useUpdateSdkClientChecksumsApolloCache } from '@/front-components/hooks/useUpdateSdkClientChecksumsApolloCache';
import { type ApplicationSdkClientChecksumsBroadcastRecord } from '@/front-components/types/ApplicationSdkClientChecksumsBroadcastRecord';
type UseOnApplicationSdkClientChecksumsUpdatedArgs = {
applicationId?: string;
skip?: boolean;
};
export const useOnApplicationSdkClientChecksumsUpdated = ({
applicationId,
skip = false,
}: UseOnApplicationSdkClientChecksumsUpdatedArgs) => {
const { updateSdkClientChecksumsApolloCache } =
useUpdateSdkClientChecksumsApolloCache({
applicationId,
});
useListenToMetadataOperationBrowserEvent<ApplicationSdkClientChecksumsBroadcastRecord>(
{
metadataName: 'application',
onMetadataOperationBrowserEvent: updateSdkClientChecksumsApolloCache,
skip,
},
);
};
@@ -0,0 +1,76 @@
import { type MetadataOperationBrowserEventDetail } from '@/browser-event/types/MetadataOperationBrowserEventDetail';
import { type ApplicationSdkClientChecksumsBroadcastRecord } from '@/front-components/types/ApplicationSdkClientChecksumsBroadcastRecord';
import { useApolloClient } from '@apollo/client/react';
import { useCallback } from 'react';
import { isDefined } from 'twenty-shared/utils';
import {
GetApplicationSdkClientChecksumsDocument,
type GetApplicationSdkClientChecksumsQuery,
} from '~/generated-metadata/graphql';
type UseUpdateSdkClientChecksumsApolloCacheArgs = {
applicationId?: string;
};
export const useUpdateSdkClientChecksumsApolloCache = ({
applicationId,
}: UseUpdateSdkClientChecksumsApolloCacheArgs) => {
const apolloClient = useApolloClient();
const updateSdkClientChecksumsApolloCache = useCallback(
(
detail: MetadataOperationBrowserEventDetail<ApplicationSdkClientChecksumsBroadcastRecord>,
) => {
if (detail.operation.type !== 'update') {
return;
}
const { updatedRecord } = detail.operation;
if (!isDefined(applicationId) || updatedRecord.id !== applicationId) {
return;
}
const { sdkClientCoreChecksum } = updatedRecord;
if (!isDefined(sdkClientCoreChecksum)) {
return;
}
const cachedData =
apolloClient.cache.readQuery<GetApplicationSdkClientChecksumsQuery>({
query: GetApplicationSdkClientChecksumsDocument,
variables: { applicationId },
});
const existingChecksums = cachedData?.applicationSdkClientChecksums;
if (!isDefined(existingChecksums)) {
void apolloClient.query({
query: GetApplicationSdkClientChecksumsDocument,
variables: { applicationId },
fetchPolicy: 'network-only',
});
return;
}
apolloClient.cache.updateQuery<GetApplicationSdkClientChecksumsQuery>(
{
query: GetApplicationSdkClientChecksumsDocument,
variables: { applicationId },
},
(existingData) => ({
...existingData,
applicationSdkClientChecksums: {
...existingChecksums,
core: sdkClientCoreChecksum,
},
}),
);
},
[apolloClient, applicationId],
);
return { updateSdkClientChecksumsApolloCache };
};
@@ -0,0 +1,4 @@
export type ApplicationSdkClientChecksumsBroadcastRecord = {
id: string;
sdkClientCoreChecksum?: string | null;
};
@@ -2,10 +2,34 @@ import { REST_API_BASE_URL } from '@/apollo/constant/rest-api-base-url';
import { getSdkClientUrls } from '@/front-components/utils/getSdkClientUrls';
describe('getSdkClientUrls', () => {
it('builds application-scoped core and metadata sdk client urls', () => {
it('builds bare urls when checksums are absent', () => {
expect(getSdkClientUrls('application-id')).toEqual({
core: `${REST_API_BASE_URL}/sdk-client/application-id/core`,
metadata: `${REST_API_BASE_URL}/sdk-client/application-id/metadata`,
metadata: `${REST_API_BASE_URL}/sdk-client/metadata`,
});
});
it('builds content-addressed urls when checksums are provided', () => {
expect(
getSdkClientUrls('application-id', {
core: 'core-checksum',
metadata: 'metadata-checksum',
}),
).toEqual({
core: `${REST_API_BASE_URL}/sdk-client/application-id/core/core-checksum`,
metadata: `${REST_API_BASE_URL}/sdk-client/metadata/metadata-checksum`,
});
});
it('falls back to the bare core url when only the metadata checksum is available', () => {
expect(
getSdkClientUrls('application-id', {
core: null,
metadata: 'metadata-checksum',
}),
).toEqual({
core: `${REST_API_BASE_URL}/sdk-client/application-id/core`,
metadata: `${REST_API_BASE_URL}/sdk-client/metadata/metadata-checksum`,
});
});
});
@@ -1,6 +1,23 @@
import { REST_API_BASE_URL } from '@/apollo/constant/rest-api-base-url';
import { isDefined } from 'twenty-shared/utils';
import { type SdkClientChecksums } from '~/generated-metadata/graphql';
export const getSdkClientUrls = (applicationId: string) => ({
core: `${REST_API_BASE_URL}/sdk-client/${applicationId}/core`,
metadata: `${REST_API_BASE_URL}/sdk-client/${applicationId}/metadata`,
});
export const getSdkClientUrls = (
applicationId: string,
checksums?: Pick<SdkClientChecksums, 'core' | 'metadata'> | null,
) => {
const applicationBaseUrl = `${REST_API_BASE_URL}/sdk-client/${applicationId}`;
const metadataBaseUrl = `${REST_API_BASE_URL}/sdk-client/metadata`;
const coreChecksum = checksums?.core;
const metadataChecksum = checksums?.metadata;
return {
core: isDefined(coreChecksum)
? `${applicationBaseUrl}/core/${coreChecksum}`
: `${applicationBaseUrl}/core`,
metadata: isDefined(metadataChecksum)
? `${metadataBaseUrl}/${metadataChecksum}`
: metadataBaseUrl,
};
};
@@ -0,0 +1,21 @@
import { QueryRunner } from 'typeorm';
import { RegisteredInstanceCommand } from 'src/engine/core-modules/upgrade/decorators/registered-instance-command.decorator';
import { FastInstanceCommand } from 'src/engine/core-modules/upgrade/interfaces/fast-instance-command.interface';
@RegisteredInstanceCommand('2.23.0', 1784625638000)
export class AddSdkClientCoreChecksumToApplicationFastInstanceCommand
implements FastInstanceCommand
{
public async up(queryRunner: QueryRunner): Promise<void> {
await queryRunner.query(
'ALTER TABLE "core"."application" ADD COLUMN IF NOT EXISTS "sdkClientCoreChecksum" text',
);
}
public async down(queryRunner: QueryRunner): Promise<void> {
await queryRunner.query(
'ALTER TABLE "core"."application" DROP COLUMN IF EXISTS "sdkClientCoreChecksum"',
);
}
}
@@ -43,11 +43,16 @@ import { AddTsVectorFieldMetadataIdToSearchFieldMetadataFastInstanceCommand } fr
import { BackfillTsVectorFieldMetadataIdOnSearchFieldMetadataSlowInstanceCommand } from 'src/database/commands/upgrade-version-command/2-18/2-18-instance-command-slow-1810000003000-backfill-ts-vector-field-metadata-id-on-search-field-metadata';
import { AddMetadataOverridesColumnFastInstanceCommand } from 'src/database/commands/upgrade-version-command/2-19/2-19-instance-command-fast-1782986475000-add-metadata-overrides-column';
import { AddLastStreamErrorToAgentChatThreadFastInstanceCommand } from 'src/database/commands/upgrade-version-command/2-19/2-19-instance-command-fast-1782996657000-add-last-stream-error-to-agent-chat-thread';
import { AddPendingQuestionMessageIdToAgentChatThreadFastInstanceCommand } from 'src/database/commands/upgrade-version-command/2-19/2-19-instance-command-fast-1782999138000-add-pending-question-to-agent-chat-thread';
import { AddLogoFileIdToApplicationFastInstanceCommand } from 'src/database/commands/upgrade-version-command/2-19/2-19-instance-command-fast-1783062755137-add-logo-file-id-to-application';
import { BackfillMetadataOverridesSlowInstanceCommand } from 'src/database/commands/upgrade-version-command/2-19/2-19-instance-command-slow-1782986476000-backfill-metadata-overrides';
import { AddTypeAndOptionsToApplicationVariablesFastInstanceCommand } from 'src/database/commands/upgrade-version-command/2-19/2-19-instance-command-fast-1783065514000-add-type-and-options-to-application-variables';
import { BackfillMetadataOverridesSlowInstanceCommand } from 'src/database/commands/upgrade-version-command/2-19/2-19-instance-command-slow-1782986476000-backfill-metadata-overrides';
import { AddCacheTokensToAgentChatThreadFastInstanceCommand } from 'src/database/commands/upgrade-version-command/2-2/2-2-instance-command-fast-1777455269302-add-cache-tokens-to-agent-chat-thread';
import { AddLogoToApplicationFastInstanceCommand } from 'src/database/commands/upgrade-version-command/2-2/2-2-instance-command-fast-1777539664664-add-logo-to-application';
import { AllowServerScopedFileFastInstanceCommand } from 'src/database/commands/upgrade-version-command/2-20/2-20-instance-command-fast-1783499671541-allow-server-scoped-file';
import { AddCalendarEndFieldMetadataIdToViewFastInstanceCommand } from 'src/database/commands/upgrade-version-command/2-22/2-22-instance-command-fast-1783956795000-add-calendar-end-field-metadata-id-to-view';
import { AddSdkClientCoreChecksumToApplicationFastInstanceCommand } from 'src/database/commands/upgrade-version-command/2-23/2-23-instance-command-fast-1784625638000-add-sdk-client-core-checksum-to-application';
import { AddAutoUpgradeToApplicationFastInstanceCommand } from 'src/database/commands/upgrade-version-command/2-23/2-23-instance-command-fast-1784297307235-add-auto-upgrade-to-application';
import { AddSubFieldNameToViewSortEarlyFastInstanceCommand } from 'src/database/commands/upgrade-version-command/2-3/2-3-instance-command-fast-1747234200000-add-sub-field-name-to-view-sort';
import { AddRelationTargetFieldMetadataIdToViewFilterEarlyFastInstanceCommand } from 'src/database/commands/upgrade-version-command/2-3/2-3-instance-command-fast-1747234300000-add-relation-target-field-metadata-id-to-view-filter';
import { AddUpgradeMigrationWorkspaceIdIndexFastInstanceCommand } from 'src/database/commands/upgrade-version-command/2-3/2-3-instance-command-fast-1777308014234-add-upgrade-migration-workspace-id-index';
@@ -89,34 +94,30 @@ import { EncryptNonSecretApplicationVariableSlowInstanceCommand } from 'src/data
import { MigrateAiModelPreferencesSlowInstanceCommand } from 'src/database/commands/upgrade-version-command/2-9/2-9-instance-command-slow-1799000010000-migrate-ai-model-preferences';
import { AddFolderImportToMessageFolderPendingSyncActionFastInstanceCommand } from './2-15/2-15-instance-command-fast-1781714499016-add-folder-import-to-message-folder-pending-sync-action';
import { AddViewKanbanColumnWidthFastInstanceCommand } from './2-15/2-15-instance-command-fast-1781900000000-add-view-kanban-column-width';
import { AddPendingQuestionMessageIdToAgentChatThreadFastInstanceCommand } from 'src/database/commands/upgrade-version-command/2-19/2-19-instance-command-fast-1782999138000-add-pending-question-to-agent-chat-thread';
import { AddWorkspaceDiscoverabilityToWorkspaceFastInstanceCommand } from './2-19/2-19-instance-command-fast-1783004140000-add-workspace-discoverability-to-workspace';
import { AddLogoToApplicationRegistrationFastInstanceCommand } from './2-19/2-19-instance-command-fast-1783069672191-add-logo-to-application-registration';
import { AddDisplayFieldsToApplicationRegistrationFastInstanceCommand } from './2-19/2-19-instance-command-fast-1783073776590-add-display-fields-to-application-registration';
import { AddStatusToFileFastInstanceCommand } from './2-19/2-19-instance-command-fast-1783082964705-add-status-to-file';
import { AddPendingMimeCheckToFileFastInstanceCommand } from './2-19/2-19-instance-command-fast-1783094691548-add-pending-mime-check-to-file';
import { DropMetadataStandardOverridesColumnFastInstanceCommand } from './2-20/2-20-instance-command-fast-1783511477234-drop-metadata-standard-overrides-column';
import { AddLogoToApplicationRegistrationFastInstanceCommand } from './2-19/2-19-instance-command-fast-1783069672191-add-logo-to-application-registration';
import { BackfillLogoOnApplicationRegistrationSlowInstanceCommand } from './2-19/2-19-instance-command-slow-1783069673191-backfill-logo-on-application-registration';
import { AddDisplayFieldsToApplicationRegistrationFastInstanceCommand } from './2-19/2-19-instance-command-fast-1783073776590-add-display-fields-to-application-registration';
import { BackfillDisplayFieldsOnApplicationRegistrationSlowInstanceCommand } from './2-19/2-19-instance-command-slow-1783073776591-backfill-display-fields-on-application-registration';
import { BackfillIsFeaturedOnApplicationRegistrationSlowInstanceCommand } from './2-19/2-19-instance-command-slow-1783120000000-backfill-is-featured-on-application-registration';
import { AllowServerScopedFileFastInstanceCommand } from 'src/database/commands/upgrade-version-command/2-20/2-20-instance-command-fast-1783499671541-allow-server-scoped-file';
import { DropMetadataStandardOverridesColumnFastInstanceCommand } from './2-20/2-20-instance-command-fast-1783511477234-drop-metadata-standard-overrides-column';
import { CreateWorkflowVersionCoreTableFastInstanceCommand } from './2-20/2-20-instance-command-fast-1783512000000-create-workflow-version-core-table';
import { BackfillNameFieldIsSystemSideEffectSlowInstanceCommand } from './2-20/2-20-instance-command-slow-1783529458168-backfill-name-field-is-system-side-effect';
import { RenameIsFeaturedToIsVettedOnApplicationRegistrationFastInstanceCommand } from './2-20/2-20-instance-command-fast-1783527064000-rename-is-featured-to-is-vetted-on-application-registration';
import { AddIsSystemSideEffectToSearchFieldMetadataFastInstanceCommand } from './2-20/2-20-instance-command-fast-1783580127637-add-is-system-side-effect-to-search-field-metadata';
import { CreateWorkflowCoreTableFastInstanceCommand } from './2-20/2-20-instance-command-fast-1783603454479-create-workflow-core-table';
import { AddGalleryImagesToApplicationRegistrationFastInstanceCommand } from './2-20/2-20-instance-command-fast-1783615890055-add-gallery-images-to-application-registration';
import { BackfillGalleryImagesOnApplicationRegistrationSlowInstanceCommand } from './2-20/2-20-instance-command-slow-1783615890056-backfill-gallery-images-on-application-registration';
import { AddWorkflowVersionSyncableColumnsFastInstanceCommand } from './2-20/2-20-instance-command-fast-1783603454480-add-workflow-version-syncable-columns';
import { BackfillWorkspaceDatabaseSchemaSlowInstanceCommand } from './2-21/2-21-instance-command-slow-1783934147089-backfill-workspace-database-schema';
import { AddGalleryImagesToApplicationRegistrationFastInstanceCommand } from './2-20/2-20-instance-command-fast-1783615890055-add-gallery-images-to-application-registration';
import { BackfillNameFieldIsSystemSideEffectSlowInstanceCommand } from './2-20/2-20-instance-command-slow-1783529458168-backfill-name-field-is-system-side-effect';
import { BackfillGalleryImagesOnApplicationRegistrationSlowInstanceCommand } from './2-20/2-20-instance-command-slow-1783615890056-backfill-gallery-images-on-application-registration';
import { AddLogoFileIdToApplicationRegistrationFastInstanceCommand } from './2-21/2-21-instance-command-fast-1783945979243-add-logo-file-id-to-application-registration';
import { AddCalendarEndFieldMetadataIdToViewFastInstanceCommand } from 'src/database/commands/upgrade-version-command/2-22/2-22-instance-command-fast-1783956795000-add-calendar-end-field-metadata-id-to-view';
import { BackfillWorkspaceDatabaseSchemaSlowInstanceCommand } from './2-21/2-21-instance-command-slow-1783934147089-backfill-workspace-database-schema';
import { AddCreatedWorkspaceActivationStatusSlowInstanceCommand } from './2-22/2-22-instance-command-slow-1784106205000-add-created-workspace-activation-status';
import { UnlistUnclaimedNpmApplicationRegistrationsSlowInstanceCommand } from './2-23/2-23-instance-command-slow-1784322591746-unlist-unclaimed-npm-application-registrations';
import { BackfillCreatedWorkspaceActivationStatusSlowInstanceCommand } from './2-23/2-23-instance-command-slow-1784286705000-backfill-created-workspace-activation-status';
import { AddKanbanAndCalendarWidgetViewTypesFastInstanceCommand } from './2-23/2-23-instance-command-fast-1784620626405-add-kanban-and-calendar-widget-view-types';
import { WidenViewCalendarIntegrityConstraintFastInstanceCommand } from './2-23/2-23-instance-command-fast-1784620626406-widen-view-calendar-integrity-constraint';
import { AddAutoUpgradeToApplicationFastInstanceCommand } from 'src/database/commands/upgrade-version-command/2-23/2-23-instance-command-fast-1784297307235-add-auto-upgrade-to-application';
import { BackfillCreatedWorkspaceActivationStatusSlowInstanceCommand } from './2-23/2-23-instance-command-slow-1784286705000-backfill-created-workspace-activation-status';
import { UnlistUnclaimedNpmApplicationRegistrationsSlowInstanceCommand } from './2-23/2-23-instance-command-slow-1784322591746-unlist-unclaimed-npm-application-registrations';
export const INSTANCE_COMMANDS = [
AddViewFieldGroupIdIndexOnViewFieldFastInstanceCommand,
@@ -236,4 +237,5 @@ export const INSTANCE_COMMANDS = [
AddKanbanAndCalendarWidgetViewTypesFastInstanceCommand,
WidenViewCalendarIntegrityConstraintFastInstanceCommand,
AddAutoUpgradeToApplicationFastInstanceCommand,
AddSdkClientCoreChecksumToApplicationFastInstanceCommand,
];
@@ -92,6 +92,7 @@ describe('ApplicationManifestApplyService', () => {
workspaceId: WORKSPACE_ID,
applicationId: APPLICATION_ID,
applicationUniversalIdentifier: 'my-app',
trigger: 'manifest-sync',
});
});
@@ -51,6 +51,7 @@ export class ApplicationManifestApplyService {
workspaceId,
applicationId: application.id,
applicationUniversalIdentifier: application.universalIdentifier,
trigger: 'manifest-sync',
});
}
@@ -163,6 +163,7 @@ export class ApplicationSyncService {
canBeUninstalled: true,
autoUpgrade: false,
isSdkLayerStale: false,
sdkClientCoreChecksum: null,
applicationRegistrationId: null,
primaryPublicDomainId: null,
createdAt: now,
@@ -134,6 +134,13 @@ export class ApplicationEntity extends WorkspaceRelatedEntity {
@Column({ nullable: false, type: 'boolean', default: false })
isSdkLayerStale: boolean;
@Column({ nullable: true, type: 'text' })
@WasIntroducedInUpgrade({
upgradeCommandName:
'2.23.0_AddSdkClientCoreChecksumToApplicationFastInstanceCommand_1784625638000',
})
sdkClientCoreChecksum: string | null;
@Column({ nullable: true, type: 'uuid' })
applicationRegistrationId: string | null;
@@ -1,20 +1,50 @@
import { UseGuards } from '@nestjs/common';
import { Parent, ResolveField } from '@nestjs/graphql';
import { Args, Parent, Query, ResolveField } from '@nestjs/graphql';
import { isAbsoluteUrl, isDefined } from 'twenty-shared/utils';
import { MetadataResolver } from 'src/engine/api/graphql/graphql-config/decorators/metadata-resolver.decorator';
import { UUIDScalarType } from 'src/engine/api/graphql/workspace-schema-builder/graphql-types/scalars';
import { ApplicationDTO } from 'src/engine/core-modules/application/dtos/application.dto';
import { SdkClientChecksumsDTO } from 'src/engine/core-modules/sdk-client/dtos/sdk-client-checksums.dto';
import { getInstalledSdkMetadataModule } from 'src/engine/core-modules/sdk-client/utils/get-installed-sdk-metadata-module.util';
import { TwentyConfigService } from 'src/engine/core-modules/twenty-config/twenty-config.service';
import { WorkspaceEntity } from 'src/engine/core-modules/workspace/workspace.entity';
import { AuthWorkspace } from 'src/engine/decorators/auth/auth-workspace.decorator';
import { NoPermissionGuard } from 'src/engine/guards/no-permission.guard';
import { WorkspaceAuthGuard } from 'src/engine/guards/workspace-auth.guard';
import { WorkspaceCacheService } from 'src/engine/workspace-cache/services/workspace-cache.service';
@UseGuards(WorkspaceAuthGuard, NoPermissionGuard)
@MetadataResolver(() => ApplicationDTO)
export class ApplicationResolver {
constructor(private readonly twentyConfigService: TwentyConfigService) {}
constructor(
private readonly twentyConfigService: TwentyConfigService,
private readonly workspaceCacheService: WorkspaceCacheService,
) {}
@Query(() => SdkClientChecksumsDTO, { nullable: true })
async applicationSdkClientChecksums(
@Args('applicationId', { type: () => UUIDScalarType })
applicationId: string,
@AuthWorkspace() workspace: WorkspaceEntity,
): Promise<SdkClientChecksumsDTO | null> {
const { flatApplicationMaps } =
await this.workspaceCacheService.getOrRecompute(workspace.id, [
'flatApplicationMaps',
]);
const application = flatApplicationMaps.byId[applicationId];
if (!isDefined(application)) {
return null;
}
return {
core: application.sdkClientCoreChecksum,
metadata: (await getInstalledSdkMetadataModule()).checksum,
};
}
// Resolves the display url of the logo bundled in the installed
// application's public assets, so clients never build file urls themselves.
@@ -65,6 +65,9 @@ export enum MetricsKeys {
AiChatTurnCompleted = 'ai-chat/turn-completed',
AiChatTurnFailed = 'ai-chat/turn-failed',
WorkspaceMetadataCacheLocalEviction = 'workspace-metadata-cache/local-eviction',
SdkClientGenerationSucceeded = 'sdk-client-generation/succeeded',
SdkClientGenerationFailed = 'sdk-client-generation/failed',
SdkClientGenerationDurationMs = 'sdk-client-generation/duration-ms',
ConnectedAccountSyncWebhookReceivedMessaging = 'connected-account-sync-webhook/received/messaging',
ConnectedAccountSyncWebhookReceivedCalendar = 'connected-account-sync-webhook/received/calendar',
}
@@ -10,9 +10,11 @@ import { FileStorageService } from 'src/engine/core-modules/file-storage/service
import { getQueueToken } from 'src/engine/core-modules/message-queue/utils/get-queue-token.util';
import { MessageQueue } from 'src/engine/core-modules/message-queue/message-queue.constants';
import { MessageQueueService } from 'src/engine/core-modules/message-queue/services/message-queue.service';
import { MetricsService } from 'src/engine/core-modules/metrics/metrics.service';
import { GENERATE_SDK_CLIENT_JOB_NAME } from 'src/engine/core-modules/sdk-client/jobs/generate-sdk-client.job-constants';
import { SdkClientGenerationService } from 'src/engine/core-modules/sdk-client/sdk-client-generation.service';
import { WorkspaceEntity } from 'src/engine/core-modules/workspace/workspace.entity';
import { WorkspaceEventBroadcaster } from 'src/engine/subscriptions/workspace-event-broadcaster/workspace-event-broadcaster.service';
import { WorkspaceCacheService } from 'src/engine/workspace-cache/services/workspace-cache.service';
describe('SdkClientGenerationService', () => {
@@ -24,6 +26,9 @@ describe('SdkClientGenerationService', () => {
>
>;
let messageQueueService: jest.Mocked<Pick<MessageQueueService, 'add'>>;
let workspaceCacheService: jest.Mocked<
Pick<WorkspaceCacheService, 'getOrRecompute'>
>;
beforeEach(async () => {
applicationService = {
@@ -32,6 +37,9 @@ describe('SdkClientGenerationService', () => {
messageQueueService = {
add: jest.fn().mockResolvedValue(undefined),
};
workspaceCacheService = {
getOrRecompute: jest.fn(),
};
const module: TestingModule = await Test.createTestingModule({
providers: [
@@ -45,13 +53,24 @@ describe('SdkClientGenerationService', () => {
provide: getRepositoryToken(WorkspaceEntity),
useValue: {} as Repository<WorkspaceEntity>,
},
{ provide: WorkspaceCacheService, useValue: {} },
{ provide: WorkspaceCacheService, useValue: workspaceCacheService },
{ provide: WorkspaceSchemaFactory, useValue: {} },
{ provide: ApplicationService, useValue: applicationService },
{
provide: getQueueToken(MessageQueue.workspaceQueue),
useValue: messageQueueService,
},
{
provide: WorkspaceEventBroadcaster,
useValue: { broadcast: jest.fn().mockResolvedValue(undefined) },
},
{
provide: MetricsService,
useValue: {
incrementCounterBy: jest.fn(),
recordHistogram: jest.fn(),
},
},
],
}).compile();
@@ -88,6 +107,7 @@ describe('SdkClientGenerationService', () => {
workspaceId,
applicationId: 'std-app-id',
applicationUniversalIdentifier: 'twenty-standard',
trigger: 'workspace-activation',
},
{
id: `sdk-client:${workspaceId}:std-app-id`,
@@ -101,6 +121,7 @@ describe('SdkClientGenerationService', () => {
workspaceId,
applicationId: 'custom-app-id',
applicationUniversalIdentifier: 'workspace-custom',
trigger: 'workspace-activation',
},
{
id: `sdk-client:${workspaceId}:custom-app-id`,
@@ -0,0 +1,257 @@
import { NotFoundException } from '@nestjs/common';
import { Test, type TestingModule } from '@nestjs/testing';
import { type Response } from 'express';
import {
SDK_CLIENT_MODULE_CACHE_CONTROL,
SDK_CLIENT_MODULE_NO_STORE_CACHE_CONTROL,
} from 'src/engine/core-modules/sdk-client/constants/sdk-client-module-cache-control';
import { SdkClientController } from 'src/engine/core-modules/sdk-client/controllers/sdk-client.controller';
import { SdkClientArchiveService } from 'src/engine/core-modules/sdk-client/sdk-client-archive.service';
import { getInstalledSdkMetadataModule } from 'src/engine/core-modules/sdk-client/utils/get-installed-sdk-metadata-module.util';
import { type WorkspaceEntity } from 'src/engine/core-modules/workspace/workspace.entity';
import { WorkspaceCacheService } from 'src/engine/workspace-cache/services/workspace-cache.service';
jest.mock(
'src/engine/core-modules/sdk-client/utils/get-installed-sdk-metadata-module.util',
() => ({
getInstalledSdkMetadataModule: jest.fn(),
}),
);
const mockGetInstalledSdkMetadataModule = jest.mocked(
getInstalledSdkMetadataModule,
);
const WORKSPACE_ID = 'workspace-1';
const APPLICATION_ID = 'app-1';
const CORE_MODULE_BUFFER = Buffer.from('core module from archive');
const PERSISTED_CORE_CHECKSUM = 'b'.repeat(64);
const workspace = { id: WORKSPACE_ID } as WorkspaceEntity;
describe('SdkClientController', () => {
let controller: SdkClientController;
let workspaceCacheService: jest.Mocked<
Pick<WorkspaceCacheService, 'getOrRecompute'>
>;
let sdkClientArchiveService: jest.Mocked<
Pick<SdkClientArchiveService, 'getClientModuleFromArchive'>
>;
let response: jest.Mocked<Pick<Response, 'setHeader' | 'send'>>;
beforeEach(async () => {
jest.clearAllMocks();
workspaceCacheService = {
getOrRecompute: jest.fn().mockResolvedValue({
flatApplicationMaps: {
byId: {
[APPLICATION_ID]: {
id: APPLICATION_ID,
universalIdentifier: 'my-app',
sdkClientCoreChecksum: PERSISTED_CORE_CHECKSUM,
},
},
},
}),
};
sdkClientArchiveService = {
getClientModuleFromArchive: jest
.fn()
.mockResolvedValue(CORE_MODULE_BUFFER),
};
response = {
setHeader: jest.fn(),
send: jest.fn(),
};
const module: TestingModule = await Test.createTestingModule({
controllers: [SdkClientController],
providers: [
{ provide: WorkspaceCacheService, useValue: workspaceCacheService },
{
provide: SdkClientArchiveService,
useValue: sdkClientArchiveService,
},
],
}).compile();
controller = module.get<SdkClientController>(SdkClientController);
});
describe('instance-wide metadata route', () => {
const INSTALLED_METADATA_BUFFER = Buffer.from('installed metadata module');
const INSTALLED_METADATA_CHECKSUM = 'a'.repeat(64);
beforeEach(() => {
mockGetInstalledSdkMetadataModule.mockResolvedValue({
moduleBuffer: INSTALLED_METADATA_BUFFER,
checksum: INSTALLED_METADATA_CHECKSUM,
});
});
it('serves the installed metadata module without touching workspace caches', async () => {
await controller.getInstanceSdkMetadataModule(
response as unknown as Response,
);
expect(workspaceCacheService.getOrRecompute).not.toHaveBeenCalled();
expect(
sdkClientArchiveService.getClientModuleFromArchive,
).not.toHaveBeenCalled();
expect(response.send).toHaveBeenCalledWith(INSTALLED_METADATA_BUFFER);
});
it('opts out of caching on the bare url', async () => {
await controller.getInstanceSdkMetadataModule(
response as unknown as Response,
);
expect(response.setHeader).toHaveBeenCalledWith(
'Cache-Control',
SDK_CLIENT_MODULE_NO_STORE_CACHE_CONTROL,
);
});
it('serves fingerprinted urls as immutable when the checksum matches the installed module', async () => {
await controller.getInstanceSdkMetadataModule(
response as unknown as Response,
INSTALLED_METADATA_CHECKSUM,
);
expect(response.setHeader).toHaveBeenCalledWith(
'Cache-Control',
SDK_CLIENT_MODULE_CACHE_CONTROL,
);
});
it('opts out of caching when the fingerprint does not match the installed module', async () => {
await controller.getInstanceSdkMetadataModule(
response as unknown as Response,
'c'.repeat(64),
);
expect(response.setHeader).toHaveBeenCalledWith(
'Cache-Control',
SDK_CLIENT_MODULE_NO_STORE_CACHE_CONTROL,
);
});
});
it('serves the metadata module from the installed package, not the archive', async () => {
const installedModuleBuffer = Buffer.from('installed metadata module');
mockGetInstalledSdkMetadataModule.mockResolvedValue({
moduleBuffer: installedModuleBuffer,
checksum: 'a'.repeat(64),
});
await controller.getSdkModule(
response as unknown as Response,
APPLICATION_ID,
'metadata',
workspace,
);
expect(
sdkClientArchiveService.getClientModuleFromArchive,
).not.toHaveBeenCalled();
expect(response.send).toHaveBeenCalledWith(installedModuleBuffer);
});
it('serves the core module from the application archive', async () => {
await controller.getSdkModule(
response as unknown as Response,
APPLICATION_ID,
'core',
workspace,
);
expect(
sdkClientArchiveService.getClientModuleFromArchive,
).toHaveBeenCalledWith({
workspaceId: WORKSPACE_ID,
applicationId: APPLICATION_ID,
applicationUniversalIdentifier: 'my-app',
moduleName: 'core',
});
expect(mockGetInstalledSdkMetadataModule).not.toHaveBeenCalled();
});
it('disables MIME sniffing and opts out of HTTP caching on the bare fallback url', async () => {
await controller.getSdkModule(
response as unknown as Response,
APPLICATION_ID,
'core',
workspace,
);
expect(response.setHeader).toHaveBeenCalledWith(
'Content-Type',
'application/javascript',
);
expect(response.setHeader).toHaveBeenCalledWith(
'X-Content-Type-Options',
'nosniff',
);
expect(response.setHeader).toHaveBeenCalledWith(
'Cache-Control',
SDK_CLIENT_MODULE_NO_STORE_CACHE_CONTROL,
);
});
it('serves fingerprinted urls as immutable when the checksum matches the persisted core checksum', async () => {
await controller.getSdkModule(
response as unknown as Response,
APPLICATION_ID,
'core',
workspace,
PERSISTED_CORE_CHECKSUM,
);
expect(response.setHeader).toHaveBeenCalledWith(
'Cache-Control',
SDK_CLIENT_MODULE_CACHE_CONTROL,
);
});
it('opts out of caching when the fingerprint does not match the persisted checksum', async () => {
await controller.getSdkModule(
response as unknown as Response,
APPLICATION_ID,
'core',
workspace,
'a'.repeat(64),
);
expect(response.setHeader).toHaveBeenCalledWith(
'Cache-Control',
SDK_CLIENT_MODULE_NO_STORE_CACHE_CONTROL,
);
});
it('rejects unknown module names', async () => {
await expect(
controller.getSdkModule(
response as unknown as Response,
APPLICATION_ID,
'evil' as never,
workspace,
),
).rejects.toBeInstanceOf(NotFoundException);
});
it('rejects unknown applications', async () => {
await expect(
controller.getSdkModule(
response as unknown as Response,
'unknown-app',
'metadata',
workspace,
),
).rejects.toBeInstanceOf(NotFoundException);
});
});
@@ -0,0 +1,4 @@
export const SDK_CLIENT_MODULE_CACHE_CONTROL =
'private, max-age=31536000, immutable';
export const SDK_CLIENT_MODULE_NO_STORE_CACHE_CONTROL = 'private, no-store';
@@ -14,7 +14,12 @@ import {
ALLOWED_SDK_MODULES,
type SdkModuleName,
} from 'src/engine/core-modules/sdk-client/constants/allowed-sdk-modules';
import {
SDK_CLIENT_MODULE_CACHE_CONTROL,
SDK_CLIENT_MODULE_NO_STORE_CACHE_CONTROL,
} from 'src/engine/core-modules/sdk-client/constants/sdk-client-module-cache-control';
import { SdkClientArchiveService } from 'src/engine/core-modules/sdk-client/sdk-client-archive.service';
import { getInstalledSdkMetadataModule } from 'src/engine/core-modules/sdk-client/utils/get-installed-sdk-metadata-module.util';
import { WorkspaceEntity } from 'src/engine/core-modules/workspace/workspace.entity';
import { AuthWorkspace } from 'src/engine/decorators/auth/auth-workspace.decorator';
import { NoPermissionGuard } from 'src/engine/guards/no-permission.guard';
@@ -29,13 +34,31 @@ export class SdkClientController {
private readonly sdkClientArchiveService: SdkClientArchiveService,
) {}
@Get(':applicationId/:moduleName')
@Get(['metadata', 'metadata/:checksum'])
@UseGuards(NoPermissionGuard)
async getInstanceSdkMetadataModule(
@Res() res: Response,
@Param('checksum') checksum?: string,
) {
const { moduleBuffer, checksum: servedModuleChecksum } =
await getInstalledSdkMetadataModule();
this.sendSdkModule({
res,
moduleBuffer,
servedModuleChecksum,
requestedChecksum: checksum,
});
}
@Get([':applicationId/:moduleName', ':applicationId/:moduleName/:checksum'])
@UseGuards(NoPermissionGuard)
async getSdkModule(
@Res() res: Response,
@Param('applicationId') applicationId: string,
@Param('moduleName') moduleName: SdkModuleName,
@AuthWorkspace() workspace: WorkspaceEntity,
@Param('checksum') checksum?: string,
) {
if (!ALLOWED_SDK_MODULES.includes(moduleName)) {
throw new NotFoundException(
@@ -56,15 +79,51 @@ export class SdkClientController {
);
}
const fileBuffer =
await this.sdkClientArchiveService.getClientModuleFromArchive({
workspaceId: workspace.id,
applicationId,
applicationUniversalIdentifier: application.universalIdentifier,
moduleName,
});
const { moduleBuffer, checksum: servedModuleChecksum } =
moduleName === 'metadata'
? await getInstalledSdkMetadataModule()
: {
moduleBuffer:
await this.sdkClientArchiveService.getClientModuleFromArchive({
workspaceId: workspace.id,
applicationId,
applicationUniversalIdentifier: application.universalIdentifier,
moduleName,
}),
checksum: application.sdkClientCoreChecksum,
};
this.sendSdkModule({
res,
moduleBuffer,
servedModuleChecksum,
requestedChecksum: checksum,
});
}
private sendSdkModule({
res,
moduleBuffer,
servedModuleChecksum,
requestedChecksum,
}: {
res: Response;
moduleBuffer: Buffer;
servedModuleChecksum: string | null;
requestedChecksum?: string;
}) {
const isChecksumMatch =
isDefined(requestedChecksum) &&
requestedChecksum === servedModuleChecksum;
res.setHeader('Content-Type', 'application/javascript');
res.send(fileBuffer);
res.setHeader('X-Content-Type-Options', 'nosniff');
res.setHeader(
'Cache-Control',
isChecksumMatch
? SDK_CLIENT_MODULE_CACHE_CONTROL
: SDK_CLIENT_MODULE_NO_STORE_CACHE_CONTROL,
);
res.send(moduleBuffer);
}
}
@@ -0,0 +1,16 @@
import { Field, ObjectType } from '@nestjs/graphql';
import { IsNotEmpty, IsOptional, IsString } from 'class-validator';
@ObjectType('SdkClientChecksums')
export class SdkClientChecksumsDTO {
@IsString()
@IsOptional()
@Field(() => String, { nullable: true })
core: string | null;
@IsString()
@IsNotEmpty()
@Field()
metadata: string;
}
@@ -1,7 +1,11 @@
import { type SdkClientGenerationTrigger } from 'src/engine/core-modules/sdk-client/types/sdk-client-generation-trigger.type';
export const GENERATE_SDK_CLIENT_JOB_NAME = 'GenerateSdkClientJob';
export type GenerateSdkClientJobData = {
workspaceId: string;
applicationId: string;
applicationUniversalIdentifier: string;
// Optional: jobs enqueued before this field existed carry no trigger
trigger?: SdkClientGenerationTrigger;
};
@@ -19,6 +19,7 @@ export class GenerateSdkClientJob {
workspaceId: data.workspaceId,
applicationId: data.applicationId,
applicationUniversalIdentifier: data.applicationUniversalIdentifier,
trigger: data.trigger,
});
}
}
@@ -165,6 +165,7 @@ export class SdkClientArchiveService {
workspaceId,
applicationId,
applicationUniversalIdentifier,
trigger: 'missing-archive',
});
}
}
@@ -1,6 +1,7 @@
import { Injectable, Logger } from '@nestjs/common';
import { InjectRepository } from '@nestjs/typeorm';
import { createHash } from 'crypto';
import * as fs from 'fs/promises';
import { printSchema } from 'graphql';
import path, { join } from 'path';
@@ -18,6 +19,9 @@ import { TemporaryDirManager } from 'src/engine/core-modules/logic-function/logi
import { InjectMessageQueue } from 'src/engine/core-modules/message-queue/decorators/message-queue.decorator';
import { MessageQueue } from 'src/engine/core-modules/message-queue/message-queue.constants';
import { MessageQueueService } from 'src/engine/core-modules/message-queue/services/message-queue.service';
import { MetricsService } from 'src/engine/core-modules/metrics/metrics.service';
import { MetricsKeys } from 'src/engine/core-modules/metrics/types/metrics-keys.type';
import { type SdkModuleName } from 'src/engine/core-modules/sdk-client/constants/allowed-sdk-modules';
import { SDK_CLIENT_PACKAGE_DIRNAME } from 'src/engine/core-modules/sdk-client/constants/sdk-client-package-dirname';
import {
SdkClientException,
@@ -27,8 +31,10 @@ import {
GENERATE_SDK_CLIENT_JOB_NAME,
type GenerateSdkClientJobData,
} from 'src/engine/core-modules/sdk-client/jobs/generate-sdk-client.job-constants';
import { type SdkClientGenerationTrigger } from 'src/engine/core-modules/sdk-client/types/sdk-client-generation-trigger.type';
import { fromWorkspaceEntityToFlat } from 'src/engine/core-modules/workspace/utils/from-workspace-entity-to-flat.util';
import { WorkspaceEntity } from 'src/engine/core-modules/workspace/workspace.entity';
import { WorkspaceEventBroadcaster } from 'src/engine/subscriptions/workspace-event-broadcaster/workspace-event-broadcaster.service';
import { WorkspaceCacheService } from 'src/engine/workspace-cache/services/workspace-cache.service';
const SDK_CLIENT_ARCHIVE_NAME = 'twenty-client-sdk.zip';
@@ -49,6 +55,8 @@ export class SdkClientGenerationService {
private readonly applicationService: ApplicationService,
@InjectMessageQueue(MessageQueue.workspaceQueue)
private readonly messageQueueService: MessageQueueService,
private readonly workspaceEventBroadcaster: WorkspaceEventBroadcaster,
private readonly metricsService: MetricsService,
) {}
async enqueueSdkClientGenerationForWorkspace(
@@ -62,52 +70,101 @@ export class SdkClientGenerationService {
await Promise.all(
[twentyStandardFlatApplication, workspaceCustomFlatApplication].map(
(application) =>
this.messageQueueService.add<GenerateSdkClientJobData>(
GENERATE_SDK_CLIENT_JOB_NAME,
{
workspaceId,
applicationId: application.id,
applicationUniversalIdentifier: application.universalIdentifier,
},
{
id: `sdk-client:${workspaceId}:${application.id}`,
retryLimit: SDK_CLIENT_GENERATION_RETRY_LIMIT,
},
),
this.enqueueSdkClientGenerationForApplication({
workspaceId,
applicationId: application.id,
applicationUniversalIdentifier: application.universalIdentifier,
trigger: 'workspace-activation',
}),
),
);
}
private async enqueueSdkClientGenerationForApplication({
workspaceId,
applicationId,
applicationUniversalIdentifier,
trigger,
}: {
workspaceId: string;
applicationId: string;
applicationUniversalIdentifier: string;
trigger: SdkClientGenerationTrigger;
}): Promise<void> {
await this.messageQueueService.add<GenerateSdkClientJobData>(
GENERATE_SDK_CLIENT_JOB_NAME,
{
workspaceId,
applicationId,
applicationUniversalIdentifier,
trigger,
},
{
id: `sdk-client:${workspaceId}:${applicationId}`,
retryLimit: SDK_CLIENT_GENERATION_RETRY_LIMIT,
},
);
}
async generateSdkClientForApplication({
workspaceId,
applicationId,
applicationUniversalIdentifier,
trigger = 'unknown',
}: {
workspaceId: string;
applicationId: string;
applicationUniversalIdentifier: string;
trigger?: SdkClientGenerationTrigger;
}): Promise<Buffer> {
const workspaceEntity = await this.workspaceRepository.findOneByOrFail({
id: workspaceId,
});
const generationStart = performance.now();
const graphqlSchema = await this.workspaceSchemaFactory.createGraphQLSchema(
fromWorkspaceEntityToFlat(workspaceEntity),
applicationId,
);
try {
const workspaceEntity = await this.workspaceRepository.findOneByOrFail({
id: workspaceId,
});
const archiveBuffer = await this.generateAndStore({
workspaceId,
applicationId,
applicationUniversalIdentifier,
schema: printSchema(graphqlSchema),
});
const graphqlSchema =
await this.workspaceSchemaFactory.createGraphQLSchema(
fromWorkspaceEntityToFlat(workspaceEntity),
applicationId,
);
this.logger.log(
`Generated SDK client for application ${applicationUniversalIdentifier}`,
);
const archiveBuffer = await this.generateAndStore({
workspaceId,
applicationId,
applicationUniversalIdentifier,
schema: printSchema(graphqlSchema),
});
return archiveBuffer;
const generationDurationMs = performance.now() - generationStart;
this.metricsService.incrementCounterBy({
key: MetricsKeys.SdkClientGenerationSucceeded,
amount: 1,
attributes: { trigger },
});
this.metricsService.recordHistogram({
key: MetricsKeys.SdkClientGenerationDurationMs,
value: generationDurationMs,
unit: 'ms',
attributes: { trigger },
});
this.logger.log(
`Generated SDK client for application ${applicationUniversalIdentifier} (trigger: ${trigger})`,
);
return archiveBuffer;
} catch (error) {
this.metricsService.incrementCounterBy({
key: MetricsKeys.SdkClientGenerationFailed,
amount: 1,
attributes: { trigger },
});
throw error;
}
}
private async generateAndStore({
@@ -145,6 +202,11 @@ export class SdkClientGenerationService {
await replaceCoreClient({ packageRoot: tempPackageRoot, schema });
const sdkClientCoreChecksum = await this.computeSdkModuleChecksum(
tempPackageRoot,
'core',
);
const archivePath = join(sourceTemporaryDir, SDK_CLIENT_ARCHIVE_NAME);
await createZipFile(tempPackageRoot, archivePath);
@@ -162,13 +224,22 @@ export class SdkClientGenerationService {
await this.applicationRepository.update(
{ id: applicationId, workspaceId },
{ isSdkLayerStale: true },
{
isSdkLayerStale: true,
sdkClientCoreChecksum,
},
);
await this.workspaceCacheService.invalidateAndRecompute(workspaceId, [
'flatApplicationMaps',
]);
await this.broadcastSdkClientCoreChecksumUpdate({
workspaceId,
applicationId,
sdkClientCoreChecksum,
});
return archiveBuffer;
} catch (error) {
throw new SdkClientException(
@@ -179,4 +250,52 @@ export class SdkClientGenerationService {
await temporaryDirManager.clean();
}
}
private async broadcastSdkClientCoreChecksumUpdate({
workspaceId,
applicationId,
sdkClientCoreChecksum,
}: {
workspaceId: string;
applicationId: string;
sdkClientCoreChecksum: string;
}): Promise<void> {
try {
await this.workspaceEventBroadcaster.broadcast({
workspaceId,
events: [
{
type: 'updated',
entityName: 'application',
recordId: applicationId,
properties: {
updatedFields: ['sdkClientCoreChecksum'],
after: {
id: applicationId,
sdkClientCoreChecksum,
},
},
},
],
});
} catch (error) {
this.logger.warn(
`Failed to broadcast SDK client core checksum update for application ${applicationId} in workspace ${workspaceId}`,
error,
);
}
}
// sha-256 (not md5) so the renderer can verify cached bundles against the URL
// checksum with WebCrypto, which has no md5 support
private async computeSdkModuleChecksum(
tempPackageRoot: string,
moduleName: SdkModuleName,
): Promise<string> {
const moduleBuffer = await fs.readFile(
join(tempPackageRoot, 'dist', `${moduleName}.mjs`),
);
return createHash('sha256').update(moduleBuffer).digest('hex');
}
}
@@ -1,12 +1,14 @@
import { Module } from '@nestjs/common';
import { Module, type OnApplicationBootstrap } from '@nestjs/common';
import { TypeOrmModule } from '@nestjs/typeorm';
import { CoreGraphQLApiModule } from 'src/engine/api/graphql/core-graphql-api.module';
import { ApplicationEntity } from 'src/engine/core-modules/application/application.entity';
import { ApplicationModule } from 'src/engine/core-modules/application/application.module';
import { MetricsModule } from 'src/engine/core-modules/metrics/metrics.module';
import { SdkClientController } from 'src/engine/core-modules/sdk-client/controllers/sdk-client.controller';
import { SdkClientArchiveService } from 'src/engine/core-modules/sdk-client/sdk-client-archive.service';
import { SdkClientGenerationService } from 'src/engine/core-modules/sdk-client/sdk-client-generation.service';
import { getInstalledSdkMetadataModule } from 'src/engine/core-modules/sdk-client/utils/get-installed-sdk-metadata-module.util';
import { WorkspaceEntity } from 'src/engine/core-modules/workspace/workspace.entity';
import { WorkspaceCacheModule } from 'src/engine/workspace-cache/workspace-cache.module';
@@ -16,9 +18,14 @@ import { WorkspaceCacheModule } from 'src/engine/workspace-cache/workspace-cache
WorkspaceCacheModule,
CoreGraphQLApiModule,
ApplicationModule,
MetricsModule,
],
controllers: [SdkClientController],
providers: [SdkClientGenerationService, SdkClientArchiveService],
exports: [SdkClientGenerationService, SdkClientArchiveService],
})
export class SdkClientModule {}
export class SdkClientModule implements OnApplicationBootstrap {
async onApplicationBootstrap(): Promise<void> {
await getInstalledSdkMetadataModule();
}
}
@@ -0,0 +1,9 @@
// What caused an SDK client generation. Emitted as a metric attribute so
// regeneration volume can be broken down by cause.
// 'unknown' covers jobs enqueued before the trigger field existed.
export type SdkClientGenerationTrigger =
| 'workspace-activation'
| 'manifest-sync'
| 'missing-archive'
| 'dev-seeder'
| 'unknown';
@@ -0,0 +1,32 @@
import { createHash } from 'crypto';
import * as fs from 'fs/promises';
import { join } from 'path';
import { SDK_CLIENT_PACKAGE_DIRNAME } from 'src/engine/core-modules/sdk-client/constants/sdk-client-package-dirname';
export type InstalledSdkMetadataModule = {
moduleBuffer: Buffer;
checksum: string;
};
// Warmed once at bootstrap (SdkClientModule.onApplicationBootstrap) and treated
// as a process-lifetime invariant: the module ships inside the server build and
// never changes at runtime. A read/hash failure blocks boot, so a running
// server always has a resolved value here.
let installedSdkMetadataModule: InstalledSdkMetadataModule | undefined;
export const getInstalledSdkMetadataModule =
async (): Promise<InstalledSdkMetadataModule> => {
if (!installedSdkMetadataModule) {
const moduleBuffer = await fs.readFile(
join(SDK_CLIENT_PACKAGE_DIRNAME, 'dist', 'metadata.mjs'),
);
installedSdkMetadataModule = {
moduleBuffer,
checksum: createHash('sha256').update(moduleBuffer).digest('hex'),
};
}
return installedSdkMetadataModule;
};
@@ -48,6 +48,7 @@ const MOCK_FLAT_APPLICATION: FlatApplication = {
updatedAt: new Date(),
deletedAt: null,
isSdkLayerStale: true,
sdkClientCoreChecksum: null,
};
const flatObjectMetadataMaps = [
@@ -43,6 +43,7 @@ const MOCK_FLAT_APPLICATION: FlatApplication = {
updatedAt: new Date(),
deletedAt: null,
isSdkLayerStale: true,
sdkClientCoreChecksum: null,
};
type GenerateMorphOrRelationFlatFieldMetadataPairTestInput = Parameters<
@@ -128,6 +128,7 @@ export class DevSeederService {
applicationId: twentyStandardFlatApplication.id,
applicationUniversalIdentifier:
twentyStandardFlatApplication.universalIdentifier,
trigger: 'dev-seeder',
});
await this.devSeederMetadataService.seed({
@@ -140,6 +141,7 @@ export class DevSeederService {
applicationId: workspaceCustomFlatApplication.id,
applicationUniversalIdentifier:
workspaceCustomFlatApplication.universalIdentifier,
trigger: 'dev-seeder',
});
await this.devSeederMetadataService.seedRelations({