3281d37bdf80098c411a4101736bfd7c5e568d8d
4 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Charles Bochet
|
0bb3660844 |
chore(twenty-sdk): shrink logic-function bundles via stubbing (#20033)
## Summary
Logic-function bundles produced by the SDK CLI were ~1.2 MB each (source
maps ~3.1 MB) because esbuild was inlining `twenty-sdk/define` and its
transitive dependencies (zod + locales, twenty-shared, etc.). Those
`define*` factories are pure build-time metadata used only by the
manifest extractor — the Lambda runtime only ever invokes
`default.config.handler`, so the factories are dead weight at runtime.
This PR shrinks the bundles to ~9.5 KB each (~99% reduction) without
changing runtime behaviour.
## What changes
- **Stub `twenty-sdk/define` at user-app build time.** New esbuild
plugin
(`packages/twenty-sdk/src/cli/utilities/build/common/plugins/stub-twenty-sdk-define.plugin.ts`)
intercepts every import of `twenty-sdk/define` during user-app builds
and replaces it with a tiny virtual module:
- Factory functions (`defineLogicFunction`,
`definePostInstallLogicFunction`, …) become `(config) => ({ success:
true, config, errors: [] })`.
- Enums and helpers become `Proxy`-based no-ops.
- Wired into both the one-shot build (`build-application.ts`) and the
watcher (`esbuild-watcher.ts`), for logic functions and front
components.
- **New runtime barrel `twenty-sdk/logic-function`.** Re-exports only
the types logic-function authors need (`InstallPayload`, `RoutePayload`,
`CronPayload`, `DatabaseEventPayload`, `LogicFunctionConfig`,
`InputJsonSchema`, …). Compiled `.mjs` is 36 bytes. Wired into Vite,
Rollup `.d.ts` bundling, `package.json#exports`, and `typesVersions`.
- **Lint enforcement.** Added an oxlint `no-restricted-imports` rule
that forbids `twenty-shared` / `twenty-shared/*` imports from
`**/*.logic-function.ts` and `**/logic-functions/**/*.ts`, with a help
message pointing at the new barrel. Applied to the `create-twenty-app`
template and to `github-connector`, `hello-world`, `postcard`.
- **Migrated existing sources.** All logic-function files across
`community/{github-connector, apollo-enrich}`, `examples/{hello-world,
postcard}`, and `internal/{twenty-for-twenty, self-hosting, exa}` now
import types from `twenty-sdk/logic-function` instead of
`twenty-sdk/define` or `twenty-shared/*`. Renamed leftover
`InstallLogicFunctionPayload` references to `InstallPayload`.
## Why this is safe
- `define*` exports from `twenty-sdk/define` are metadata factories
whose call expressions are statically inspected by the manifest
extractor (`manifest-extract-config.ts`). They're never evaluated at
runtime — the Lambda executor only walks `default.config.handler`
(`logic-function-drivers/constants/executor/index.mjs`).
- The stub keeps the same call shape (`{ success, config, errors }`), so
any logic-function module that re-exports
`defineX(config).config.handler` still resolves to the user's handler at
runtime.
- Front-component bundles are unaffected by the stub because the
pre-existing JSX transform plugin
(`jsx-transform-to-remote-dom-worker-format-plugin.ts`) unwraps
`defineFrontComponent(...)` earlier in the pipeline. That's intentional
— front-component bloat is React/Preact, not in scope here.
## Measurements (github-connector)
| Asset | Before | After |
|---|---|---|
| `*.logic-function.mjs` | ~1.2 MB | ~9.5 KB |
| `*.logic-function.mjs.map` | ~3.1 MB | ~22 KB |
|
||
|
Félix Malfait
|
4f938aa097 |
feat(app): infrastructure for pre-installed apps (#19973)
**PR 1 of 2.** Follow-up PR ships the Exa app, sets it as a default
pre-installed app, and removes the current `WebSearchTool` /
`WebSearchService` / `ExaDriver`. This PR adds the plumbing; no
user-visible change yet.
## Summary
- Server admins can declare a list of npm app packages to auto-install
on every new workspace and backfill onto existing workspaces via CLI.
- Server-level secrets (like Exa's API key) live on the
`ApplicationRegistration` (one row per server, encrypted) and are
injected into logic function execution env at runtime. No more
per-workspace storage of global secrets.
- A generic `POST /app/billing/charge` endpoint lets app logic functions
emit workspace usage events for metered features. Exa uses it in PR 2;
future apps (call recorder, etc.) reuse it.
- `LogicFunctionToolProvider` tool name prefix changes `logic_function_`
→ `app_`. Shorter, accurate (they come from installed apps).
## What's in this PR
**Logic function executor — server-level variables**
- `LogicFunctionExecutorService.getExecutionEnvVariables` now resolves
env vars in the order: hardcoded defaults →
`ApplicationRegistrationVariable[]` (server-level) →
`ApplicationVariable[]` (workspace-level override). The manifest
`serverVariables` schema has existed; this closes the loop.
**Config**
- `PRE_INSTALLED_APPS` — comma-separated list of npm packages. Default:
empty.
**\`PreInstalledAppsService\`** (new module)
- \`onApplicationBootstrap()\` — fetches each package's manifest from
the app registry CDN, upserts an \`ApplicationRegistration\`, and seeds
declared \`serverVariables\` from matching env vars (e.g.
\`EXA_API_KEY\` env → encrypted registration variable).
- \`installOnWorkspace(workspaceId)\` — installs all pre-installed apps
on a single workspace. Tolerates per-app failures.
**Auto-install on new workspace activation**
- \`WorkspaceService.prefillCreatedWorkspaceRecords\` invokes
\`installOnWorkspace\` after prefilling standard records. Non-blocking
on failure.
**Backfill CLI command**
- \`install-pre-installed-apps\` — iterates active and suspended
workspaces, installs pre-installed apps that aren't yet installed.
Idempotent. Run after changing \`PRE_INSTALLED_APPS\`.
**App billing endpoint**
- \`POST /app/billing/charge\`. Authenticated via \`APPLICATION_ACCESS\`
token (already injected into logic function execution env as
\`DEFAULT_APP_ACCESS_TOKEN\`). Body: \`{ creditsUsedMicro, quantity,
unit, operationType, resourceContext? }\`. Emits \`USAGE_RECORDED\` with
\`applicationId\` as \`resourceId\`. Generic — reusable by any app.
**Tool name prefix**
- \`LogicFunctionToolProvider.buildLogicFunctionToolName\` now produces
\`app_<name>\` instead of \`logic_function_<name>\`. Only affects tools
sourced from logic functions; other tool providers unchanged.
## Stats
- 16 files, +501 / −2
- 7 new files (1 command, 1 service × 2, 1 controller, 1 DTO, 2 modules)
- Typecheck: 7 pre-existing errors, zero new
- Prettier clean
## Behavior deltas
- **\`PRE_INSTALLED_APPS\` default = empty**: existing servers see no
change on merge.
- **\`ApplicationRegistrationVariable\` is now read by the executor**:
apps that were using manifest \`serverVariables\` but expecting them to
be ignored by the executor will now see them injected. No apps ship with
\`isTool: true\` logic functions today, so this is latent — first
consumer is Exa in PR 2.
- **Tool prefix**: currently no logic-function tools are named
\`logic_function_*\` in any production flow. The prefix change affects
only future tools emitted by \`LogicFunctionToolProvider\`.
## Risks
- **CDN unavailability at startup**: if the app registry CDN is down,
\`ensureRegistrationsExist\` logs warnings but doesn't block server
start. Installation on new workspaces during this window will find no
registrations and log a non-blocking error. Backfill command can retry
after CDN recovers.
- **Cold-start overhead**: \`ensureRegistrationsExist\` is called once
per process on bootstrap. Current configurable default is empty, so zero
overhead. When an admin sets \`PRE_INSTALLED_APPS\`, they accept one
HTTP call per package at boot.
- **Server-level variables flow**:
\`ApplicationRegistrationVariable.encryptedValue\` is shared by all
workspaces of a server. Appropriate for a single-tenant Exa key. Not
appropriate for per-tenant keys — those go in workspace-level
\`ApplicationVariable\` and override.
## Test plan
- [ ] \`npx nx typecheck twenty-server\` passes (verified: 7
pre-existing unrelated errors, zero new)
- [ ] Set \`PRE_INSTALLED_APPS=@twenty-apps/hello-world\` (or any real
npm-published app), \`HELLO_WORLD_API_KEY=xxx\`, restart server:
\`ApplicationRegistration\` row is upserted,
\`ApplicationRegistrationVariable\` for HELLO_WORLD_API_KEY is populated
(encrypted).
- [ ] Create a new workspace: the app is auto-installed,
\`ApplicationEntity\` row created, \`LogicFunctionEntity\` rows created.
- [ ] Existing workspace: run \`yarn nx run twenty-server:command
install-pre-installed-apps\`: apps install across all workspaces,
idempotent on re-run.
- [ ] Trigger a logic function that reads
\`process.env.HELLO_WORLD_API_KEY\`: value resolves from the
server-level \`ApplicationRegistrationVariable\`.
- [ ] Log a charge from the handler: \`POST /app/billing/charge\` with
\`Authorization: Bearer \$DEFAULT_APP_ACCESS_TOKEN\` body
\`{creditsUsedMicro: 1000, quantity: 1, unit: "INVOCATION",
operationType: "WEB_SEARCH"}\` → returns \`{success: true}\`,
\`USAGE_RECORDED\` event emitted with correct
\`resourceId=applicationId\`.
- [ ] Tool name generated by \`LogicFunctionToolProvider\` starts with
\`app_\`.
## What's NOT in this PR (PR 2 scope)
- The Exa app itself (\`packages/twenty-apps/...\` directory)
- Removing \`WebSearchTool\`, \`WebSearchService\`, \`ExaDriver\`,
\`web-search\` module
- Removing \`WEB_SEARCH_DRIVER\` config var
- Removing the current \`exa_web_search\` entry in
\`ActionToolProvider\`
- Chat preload list updated to \`app_exa_web_search\`
- Frontend \`getToolDisplayMessage\` branch for \`app_exa_web_search\`
- Setting \`PRE_INSTALLED_APPS\` default to include \`@twenty-apps/exa\`
---------
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
Charles Bochet
|
eb1ca1b9ec |
perf(sdk): split twenty-sdk barrel into per-purpose subpaths to cut logic-function bundle ~700x (#19834)
## Summary
Logic-function bundles produced by the twenty-sdk CLI were ~1.18 MB even
for a one-line handler. Root cause: the SDK shipped as a single bundled
barrel (`twenty-sdk` → `dist/index.mjs`) that co-mingled server-side
definition factories with the front-component runtime, validation (zod),
and React. With no `\"sideEffects\"` declaration on the SDK package,
esbuild had to assume every module-level statement could have side
effects and refused to drop unused code.
This PR restructures the SDK so consumers' bundlers can tree-shake at
the leaf level:
- **Reorganized SDK source.** All server-side definition factories now
live under `src/sdk/define/` (agents, application, fields,
logic-functions, objects, page-layouts, roles, skills, views,
navigation-menu-items, etc.). All front-component runtime
(components, hooks, host APIs, command primitives) lives under
`src/sdk/front-component/`. The legacy bare `src/sdk/index.ts` is
removed; the bare `twenty-sdk` entry no longer exists.
- **Split the build configs by purpose / runtime env.** Replaced
`vite.config.sdk.ts` with two purpose-specific configs:
- `vite.config.define.ts` — node target, externals from package
`dependencies`, emits to `dist/define/**`
- `vite.config.front-component.ts` — browser/React target, emits to
`dist/front-component/**`
Both use `preserveModules: true` so each leaf ships as its own `.mjs`.
- **\`\"sideEffects\": false\`** on `twenty-sdk` so esbuild can drop
unreferenced re-exports.
- **\`package.json\` exports + \`typesVersions\`** updated: dropped the
bare \`.\` entry, added \`./front-component\`, and pointed \`./define\`
at the new per-module dist layout.
- **Migrated every internal/example/community app** to the new subpath
imports (`twenty-sdk/define`, `twenty-sdk/front-component`,
`twenty-sdk/ui`).
- **Added \`bundle-investigation\` internal app** that reproduces the
bundle bloat and demonstrates the fix.
- Cleaned up dead \`twenty-sdk/dist/sdk/...\` references in the
front-component story builder, the call-recording app, and the SDK
tsconfig.
## Bundle size impact
Measured with esbuild using the same options as the SDK CLI
(\`packages/twenty-apps/internal/bundle-investigation\`):
| Variant | Imports | Before | After |
| ----------------------- |
------------------------------------------------------- | ---------- |
--------- |
| \`01-bare\` | \`defineLogicFunction\` from \`twenty-sdk/define\` |
1177 KB | **1.6 KB** |
| \`02-with-sdk-client\` | + \`CoreApiClient\` from
\`twenty-client-sdk/core\` | 1177 KB | **1.9 KB** |
| \`03-fetch-issues\` | + GitHub GraphQL fetch + JWT signing + 2
mutations | 1181 KB | **5.8 KB** |
| \`05-via-define-subpath\` | same as \`01\`, via the public subpath |
1177 KB | **1.7 KB** |
That's a ~735× reduction on the bare baseline. Knock-on benefits for
Lambda warm + cold starts, S3 upload size, and \`/tmp\` disk usage in
warm containers.
## Test plan
- [x] \`npx nx run twenty-sdk:build\` succeeds
- [x] \`npx nx run twenty-sdk:typecheck\` passes
- [x] \`npx nx run twenty-sdk:test:unit\` passes (31 files / 257 tests)
- [x] \`npx nx run-many -t typecheck
--projects=twenty-front,twenty-server,twenty-front-component-renderer,twenty-sdk,twenty-shared,bundle-investigation\`
passes
- [x] \`node
packages/twenty-apps/internal/bundle-investigation/scripts/build-variants.mjs\`
produces the sizes above
- [ ] CI green
Made with [Cursor](https://cursor.com)
|
||
|
Charles Bochet
|
6e259d3ded |
Inline twenty-shared types in SDK declarations (#19605)
## Summary - `twenty-shared` is private and never published to npm, so SDK consumers couldn't resolve type imports like `from 'twenty-shared/types'` in the generated `.d.ts` files - Replace `vite.config.sdk.ts` with `rollup-plugin-dts` which compiles `src/sdk/index.ts` directly into a self-contained `dist/sdk/index.d.ts` with all `twenty-shared` types inlined - The JS output from `vite.config.sdk.ts` (`dist/sdk/*.js`) was unused — the main export already maps to `dist/index.mjs` from the node Vite config ## Changes - **Deleted** `vite.config.sdk.ts` — its preserved-module JS output wasn't referenced by any `package.json` export - **Added** `rollup.config.dts.mjs` — uses `rollup-plugin-dts` to compile SDK types from source with `twenty-shared` inlined (~850ms) - **Updated** `project.json` — build/dev/build:sdk targets now use rollup instead of the removed vite config - **Updated** `tsconfig.json` — removed `vite.config.sdk.ts` from include ## Test plan - [ ] Run `npx nx build twenty-sdk` and verify `dist/sdk/index.d.ts` contains no `twenty-shared` references - [ ] Verify `dist/index.mjs` and `dist/index.cjs` are still produced correctly - [ ] Verify CLI (`dist/cli.cjs`) still works - [ ] Verify `npx nx build:sdk twenty-sdk` works standalone Made with [Cursor](https://cursor.com) |