Commit Graph

24 Commits

Author SHA1 Message Date
Paul Rastoin 4ea2e32366 Refactor twenty client sdk provisioning for logic function and front-component (#18544)
## 1. The `twenty-client-sdk` Package (Source of Truth)

The monorepo package at `packages/twenty-client-sdk` ships with:
- A **pre-built metadata client** (static, generated from a fixed
schema)
- A **stub core client** that throws at runtime (`CoreApiClient was not
generated...`)
- Both ESM (`.mjs`) and CJS (`.cjs`) bundles in `dist/`
- A `package.json` with proper `exports` map for
`twenty-client-sdk/core`, `twenty-client-sdk/metadata`, and
`twenty-client-sdk/generate`

## 2. Generation & Upload (Server-Side, at Migration Time)

**When**: `WorkspaceMigrationRunnerService.run()` executes after a
metadata schema change.

**What happens in `SdkClientGenerationService.generateAndStore()`**:
1. Copies the stub `twenty-client-sdk` package from the server's assets
(resolved via `SDK_CLIENT_PACKAGE_DIRNAME` — from
`dist/assets/twenty-client-sdk/` in production, or from `node_modules`
in dev)
2. Filters out `node_modules/` and `src/` during copy — only
`package.json` + `dist/` are kept (like an npm publish)
3. Calls `replaceCoreClient()` which uses `@genql/cli` to introspect the
**application-scoped** GraphQL schema and generates a real
`CoreApiClient`, then compiles it to ESM+CJS and overwrites
`dist/core.mjs` and `dist/core.cjs`
4. Archives the **entire package** (with `package.json` + `dist/`) into
`twenty-client-sdk.zip`
5. Uploads the single archive to S3 under
`FileFolder.GeneratedSdkClient`
6. Sets `isSdkLayerStale = true` on the `ApplicationEntity` in the
database

## 3. Invalidation Signal

The `isSdkLayerStale` boolean column on `ApplicationEntity` is the
invalidation mechanism:
- **Set to `true`** by `generateAndStore()` after uploading a new client
archive
- **Checked** by both logic function drivers before execution — if
`true`, they rebuild their local layer
- **Set back to `false`** by `markSdkLayerFresh()` after the driver has
successfully consumed the new archive

Default is `false` so existing applications without a generated client
aren't affected.

## 4a. Logic Functions — Local Driver

**`ensureSdkLayer()`** is called before every execution:
1. Checks if the local SDK layer directory exists AND `isSdkLayerStale`
is `false` → early return
2. Otherwise, cleans the local layer directory
3. Calls `downloadAndExtractToPackage()` which streams the zip from S3
directly to disk and extracts the full package into
`<tmpdir>/sdk/<workspaceId>-<appId>/node_modules/twenty-client-sdk/`
4. Calls `markSdkLayerFresh()` to set `isSdkLayerStale = false`

**At execution time**, `assembleNodeModules()` symlinks everything from
the deps layer's `node_modules/` **except** `twenty-client-sdk`, which
is symlinked from the SDK layer instead. This ensures the logic
function's `import ... from 'twenty-client-sdk/core'` resolves to the
generated client.

## 4b. Logic Functions — Lambda Driver

**`ensureSdkLayer()`** is called during `build()`:
1. Checks if `isSdkLayerStale` is `false` and an existing Lambda layer
ARN exists → early return
2. Otherwise, deletes all existing layer versions for this SDK layer
name
3. Calls `downloadArchiveBuffer()` to get the raw zip from S3 (no disk
extraction)
4. Calls `reprefixZipEntries()` which streams the zip entries into a
**new zip** with the path prefix
`nodejs/node_modules/twenty-client-sdk/` — this is the Lambda layer
convention path. All done in memory, no disk round-trip
5. Publishes the re-prefixed zip as a new Lambda layer via
`publishLayer()`
6. Calls `markSdkLayerFresh()`

**At function creation**, the Lambda is created with **two layers**:
`[depsLayerArn, sdkLayerArn]`. The SDK layer is listed last so it
overwrites the stub `twenty-client-sdk` from the deps layer (later
layers take precedence in Lambda's `/opt` merge).

## 5. Front Components

Front components are built by `app:build` with `twenty-client-sdk/core`
and `twenty-client-sdk/metadata` as **esbuild externals**. The stored
`.mjs` in S3 has unresolved bare import specifiers like `import {
CoreApiClient } from 'twenty-client-sdk/core'`.

SDK import resolution is split between the **frontend host** (fetching &
caching SDK modules) and the **Web Worker** (rewriting imports):

**Server endpoints**:
- `GET /rest/front-components/:id` —
`FrontComponentService.getBuiltComponentStream()` returns the **raw
`.mjs`** directly from file storage. No bundling, no SDK injection.
- `GET /rest/sdk-client/:applicationId/:moduleName` —
`SdkClientController` reads a single file (e.g. `dist/core.mjs`) from
the generated SDK archive via
`SdkClientGenerationService.readFileFromArchive()` and serves it as
JavaScript.

**Frontend host** (`FrontComponentRenderer` in `twenty-front`):
1. Queries `FindOneFrontComponent` which returns `applicationId`,
`builtComponentChecksum`, `usesSdkClient`, and `applicationTokenPair`
2. If `usesSdkClient` is `true`, renders
`FrontComponentRendererWithSdkClient` which calls the
`useApplicationSdkClient` hook
3. `useApplicationSdkClient({ applicationId, accessToken })` checks the
Jotai atom family cache for existing blob URLs. On cache miss, fetches
both SDK modules from `GET /rest/sdk-client/:applicationId/core` and
`/metadata`, creates **blob URLs** for each, and stores them in the atom
family
4. Once the blob URLs are cached, passes them as `sdkClientUrls`
(already blob URLs, not server URLs) to `SharedFrontComponentRenderer` →
`FrontComponentWorkerEffect` → worker's `render()` call via
`HostToWorkerRenderContext`

**Worker** (`remote-worker.ts` in `twenty-sdk`):
1. Fetches the raw component `.mjs` source as text
2. If `sdkClientUrls` are provided and the source contains SDK import
specifiers (`twenty-client-sdk/core`, `twenty-client-sdk/metadata`),
**rewrites** the bare specifiers to the blob URLs received from the host
(e.g. `'twenty-client-sdk/core'` → `'blob:...'`)
3. Creates a blob URL for the rewritten source and `import()`s it
4. Revokes only the component blob URL after the module is loaded — the
SDK blob URLs are owned and managed by the host's Jotai cache

This approach eliminates server-side esbuild bundling on every request,
caches SDK modules per application in the frontend, and keeps the
worker's job to a simple string rewrite.

## Summary Diagram

```
app:build (SDK)
  └─ twenty-client-sdk stub (metadata=real, core=stub)
       │
       ▼
WorkspaceMigrationRunnerService.run()
  └─ SdkClientGenerationService.generateAndStore()
       ├─ Copy stub package (package.json + dist/)
       ├─ replaceCoreClient() → regenerate core.mjs/core.cjs
       ├─ Zip entire package → upload to S3
       └─ Set isSdkLayerStale = true
              │
     ┌────────┴────────────────────┐
     ▼                             ▼
Logic Functions               Front Components
     │                             │
     ├─ Local Driver               ├─ GET /rest/sdk-client/:appId/core
     │   └─ downloadAndExtract     │    → core.mjs from archive
     │      → symlink into         │
     │        node_modules         ├─ Host (useApplicationSdkClient)
     │                             │    ├─ Fetch SDK modules
     └─ Lambda Driver              │    ├─ Create blob URLs
         └─ downloadArchiveBuffer  │    └─ Cache in Jotai atom family
            → reprefixZipEntries   │
            → publish as Lambda    ├─ GET /rest/front-components/:id
              layer                │    → raw .mjs (no bundling)
                                   │
                                   └─ Worker (browser)
                                        ├─ Fetch component .mjs
                                        ├─ Rewrite imports → blob URLs
                                        └─ import() rewritten source
```

## Next PR
- Estimate perf improvement by implementing a redis caching for front
component client storage ( we don't even cache front comp initially )
- Implem frontent blob invalidation sse event from server

---------

Co-authored-by: Charles Bochet <charlesBochet@users.noreply.github.com>
2026-03-24 18:10:25 +00:00
Paul Rastoin 06bdb5ad6a [SDK] Agent in manifest (#18431)
# Introduction
Adding agent in the manifest, required for twenty standard app
extraction out of twenty-server
2026-03-09 11:10:14 +00:00
Félix Malfait 012d819557 OAuth Client — Unified ApplicationRegistration, OAuth server, and frontend (#18267)
## Summary

Consolidates three separate PRs (#18260, #18261, #18262) into a single
unified branch with all review feedback addressed:

### New features
- **ApplicationRegistration entity** — server-level registration for
OAuth apps with encrypted server variables
- **OAuth 2.0 server** — authorization code, client credentials, refresh
token grants with PKCE support
- **OAuth discovery endpoint** —
`.well-known/oauth-authorization-server` metadata
- **Frontend UI** — app registration details page with credential
management, redirect URI editing, and server variable configuration
- **CLI integration** — `twenty dev` auto-registers apps and stores
OAuth credentials locally
- **Authorize consent screen** — OAuth consent page at `/authorize`
showing requested scopes

### Review feedback addressed

**Renames (PR #18260):**
- `appRegistration` → `applicationRegistration` (entity, tables, files,
imports, GraphQL types)
- `appRegistrationVariable` → `applicationRegistrationVariable`
- `clientId` → `oAuthClientId`, `clientSecretHash` →
`oAuthClientSecretHash`, `redirectUris` → `oAuthRedirectUris`, `scopes`
→ `oAuthScopes`

**Security fixes (PR #18261):**
- Fixed redirect URI validation bypass when `oAuthRedirectUris` is an
empty array
- Fixed workspace isolation in `clientCredentialsGrant` — now uses
`find()` with explicit handling for multiple installations
- Added error logging in refresh token `catch` block instead of silently
swallowing

**Code quality (PR #18262):**
- Split `VersionDistributionEntry` into its own file (one export per
file)
- Split GraphQL queries and mutations into individual files with a
shared fragment
- Removed unused `OAuth` entry from `AuthProviderEnum`
- Added loading state to `handleRotateSecret`
- Removed 27 narration-style comments from test files
- Added proper guards (`PublicEndpointGuard`, `NoPermissionGuard`) to
controllers and resolvers

## Test plan

- [ ] Verify `twenty dev` registers an app and stores OAuth credentials
- [ ] Test OAuth authorization code flow end-to-end (authorize → token →
API call)
- [ ] Test client credentials grant
- [ ] Verify redirect URI validation rejects requests when no URIs are
registered
- [ ] Verify app registration detail page renders correctly
- [ ] Test secret rotation with loading state
- [ ] Verify server variable editing and saving
- [ ] Run `npx nx database:reset twenty-server` to validate migration

Closes #18260, #18261, #18262


Made with [Cursor](https://cursor.com)

---------

Co-authored-by: claude[bot] <41898282+claude[bot]@users.noreply.github.com>
2026-02-28 14:07:49 +01:00
nitin 887371054a Host-remote refresh token implementation (#18044) 2026-02-24 19:37:29 +01:00
martmull 3bc887e12e Add default relation to standard object on custom object in manifest (#18033)
add default relation fields when creating an object from an application

---------

Co-authored-by: prastoin <paul@twenty.com>
2026-02-20 11:49:06 +01:00
Paul Rastoin 175df59c21 Support Skill in manifest (#18092)
# Introduction
Support skill in manifest, pre-requisite for the twenty standard app
migration
2026-02-20 11:45:42 +01:00
Raphaël Bosi bd03073b6d [FRONT COMPONENTS] Declare command menu items in front components (#18047)
## Description

- Adds support for declaring command menu items directly within
`defineFrontComponent` via an optional command config property
- Introduces a new `CommandMenuItemManifest` type in twenty-shared and
wires it through the manifest build pipeline

## Example Of usage

```tsx
import { defineFrontComponent } from "twenty-sdk";

const TestAction = () => {
  return <div>Test Action</div>;
};

export default defineFrontComponent({
  universalIdentifier: "6c289461-0007-4a62-a99f-69e5c11a4ce7",
  name: "test-action",
  description: "Test Action",
  component: TestAction,
  command: {
    universalIdentifier: "c07df864-495f-46f3-9f5b-9d3ce2589e9b",
    label: "Run My Action",
    icon: "IconBolt",
    isPinned: false,
  },
});

```

## Video QA


https://github.com/user-attachments/assets/f910fc6a-44a9-45d1-87c5-f0ce64bb3878
2026-02-19 15:23:46 +01:00
Charles Bochet c3781e87cc Sync page Layout (#18034)
## Sync page layouts, tabs, and widgets

Adds the ability for SDK applications to synchronize `pageLayout`,
`pageLayoutTab`, and `pageLayoutWidget` entities, following the same
pattern established in #18003 for views and navigation menu items.

### Changes

**`twenty-shared`**
- New `PageLayoutManifest`, `PageLayoutTabManifest`, and
`PageLayoutWidgetManifest` types with a hierarchical structure (page
layout → tabs → widgets)
- Added `pageLayouts: PageLayoutManifest[]` to the `Manifest` type

**`twenty-sdk`**
- New `definePageLayout()` SDK function with validation for
universalIdentifier, name, and nested tabs/widgets
- Wired into the manifest extraction and build pipeline
(`DefinePageLayout` target function, `PageLayouts` entity key)
- Exported from the SDK entry point

**`twenty-server`**
- Added `pageLayout`, `pageLayoutTab`, `pageLayoutWidget` to
`APPLICATION_MANIFEST_METADATA_NAMES`
- New conversion utilities: manifest → universal flat entity for all
three entity types
- Updated `computeApplicationManifestAllUniversalFlatEntity
2026-02-18 17:55:04 +01:00
Thomas Trompette b4e924b671 Sync views and navigation items (#18003)
Both objects are necessary to fully enjoy objects within applications

<img width="770" height="311" alt="Capture d’écran 2026-02-17 à 15 19
43"
src="https://github.com/user-attachments/assets/48c51fa4-63f4-45b2-a40a-df73f3aa79be"
/>
2026-02-17 16:32:41 +01:00
martmull da064d5e88 Support define is tool logic function (#17926)
- supports isTool and timeout settings in defineLogicFunction in apps
and in setting tabs definition
- compute for all toolInputSchema for logic funciton, in settings and in
code steps

<img width="991" height="802" alt="image"
src="https://github.com/user-attachments/assets/05dc1221-cac9-45a3-87b0-3b13161446fd"
/>
2026-02-16 10:43:29 +01:00
Paul Rastoin b2f7c745f8 Solo transaction application synchronization service refactor (#17864)
# Introduction
Refactoring the application sync service to be making a single validate
build and run transaction instead of calling all the services n times
thanks to the universal workspace migration refactor that allow doing so

## What's next
Migrating all below entities to by syncableEntities so they can be
universalised too ( right now they're still calling the services n times
and won't be returned in the workspace migration )
-
[objectPermission](https://github.com/twentyhq/core-team-issues/issues/2223)
-
[fieldPermission](https://github.com/twentyhq/core-team-issues/issues/2224)
-
[permissionFlag](https://github.com/twentyhq/core-team-issues/issues/2225)
2026-02-11 16:30:01 +00:00
martmull 9162685b2e Reorganize logic function files (#17766)
reorganize according to

<img width="1243" height="725" alt="Pasted Graphic"
src="https://github.com/user-attachments/assets/ba65dd10-8eec-4b13-ad49-9726edd3b79c"
/>

Not working yet
2026-02-09 12:36:39 +01:00
martmull 65678ed99f Upload files instead ofsources (#17608) 2026-02-03 12:11:10 +01:00
Marie 0001c2e7d0 [Apps] Apps marketplace (first draft) (#17562)
https://github.com/user-attachments/assets/c4e63edb-6e98-44bc-841f-ee110ae712d4

How it works
- `manifest.json` files are now committed when apps are published to our
repo
- to display available apps, from the server, we read into our github
repo, using a pod-scoped cache
- feature flagged
- app installation will be behind permission gate MARKETPLACE_APPS

Limitations and what is yet to develop
- content and settings tabs
- installed apps tab
- app installation
- test and potentially fix reading from .manifest.json once [Reshape
manifest
structure](https://github.com/twentyhq/core-team-issues/issues/2183) is
done. additional work is expected on assets notably. (couldnt properly
do it here as manifest.json will only be committed after this pr)
- we only read in community/ folder for now - we may want tochange that
- the cache is rather artisanal for now and scoped by pod - we may want
to change that
2026-01-30 17:32:32 +00:00
martmull f46da3eefd Update manifest structure (#17547)
Move all sync entities in an `entities` key. Rename functions to
logicFunctions

```json
{
  application: {
    ...
  },
  entities: {
    objects: [],
    logicFunctions: [],
    ...
  }
}
```
2026-01-30 16:26:45 +01:00
martmull 3412992e99 2162 Add asset watcher in twenty-sdk dev mode (#17513)
- assets are pushed in .twenty/output
- assets are uploaded in FileFolder.Assets
- not handled yet by the sync-manifest endpoint
2026-01-29 09:08:44 +00:00
Charles Bochet da6f1bbef3 Rename serverlessFunction to logicFunction (#17494)
## Summary

Rename "Serverless Function" to "Logic Function" across the codebase for
clearer naming.

### Environment Variable Changes

| Old | New |
|-----|-----|
| `SERVERLESS_TYPE` | `LOGIC_FUNCTION_TYPE` |
| `SERVERLESS_LAMBDA_REGION` | `LOGIC_FUNCTION_LAMBDA_REGION` |
| `SERVERLESS_LAMBDA_ROLE` | `LOGIC_FUNCTION_LAMBDA_ROLE` |
| `SERVERLESS_LAMBDA_SUBHOSTING_URL` |
`LOGIC_FUNCTION_LAMBDA_SUBHOSTING_URL` |
| `SERVERLESS_LAMBDA_ACCESS_KEY_ID` |
`LOGIC_FUNCTION_LAMBDA_ACCESS_KEY_ID` |
| `SERVERLESS_LAMBDA_SECRET_ACCESS_KEY` |
`LOGIC_FUNCTION_LAMBDA_SECRET_ACCESS_KEY` |

### Breaking Changes

- Environment variables must be updated in production deployments
- Database migration renames `serverlessFunction` → `logicFunction`
tables
2026-01-28 01:42:19 +01:00
martmull f4ca69a474 Implement dev mode nice UI (#17471)
Implement a nice terminal UI for dev mode using INK

<img width="1512" height="721" alt="image"
src="https://github.com/user-attachments/assets/79a71f37-1b31-4761-9e8d-718ef029ceb8"
/>
2026-01-27 15:34:28 +00:00
Charles Bochet 29c0c952e7 Add Front components to SDK (#17259)
## Add `frontComponent` entity type to SDK

This PR adds a new `frontComponent` entity type at parity with
`serverlessFunction`. Front components are React components that can be
defined and bundled as part of Twenty applications.

### Changes

#### New Entity Type
- Added `FRONT_COMPONENT` to `SyncableEntity` enum
- Front components use `*.front-component.tsx` file naming convention
- CLI command `twenty app:add` now includes `front-component` as an
option

#### SDK Application Layer (`twenty-sdk`)
- Added `defineFrontComponent()` function for defining front component
configurations with validation
- Added `FrontComponentConfig` type for component configuration
- Added `getFrontComponentBaseFile()` template generator for scaffolding
new front components
- Added `loadFrontComponentModule()` to load front component modules and
extract component metadata

#### Shared Types (`twenty-shared`)
- Added `FrontComponentManifest` type with `universalIdentifier`,
`name`, `description`, `componentPath`, and `componentName` fields
- Updated `ApplicationManifest` to include optional `frontComponents`
array

#### Manifest Build System
- Updated `manifest-build.ts` to discover and load
`*.front-component.tsx` files
- Updated `manifest-validate.ts` to validate front components and check
for duplicate IDs
- Updated `manifest-display.ts` to display front component count in
build summary
- Updated `manifest-plugin.ts` to display front component entry points
and watch `.tsx` files

#### Template (`create-twenty-app`)
- New applications now include a sample
`hello-world.front-component.tsx` file

#### Tests
- Added unit tests for `defineFrontComponent()`
- Added unit tests for `getFrontComponentBaseFile()`
2026-01-20 05:54:14 +00:00
martmull d55b83375d Allow app to extend object (#17116)
- exports `extendObject` utils from `twenty-sdk`
- define `*.object-extension.ts` filename pattern to extend and object
- handle object-extension configs in manifest
2026-01-13 15:07:14 +00:00
Marie 3ada8e5168 [Extensibility] Support relation fields (#17056)
Closes https://github.com/twentyhq/core-team-issues/issues/2043
2026-01-12 12:54:30 +00:00
Félix Malfait 0173e40a20 feat: Serverless Functions as AI Tools (#16919)
## Summary

This PR enables serverless functions to be exposed as AI tools, allowing
them to be used by AI agents.

### Changes

- Added new `SERVERLESS_FUNCTION` tool category
- Added `toolDescription`, `toolInputSchema`, and `toolOutputSchema`
fields to serverless functions
- Created database migration for the new schema columns
- Added tool index query and resolver for fetching available tools
- Added Settings AI page tabs (Skills, Tools, Settings) with new tools
table
- Added utility to convert tool schema to JSON schema format
- Updated frontend to display tools in the settings page

### Implementation Details

- Serverless functions can now define tool metadata (description,
input/output schemas)
- These functions are automatically registered in the tool registry
- The tool index endpoint allows querying available tools with their
schemas
- Settings page now has a dedicated Tools tab showing all available
tools
2026-01-05 11:13:06 +01:00
martmull e289f3056e 1895 extensibility v1 application tokens 3 (#16504)
- moves applicationRoleId to application entity
- add new `APPLICATION` FieldActorSource and `APPLICATION`
JwtTokenTypeEnum value
- create a new token with applicationId when executing a function
- when applicationId is in token, check for application.defaultRole
permissions
-use twenty-shared types in `twenty-sdk/application`
- create a new import from generate called "Twenty" that you can use
directly without having to set TWENTY_API_KEY AND TWENTY_API_URL (keep
metadata or core parameter only)
- provide to serverless unique one time BEARER TOKEN to run it

Result
<img width="977" height="566" alt="image"
src="https://github.com/user-attachments/assets/e78428a0-5b13-4975-aa13-58ee3b32450c"
/>

<img width="910" height="596" alt="image"
src="https://github.com/user-attachments/assets/6ec72bf5-7655-4093-a45e-ad269595a324"
/>

<img width="741" height="568" alt="image"
src="https://github.com/user-attachments/assets/7683944c-fd79-4417-8fb2-8e4815cc112f"
/>
2025-12-15 17:44:23 +01:00
martmull 7f1e69740a 1895 extensibility v1 application tokens (#16365)
First PR to implement application tokens
- add new application role in twenty-server
- move duplicated constants and types to twenty-shared
- will  add role configuration utils into twenty-sdk in another PR
2025-12-08 10:42:46 +01:00