Commit Graph

9 Commits

Author SHA1 Message Date
martmull 7ebcbe8801 Unify oAuth success and failure screen with autorize page (#20746)
Goal, matching authorize page design

<img width="2062" height="1376" alt="image"
src="https://github.com/user-attachments/assets/93d0f77a-c769-4a32-b41b-16459378314f"
/>


## Before

<img width="1540" height="942" alt="image"
src="https://github.com/user-attachments/assets/4de64f37-8519-4fdc-9388-70d98a69663e"
/>


## After

<img width="962" height="657" alt="image"
src="https://github.com/user-attachments/assets/c4fd19aa-13e3-452e-b6b7-c10202cb1edf"
/>

<img width="705" height="437" alt="image"
src="https://github.com/user-attachments/assets/e1f743f1-a862-4629-a3d0-c8e62e429e04"
/>
2026-05-20 07:43:34 +00:00
martmull 0cc2194399 Simplify create-twenty-app command (#20512)
## Simplify `create-twenty-app` for zero-interaction use

Makes `npx create-twenty-app@latest my-app` a fully non-interactive,
single-command experience suitable for automated environments (Codex,
Claude plugins).

  ### Changes

- **Remove all interactive prompts** — app name, display name,
description, and scaffold confirmation are now derived from CLI args
with sensible defaults. `inquirer` dependency removed
   entirely.
- **Replace OAuth with API key auth** — use the seeded dev API key
(`DEV_API_KEY`) to authenticate against the Docker instance as
`tim@apple.dev`, eliminating the browser-based OAuth
  flow.
- **Docker-first with early validation** — check Docker is installed
before scaffolding; if missing, print the install URL and exit. Detect
alternative runtimes (Podman, nerdctl).
- **Parallel image pull** — `docker pull` runs in the background during
scaffold + dependency install, saving 10-30s on typical runs.
- **Always pull latest image** — ensures the dev server is up-to-date on
every run.
- **Stop detecting port 3000** — only check port 2020 (Docker instance).
- **Update CLI flags** — remove `--skip-local-instance` and `--yes`; add
`--skip-docker`.
- **Update CI workflows and docs** — align e2e workflows, package
README, and template README/cd.yml with the new flow.
2026-05-13 16:44:27 +00:00
Paul Rastoin 4ea2e32366 Refactor twenty client sdk provisioning for logic function and front-component (#18544)
## 1. The `twenty-client-sdk` Package (Source of Truth)

The monorepo package at `packages/twenty-client-sdk` ships with:
- A **pre-built metadata client** (static, generated from a fixed
schema)
- A **stub core client** that throws at runtime (`CoreApiClient was not
generated...`)
- Both ESM (`.mjs`) and CJS (`.cjs`) bundles in `dist/`
- A `package.json` with proper `exports` map for
`twenty-client-sdk/core`, `twenty-client-sdk/metadata`, and
`twenty-client-sdk/generate`

## 2. Generation & Upload (Server-Side, at Migration Time)

**When**: `WorkspaceMigrationRunnerService.run()` executes after a
metadata schema change.

**What happens in `SdkClientGenerationService.generateAndStore()`**:
1. Copies the stub `twenty-client-sdk` package from the server's assets
(resolved via `SDK_CLIENT_PACKAGE_DIRNAME` — from
`dist/assets/twenty-client-sdk/` in production, or from `node_modules`
in dev)
2. Filters out `node_modules/` and `src/` during copy — only
`package.json` + `dist/` are kept (like an npm publish)
3. Calls `replaceCoreClient()` which uses `@genql/cli` to introspect the
**application-scoped** GraphQL schema and generates a real
`CoreApiClient`, then compiles it to ESM+CJS and overwrites
`dist/core.mjs` and `dist/core.cjs`
4. Archives the **entire package** (with `package.json` + `dist/`) into
`twenty-client-sdk.zip`
5. Uploads the single archive to S3 under
`FileFolder.GeneratedSdkClient`
6. Sets `isSdkLayerStale = true` on the `ApplicationEntity` in the
database

## 3. Invalidation Signal

The `isSdkLayerStale` boolean column on `ApplicationEntity` is the
invalidation mechanism:
- **Set to `true`** by `generateAndStore()` after uploading a new client
archive
- **Checked** by both logic function drivers before execution — if
`true`, they rebuild their local layer
- **Set back to `false`** by `markSdkLayerFresh()` after the driver has
successfully consumed the new archive

Default is `false` so existing applications without a generated client
aren't affected.

## 4a. Logic Functions — Local Driver

**`ensureSdkLayer()`** is called before every execution:
1. Checks if the local SDK layer directory exists AND `isSdkLayerStale`
is `false` → early return
2. Otherwise, cleans the local layer directory
3. Calls `downloadAndExtractToPackage()` which streams the zip from S3
directly to disk and extracts the full package into
`<tmpdir>/sdk/<workspaceId>-<appId>/node_modules/twenty-client-sdk/`
4. Calls `markSdkLayerFresh()` to set `isSdkLayerStale = false`

**At execution time**, `assembleNodeModules()` symlinks everything from
the deps layer's `node_modules/` **except** `twenty-client-sdk`, which
is symlinked from the SDK layer instead. This ensures the logic
function's `import ... from 'twenty-client-sdk/core'` resolves to the
generated client.

## 4b. Logic Functions — Lambda Driver

**`ensureSdkLayer()`** is called during `build()`:
1. Checks if `isSdkLayerStale` is `false` and an existing Lambda layer
ARN exists → early return
2. Otherwise, deletes all existing layer versions for this SDK layer
name
3. Calls `downloadArchiveBuffer()` to get the raw zip from S3 (no disk
extraction)
4. Calls `reprefixZipEntries()` which streams the zip entries into a
**new zip** with the path prefix
`nodejs/node_modules/twenty-client-sdk/` — this is the Lambda layer
convention path. All done in memory, no disk round-trip
5. Publishes the re-prefixed zip as a new Lambda layer via
`publishLayer()`
6. Calls `markSdkLayerFresh()`

**At function creation**, the Lambda is created with **two layers**:
`[depsLayerArn, sdkLayerArn]`. The SDK layer is listed last so it
overwrites the stub `twenty-client-sdk` from the deps layer (later
layers take precedence in Lambda's `/opt` merge).

## 5. Front Components

Front components are built by `app:build` with `twenty-client-sdk/core`
and `twenty-client-sdk/metadata` as **esbuild externals**. The stored
`.mjs` in S3 has unresolved bare import specifiers like `import {
CoreApiClient } from 'twenty-client-sdk/core'`.

SDK import resolution is split between the **frontend host** (fetching &
caching SDK modules) and the **Web Worker** (rewriting imports):

**Server endpoints**:
- `GET /rest/front-components/:id` —
`FrontComponentService.getBuiltComponentStream()` returns the **raw
`.mjs`** directly from file storage. No bundling, no SDK injection.
- `GET /rest/sdk-client/:applicationId/:moduleName` —
`SdkClientController` reads a single file (e.g. `dist/core.mjs`) from
the generated SDK archive via
`SdkClientGenerationService.readFileFromArchive()` and serves it as
JavaScript.

**Frontend host** (`FrontComponentRenderer` in `twenty-front`):
1. Queries `FindOneFrontComponent` which returns `applicationId`,
`builtComponentChecksum`, `usesSdkClient`, and `applicationTokenPair`
2. If `usesSdkClient` is `true`, renders
`FrontComponentRendererWithSdkClient` which calls the
`useApplicationSdkClient` hook
3. `useApplicationSdkClient({ applicationId, accessToken })` checks the
Jotai atom family cache for existing blob URLs. On cache miss, fetches
both SDK modules from `GET /rest/sdk-client/:applicationId/core` and
`/metadata`, creates **blob URLs** for each, and stores them in the atom
family
4. Once the blob URLs are cached, passes them as `sdkClientUrls`
(already blob URLs, not server URLs) to `SharedFrontComponentRenderer` →
`FrontComponentWorkerEffect` → worker's `render()` call via
`HostToWorkerRenderContext`

**Worker** (`remote-worker.ts` in `twenty-sdk`):
1. Fetches the raw component `.mjs` source as text
2. If `sdkClientUrls` are provided and the source contains SDK import
specifiers (`twenty-client-sdk/core`, `twenty-client-sdk/metadata`),
**rewrites** the bare specifiers to the blob URLs received from the host
(e.g. `'twenty-client-sdk/core'` → `'blob:...'`)
3. Creates a blob URL for the rewritten source and `import()`s it
4. Revokes only the component blob URL after the module is loaded — the
SDK blob URLs are owned and managed by the host's Jotai cache

This approach eliminates server-side esbuild bundling on every request,
caches SDK modules per application in the frontend, and keeps the
worker's job to a simple string rewrite.

## Summary Diagram

```
app:build (SDK)
  └─ twenty-client-sdk stub (metadata=real, core=stub)
       │
       ▼
WorkspaceMigrationRunnerService.run()
  └─ SdkClientGenerationService.generateAndStore()
       ├─ Copy stub package (package.json + dist/)
       ├─ replaceCoreClient() → regenerate core.mjs/core.cjs
       ├─ Zip entire package → upload to S3
       └─ Set isSdkLayerStale = true
              │
     ┌────────┴────────────────────┐
     ▼                             ▼
Logic Functions               Front Components
     │                             │
     ├─ Local Driver               ├─ GET /rest/sdk-client/:appId/core
     │   └─ downloadAndExtract     │    → core.mjs from archive
     │      → symlink into         │
     │        node_modules         ├─ Host (useApplicationSdkClient)
     │                             │    ├─ Fetch SDK modules
     └─ Lambda Driver              │    ├─ Create blob URLs
         └─ downloadArchiveBuffer  │    └─ Cache in Jotai atom family
            → reprefixZipEntries   │
            → publish as Lambda    ├─ GET /rest/front-components/:id
              layer                │    → raw .mjs (no bundling)
                                   │
                                   └─ Worker (browser)
                                        ├─ Fetch component .mjs
                                        ├─ Rewrite imports → blob URLs
                                        └─ import() rewritten source
```

## Next PR
- Estimate perf improvement by implementing a redis caching for front
component client storage ( we don't even cache front comp initially )
- Implem frontent blob invalidation sse event from server

---------

Co-authored-by: Charles Bochet <charlesBochet@users.noreply.github.com>
2026-03-24 18:10:25 +00:00
Paul Rastoin 75bb3a904d [SDK] Refactor clients (#18433)
# Intoduction

Closes https://github.com/twentyhq/core-team-issues/issues/2289

In this PR all the clients becomes available under `twenty-sdk/clients`,
this is a breaking change but generated was too vague and thats still
the now or never best timing to do so

## CoreClient
The core client is now shipped with a default stub empty class for both
the schema and the client
Allowing its import, will still raises typescript errors when consumed
as generated but not generated

## MetadataClient
The metadata client is workspace agnostic, it's now generated and
commited in the repo. added a ci that prevents any schema desync due to
twenty-server additions

Same behavior than for the twenty-front generated graphql schema
2026-03-09 15:32:13 +00:00
martmull 0091ef5f6c Sync built files (#17379)
as title, upload built files to local storage

---------

Co-authored-by: Charles Bochet <charles@twenty.com>
2026-01-23 13:43:53 +01:00
Charles Bochet 351e2030ff Rework watcher (#17284)
Heavy Refactoring of the watcher, sorry about this one, I'll keep
iterating on it.

In a nutshell:
- app-dev.ts is maintaining 3 watchers in parallel: manifest, function
and frontComponent
2026-01-20 17:25:44 +01:00
martmull 3958664e87 Move assets folder at root level (#17238)
fix assets management
2026-01-19 16:01:54 +01:00
Charles Bochet e61b92132c Rework folder structure (#17230)
<img width="700" height="771" alt="image"
src="https://github.com/user-attachments/assets/b784fbe6-f976-4c9a-84c1-3d90a9010665"
/>

Second one :)
2026-01-19 14:08:05 +00:00
martmull e498367e2f Merge twenty-cli into twenty-sdk (#16150)
- Moves twenty-cli content into twenty-sdk
- add a new twenty-sdk:0.1.0 version
- this new twenty-sdk exports a cli command called 'twenty' (like
twenty-cli before)
- deprecates twenty-cli
- simplify app init command base-project
- use `twenty-sdk:0.1.0` in base project
- move the "twenty-sdk/application" barrel to "twenty-sdk"
- add `create-twenty-app` package

<img width="1512" height="919" alt="image"
src="https://github.com/user-attachments/assets/007bef45-4e71-419a-9213-cebed376adbf"
/>

<img width="1506" height="929" alt="image"
src="https://github.com/user-attachments/assets/3de2fec6-1624-4923-ae13-f4e1cf165eb5"
/>
2025-12-01 11:44:35 +01:00