80fb91c033dfd367f17ba58e2095526faa016e70
66 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
|
|
997b2c38de |
Add cookie-session integration test suite (#23715)
Stacked on #23642. Integration suite for the cookie-session surface, organized as one successful/failing spec pair per stage of the session lifecycle. 14 spec files, ~36 tests, all over real HTTP against the booted app. ## Coverage by stage **1. Session creation on auth exchanges** (`successful-`/`failing-session-creation`) Flag gating (default off: tokens, no cookie, no row); httpOnly cookie snapshot with 180d expiry window; SHA-256 hash-at-rest with the row bound to the apple seed workspace; scripted sign-ins without an Origin header still get the cookie; login-CSRF refuses the cookie for disallowed origins while returning the token pair; sign-in over an existing session revokes it as `SUPERSEDED`; a failed credentials exchange mints nothing. **2. Cookie delivery** (`successful-session-cookie-delivery`, `secure-deployment-session-cookie`) The runtime side door (`AUTH_COOKIE_SAME_SITE=none` forces the secure path) pins the `__Host-`/`Secure`/`SameSite=None` variant in the default CI run. The exact production combination (`__Host-`, `Secure`, `SameSite=Lax`) is covered by a dedicated spec that requires the app to boot with an https `SERVER_URL`: the secure branch is decided by config, never the transport, so no TLS is needed. It skips itself on plain-http boots; CI runs it as an extra step on one shard with `SERVER_URL=https://localhost:3000`, including the `__Host-` round-trip and the plain-cookie-name downgrade refusal. **3. Per-request authentication and the CSRF read gate** (`successful-`/`failing-session-cookie-authentication`) A cookie-only request resolves the seeded user; a `sess_` token presented as Bearer is rejected; cookie-authenticated unsafe requests with a disallowed or missing Origin get 403 `CSRF_ORIGIN_MISMATCH`; an unknown session token is unauthenticated and its dead cookie is cleared. **3b. Workspace binding** (`successful-session-workspace-binding`) Tim signs into both seeded workspaces (apple and yc); each session row is bound to the workspace its exchange selected (`workspaceId` and `userWorkspaceId` pinned to the seed ids), and each cookie resolves to its own workspace context, with no request-side input able to pivot a session across workspaces. **3c. Credentialed CORS** (`cors-credentialed-origins`) Allowlisted origins get the reflected `Access-Control-Allow-Origin` plus `Access-Control-Allow-Credentials: true` and `Vary: Origin`, preflight included; other origins keep the public wildcard. See tooling notes: this surface was previously untestable. **4. Sessions API** (`successful-`/`failing-user-sessions-api`) `currentUserSessions` marks exactly the presented session as current; `revokeUserSession` revokes by id (`USER_REVOKED`) and drops it from the listing; `revokeAllOtherUserSessions` spares the presented session; cross-user revocation and unauthenticated listing are refused. **5. Exits** (`successful-sign-out`, `failing-session-expiration`) `signOut` revokes with `USER_SIGN_OUT`, clears the cookie, and reuse fails immediately (cache invalidated, not TTL-bound); a cookie-less sign-out clears nothing, so a cross-site POST cannot log a visitor out; absolute-lifetime and idle-timeout expiry both reject and clear the cookie. **7. Cleanup cron** (`user-session-cleanup-cron`) Both halves run in-process against fixtures spanning the 30d retention boundary. Sessions: expired/revoked-beyond-retention deleted; active, recently-expired, and idle-expired rows survive (the idle case pins the known predicate gap). Refresh tokens: old-expired and old-revoked deleted, fresh kept, and a long-expired token of another type survives, pinning the `type` filter that keeps the shared `appToken` table safe from the hard-delete. Not covered here by design: the impersonation park/restore sub-funnel (stage 6, follow-up) and the client-side funnel (stage 8, front-end scope). Password-change revocation and the renewal bridge are also left to follow-ups. ## How the flag is flipped `AUTH_COOKIE_SESSIONS_ENABLED` (and `AUTH_COOKIE_SAME_SITE` for the secure side door) are toggled at runtime through the admin panel config API, reusing the `twenty-config` test utils: `DatabaseConfigDriver.set` updates its cache synchronously and `TwentyConfigService` consults the DB driver before the env driver. No `.env.test` change, no app reboot, runs in the default CI environment without the `ci:auth-cookie-sessions` label. `SERVER_URL` is env-only, hence the dedicated CI step for the production secure-deployment spec. ## Shared tooling changes - **`applyCredentialedCors` extraction (src change)**: the integration harness booted with Nest's wildcard `cors: true`, not the credentialed-allowlist setup living in `main.ts`, so the CORS surface was untestable by construction. The setup moved into `applyCredentialedCors`, now called by both the production bootstrap and `createApp`, making the harness's CORS behavior the deployed one. Behavior-neutral for production. - `makeMetadataAPIRequest` accepts an explicit `null` token for unauthenticated requests. Passing `undefined` silently fell back to the default admin token (parameter defaults apply to `undefined`), which made supposedly public requests Bearer-authenticated, bypassing both the cookie auth path and the CSRF middleware. Existing call sites are unaffected. - The `GetLoginTokenFromCredentials` / `GetAuthTokensFromLoginToken` documents moved into shared query factories; the workspace-origin builder is extracted and generalized to any seeded subdomain (`buildWorkspaceOriginForSubdomain`, reused by `getAccessTokenForCredentials`). - Suite-local helpers: `signInWithCookieCapture` (full credentials exchange returning the raw supertest response, with a `workspaceSubdomain` option), `postMetadataOperationWithHeaders` (Origin/Cookie header control), cookie extraction for both cookie names, clearing-cookie detection, snapshot normalization (token and expiry redacted), and shared `ALLOWED_ORIGIN`/`DISALLOWED_ORIGIN` constants derived from `FRONTEND_URL`. Verified locally: full suite green in CI mode on both plain-http and https-`SERVER_URL` boots; oxlint and tsc clean. --------- Co-authored-by: Félix Malfait <felix.malfait@gmail.com> |
||
|
|
6e1e98f4ab |
fix(server): shard the server-test unit job to stop the intermittent crash (#23009)
## Problem `server-test` fails intermittently: exit 1 with **no `FAIL` line and no `Test Suites:/Tests:` summary** — the jest run is aborted mid-way, before the reporter's `onRunComplete`. ## Root cause The `test` target runs the entire unit suite (~6,600 tests) in **one in-band jest process on a single runner VM** (`nx.json` sets `maxWorkers: 1` for the `ci` configuration). A few minutes in, that process is killed by an **external `SIGKILL`** — confirmed *not* OOM (~15 GB free at kill time, no cgroup `oom_kill`) and *not* an in-process crash (a Node diagnostic report armed with `--report-on-fatalerror` + `--report-uncaught-exception` writes nothing). The whole-run kill is why it fails intermittently with no summary. `maxWorkers=2` on one VM still dies, so the threshold is **per-VM**, not per-process. ## Fix Shard the unit suite across VMs, the same way `server-integration-test` already does: - A `twenty-server` `test:ci` target runs jest directly (so `--shard` forwards) with `dependsOn: ["^build"]` so the workspace deps are built. - `server-test` becomes a 4-way matrix; each shard runs a quarter of the suite, well under the kill threshold. - `ci-server-status-check` already aggregates `server-test`, so required checks are unchanged. Also provides two mocks a completed run needs but the SIGKILL had been masking in `ApplicationRegistrationService.upsertFromCatalog` unit tests: the `MetricsService` provider and `applicationRegistrationRepository.createQueryBuilder`. |
||
|
|
97871131a1 |
fix(server): mitigate integration-test OOM flakiness (#21588)
## Problem `server-integration-test` shards have been failing intermittently across unrelated PRs with a distinctive signature: the shard exits code 1 with **no jest assertion failure, no `Test Suites:` summary, and no V8 `JavaScript heap out of memory` error** — the process just dies mid-run. Failures hit random shards and clear on re-run (e.g. an unrelated branch failed shard 6 once, then passed 3× on identical code), while the `merge_group` gate stays green. ### Root cause Each shard runs a **single in-band jest process** that boots one shared NestJS app (`globalSetup` → `app.listen`) and holds it for the entire shard, driving heavy metadata migrations + cache rebuilds in that one process. `NODE_OPTIONS=--max-old-space-size=12288` let V8 grow to 12 GB — *above* the `ubuntu-latest` runner's available RAM (16 GB, shared with Postgres/Redis/ClickHouse). V8 therefore deferred aggressive GC and grew past physical memory, so the **OS OOM-killer killed the process before V8 hit its own ceiling** — which is why there's no heap error and no jest summary, just a silent exit. ## Changes (CI/test-only — prod runtime untouched) - **Lower the integration jest heap cap `12288` → `6144`** so V8 self-limits below physical headroom instead of being OS-killed. Counterintuitively safer: a real leak now surfaces as a *visible* heap error naming the test, rather than a silent death. (`database:reset` keeps 12288 — it runs alone, before jest.) - **Add `--logHeapUsage`** to the integration jest runs to expose the per-file heap trend for confirming/pinpointing the growth. - **Split integration tests across 16 shards (was 10)** to lower the peak working set per shard. - **Make perf logging a first-class `LoggerService` tool** (per @prastoin's review): add `LoggerService.perf()` and unify the existing `time()`/`timeEnd()` helpers into `perfTime()`/`perfTimeEnd()` (now routed through the driver), all gated by a new `PERF_LOG_ENABLED` config var. It **defaults on** so real environments keep emitting the install-perf logs, and `.env.test` sets it `false` to mute the per-action flood in integration tests. The `application-manifest` and `validate-build` services were moved from the built-in `Logger` to `LoggerService` to use it. ## Notes - `--max-old-space-size` lives only in the `test:integration` nx target; it is **not** the prod server heap setting, so prod is unaffected. - This is mitigation. If `--logHeapUsage` shows monotonic growth across files, there's a real accumulation in the long-lived app (retained flat-maps / metadata cache) worth a follow-up heap-snapshot fix. <!-- This is an auto-generated description by cubic. --> <a href="https://cubic.dev/pr/twentyhq/twenty/pull/21588?utm_source=github" target="_blank" rel="noopener noreferrer" data-no-image-dialog="true"><picture><source media="(prefers-color-scheme: dark)" srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source media="(prefers-color-scheme: light)" srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img alt="Review in cubic" src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a> <!-- End of auto-generated description by cubic. --> |
||
|
|
a84a4c1ab7 |
fix(server): load integration jest config transpile-only; drop tsx pin (#21563)
## Context Follow-up to [#21559](https://github.com/twentyhq/twenty/pull/21559) (the esbuild 0.28.1 security bump). That PR had to pin `tsx` to `4.21.0` to avoid a CI-only `server-integration-test` failure. This removes the need for that pin by fixing the root cause. ## Root cause The integration-test command boots jest with `NODE_OPTIONS="--import tsx/esm"`, while jest *also* compiles `jest-integration.config.ts` with **ts-node, type-checking on**. Two TypeScript transformers run over the same file: - tsx's loader transpiles `node-environment.interface.ts` via esbuild, downleveling the enum to `var NodeEnvironment = (…)(NodeEnvironment || {})`. - jest's ts-node then *type-checks that downleveled output* and rejects it with `TS7022: 'NodeEnvironment' … referenced directly or indirectly in its own initializer`. It's not a real type error and not esbuild's fault — esbuild's output is valid JS, just not valid TS to re-type-check. It only surfaced once `tsx` resolved to `4.22.x` (whose loader feeds that output into ts-node), which is why #21559 pinned tsx to 4.21.0. Verified in isolation: ts-node type-checking esbuild's downleveled enum → `TS7022`; the same under `transpileOnly`/`TS_NODE_TRANSPILE_ONLY=true` → clean. ## Fix Run the integration jest config **transpile-only** (`TS_NODE_TRANSPILE_ONLY=true` on the `test:integration` target, base + `with-db-reset`). The config file doesn't need type-checking at boot, and jest's ts-node now emits JS without re-type-checking esbuild's output — eliminating the whole class of tsx/esbuild-downleveling sensitivity. With the collision gone, drop the workaround from the root `package.json`: - removed the `tsx: 4.21.0` resolution - removed the `tsx/esbuild: 0.28.1` resolution `tsx`'s `^4.x` ranges now resolve to **4.22.4**, which pins esbuild `~0.28.0` → **0.28.1** on its own, so esbuild stays 0.28.1 across the lockfile with no resolution. The `//resolutions` doc block is updated accordingly. ## Verification - `yarn install` clean; lockfile has only esbuild 0.28.1; tsx resolves to 4.22.4. - `jest --config ./jest-integration.config.ts --listTests` with tsx 4.22.4 + `TS_NODE_TRANSPILE_ONLY=true` loads the config and lists all 420 suites. - CI `server-integration-test` is the real validator (the failure was CI-only). <!-- This is an auto-generated description by cubic. --> <a href="https://cubic.dev/pr/twentyhq/twenty/pull/21563?utm_source=github" target="_blank" rel="noopener noreferrer" data-no-image-dialog="true"><picture><source media="(prefers-color-scheme: dark)" srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source media="(prefers-color-scheme: light)" srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img alt="Review in cubic" src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a> <!-- End of auto-generated description by cubic. --> |
||
|
|
323e66433e |
lint: migrate prettier to oxfmt (#20783)
Most changes are `implements` being unwrapped this is not a oxfmt regression Prettier in 3.7 (we're on 3.1) changed this behaviour prettier blog [post](https://prettier.io/blog/2025/11/27/3.7.0#change-18094) This unifies our linting tooling --------- Co-authored-by: github-actions <github-actions@twenty.com> Co-authored-by: Charles Bochet <charles@twenty.com> |
||
|
|
a3b0a34207 |
Fix lint:diff-with-main oxlint rules build dependency (#20389)
## Summary Closes #20382. `lint:diff-with-main` can load `.oxlintrc.json` files that reference `../twenty-oxlint-rules/dist/oxlint-plugin.mjs`, but the diff-lint targets did not build `twenty-oxlint-rules` first. On fresh clones, that generated plugin file is missing and oxlint fails before linting. This PR adds `twenty-oxlint-rules:build` before diff lint for: - the root `lint:diff-with-main` target default - the custom `twenty-front:lint:diff-with-main` target - the custom `twenty-server:lint:diff-with-main` target It also adds regression coverage for: - the default diff-lint target dependency - the custom front/server diff-lint target dependencies - preserving `twenty-website-new` custom dependencies because it does not load the Twenty oxlint plugin ## Tests - `npx vitest run --config packages/twenty-oxlint-rules/vitest.config.mts workspace/lint-diff-with-main-targets.spec.ts` - `node_modules/.bin/nx test twenty-oxlint-rules` - `node_modules/.bin/nx typecheck twenty-oxlint-rules` - `node_modules/.bin/nx build twenty-oxlint-rules` - `node_modules/.bin/nx lint:diff-with-main twenty-server` - `node_modules/.bin/nx lint:diff-with-main twenty-front` - `npx oxlint -c packages/twenty-oxlint-rules/.oxlintrc.json packages/twenty-oxlint-rules/workspace/lint-diff-with-main-targets.spec.ts` - `git diff --check` ## Notes - `twenty-website-new:lint:diff-with-main` dependency shape remains unchanged. Full local execution is blocked by missing local `unzip`, which the existing `check-lottie-frames` script requires. ## Docs / Changelog No docs or manual changelog update needed. This fixes Nx task wiring for an existing documented command. --------- Co-authored-by: Charles Bochet <charles@twenty.com> |
||
|
|
88394c25ef |
Bump twenty current version (#20241)
# Introduction This PR introduces a workflow and nx command that allow bumping to a given version or incrementing the current `TWENTY_CURRENT_VERSION` Combined with accurate on point cd triggered and CI upgrade sequence guard mutation workflow the window where a PR can corrupt an already released twenty version is mitigated |
||
|
|
7dfc556250 |
refactor messaging jobs (#19626)
Cleans up the code quality by migrating from Raw SQL to TypeORM entities. The previous implementation was necessary to do cross‑schema table joins but since we've migrated to the core schema we don't need it anymore. - Also extracted `toIsoStringOrNull` to a utility it was duplicated several times - Moved `isThrottled` logic from job handler to cron enqueuer |
||
|
|
21142d98fe |
Implement cross version upgrade (#19559)
# Introduction Refactoring the upgrade engine to handle cross version upgrade, completely getting rid of the semver `version` at db and runtime level It remains a visual a listing indicator for or CD process but also during devenv in order to prepare next release Will write a release process runbook documentation on how to handle upgrade step patch, command insertion etc as it needs to be cascaded across all the involved supported version **The upgrade sequence model:** The sequence is a flat, ordered array of upgrade steps (`UpgradeStep[]`), built from the registry by chaining all versions in order, each version contributing its fast-instance → slow-instance → workspace commands sorted by timestamp. Version is metadata for logging, not used in the algorithm. **Segments:** The sequence naturally splits into alternating segments of contiguous instance steps and contiguous workspace steps. The runner processes segments in order: - **Instance segment:** Run sequentially from the instance cursor. Each step runs once globally. - **Workspace segment:** Each workspace independently walks from its own cursor through the end of the segment. Workspaces are independent within a segment — they can be at different positions. - **Synchronization (workspace → instance):** The runner blocks before entering an instance segment. All active/suspended workspaces must have completed the last workspace step of the preceding workspace segment. If any workspace failed, abort. This is the only explicit synchronization point. - Instance → workspace ordering is implicit — the runner processes segments sequentially, so the instance segment naturally completes before the workspace segment begins. full docs https://gist.github.com/prastoin/e62106d455fd72d6b6ebada8351e5492 ## Version constants & type-level deprecation Version management is split into three atomic constants: `TWENTY_PREVIOUS_VERSIONS`, `TWENTY_CURRENT_VERSION`, and `TWENTY_NEXT_VERSIONS`. Two derived constants compose them: `CROSS_UPGRADE_SUPPORTED_VERSIONS` (previous + current — what the engine runs) and `ALL_TWENTY_VERSIONS` (the full ordered tuple including next). The registry service validates at module init that no version is duplicated across constants and that at least one previous version exists. A `DeprecatedSinceVersion<RemoveAtVersion, T>` type utility resolves to `T` while `TWENTY_CURRENT_VERSION` is below `RemoveAtVersion`, and to `never` once it reaches it — turning deprecation into a compile-time guarantee via `IndexOf` and `IsGreaterOrEqual` generics in `twenty-shared`. ### `workspace.version` column deprecation The column is replaced by cursor-based state inference from `UpgradeMigration` records, but cannot be dropped in 1.22: workspaces activated during 1.21 predate the cursor system and need their initial cursor backfilled first (`backfillWorkspaceCreatedIn1_21_0Cursors`). This backfill itself depends on a new `isInitial` column on `UpgradeMigration`, bootstrapped via a targeted TypeORM migration before the upgrade sequence runs. Both functions and the entity field are typed with `DeprecatedSinceVersion<'1.23.0', ...>`. When `TWENTY_CURRENT_VERSION` reaches `1.23.0`, compile errors force their removal — and the pre-declared `DropWorkspaceVersionColumnFastInstanceCommand` takes over to drop the column. ## What's next - ci cross version upgrade ( wip ) - banner asking to contact twenty administrator if workspace is outdated - upgrade healthcheck cli ## New unit/integ test pattern Create a dedicated `createNestApp` that consumes a real database in order not to have to mack any database interaction to the `upgradeMigrations` allowing full coverage of the whole `upgradeRunnerService.run` core logic |
||
|
|
5a22cc88c5 |
database:reset depends on database:init that runs slow instance commands (#19557)
```ts
Nest] 46347 - 04/10/2026, 3:37:43 PM LOG [InstanceLoader] LogicFunctionModule dependencies initialized
[Nest] 46347 - 04/10/2026, 3:37:43 PM LOG [InstanceLoader] ApplicationUpgradeModule dependencies initialized
[Nest] 46347 - 04/10/2026, 3:37:43 PM LOG [InstanceLoader] WorkspaceModule dependencies initialized
[Nest] 46347 - 04/10/2026, 3:37:43 PM LOG [InstanceLoader] IteratorActionModule dependencies initialized
[Nest] 46347 - 04/10/2026, 3:37:43 PM LOG [InstanceLoader] DelayActionModule dependencies initialized
[Nest] 46347 - 04/10/2026, 3:37:43 PM LOG [InstanceLoader] RestApiCoreModule dependencies initialized
[Nest] 46347 - 04/10/2026, 3:37:43 PM LOG [InstanceLoader] RecordCRUDActionModule dependencies initialized
[Nest] 46347 - 04/10/2026, 3:37:43 PM LOG [InstanceLoader] WorkflowTriggerModule dependencies initialized
[Nest] 46347 - 04/10/2026, 3:37:43 PM LOG [InstanceLoader] TimelineMessagingModule dependencies initialized
[Nest] 46347 - 04/10/2026, 3:37:43 PM LOG [InstanceLoader] WorkflowVersionModule dependencies initialized
[Nest] 46347 - 04/10/2026, 3:37:43 PM LOG [InstanceLoader] UserModule dependencies initialized
[Nest] 46347 - 04/10/2026, 3:37:43 PM LOG [InstanceLoader] WorkflowToolsModule dependencies initialized
[Nest] 46347 - 04/10/2026, 3:37:43 PM LOG [InstanceLoader] ApplicationOAuthModule dependencies initialized
[Nest] 46347 - 04/10/2026, 3:37:43 PM LOG [InstanceLoader] BillingWebhookModule dependencies initialized
[Nest] 46347 - 04/10/2026, 3:37:43 PM LOG [InstanceLoader] AiAgentExecutionModule dependencies initialized
[Nest] 46347 - 04/10/2026, 3:37:43 PM LOG [InstanceLoader] AiAgentActionModule dependencies initialized
[Nest] 46347 - 04/10/2026, 3:37:43 PM LOG [InstanceLoader] ToolProviderModule dependencies initialized
[Nest] 46347 - 04/10/2026, 3:37:43 PM LOG [InstanceLoader] AiAgentMonitorModule dependencies initialized
[Nest] 46347 - 04/10/2026, 3:37:43 PM LOG [InstanceLoader] WorkflowExecutorModule dependencies initialized
[Nest] 46347 - 04/10/2026, 3:37:43 PM LOG [InstanceLoader] WorkflowRunnerModule dependencies initialized
[Nest] 46347 - 04/10/2026, 3:37:43 PM LOG [InstanceLoader] McpModule dependencies initialized
[Nest] 46347 - 04/10/2026, 3:37:43 PM LOG [InstanceLoader] AiChatModule dependencies initialized
[Nest] 46347 - 04/10/2026, 3:37:43 PM LOG [InstanceLoader] WorkflowApiModule dependencies initialized
[Nest] 46347 - 04/10/2026, 3:37:43 PM LOG [InstanceLoader] AuthModule dependencies initialized
[Nest] 46347 - 04/10/2026, 3:37:43 PM LOG [DatabaseConfigDriver] [INIT] Loading initial config variables from database
[Nest] 46347 - 04/10/2026, 3:37:43 PM LOG [DatabaseConfigDriver] [INIT] Config variables loaded: 1 values found in DB, 83 falling to env vars/defaults
[Nest] 46347 - 04/10/2026, 3:37:43 PM LOG [UpgradeCommandRegistryService] Registered 3 fast instance, 0 slow instance, and 14 workspace command(s) for 1.21.0
[Nest] 46347 - 04/10/2026, 3:37:43 PM LOG [UpgradeCommandRegistryService] Registered 4 fast instance, 1 slow instance, and 3 workspace command(s) for 1.22.0
[Nest] 46347 - 04/10/2026, 3:37:43 PM LOG [GenerateInstanceCommandCommand] Generating fast instance command for version 1.22.0...
[Nest] 46347 - 04/10/2026, 3:37:43 PM WARN [GenerateInstanceCommandCommand] No changes in database schema were found - cannot generate a migration.
———————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————
NX Successfully ran target database:migrate:generate for project twenty-server and 8 tasks it depends on (11s)
With additional flags:
--name=martmull
```
|
||
|
|
e411f076f1 |
Replace typeorm binary by database:migrate:generate (#19515)
|
||
|
|
23874848a4 |
Instance commands and upgrade_migrations table (#19356)
# Introduction Now only using typeorm to generate migrations up and down statement We handle and maintain our own migration table history ## What's new Now all the instance commands will live within the same module and folder than the upgrade commands Sequentiality comes from the timestamp located in the filename Same sequentiality also applies to the workspace commands in the future, for the moment still expected a as code explicit declaration ( below screen is an example see below section ) <img width="1382" height="634" alt="image" src="https://github.com/user-attachments/assets/5610a246-4eae-485e-99f4-98fb89ad5ac8" /> ## Existing 1.21 migrations We won't start following this pattern in 1.21 yet at least not with the migration that has already been released as typeorm migrations in cloud production as they would rerun ## Small duplication Duplicating the legacy typeorm and instance commands run in the `run-instance-commands` to avoid any merge of interest for the moment ## Concurrency Not handling any run in parrallel of the upgrade for the moment |
||
|
|
e062343802 |
Refactor typeorm migration lifecycle and generation (#19275)
# Introduction Typeorm migration are now associated to a given twenty-version from the `UPGRADE_COMMAND_SUPPORTED_VERSIONS` that the current twenty core engine handles This way when we upgrade we retrieve the migrations that need to be run, this will be useful for the cross-version incremental upgrade so we preserve sequentiality ## What's new To generate ```sh npx nx database:migrate:generate twenty-server -- --name add-index-to-users ``` To apply all ```sh npx nx database:migrate twenty-server ``` ## Next Introduce slow and fast typeorm migration in order to get rid of the save point pattern in our code base Create a clean and dedicated `InstanceUpgradeService` abstraction |
||
|
|
281bb6d783 |
Guard yarn database:migrate:prod (#19008)
## Motivations A lot of self hosters hands up using the `yarn database:migrated:prod` either manually or through AI assisted debug while they try to upgrade an instance while their workspace is still blocked in a previous one Leading to their whole database permanent corruption ## What happened Replaced the direct call the the typeorm cli to a command calling it programmatically, adding a layer of security in case a workspace seems to be blocked in a previous version than the one just before the one being installed ( e.g 1.0 when you try to upgrade from 1.1 to 1.2 ) For our cloud we still need a way to bypass this security explaining the -f flag ## Remark Centralized this logic and refactored creating new services `WorkspaceVersionService` and `CoreEngineVersionService` that will become useful for the upcoming upgrade refactor Related to https://github.com/twentyhq/twenty-infra/pull/529 |
||
|
|
4ea2e32366 |
Refactor twenty client sdk provisioning for logic function and front-component (#18544)
## 1. The `twenty-client-sdk` Package (Source of Truth)
The monorepo package at `packages/twenty-client-sdk` ships with:
- A **pre-built metadata client** (static, generated from a fixed
schema)
- A **stub core client** that throws at runtime (`CoreApiClient was not
generated...`)
- Both ESM (`.mjs`) and CJS (`.cjs`) bundles in `dist/`
- A `package.json` with proper `exports` map for
`twenty-client-sdk/core`, `twenty-client-sdk/metadata`, and
`twenty-client-sdk/generate`
## 2. Generation & Upload (Server-Side, at Migration Time)
**When**: `WorkspaceMigrationRunnerService.run()` executes after a
metadata schema change.
**What happens in `SdkClientGenerationService.generateAndStore()`**:
1. Copies the stub `twenty-client-sdk` package from the server's assets
(resolved via `SDK_CLIENT_PACKAGE_DIRNAME` — from
`dist/assets/twenty-client-sdk/` in production, or from `node_modules`
in dev)
2. Filters out `node_modules/` and `src/` during copy — only
`package.json` + `dist/` are kept (like an npm publish)
3. Calls `replaceCoreClient()` which uses `@genql/cli` to introspect the
**application-scoped** GraphQL schema and generates a real
`CoreApiClient`, then compiles it to ESM+CJS and overwrites
`dist/core.mjs` and `dist/core.cjs`
4. Archives the **entire package** (with `package.json` + `dist/`) into
`twenty-client-sdk.zip`
5. Uploads the single archive to S3 under
`FileFolder.GeneratedSdkClient`
6. Sets `isSdkLayerStale = true` on the `ApplicationEntity` in the
database
## 3. Invalidation Signal
The `isSdkLayerStale` boolean column on `ApplicationEntity` is the
invalidation mechanism:
- **Set to `true`** by `generateAndStore()` after uploading a new client
archive
- **Checked** by both logic function drivers before execution — if
`true`, they rebuild their local layer
- **Set back to `false`** by `markSdkLayerFresh()` after the driver has
successfully consumed the new archive
Default is `false` so existing applications without a generated client
aren't affected.
## 4a. Logic Functions — Local Driver
**`ensureSdkLayer()`** is called before every execution:
1. Checks if the local SDK layer directory exists AND `isSdkLayerStale`
is `false` → early return
2. Otherwise, cleans the local layer directory
3. Calls `downloadAndExtractToPackage()` which streams the zip from S3
directly to disk and extracts the full package into
`<tmpdir>/sdk/<workspaceId>-<appId>/node_modules/twenty-client-sdk/`
4. Calls `markSdkLayerFresh()` to set `isSdkLayerStale = false`
**At execution time**, `assembleNodeModules()` symlinks everything from
the deps layer's `node_modules/` **except** `twenty-client-sdk`, which
is symlinked from the SDK layer instead. This ensures the logic
function's `import ... from 'twenty-client-sdk/core'` resolves to the
generated client.
## 4b. Logic Functions — Lambda Driver
**`ensureSdkLayer()`** is called during `build()`:
1. Checks if `isSdkLayerStale` is `false` and an existing Lambda layer
ARN exists → early return
2. Otherwise, deletes all existing layer versions for this SDK layer
name
3. Calls `downloadArchiveBuffer()` to get the raw zip from S3 (no disk
extraction)
4. Calls `reprefixZipEntries()` which streams the zip entries into a
**new zip** with the path prefix
`nodejs/node_modules/twenty-client-sdk/` — this is the Lambda layer
convention path. All done in memory, no disk round-trip
5. Publishes the re-prefixed zip as a new Lambda layer via
`publishLayer()`
6. Calls `markSdkLayerFresh()`
**At function creation**, the Lambda is created with **two layers**:
`[depsLayerArn, sdkLayerArn]`. The SDK layer is listed last so it
overwrites the stub `twenty-client-sdk` from the deps layer (later
layers take precedence in Lambda's `/opt` merge).
## 5. Front Components
Front components are built by `app:build` with `twenty-client-sdk/core`
and `twenty-client-sdk/metadata` as **esbuild externals**. The stored
`.mjs` in S3 has unresolved bare import specifiers like `import {
CoreApiClient } from 'twenty-client-sdk/core'`.
SDK import resolution is split between the **frontend host** (fetching &
caching SDK modules) and the **Web Worker** (rewriting imports):
**Server endpoints**:
- `GET /rest/front-components/:id` —
`FrontComponentService.getBuiltComponentStream()` returns the **raw
`.mjs`** directly from file storage. No bundling, no SDK injection.
- `GET /rest/sdk-client/:applicationId/:moduleName` —
`SdkClientController` reads a single file (e.g. `dist/core.mjs`) from
the generated SDK archive via
`SdkClientGenerationService.readFileFromArchive()` and serves it as
JavaScript.
**Frontend host** (`FrontComponentRenderer` in `twenty-front`):
1. Queries `FindOneFrontComponent` which returns `applicationId`,
`builtComponentChecksum`, `usesSdkClient`, and `applicationTokenPair`
2. If `usesSdkClient` is `true`, renders
`FrontComponentRendererWithSdkClient` which calls the
`useApplicationSdkClient` hook
3. `useApplicationSdkClient({ applicationId, accessToken })` checks the
Jotai atom family cache for existing blob URLs. On cache miss, fetches
both SDK modules from `GET /rest/sdk-client/:applicationId/core` and
`/metadata`, creates **blob URLs** for each, and stores them in the atom
family
4. Once the blob URLs are cached, passes them as `sdkClientUrls`
(already blob URLs, not server URLs) to `SharedFrontComponentRenderer` →
`FrontComponentWorkerEffect` → worker's `render()` call via
`HostToWorkerRenderContext`
**Worker** (`remote-worker.ts` in `twenty-sdk`):
1. Fetches the raw component `.mjs` source as text
2. If `sdkClientUrls` are provided and the source contains SDK import
specifiers (`twenty-client-sdk/core`, `twenty-client-sdk/metadata`),
**rewrites** the bare specifiers to the blob URLs received from the host
(e.g. `'twenty-client-sdk/core'` → `'blob:...'`)
3. Creates a blob URL for the rewritten source and `import()`s it
4. Revokes only the component blob URL after the module is loaded — the
SDK blob URLs are owned and managed by the host's Jotai cache
This approach eliminates server-side esbuild bundling on every request,
caches SDK modules per application in the frontend, and keeps the
worker's job to a simple string rewrite.
## Summary Diagram
```
app:build (SDK)
└─ twenty-client-sdk stub (metadata=real, core=stub)
│
▼
WorkspaceMigrationRunnerService.run()
└─ SdkClientGenerationService.generateAndStore()
├─ Copy stub package (package.json + dist/)
├─ replaceCoreClient() → regenerate core.mjs/core.cjs
├─ Zip entire package → upload to S3
└─ Set isSdkLayerStale = true
│
┌────────┴────────────────────┐
▼ ▼
Logic Functions Front Components
│ │
├─ Local Driver ├─ GET /rest/sdk-client/:appId/core
│ └─ downloadAndExtract │ → core.mjs from archive
│ → symlink into │
│ node_modules ├─ Host (useApplicationSdkClient)
│ │ ├─ Fetch SDK modules
└─ Lambda Driver │ ├─ Create blob URLs
└─ downloadArchiveBuffer │ └─ Cache in Jotai atom family
→ reprefixZipEntries │
→ publish as Lambda ├─ GET /rest/front-components/:id
layer │ → raw .mjs (no bundling)
│
└─ Worker (browser)
├─ Fetch component .mjs
├─ Rewrite imports → blob URLs
└─ import() rewritten source
```
## Next PR
- Estimate perf improvement by implementing a redis caching for front
component client storage ( we don't even cache front comp initially )
- Implem frontent blob invalidation sse event from server
---------
Co-authored-by: Charles Bochet <charlesBochet@users.noreply.github.com>
|
||
|
|
2dfa742543 |
chore: improve i18n workflow to prevent stale compiled translations (#18850)
## Summary - **Add `lingui:compile` to Dockerfile** before both the server and frontend build stages, ensuring compiled translation catalogs are always fresh regardless of git state - **Add `repository-dispatch` to i18n workflows** (`i18n-push.yaml` and `i18n-pull.yaml`) to trigger reactive automerge in `twenty-infra` when the i18n PR is ready, replacing the 15-minute polling approach ## Context Users sometimes see "Uncompiled message detected" errors because releases can be cut from `main` before the i18n PR (with freshly compiled translation catalogs) has been merged. This creates a race condition between new translatable strings landing on `main` and their compiled catalogs being available. These changes fix this in two ways: 1. **Safety net in builds**: Every Docker build now compiles translations before building, so even if compiled catalogs in git are stale, the build artifact is always correct 2. **Faster i18n PR merges**: Instead of a 15-minute cron polling for i18n PRs, the workflows now notify `twenty-infra` immediately when translations are ready, reducing merge latency from ~15 minutes to ~1 minute Companion PR in twenty-infra: twentyhq/twenty-infra (feat/i18n-reactive-automerge) ## Test plan - [ ] Verify `TWENTY_INFRA_TOKEN` secret is available to i18n workflows - [ ] Docker build still succeeds with the added `lingui:compile` steps - [ ] i18n-push triggers automerge in twenty-infra after pushing changes - [ ] i18n-pull triggers automerge in twenty-infra after pulling translations Made with [Cursor](https://cursor.com) |
||
|
|
994215e0dc |
Update store on data model mutation (#18684)
- enrich SSE events with relations - remove queries from sse metadata events - on sse event, manage store - on object/field metadata changes, manage store TODO left: - fix other metadata items --------- Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: github-actions <github-actions@twenty.com> Co-authored-by: Charles Bochet <charles@twenty.com> |
||
|
|
c9deab4373 |
[COMMAND MENU ITEMS] Remove standard front components (#18581)
All standard command menu items will link to an engine component instead of standard front components. |
||
|
|
2de022afcf |
Add standard command menu items (#18527)
## Add standard command menu items
### Summary
This PR introduces standard command menu items, migrating hardcoded
command menu actions to the backend command menu item architecture
powered by front components. It adds a new `twenty-standard-application`
package that defines, builds, and registers front components as standard
command menu items, gated behind the `IS_COMMAND_MENU_ITEM_ENABLED`
feature flag.
### Description
- **New `twenty-standard-application` package**: Contains front
component definitions with an esbuild-based build pipeline that
generates minified `.mjs` bundles and a manifest with checksums.
- **Server-side registration**: New constants register all items with
metadata (labels, icons, positions, availability types, conditional
expressions). A `StandardFrontComponentUploadService` uploads built
components to file storage.
- **`FALLBACK` availability type**: New enum value for command menu
items that appear as fallback options (e.g., "Search Records" fallback).
- **`CommandMenuContextApi` refactor**
- **Conditional availability enhancements**: New array-based helper
functions for evaluating multi-record conditions.
- **Frontend wiring** (twenty-front):
`useCommandMenuItemFrontComponentCommands`
## Next steps
Only simple commands have been implemented for now:
- **Navigation (9)** -- `CommandLink`: go-to-companies,
go-to-dashboards, go-to-notes, go-to-opportunities, go-to-people,
go-to-runs, go-to-settings, go-to-tasks, go-to-workflows
- **Side panel (4)** -- `CommandOpenSidePanelPage`: ask-ai,
search-records, search-records-fallback, view-previous-ai-chats
We still have to implement front components for all the following
commands:
All have placeholder `execute` logic (`async () => {}`) with a `// TODO:
implement execute logic` comment:
**Record (22)**
- `add-to-favorites`, `remove-from-favorites`
- `create-new-record`, `create-new-view`
- `delete-single-record`, `delete-multiple-records`
- `destroy-single-record`, `destroy-multiple-records`
- `restore-single-record`, `restore-multiple-records`
- `export-from-record-index`, `export-from-record-show`,
`export-multiple-records`, `export-note-to-pdf`, `export-view`
- `hide-deleted-records`, `see-deleted-records`
- `import-records`, `merge-multiple-records`, `update-multiple-records`
- `navigate-to-next-record`, `navigate-to-previous-record`
**Page layout (3)** -- `cancel-record-page-layout`,
`edit-record-page-layout`, `save-record-page-layout`
**Dashboard (4)** -- `cancel-dashboard-layout`, `duplicate-dashboard`,
`edit-dashboard-layout`, `save-dashboard-layout`
**Workflow (10)** -- `activate-workflow`, `add-node-workflow`,
`deactivate-workflow`, `discard-draft-workflow`, `duplicate-workflow`,
`see-active-version-workflow`, `see-runs-workflow`,
`see-versions-workflow`, `test-workflow`, `tidy-up-workflow`
**Workflow version (4)** -- `see-runs-workflow-version`,
`see-versions-workflow-version`, `see-workflow-workflow-version`,
`use-as-draft-workflow-version`
**Workflow run (3)** -- `see-version-workflow-run`,
`see-workflow-workflow-run`, `stop-workflow-run`
|
||
|
|
d37ed7e07c |
Optimize merge queue to only run E2E and integrate prettier into lint (#18459)
## Summary - **Merge queue optimization**: Created a dedicated `ci-merge-queue.yaml` workflow that only runs Playwright E2E tests on `ubuntu-latest-8-cores`. Removed `merge_group` trigger from all 7 existing CI workflows (front, server, shared, website, sdk, zapier, docker-compose). The merge queue goes from ~30+ parallel jobs to a single focused E2E job. - **Label-based merge queue simulation**: Added `run-merge-queue` label support so developers can trigger the exact merge queue E2E pipeline on any open PR before it enters the queue. - **Prettier in lint**: Chained `prettier --check` into `lint` and `prettier --write` into `lint --configuration=fix` across `nx.json` defaults, `twenty-front`, and `twenty-server`. Prettier formatting errors are now caught by `lint` and fixed by `lint:fix` / `lint:diff-with-main --configuration=fix`. ## After merge (manual repo settings) Update GitHub branch protection required status checks: 1. Remove old per-workflow merge queue checks (`ci-front-status-check`, `ci-e2e-status-check`, `ci-server-status-check`, etc.) 2. Add `ci-merge-queue-status-check` as the required check for the merge queue |
||
|
|
9d57bc39e5 |
Migrate from ESLint to OxLint (#18443)
## Summary Fully replaces ESLint with OxLint across the entire monorepo: - **Replaced all ESLint configs** (`eslint.config.mjs`) with OxLint configs (`.oxlintrc.json`) for every package: `twenty-front`, `twenty-server`, `twenty-emails`, `twenty-ui`, `twenty-shared`, `twenty-sdk`, `twenty-zapier`, `twenty-docs`, `twenty-website`, `twenty-apps/*`, `create-twenty-app` - **Migrated custom lint rules** from ESLint plugin format to OxLint JS plugin system (`@oxlint/plugins`), including `styled-components-prefixed-with-styled`, `no-hardcoded-colors`, `sort-css-properties-alphabetically`, `graphql-resolvers-should-be-guarded`, `rest-api-methods-should-be-guarded`, `max-consts-per-file`, and Jotai-related rules - **Migrated custom rule tests** from ESLint `RuleTester` + Jest to `oxlint/plugins-dev` `RuleTester` + Vitest - **Removed all ESLint dependencies** from `package.json` files and regenerated lockfiles - **Updated Nx targets** (`lint`, `lint:diff-with-main`, `fmt`) in `nx.json` and per-project `project.json` to use `oxlint` commands with proper `dependsOn` for plugin builds - **Updated CI workflows** (`.github/workflows/ci-*.yaml`) — no more ESLint executor - **Updated IDE setup**: replaced `dbaeumer.vscode-eslint` with `oxc.oxc-vscode` extension, configured `source.fixAll.oxc` and format-on-save with Prettier - **Replaced all `eslint-disable` comments** with `oxlint-disable` equivalents across the codebase - **Updated docs** (`twenty-docs`) to reference OxLint instead of ESLint - **Renamed** `twenty-eslint-rules` package to `twenty-oxlint-rules` ### Temporarily disabled rules (tracked in `OXLINT_MIGRATION_TODO.md`) | Rule | Package | Violations | Auto-fixable | |------|---------|-----------|-------------| | `twenty/sort-css-properties-alphabetically` | twenty-front | 578 | Yes | | `typescript/consistent-type-imports` | twenty-server | 3814 | Yes | | `twenty/max-consts-per-file` | twenty-server | 94 | No | ### Dropped plugins (no OxLint equivalent) `eslint-plugin-project-structure`, `lingui/*`, `@stylistic/*`, `import/order`, `prefer-arrow/prefer-arrow-functions`, `eslint-plugin-mdx`, `@next/eslint-plugin-next`, `eslint-plugin-storybook`, `eslint-plugin-react-refresh`. Partial coverage for `jsx-a11y` and `unused-imports`. ### Additional fixes (pre-existing issues exposed by merge) - Fixed `EmailThreadPreview.tsx` broken import from main rename (`useOpenEmailThreadInSidePanel`) - Restored truthiness guard in `getActivityTargetObjectRecords.ts` - Fixed `AgentTurnResolver` return types to match entity (virtual `fileMediaType`/`fileUrl` are resolved via `@ResolveField()`) ## Test plan - [x] `npx nx lint twenty-front` passes - [x] `npx nx lint twenty-server` passes - [x] `npx nx lint twenty-docs` passes - [x] Custom oxlint rules validated with Vitest: `npx nx test twenty-oxlint-rules` - [x] `npx nx typecheck twenty-front` passes - [x] `npx nx typecheck twenty-server` passes - [x] CI workflows trigger correctly with `dependsOn: ["twenty-oxlint-rules:build"]` - [x] IDE linting works with `oxc.oxc-vscode` extension |
||
|
|
0d4fe4575b |
Various SDK improvements (#18115)
## Summary - **Refactor frontend metadata loading architecture**: Split the monolithic `EagerMetadataLoadEffect` into focused provider effects (`UserMetadataProviderEffect`, `ObjectMetadataProviderEffect`, `ViewMetadataProviderEffect`) orchestrated by `MetadataProviderEffects`. Replaced `UserProvider` + `ObjectMetadataItemsProvider` with a single `MetadataGater` that gates rendering on `isAppMetadataReadyState`. The metadata store now validates view-object consistency before promoting views, and `updateDraft` skips no-op updates via deep equality checks. - **SDK CLI improvements**: Added `app:typecheck` command, improved error handling in API sync (extracts GraphQL error messages), added `serializeError` utility for human-readable error output, added `error` file status to dev mode orchestrator with UI support, and fixed ClickHouse migration/seed commands to use `transpile-only`. |
||
|
|
e6d3dc07e2 |
Fix EMFILE: too many open files, watch on macOS (#17901)
## Summary Fixes `EMFILE: too many open files, watch` crash that most of the team is hitting on macOS when running `yarn start` or `npx nx start twenty-server`. Adds `rimraf dist` before `nest start --watch` in the `start` and `start:debug` targets, so the watcher starts with a clean output directory. ## Root cause The NestJS SWC compiler (`@nestjs/cli@11`) creates **three overlapping chokidar watchers** when `nest start --watch` runs: | Watcher | Watches | Purpose | Handles | |---|---|---|---| | SWC CLI (`@swc/cli`) | `src/` | Detects file changes → recompiles | ~1,730 | | NestJS `watchFilesInSrcDir` | `src/` | Workaround: SWC misses new files | shared with above | | NestJS `watchFilesInOutDir` | **`dist/`** | Detects compiled `.js` → restarts server | **~3,548** | `@nestjs/cli@11` ships with **chokidar v4**, which dropped macOS `fsevents` support and uses `fs.watch()` instead — creating **one file descriptor per directory**. Chokidar v3 used a single `fsevents` kernel subscription per directory tree. Total: **~5,000+ `fs.watch()` handles**, far exceeding the default macOS `ulimit -n` of ~2,560. ### Why it broke now PR #17851 (`15fc850212`) changed the `start` target from `dependsOn: ["build"]` to `dependsOn: ["^build"]`, removing the `rimraf dist && nest build` pre-step. Without that cleanup, `dist/` accumulated stale directories from code reorganizations (e.g. `application-layer/` → `application/` rename), growing to ~3,548 directories vs ~1,730 in a clean build. ## What this PR does Adds `rimraf dist &&` before `nest start --watch` in the `start` and `start:debug` commands. This ensures `dist/` starts empty and only contains directories matching the current `src/` structure (~1,730), keeping watcher count in the ~3,400 range. We still get the startup speed improvement from #17851 (no redundant full SWC build), since `rimraf dist` is ~instant while the removed `nest build` step took 30-60s. ## Future considerations As the codebase grows, even a clean `dist/` will eventually approach the macOS default `ulimit -n` (~2,560). Options to consider if that happens: 1. **Yarn resolution to force chokidar 3.6.0** — restores `fsevents`, reducing watcher count from ~5,000 to ~3-5. This is what Vite 7 does internally. Simple and effective, but pins to an older major version. 2. **Patch `@nestjs/cli`** to skip the `dist/` watcher — the `watchFilesInOutDir` watcher accounts for ~65% of all handles and only exists because NestJS doesn't have a direct hook into SWC's compilation-complete event. Could be removed via `yarn patch`. 3. **Replace `nest start --watch` entirely** — use `node --watch-path=src` (Node 22+) with `@swc-node/register` for on-the-fly compilation. Uses a single native watcher regardless of directory count. Requires rethinking asset copying (`watchAssets` in `nest-cli.json`). 4. **Wait for upstream fix** — NestJS CLI should either re-add `fsevents` support or use Node's recursive `fs.watch()` option (available since Node 20) instead of per-directory watchers. ## Test plan - [ ] Run `npx nx start twenty-server` on macOS — server starts without EMFILE error - [ ] Run `npx nx start:debug twenty-server` — debug mode starts without EMFILE error - [ ] Edit a `.ts` file while server is running — hot reload still works - [ ] Run `yarn start` (frontend + backend + worker) — no crashes Made with [Cursor](https://cursor.com) Co-authored-by: Cursor <cursoragent@cursor.com> |
||
|
|
15fc850212 |
Remove redundant self-build from Nx targets that compile on the fly (#17851)
## Summary - Changed 6 Nx targets (`start`, `start:debug`, `typeorm`, `ts-node`, `database:migrate`, `database:migrate:revert`) from `dependsOn: ["build"]` to `dependsOn: ["^build"]` to eliminate a redundant full SWC compilation step (~30-60s per invocation). - These targets all use `nest start --watch`, `ts-node`, or TypeORM's CLI (which runs under ts-node), so they already compile TypeScript themselves. The `build` dependency was causing `rimraf dist && nest build` to run first, only for the target's own command to recompile everything again. - Fixed `start:debug` to call `nest start --watch --debug` directly instead of routing through `nx start --debug`, which would trigger yet another build cycle. Note: targets that run pre-compiled code from `dist/` (like `database:reset`) intentionally keep `dependsOn: ["build"]`. ## Test plan - [ ] Run `npx nx start twenty-server` and verify the server starts with only one SWC compilation pass instead of two - [ ] Run `npx nx start:debug twenty-server` and verify the debugger attaches correctly - [ ] Run `npx nx run twenty-server:database:migrate` and verify migrations run correctly - [ ] Run `npx nx run twenty-server:database:reset twenty-server` and verify it still builds before running (unchanged) Made with [Cursor](https://cursor.com) Co-authored-by: Cursor <cursoragent@cursor.com> |
||
|
|
52fe21c04b |
Fix ci + improvements (#17795)
as title |
||
|
|
8dec37b826 |
feat: migrate typecheck to tsgo for faster type checking (#17331)
## Summary - Switch `twenty-server` and `fireflies` typecheck from tsc to tsgo (~75x faster) - Enable tsgo in VSCode via `typescript.experimental.useTsgo` setting - Add `@ts-nocheck` to `remove-step.spec.ts` to work around tsgo performance issue with deep spread operations ## Performance | Package | Before (tsc) | After (tsgo) | |---------|-------------|--------------| | `twenty-server` | ~150s | ~2s | | `twenty-front` | ~40s | ~2s | ## Related - Workaround for: https://github.com/microsoft/typescript-go/issues/2551 ## Test plan - [x] `npx nx typecheck twenty-server` passes - [x] `npx nx typecheck twenty-front` passes - [x] `npx nx run-many --target=typecheck --exclude=fireflies` passes (fireflies has pre-existing type errors) |
||
|
|
59c7295033 |
Identify standard role (#17234)
# Introduction Related to https://github.com/twentyhq/core-team-issues/issues/1989 1/ Migration, applicationId and universalIdentifier are required on entity ( save point migration + upgrade command fallback pattern ) 2/ Backfill using previous standard ids ## Test tested prod extract ## Note Added build to typeorm nx generate migration command so we never forgot to build server before |
||
|
|
dc93cf4c59 |
feat: add TypeScript Go (tsgo) for faster type checking (#17211)
## Summary - Add `@typescript/native-preview` (tsgo) for dramatically faster type checking on frontend projects - Configure tsgo as default for frontend projects (twenty-front, twenty-ui, twenty-shared, etc.) - Keep tsc for twenty-server (faster for NestJS decorator-heavy code) - Fix type imports for tsgo compatibility (DOMPurify, AxiosInstance) - Remove deprecated `baseUrl` from tsconfigs where safe ## Performance Results | Project | tsgo | tsc -b | Speedup | |---------|------|--------|---------| | **twenty-front** | 1.4s | 60.7s | **43x faster** | | **twenty-server** | 2m42s | 1m10s | tsc is faster (decorators) | tsgo excels at modern React/JSX codebases but struggles with decorator-heavy NestJS backends, so we use the optimal checker for each. ## Usage ```bash # Default (tsgo for frontend, tsc for backend) nx typecheck twenty-front nx typecheck twenty-server # Force tsc fallback if needed nx typecheck twenty-front --configuration=tsc # Force tsgo on backend (slower, not recommended) nx typecheck twenty-server --configuration=tsgo ``` ## Test plan - [x] `nx typecheck twenty-front` passes with tsgo - [x] `nx typecheck twenty-server` passes with tsc - [x] `nx run-many -t typecheck --exclude=fireflies` passes - [ ] CI tests pass |
||
|
|
2c8d3f02e1 |
feat: upgrade to Storybook version 10 (#17110)
Upgraded to Storybook 10. We still use `@storybook/test-runner` for testing since it appears it'd require more work to move from Jest to Vitest than I initially anticipated, but I completed this PR to fix `storybook:serve:dev` - it takes time to load, but it works the way it used to with Storybook 8. https://github.com/user-attachments/assets/7afc32c6-4bcf-4b37-b83b-8d00d28dda15 |
||
|
|
73d2027391 |
fix: centralize lint:changed configuration in nx.json (#16877)
## Summary
The `lint:changed` command was not using the correct ESLint config for
`twenty-server`, causing it to use the root `eslint.config.mjs` instead
of the package-specific one. This PR fixes the issue and renames the
command to `lint:diff-with-main` for clarity.
## Problem
- `twenty-front` correctly specified `--config
packages/twenty-front/eslint.config.mjs`
- `twenty-server` just called `npx eslint` without specifying a config
- This meant `twenty-server` was missing important rules like:
- `@typescript-eslint/no-explicit-any: 'error'`
- `@stylistic/*` rules (linebreak-style, padding, etc.)
- `import/order` with NestJS patterns
- Custom workspace rules (`@nx/workspace-inject-workspace-repository`,
etc.)
## Solution
1. **Renamed** `lint:changed` to `lint:diff-with-main` to be explicit
about what the command does
2. **Centralized** the configuration in `nx.json` targetDefaults:
- Uses `{projectRoot}` interpolation for paths (resolved by Nx at
runtime)
- Each package automatically uses its own ESLint config
- Packages can override specific options (e.g., file extension pattern)
3. **Simplified** `project.json` files by inheriting from defaults
## Usage
```bash
# Lint only files changed vs main branch
npx nx lint:diff-with-main twenty-front
npx nx lint:diff-with-main twenty-server
# Auto-fix files changed vs main
npx nx lint:diff-with-main twenty-front --configuration=fix
```
## Changes
- **nx.json**: Added `lint:diff-with-main` target default with
configurable pattern
- **twenty-front/project.json**: Simplified to inherit from defaults
- **twenty-server/project.json**: Overrides pattern to include `.json`
files
- **CLAUDE.md** and **.cursor/rules**: Updated documentation
|
||
|
|
1088f7bbab |
feat: add lingui/no-unlocalized-strings ESLint rule and fix translations (#16610)
## Summary
This PR adds the `lingui/no-unlocalized-strings` ESLint rule to detect
untranslated strings and fixes translation issues across multiple
components.
## Changes
### ESLint Configuration (`eslint.config.react.mjs`)
- Added comprehensive `ignore` patterns for non-translatable strings
(CSS values, HTML attributes, technical identifiers)
- Added `ignoreNames` for props that don't need translation (className,
data-*, aria-*, etc.)
- Added `ignoreFunctions` for console methods, URL APIs, and other
non-user-facing functions
- Disabled rule for debug files, storybook, and test files
### Components Fixed (~19 files)
- Object record components (field inputs, pickers, merge dialogs)
- Settings components (accounts, admin panel)
- Serverless function components
- Record table and title cell components
## Status
🚧 **Work in Progress** - ~124 files remaining to fix
This PR is being submitted as draft to allow progressive fixing of
remaining translation issues.
## Testing
- Run `npx eslint "src/**/*.tsx"` in `packages/twenty-front` to check
remaining issues
|
||
|
|
0145920d7e |
e2e tests (#16533)
In this PR, - current basic E2E tests are fixed, and some were added, covering some basic scenarios - some tests avec been commented out, until we decide whether they are worth fixing The next steps are - evaluate the flakiness of the tests. Once they've proved not to be flaky, we should add more tests + re-write the current ones not using aria-label (cf @lucasbordeau indication). - We will add them back to the development flow |
||
|
|
75bba5a8ee |
Standard Agent, Role, Role target (#16499)
# Introduction In this pullrequest have been migrated to the flat standard entities: Role, Agent and RoleTargets. ## What happens - Removed createStandardMorph tool util in favor of dynamic typing of `createMorphOrRelationStandardField` - Implemented a command to remove standard agents and their role that has been removed in https://github.com/twentyhq/twenty/pull/16513 also added a default role target to data manipulator role to the only remaining agent - Implemented an agent deleteMany service handler |
||
|
|
5dfb66917c |
Upgrade NestJS from 10.x to 11.x (#15836)
## Overview This PR upgrades all NestJS dependencies from version 10.x to 11.x, following the [official migration guide](https://docs.nestjs.com/migration-guide). This builds on top of the v9 to v10 upgrade completed in PR #15835. ## Changes ### Dependencies Updated **Core packages (10.x → 11.x):** - `@nestjs/common`: 10.4.16 → 11.0.8 - `@nestjs/core`: 10.4.16 → 11.0.8 - `@nestjs/platform-express`: 10.4.16 → 11.0.8 - `@nestjs/config`: 3.2.3 → 3.3.0 - `@nestjs/passport`: 10.0.3 → 11.0.0 - `@nestjs/axios`: 3.0.2 → 3.1.2 - `@nestjs/schedule`: ^3.0.0 → ^4.1.1 - `@nestjs/serve-static`: 4.0.2 → 5.0.1 - `@nestjs/cache-manager`: ^2.2.1 → ^2.3.0 - `@nestjs/jwt`: 10.2.0 → 11.0.0 - `@nestjs/typeorm`: 10.0.2 → 11.0.0 - `@nestjs/terminus`: 11.0.0 (already on v11) - `@nestjs/event-emitter`: 2.1.0 (compatible) **DevDependencies:** - `@nestjs/testing`: ^10.4.16 → ^11.0.8 - `@nestjs/schematics`: ^10.1.0 → ^11.0.2 - `@nestjs/cli`: 10.3.0 → 11.0.0 ### Code Changes **Fixed: TwentyConfigModule conditional imports** - Updated `TwentyConfigModule.forRoot()` to use spread operator for conditional imports - Fixes TypeScript error with NestJS 11's stricter DynamicModule type checking **Cleanup: Removed unused package** - Removed `@revertdotdev/revert-react` (not being used anywhere in the codebase) ## Breaking Changes Addressed ### 1. ✅ Reflector Type Inference - **Impact**: None - codebase only uses `reflector.get()` method - **Analysis**: Does not use `getAllAndMerge()` or `getAllAndOverride()` (the methods with breaking changes) - **Files reviewed**: feature-flag.guard.ts, message-queue-metadata.accessor.ts, workspace-query-hook-metadata.accessor.ts ### 2. ✅ Lifecycle Hooks Execution Order - **Change**: Termination hooks (`OnModuleDestroy`, `BeforeApplicationShutdown`, `OnApplicationShutdown`) now execute in REVERSE order - **Analysis**: Reviewed all lifecycle hook implementations - Redis client cleanup - Database connection cleanup (GlobalWorkspaceDataSource) - BullMQ queue/worker cleanup - Cache storage cleanup - **Result**: Dependency order is safe - services using connections clean up before the connections themselves ### 3. ✅ Middleware Registration Order - **Change**: Global middleware now executes first regardless of import order - **Analysis**: Middleware is not registered as global, so execution order remains consistent - **Files reviewed**: app.module.ts, middleware.module.ts ## Testing All tests passing and build successful: **Unit Tests (283+ tests):** - ✅ Health module: 38 tests passed - ✅ Auth module: 115 tests passed (passport v11 integration) - ✅ REST API: 90 tests passed (middleware and express platform) - ✅ Feature flags: 17 tests passed (Reflector usage) - ✅ Workspace: 23 tests passed **Build & Quality:** - ✅ Type checking: Passed - ✅ Linting: Passed - ✅ Build: 3,683 files compiled successfully ## Verification Tested critical NestJS functionality: - ✅ Authentication & Security (JWT, OAuth, guards) - ✅ HTTP Platform (Express integration, REST endpoints) - ✅ Dependency Injection (Services, factories, providers) - ✅ Cache Management (Redis with @nestjs/cache-manager) - ✅ GraphQL (Query runners, resolvers) - ✅ Configuration (Environment config) - ✅ Scheduling (Cron jobs with @nestjs/schedule v4) - ✅ Lifecycle Hooks (Module initialization and cleanup) - ✅ Reflector (Metadata reflection in guards) ## Related PRs - #15835 - Upgrade NestJS from 9.x to 10.x (completed) <!-- CURSOR_SUMMARY --> --- > [!NOTE] > Upgrades NestJS to v11 and updates routing patterns, auth strategies, GraphQL schema options, and build/dist paths (scripts, Docker, Nx, migrations, assets), plus enables Devtools in development. > > - **Backend (NestJS 11 upgrade)**: > - Bump `@nestjs/*` packages (core, platform-express, jwt, passport, typeorm, serve-static, schedule, cli/testing/schematics) to v11. > - Update REST/route-trigger/file controllers to new wildcard syntax (`*path`). > - Refactor OAuth (Google/Microsoft) and SAML strategies (abstract base + explicit `validate`); minor typings. > - Enable `DevtoolsModule` in development. > - **GraphQL**: > - Add `buildSchemaOptions.orphanedTypes` for client-config types; keep Yoga/Sentry setup. > - **Build/Runtime & Config**: > - Standardize dist layout (remove `src` in paths): update scripts, Docker `CMD`, Nx `project.json`, render scripts, TypeORM migration paths, asset resolution. > - Adjust `nest-cli.json` (watchOptions, asset globs, migrations outDir, monorepo/root). > - Improve config module imports (spread conditional); tsconfig excludes `node_modules`. > - Minor Nx default: `start` target caching disabled. > > <sup>Written by [Cursor Bugbot](https://cursor.com/dashboard?tab=bugbot) for commit 1139fd85a97d0c72314d416d07464cc3c9942783. This will update automatically on new commits. Configure [here](https://cursor.com/dashboard?tab=bugbot).</sup> <!-- /CURSOR_SUMMARY --> |
||
|
|
e91b0a4b15 |
[CLI-E2E-CI] Fix dependency graph (#15167)
# Introduction Fixed the build dependency leading to twenty-server start failing before building, Removed redundant steps Make everything run on test db |
||
|
|
7c661f47fd |
e2e test environment fortwenty-cli (#15123)
# Introduction Defining very first basis of the twenty-cli e2e testing env. Dynamically generating tests cases based on a list of applications names that will be matched to stored twenty-apps and run install delete and reinstall with their configuration on the same instance We could use a glob pattern with a specific e2e configuration in every apps but right now overkill ## Notes - We should define typescript path aliasing to ease import devxp - parse the config using a zod object ## Some vision on test granularity Right now we only check that the server sent back success or failure on below operation. In the future the synchronize will return a report of what has been installed per entity exactly. We will be able to snapshot everything in order to detect regressions We should also be testing the cli directly for init and other stuff in the end, we could get some inspiration from what's done in preconstruct e2e tests with an on heap virtual file system ## Conclusion Any suggestions are more than welcomed ! close https://github.com/twentyhq/core-team-issues/issues/1721 |
||
|
|
e54aa9d925 |
Fix rimraf not cleaning dist properly (#14903)
trying to solve build errors when launching commands
```console
> nx run twenty-shared:generateBarrels [existing outputs match the cache, left as is]
> nx run twenty-shared:build [existing outputs match the cache, left as is]
> nx run twenty-emails:build [existing outputs match the cache, left as is]
> nx run twenty-server:typecheck
> tsc -b tsconfig.json --incremental
> nx run twenty-server:build
> rimraf dist
> nest build --path ./tsconfig.build.json
> SWC Running...
[Error: ENOTEMPTY: directory not empty, rmdir '/Users/martinmuller/Desktop/twenty/packages/twenty-server/dist/src/modules'] {
errno: -66,
code: 'ENOTEMPTY',
syscall: 'rmdir',
path: '/Users/martinmuller/Desktop/twenty/packages/twenty-server/dist/src/modules'
}
```
|
||
|
|
584389e17f |
add code checksum to flat serverless function (#14721)
## Context Now adding serverless code sync within the migration v2 logic itself. To do that we need to follow the - Prepare flat input - Build migration - Run migration steps where now the serverless function entity will store in DB and cache a checksum of its code and the flat input will contain the code with the checksum. Build will compare checksum and create an update action containing the code if it has changed and the migration will now run the corresponding services (instead of calling those in the parent serverless service exposed in the API) allowing us to keep that logic functional for other use cases such as import/export, twenty-cli and twenty upgrades. |
||
|
|
16fcaacbdb |
Translation cleanup (#13411)
Remove dead translations |
||
|
|
29f7b74756 |
fix(ci): reorganize workflow steps and move cache saving to correct s… (#13083)
…tage |
||
|
|
a68895189c |
Deprecate old relations completely (#12482)
# What Fully deprecate old relations because we have one bug tied to it and it make the codebase complex # How I've made this PR: 1. remove metadata datasource (we only keep 'core') => this was causing extra complexity in the refactor + flaky reset 2. merge dev and demo datasets => as I needed to update the tests which is very painful, I don't want to do it twice 3. remove all code tied to RELATION_METADATA / relation-metadata.resolver, or anything tied to the old relation system 4. Remove ONE_TO_ONE and MANY_TO_MANY that are not supported 5. fix impacts on the different areas : see functional testing below # Functional testing ## Functional testing from the front-end: 1. Database Reset ✅ 2. Sign In ✅ 3. Workspace sign-up ✅ 5. Browsing table / kanban / show ✅ 6. Assigning a record in a one to many / in a many to one ✅ 7. Deleting a record involved in a relation ✅ => broken but not tied to this PR 8. "Add new" from relation picker ✅ => broken but not tied to this PR 9. Creating a Task / Note, Updating a Task / Note relations, Deleting a Task / Note (from table, show page, right drawer) ✅ => broken but not tied to this PR 10. creating a relation from settings (custom / standard x oneToMany / manyToOne) ✅ 11. updating a relation from settings should not be possible ✅ 12. deleting a relation from settings (custom / standard x oneToMany / manyToOne) ✅ 13. Make sure timeline activity still work (relation were involved there), espacially with Task / Note => to be double checked ✅ => Cannot convert undefined or null to object 14. Workspace deletion / User deletion ✅ 15. CSV Import should keep working ✅ 16. Permissions: I have tested without permissions V2 as it's still hard to test v2 work and it's not in prod yet ✅ 17. Workflows global test ✅ ## From the API: 1. Review open-api documentation (REST) ✅ 2. Make sure REST Api are still able to fetch relations ==> won't do, we have a coupling Get/Update/Create there, this requires refactoring 3. Make sure REST Api is still able to update / remove relation => won't do same ## Automated tests 1. lint + typescript ✅ 2. front unit tests: ✅ 3. server unit tests 2 ✅ 4. front stories: ✅ 5. server integration: ✅ 6. chromatic check : expected 0 7. e2e check : expected no more that current failures ## Remove // Todos 1. All are captured by functional tests above, nothing additional to do ## (Un)related regressions 1. Table loading state is not working anymore, we see the empty state before table content 2. Filtering by Creator Tim Ap return empty results 3. Not possible to add Tasks / Notes / Files from show page # Result ## New seeds that can be easily extended <img width="1920" alt="image" src="https://github.com/user-attachments/assets/d290d130-2a5f-44e6-b419-7e42a89eec4b" /> ## -5k lines of code ## No more 'metadata' dataSource (we only have 'core) ## No more relationMetadata (I haven't drop the table yet it's not referenced in the code anymore) ## We are ready to fix the 6 months lag between current API results and our mocked tests ## No more bug on relation creation / deletion --------- Co-authored-by: Weiko <corentin@twenty.com> Co-authored-by: Félix Malfait <felix@twenty.com> |
||
|
|
322c8a1852 |
Upgrade to Node22 (#12488)
BlocknoteJS requires an ESM module where our server is CJS, this forced us to pin the server-util version, which led us to force the resolution of several packages, leading to bugs downstream. From Node 22.12 Node supports requiring ESM modules (available from Node 22.0 with a flag). So I upgrade the module. I picked Node 22 and not Node 23 or Node 24 because 22 is the LTS and we don't plan to change node versions frequently. If you remain on Node 18, things should still mostly work, except if you edit a Rich Text field. I also starting changing the default runtime for Serverless Functions which isn't directly related. This means new serverless functions will be created on Node 22, but we will still need another PR to migrate existing serverless functions before September (end of support by AWS). (In this PR I also remove the upgrade commands from 0.43 since they rely on Blocknote and I didn't want to have to deal with this) --------- Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com> |
||
|
|
49b7f5255f |
Update what is being audit logged (#11833)
No need to audit log workflow runs as it's already a form of audit log. Add more audit log for other objects Rename MessagingTelemetry to MessagingMonitoring Merge Analytics and Audit in one (Audit) --------- Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com> |
||
|
|
587281a541 | feat(analytics): add clickhouse (#11174) | ||
|
|
de91a5e39e | chore(twenty-server): remove eslint warn + add maxWarning 0 (#10103) | ||
|
|
f40d7e2ba8 |
Deprecate message queue type (#10040)
Not removing all the code for now, maybe we should 🤔
|
||
|
|
0a8d75bc07 |
[FIX] Nx project's scope build dependency management (#10010)
# Introduction Defined `dependsOn` for each nx project's configuration that has a dependency to another local package ( ui, shared ). As follows: ```json "dependsOn": ["^build"] ``` Where the `^` symbol means "all dependencies of this project" Now on a fresh repo, no built or install deps after install dependencies you can directly hit `npx nx build PROJECT_NAME` closes https://github.com/twentyhq/core-team-issues/issues/371 Related what was failing [run](https://github.com/twentyhq/twenty-infra/actions/runs/13141544809/job/36669643182) Cancelled before deploy, attested build was correct within the publish and digest |
||
|
|
549c3faf71 |
Add server translation (#9847)
First proof of concept for server-side translation. The goal was to translate one metadata item: <img width="939" alt="Screenshot 2025-01-26 at 08 18 41" src="https://github.com/user-attachments/assets/e42a3f7f-f5e3-4ee7-9be5-272a2adccb23" /> |
||
|
|
e0ea5b35d9 |
[SERVER] Typecheck the twenty-server (#9736)
# Introduction It seems like that within the `twenty-server` `nx-project` there's no entry for the `typecheck` command. It also seems to silently fail along the `start` command even if it depends on it. Looking at this [run](https://github.com/twentyhq/twenty/actions/runs/12865329787/job/35865644811?pr=9736) we can see that even when containing following diff it still does not fail ```ts // twenty-server/main.ts // @ts-expected-error: Manually triggering TSC error should be deleted afterwards const bootstrap = async () => { ``` By adding the `typecheck` entry in the project the same run will now [fails](https://github.com/twentyhq/twenty/actions/runs/12865516249/job/35866224325)  # Notes - There're no existing TSC errors even if wasn't guarded by the CI rn 😎 - The ci will take dozen of more seconds to run, should keep an eye on // with nx implem checking if it works as expected |
||
|
|
85c04c8931 |
Performance improvement to dev xp (#9294)
The DX is not great when you need to do a lot of database resets/command. Should we disable Typescript validation to speed things up? With this and caching database:reset takes 1min instead of 2 on my machine. See also: https://github.com/typeorm/typeorm/issues/4136 And #9291 / #9293 --------- Co-authored-by: Lucas Bordeau <bordeau.lucas@gmail.com> |