Commit Graph

105 Commits

Author SHA1 Message Date
Paul Rastoin 6a8457d8c5 fix(ci): diff upgrade mutation guard against the merged base, not stale PR base (#23278)
The `server-previous-version-upgrade-mutation-guard` check falsely flags
upgrade commands that landed on `main` as being added/modified by
unrelated PRs (e.g. [run on
#23207](https://github.com/twentyhq/twenty/actions/runs/30097349274/job/89494554178)
flagged six `2-23`/`2-24` files the PR never touched).

## Cause

The guard action diffs `git diff "$BASE_SHA" HEAD` where the PR caller
passes `base_sha: github.event.pull_request.base.sha`.

On a `pull_request` event `HEAD` is the `refs/pull/N/merge` ref, whose
first parent is the *current* tip of `main` it was merged with. But
`pull_request.base.sha` is pinned to the base at the last branch sync
and lags behind. Any upgrade command merged into `main` after that point
exists in `HEAD` but not in the stale base, so the two-dot diff
attributes it to the PR.

Verified against the real merge ref for #23207: diffing against
`base.sha` reproduces the six false offenders from the failing run;
diffing against the merge ref first parent is clean.

## Fix

Derive the base from the merge ref first parent (`HEAD^1`, the actual
merged `main` tip), falling back to `base.sha` only when `HEAD` is not a
merge commit (non-mergeable PR).

The merge-queue caller in `ci-merge-queue.yaml` is unaffected: it passes
`merge_group.base_sha`, which already matches its checkout.

<!-- This is an auto-generated description by cubic. -->
<a
href="https://cubic.dev/pr/twentyhq/twenty/pull/23278?utm_source=github"
target="_blank" rel="noopener noreferrer"
data-no-image-dialog="true"><picture><source
media="(prefers-color-scheme: dark)"
srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source
media="(prefers-color-scheme: light)"
srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img
alt="Review in cubic"
src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a>
<!-- End of auto-generated description by cubic. -->
2026-07-24 16:01:36 +02:00
Paul Rastoin 29747f5d6b ci: report required status checks on merge_group so the queue isn't blocked (#23275)
Follow-up to #23216, which added the `merge_group`-triggered
`upgrade-mutation-guard`. This makes the merge queue actually usable.

## Why

The merge queue waits for every **required status check** to report a
conclusion on the `merge_group` candidate commit, and there is no
queue-only subset: it uses the branch's required status checks. Our
required `ci-*-status-check` contexts only trigger on `pull_request`, so
on a queued PR they sit at "Expected - Waiting for status to be
reported" and block the queue until the status-check timeout (60 min),
which then counts them as failed.

Only `upgrade-mutation-guard` (from #23216) triggers on `merge_group`,
so today it is the only check that reports in the queue.

## What

Add a `merge_group` trigger to each of the seven required-check
workflows and short-circuit the expensive work so the check reports
success in seconds, while the full suite keeps running on `pull_request`
to gate PRs. `upgrade-mutation-guard` stays the only check the queue
genuinely validates against `main`.

Mechanism: on `merge_group` the root jobs skip, everything downstream
cascades to `skipped`, and the `always()`-gated `*-status-check` job
runs, sees no failing needs, and succeeds. Kept as the same job in the
same workflow so the required-check context is byte-identical to the
PR-level one (a separate pass-through workflow could register a
different context and not satisfy branch protection).

Per workflow:

- **ci-front**: trigger only. It already cascades -
`changed-files-check` is `pull_request`-only and `front-sb-build` gates
on `push || any_changed`, so nothing runs on `merge_group`.
- **ci-server**: trigger + `if: github.event_name != 'merge_group'` on
the three ungated root jobs (`changed-files-check`,
`upgrade-changed-files-check`,
`server-previous-version-upgrade-mutation-guard`). The guard would
otherwise fail on `merge_group` since `pull_request.base.sha` is empty
there; the queue-side guard in `ci-merge-queue.yaml` already covers that
case.
- **ci-sdk / ci-website / ci-test-docker-compose**: trigger + the same
guard on `changed-files-check`.
- **ci-twenty-apps**: trigger + the guard on `discover` (its
`ci`/`integration` jobs gate on `discover` output, so they cascade off).

## Settings note

This complements the branch-protection changes for the queue (enable the
queue, max group size 1, per #23216). If the branch has **other**
required checks beyond these seven whose workflows are
`pull_request`-only, they need the same `merge_group` treatment or the
queue will wait on them too.

---
_Generated by [Claude
Code](https://claude.ai/code/session_015mZozbvyvha1S6wsEVLBnF)_

<!-- This is an auto-generated description by cubic. -->
<a
href="https://cubic.dev/pr/twentyhq/twenty/pull/23275?utm_source=github"
target="_blank" rel="noopener noreferrer"
data-no-image-dialog="true"><picture><source
media="(prefers-color-scheme: dark)"
srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source
media="(prefers-color-scheme: light)"
srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img
alt="Review in cubic"
src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a>
<!-- End of auto-generated description by cubic. -->
2026-07-24 15:18:26 +02:00
Paul Rastoin bb4e427196 ci: run upgrade mutation guard in the merge queue against main (#23216)
Follow-up to #23215 (merged). Rebased on `main`.

## Why

#23215 fixes an instance of a class of bug: an upgrade command whose
version is chosen at `generate:instance-command` time from
`TWENTY_CURRENT_VERSION`, then left behind when `main` bumps the version
before the PR merges (base-drift). The command ships one minor early and
instances already on the newer version skip it forever.

The existing `server-previous-version-upgrade-mutation-guard` in
`ci-server.yaml` runs on `pull_request`, so it validates against the
PR's base. When the base is stale (main moved after the branch was cut),
the guard reads the branch's own `TWENTY_CURRENT_VERSION` and the check
passes even though the command is now a version behind main. That is
exactly how the original bug slipped through.

## What

The version-directory and append-only-timestamp validation is extracted
into a shared composite action,
`.github/actions/upgrade-mutation-guard`, diffed against a
caller-supplied `base_sha`. It is called from two places:

- **`ci-server.yaml`** (PR-level guard, `base = pull_request.base.sha`)
for fast feedback. The job keeps its existing name/check. This replaces
~290 lines of inline shell.
- **`ci-merge-queue.yaml`** (new, `merge_group`-triggered, `base =
merge_group.base_sha`). GitHub builds each merge-queue candidate on top
of the current tip of `main`, so the checks read
`TWENTY_CURRENT_VERSION` and the existing per-directory timestamps from
main's real state at merge time.

Because the candidate is rebased onto main, base-drift is caught by
construction: the same validation simply runs where the base is
guaranteed current. No origin/main comparison hack; the logic now lives
in one place.

## Bypass semantics

The guard has two independent checks, and they are treated differently
on purpose:

- **Version-directory check** keeps its
`ci:allow-previous-version-upgrade-mutation` bypass, a deliberate,
reviewed escape hatch for legitimately touching a previous-version
directory. The PR-level guard reads the label directly; the merge-queue
guard resolves it from the queued PR (the `merge_group` event carries no
labels) and passes it to the composite action, which skips only the
version-directory step.
- **Timestamp / append-only check has no bypass.** The old
`ci:allow-upgrade-command-timestamp-exception` label is removed. A fake
or out-of-order timestamp rewinds the upgrade cursor and re-hides
already-applied columns, so there is no "allowed" version of it: the
timestamp just has to be configured correctly (real epoch millis,
strictly greater than every existing command in the same version
directory). If a blocking existing max is itself a fabricated future
timestamp, re-slot that command to its real merge epoch rather than
reaching for a bypass.

Preventing previous-version mutation is the guard's primary purpose. In
the merge queue the guard job always runs and skips only the
version-directory step when the bypass label is set, so it reports a
real success/failure (the required check never resolves to a skipped
state, and a label-lookup failure fails closed) and the timestamp check
always runs.

## Requires a settings change (not in this diff)

Enabling the merge queue and marking the check required are
branch-protection settings, not file changes. After merge, an admin
needs to:

1. Enable the merge queue for `main` in branch protection.
2. Add `CI - Merge Queue / upgrade-mutation-guard` to the merge queue's
required checks.

## Notes

- Composite action, not a `workflow_call` reusable workflow,
deliberately: converting the `ci-server.yaml` job to a reusable-workflow
call would rename its status check to
`server-previous-version-upgrade-mutation-guard / ` and break that
required-check mapping in branch protection. A composite action dedups
the logic while keeping both callers' check names intact.

---------

Co-authored-by: Paul Rastoin <paul.rastoin@gmail.com>
2026-07-24 14:37:45 +02:00
Félix Malfait 6e1e98f4ab fix(server): shard the server-test unit job to stop the intermittent crash (#23009)
## Problem

`server-test` fails intermittently: exit 1 with **no `FAIL` line and no
`Test Suites:/Tests:` summary** — the jest run is aborted mid-way,
before the reporter's `onRunComplete`.

## Root cause

The `test` target runs the entire unit suite (~6,600 tests) in **one
in-band jest process on a single runner VM** (`nx.json` sets
`maxWorkers: 1` for the `ci` configuration). A few minutes in, that
process is killed by an **external `SIGKILL`** — confirmed *not* OOM
(~15 GB free at kill time, no cgroup `oom_kill`) and *not* an in-process
crash (a Node diagnostic report armed with `--report-on-fatalerror` +
`--report-uncaught-exception` writes nothing). The whole-run kill is why
it fails intermittently with no summary. `maxWorkers=2` on one VM still
dies, so the threshold is **per-VM**, not per-process.

## Fix

Shard the unit suite across VMs, the same way `server-integration-test`
already does:

- A `twenty-server` `test:ci` target runs jest directly (so `--shard`
forwards) with `dependsOn: ["^build"]` so the workspace deps are built.
- `server-test` becomes a 4-way matrix; each shard runs a quarter of the
suite, well under the kill threshold.
- `ci-server-status-check` already aggregates `server-test`, so required
checks are unchanged.

Also provides two mocks a completed run needs but the SIGKILL had been
masking in `ApplicationRegistrationService.upsertFromCatalog` unit
tests: the `MetricsService` provider and
`applicationRegistrationRepository.createQueryBuilder`.
2026-07-20 06:39:13 +02:00
Paul Rastoin cb95410a51 ci: test twenty-apps install against latest dockerhub and local server + new trigger (#22636)
## Why

App installability can silently regress from two directions, and today
CI only covers one of them:

1. A **server** change (about to merge from the monorepo) breaks the
ability to install the **current public apps** — a
backward-compatibility regression users would hit on upgrade.
2. An **app** change breaks against a server **built from the current
monorepo files** (not just the last published image), so the app and the
upcoming server drift apart before either ships.

Both are compatibility guarantees between the server and the app
catalog. Today they are only tested from the app side, against the
latest published image. This PR makes CI enforce the contract from both
sides:

- Any server PR must keep **every** current public app installable.
- Any app PR is exercised against both the **released** server (its
integration suite — what users run today) and the **upcoming**
(monorepo) server (integration plus deploy + install).

## What

Shared building blocks so both CIs exercise the same paths instead of
duplicating them:

- **`spawn-twenty-server`** (composite action) — returns a running
server (`server-url` + `api-key`) from either the latest published
Docker Hub image or a server built from the monorepo. Both sources
expose the same contract, so callers never branch on how the server came
up.
- **`test-twenty-app`** (composite action) — exercises one app against a
given server, delegating deploy + install to the shared
`deploy-twenty-app` / `install-twenty-app` actions.
- **`discover-apps`** (reusable workflow) — the single source of truth
for the app matrix. Parameterized by `scope` (`public` vs
`internal-and-public`) and `changed-only`, so both CIs derive their
matrix from the filesystem instead of a hand-maintained list. Discovery
stays automatic: a newly added public app is picked up with no CI edit,
which is what keeps the "every public app" guarantee honest.

Wired in:

- **CI Server** gains a `server-apps-install-smoke` matrix that installs
every public app (`discover-apps` with `scope: public, changed-only:
false`) against the about-to-merge server, gated in
`ci-server-status-check` so a regression blocks merge.
- **CI Twenty Apps** discovers changed apps (`scope:
internal-and-public, changed-only: true`) and runs each against both
server sources — the released image and the monorepo build.

## Why the coverage differs per side (not "always everything")

`test-twenty-app` has three explicit modes —
`installation-and-integration-test` (integration + deploy + install),
`integration-test-only` (suite only), `installation-only` (deploy +
install only) — because the useful signal depends on what actually
changed:

- **App PR against the monorepo server →
`installation-and-integration-test`.** The app changed, so run its whole
suite against the upcoming server, install included.
- **App PR against the released server → `integration-test-only`.**
Checks the app's own suite against what users run today; install against
the released image is left to the SDK e2e path.
- **Server PR → `installation-only`, across all apps.** The apps did not
change; the only question is "can each one still be installed." Running
every app's full integration suite on every server PR would be far
slower and largely redundant. Installation-only keeps this broad (the
whole catalog) and cheap enough to always run and block merge.

The tradeoff is deliberate: broad but shallow where nothing in the app
changed, deep where it did.

## Notes / trade-offs

- On app-only PRs the `local` source pays a full server build per app
(the `server-build` cache is only warm on server PRs). Could be
optimized later with a shared warm-up job.
- SDK-local (Verdaccio) install testing stays in
`ci-create-app-e2e-minimal`; this PR's `local` source targets the server
build.

<a
href="https://cubic.dev/pr/twentyhq/twenty/pull/22636?utm_source=github"
rel="nofollow noreferrer noopener" target="_blank">``&lt;img alt="Review
in cubic"
src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"&gt;``</a>
2026-07-10 19:18:53 +02:00
Félix Malfait 024a9b4d94 fix(upgrade): re-slot all 2-19 upgrade commands to their real merge epochs and guard timestamps in CI (#22498)
# Fix broken 2-19 upgrade command sequence (dev incident:
`lastStreamError` / `workspaceDiscoverability`)

## Incident

The dev environment (tracking main) throws:
- `Property "lastStreamError" was not found in "AgentChatThreadEntity"`
- `Cannot return null for non-nullable field
Workspace.workspaceDiscoverability`

## Root cause

The 2-19 upgrade commands were committed with **fabricated future
timestamps** (year-2027 epochs like `1820000000000`). The upgrade cursor
(`upgrade-aware-entity-metadata.adapter.ts`) tracks a **single**
most-recent applied step: it looks up the latest
`core."upgradeMigration"` row's name in the sequence (sorted by
timestamp within kind) and hides every `@WasIntroducedInUpgrade` column
at or past that index.

Two ways this breaks, and both were live on main:
1. **Cursor regression**: a command merged *later* with a *smaller*
timestamp (e.g. `pendingQuestion` at `1811…` after `metadata-overrides`
at `1820…` had run) sorts *before* already-applied steps. When migrate
runs, the "latest" row now points earlier in the sequence, re-hiding
columns that were already applied.
2. **Migrate not running at all** (Felix's hypothesis): if the deploy
pipeline skipped `database:migrate:prod`, none of the 2-19 rows exist
and every 2-19-gated column is hidden.

Both hypotheses have the same fix path; the discriminating query is in
the verification section below.

## Fix

**Real timestamps** (per maintainer direction — no more fabricated
epochs):

| Command | Old (fabricated) | New (real merge epoch) | Introduced by |
|---|---|---|---|
| workspace: backfill-workspace-custom-application-registration |
`1820000000000` | `1782853718000` | #22378 |
| fast: add-metadata-overrides-column | `1820000100000` |
`1782986475000` | #22417 |
| slow: backfill-metadata-overrides | `1820000110000` | `1782986476000`
(+1s to order after its fast pair) | #22417 |
| fast: add-last-stream-error-to-agent-chat-thread | `1821000000000` |
`1782996657000` | #22434 |
| fast: add-pending-question-to-agent-chat-thread | `1811000000000` |
`1782999138000` | #22346 |
| fast: add-workspace-discoverability-to-workspace | `1820000001000` |
`1783004140000` | #22423 |

Each value is the committer epoch of the squash-merge commit that
introduced the command on main (verified via `git log --diff-filter=A`).
Sorted by real time, the fast sequence is strictly increasing, so the
cursor can no longer regress.

**Idempotency**: renaming a command changes its step name, so every one
of these re-runs on any instance that already applied it under the old
name (dev cluster, edge self-hosters — 2.19 is unreleased, so tagged
releases are unaffected). All six are now safe to re-run:
- `lastStreamError`, `pendingQuestion`, `metadata-overrides` fast: `ADD
COLUMN IF NOT EXISTS` (already were)
- `metadata-overrides` slow backfill: `WHERE … IS NULL` guard (already
was)
- workspace command: skips when `applicationRegistrationId` is already
set (already did)
- `workspaceDiscoverability`: **made idempotent in this PR** — `CREATE
TYPE` wrapped in a `duplicate_object` handler, `ADD COLUMN IF NOT
EXISTS`

**CI guard** (replaces the append-only check added earlier on this
branch):
- Timestamps must be **real**: within `[now − 60 days, now + 2 days]`.
This is the check that would have prevented the original sin — 2027
epochs can never pass.
- Still **append-only** within the version directory, but computed from
`git diff --name-status --find-renames` so renamed/copied files are
checked too (cubic's P2), and files the PR deletes/renames away no
longer count toward the existing max (otherwise a re-slotting PR like
this one could never pass its own guard).
- Covers `workspace-command-<ts>-` filenames, not just
`instance-command-fast|slow-<ts>-`; skips `.spec.ts` files.
- Failure message documents the escape path (re-slot the fabricated
blocker to its real epoch + make it idempotent) and a bypass label
`ci:allow-upgrade-command-timestamp-exception` for deliberate
exceptions.

## Deploy sequencing (important)

After this merges and deploys, `database:migrate:prod` **must run**
before the API pods are relied on: the old step names no longer exist in
the sequence, so until the renamed commands run once, the cursor
resolves to 0 and *every* gated column is hidden. The commands are
idempotent, so the re-run is harmless. Running API/worker pods only
compute the cursor at boot — restart them after migrate.

## Verification / diagnosis on dev

```sql
SELECT name, status, "createdAt"
FROM core."upgradeMigration"
WHERE "workspaceId" IS NULL
ORDER BY "createdAt" DESC
LIMIT 15;
```
- Latest rows named `…_182xxxxxxxxxx` (fabricated) and completed →
migrate ran, cursor regressed (hypothesis 1).
- No 2-19 rows at all → migrate never ran for 2-19 (hypothesis 2).
- After the fix: latest row should be
`2.19.0_AddWorkspaceDiscoverabilityToWorkspaceFastInstanceCommand_1783004140000`,
status `completed`.

https://claude.ai/code/session_01Lyi6zTema2FMVVh8MD6c38
2026-07-02 23:38:38 +02:00
Paul Rastoin 5e5c8e0956 ci(server,emails): run lingui extract & compile on PRs to gate i18n breakage (#22086)
## Why

Translations for `twenty-server` and `twenty-emails` are only extracted
**after merge** — in `i18n-push.yaml` (push to `main`). Their PR
workflows (`ci-server.yaml`, `ci-emails.yaml`) never run `lingui
extract`, so a change that crashes extraction passes every PR check and
only fails post-merge — the same process gap that let the frontend crash
through (fixed in #22080, frontend gate in #22084).

Both projects have a `lingui:extract` target and are extracted by
`i18n-push.yaml`, so the equivalent guard applies.

## What

- **`ci-server.yaml`** — add `lingui:extract` to the existing
`server-lint-typecheck` job's `nx-affected` tasks (`tag:
scope:backend`). That job already builds `twenty-shared` and is already
part of `ci-server-status-check`, so no new job/wiring is needed:
  ```
  tasks: lint,typecheck  ->  tasks: lint,typecheck,lingui:extract
  ```
- **`ci-emails.yaml`** — add a `lingui:extract` step to the
`emails-test` job (this workflow has no `nx-affected` job, so a direct
target run fits):
  ```yaml
  - name: Extract translations (lingui)
    run: npx nx run twenty-emails:lingui:extract
  ```

Both run the same extraction command as the post-merge `i18n-push`
workflow, so failures are caught before merge.

## Notes

- `lingui extract` exits non-zero on extraction failures (verified on
the frontend crash: exit code 1), so these steps genuinely fail the job.
- Both jobs already gate on `changed-files-check`, so extract only runs
when the respective package changes.
- `nx affected -t=lingui:extract` only runs for projects that have the
target; `lingui:extract`'s `^build` dependency is resolved automatically
by nx.

Completes the extract-gate coverage started for the frontend in #22084.


<!-- This is an auto-generated description by cubic. -->
<a
href="https://cubic.dev/pr/twentyhq/twenty/pull/22086?utm_source=github"
target="_blank" rel="noopener noreferrer"
data-no-image-dialog="true"><picture><source
media="(prefers-color-scheme: dark)"
srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source
media="(prefers-color-scheme: light)"
srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img
alt="Review in cubic"
src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a>
<!-- End of auto-generated description by cubic. -->
2026-06-24 15:55:57 +02:00
Paul Rastoin 8842a80a44 ci(server): cross-version upgrade check on PRs (v1.22 → from source) (#22065)
## What

Adds a pre-merge CI check that proves a database created and seeded by
the **oldest supported release** (`twentycrm/twenty:v1.22` from Docker
Hub) can be upgraded by the **current version built from source**, and
that the upgraded instance comes up healthy with its data still
queryable.

Runs only on PRs touching the upgrade path (`upgrade-version-command/**`
+ `core-modules/upgrade/**`), and blocks the PR via
`ci-server-status-check`.

## How

New reusable workflow `ci-cross-version-upgrade.yaml` (`workflow_call` +
`workflow_dispatch`), called from `ci-server.yaml` after `server-build`
so the build cache is populated in-run:

1. **Services** — `postgres:16` (prod parity) + `redis:7` on a docker
network.
2. **Old version** — pull `twentycrm/twenty:v1.22`, boot it against the
DB, `workspace:seed:dev`, sanity-check the seed via `psql`.
3. **New version (from source)** — restore the `server-build` nx cache
(best-effort: a miss just cold-builds), `nx build`, run the `upgrade`
command against the same DB, `start:ci`, poll `/healthz`.
4. **Smoke** — assert `upgrade:status` shows `Instance: Up to date` / `0
behind, 0 failed`, then run companies/people/metadata GraphQL queries.

The job is always invoked but gated by a `skip` input (computed from the
upgrade-paths `changed-files` check), with a `no-op` job reporting
success when skipped — so the status check always resolves instead of
leaving a dangling skipped job, mirroring the twenty-infra pattern.

Unlike the equivalent post-merge gate in infra-twenty, this is
**pre-merge**, uses **native PR path filtering** (no compare API), and
**reuses the from-source build cache** instead of pulling an ECR image —
no cross-repo plumbing, no skipped-commit gap.

## Security note

No credentials are committed. `APP_SECRET` is generated fresh per run
(`openssl rand`, `::add-mask::`'d) and shared between the old container
and the from-source server within the job; the smoke-test API token is
minted at runtime via `workspace:generate-api-key` against the upgraded
server and masked in logs.

## Verified with a real run

Validated end-to-end by temporarily touching the upgrade path to trigger
the job (trigger commit since dropped), in [CI Server run
`28097035272`](https://github.com/twentyhq/twenty/actions/runs/28097035272)
→ [`cross-version-upgrade`
job](https://github.com/twentyhq/twenty/actions/runs/28097035272/job/83189453451)
 **all steps green**:

- v1.22 container boot → `workspace:seed:dev` → `psql` seed sanity check

- from-source build (nx cache restored) → `upgrade` → **56 workspace(s)
succeeded, 0 failed** 
- server healthy → API token minted at runtime via
`workspace:generate-api-key` 
- `upgrade:status` → `Instance: Up to date`, `0 behind, 0 failed` 
- companies / people / metadata GraphQL smoke queries 
- `no-op` job correctly skipped (real job ran because the gate matched)


The three assumptions originally flagged for first-run all held; one bug
was found and fixed in the process — `upgrade:status` colorizes via
`chalk` even with `NO_COLOR`, so the assertion now strips ANSI escapes
before grepping.

> Note: the overall `ci-server` run shows a failure from an **unrelated
flaky integration test** (`if-else-workflow.integration-spec.ts`,
`column workspaceMember.region does not exist` in shard 11). The same
trigger commit passed all 16 integration shards in the prior run — it's
a pre-existing flake, not caused by this PR.

## Note

Still keeping the equivalent one inside infra-twenty as an final
bottleneck just in case

<!-- This is an auto-generated description by cubic. -->
<a
href="https://cubic.dev/pr/twentyhq/twenty/pull/22065?utm_source=github"
target="_blank" rel="noopener noreferrer"
data-no-image-dialog="true"><picture><source
media="(prefers-color-scheme: dark)"
srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source
media="(prefers-color-scheme: light)"
srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img
alt="Review in cubic"
src="https://www.cubic.dev/buttons/review-in-cubic-light.svg"></picture></a>
<!-- End of auto-generated description by cubic. -->
2026-06-24 14:20:38 +02:00
Charles Bochet 495907c781 ci: remove merge queue, run e2e on push to main (#21722)
## What

Removes the GitHub merge queue and runs what the queue used to gate —
the E2E (Playwright) suite — directly on push to `main`. If that run
fails on `main`, we ping engineering via webhook.

## Why

In the queue, only the `e2e-test` job in `ci-merge-queue.yaml` ran real
work — every other CI workflow's `merge_group` path skipped its
`changed-files-check` and tests, so the queue's status checks for those
were effectively green no-ops. The expensive thing actually gated was
E2E. Moving it to `push: main` validates the merged state post-merge
without the queue's batching overhead.

## Changes

- **Rename** `ci-merge-queue.yaml` → `ci-e2e-main.yaml` (`name: CI E2E
Main`).
- `e2e-test` now triggers on `push` to `main` (the `run-merge-queue` PR
label is kept as a manual opt-in for running E2E on a PR).
  - Status-check job renamed `ci-e2e-main-status-check`.
- New `notify-main-ci-failure` job: on a failed **main push**, `POST`s
to `https://engineering.twenty.com/s/main-ci-failing` with the commit
SHA, actor, and run URL.
- **Strip dead merge-queue config** from the other CI workflows: removed
the `merge_group:` triggers and the now-unreachable `if:
github.event_name != 'merge_group'` guards from `ci-server`,
`ci-shared`, `ci-sdk`, `ci-front-component-renderer`,
`ci-test-docker-compose`, `ci-website`, and the `merge_group:` trigger
from `ci-front`, `ci-ui`, `ci-new-ui`.

## Required follow-up (not in this PR)

The merge queue itself is a **repo setting**, not code. After this
merges, disable **"Require merge queue"** on the `main` ruleset/branch
protection (Settings → Rules), otherwise GitHub keeps batching. Required
status checks tied to the old queue should also be dropped/updated.

## Behavior change

E2E now runs **after** merge rather than blocking it in the queue — a
bad change lands on `main` and then alerts (the webhook is the
mitigation), instead of being held back pre-merge.

<!-- This is an auto-generated description by cubic. -->
<a
href="https://cubic.dev/pr/twentyhq/twenty/pull/21722?utm_source=github"
target="_blank" rel="noopener noreferrer"
data-no-image-dialog="true"><picture><source
media="(prefers-color-scheme: dark)"
srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source
media="(prefers-color-scheme: light)"
srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img
alt="Review in cubic"
src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a>
<!-- End of auto-generated description by cubic. -->
2026-06-17 18:33:07 +02:00
Charles Bochet 97871131a1 fix(server): mitigate integration-test OOM flakiness (#21588)
## Problem

`server-integration-test` shards have been failing intermittently across
unrelated PRs with a distinctive signature: the shard exits code 1 with
**no jest assertion failure, no `Test Suites:` summary, and no V8
`JavaScript heap out of memory` error** — the process just dies mid-run.
Failures hit random shards and clear on re-run (e.g. an unrelated branch
failed shard 6 once, then passed 3× on identical code), while the
`merge_group` gate stays green.

### Root cause

Each shard runs a **single in-band jest process** that boots one shared
NestJS app (`globalSetup` → `app.listen`) and holds it for the entire
shard, driving heavy metadata migrations + cache rebuilds in that one
process. `NODE_OPTIONS=--max-old-space-size=12288` let V8 grow to 12 GB
— *above* the `ubuntu-latest` runner's available RAM (16 GB, shared with
Postgres/Redis/ClickHouse). V8 therefore deferred aggressive GC and grew
past physical memory, so the **OS OOM-killer killed the process before
V8 hit its own ceiling** — which is why there's no heap error and no
jest summary, just a silent exit.

## Changes (CI/test-only — prod runtime untouched)

- **Lower the integration jest heap cap `12288` → `6144`** so V8
self-limits below physical headroom instead of being OS-killed.
Counterintuitively safer: a real leak now surfaces as a *visible* heap
error naming the test, rather than a silent death. (`database:reset`
keeps 12288 — it runs alone, before jest.)
- **Add `--logHeapUsage`** to the integration jest runs to expose the
per-file heap trend for confirming/pinpointing the growth.
- **Split integration tests across 16 shards (was 10)** to lower the
peak working set per shard.
- **Make perf logging a first-class `LoggerService` tool** (per
@prastoin's review): add `LoggerService.perf()` and unify the existing
`time()`/`timeEnd()` helpers into `perfTime()`/`perfTimeEnd()` (now
routed through the driver), all gated by a new `PERF_LOG_ENABLED` config
var. It **defaults on** so real environments keep emitting the
install-perf logs, and `.env.test` sets it `false` to mute the
per-action flood in integration tests. The `application-manifest` and
`validate-build` services were moved from the built-in `Logger` to
`LoggerService` to use it.

## Notes

- `--max-old-space-size` lives only in the `test:integration` nx target;
it is **not** the prod server heap setting, so prod is unaffected.
- This is mitigation. If `--logHeapUsage` shows monotonic growth across
files, there's a real accumulation in the long-lived app (retained
flat-maps / metadata cache) worth a follow-up heap-snapshot fix.

<!-- This is an auto-generated description by cubic. -->
<a
href="https://cubic.dev/pr/twentyhq/twenty/pull/21588?utm_source=github"
target="_blank" rel="noopener noreferrer"
data-no-image-dialog="true"><picture><source
media="(prefers-color-scheme: dark)"
srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source
media="(prefers-color-scheme: light)"
srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img
alt="Review in cubic"
src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a>
<!-- End of auto-generated description by cubic. -->
2026-06-15 15:54:51 +02:00
neo773 565995e715 security: harden CI against supply-chain attacks (#20476)
- Pin all third-party actions to SHA
- Gate claude.yml triggers to internal authors with Harden-Runner egress
audit
- Ignore fork-PR lifecycle scripts
- Narrow cross-repo dispatch payloads
- Add 7d npm release-age gate
- Add CODEOWNERS on .github/** and .yarnrc.yml

---------

Co-authored-by: prastoin <paul@twenty.com>
2026-05-12 12:20:29 +00:00
Paul Rastoin 1e7c1691f4 [CI] Prevent previous version upgrade sequence mutation (#20075)
# Introduction
Prevent any PR to target a previous already released twenty version by
mistake.

Especially useful for existing opened PR introducing commands into an
upgrade that has just been released leading to a
`TWENTY_CURRENT_VERSION` bump

<img width="3150" height="1158" alt="image"
src="https://github.com/user-attachments/assets/b83d211f-a061-4d63-ae7a-354d7851ec08"
/>

## Bypass
If intentional add `ci:allow-previous-version-upgrade-mutation` label to
the PR and re-run the failed job

<img width="3150" height="1158" alt="image"
src="https://github.com/user-attachments/assets/f94ee630-d87b-4477-9e50-bf6773a8a280"
/>

This will require a brand new ci from a commit introduced after the
label has been added
2026-04-27 11:15:49 +00:00
Félix Malfait 75848ff8ea feat: move admin panel to dedicated /admin-panel GraphQL endpoint (#19852)
## Summary

Splits admin-panel resolvers off the shared `/metadata` GraphQL endpoint
onto a dedicated `/admin-panel` endpoint. The backend plumbing mirrors
the existing `metadata` / `core` pattern (new scope, decorator, module,
factory), and admin types now live in their own
`generated-admin/graphql.ts` on the frontend — dropping 877 lines of
admin noise from `generated-metadata`.

## Why

- **Smaller attack surface on `/metadata`** — every authenticated user
hits that endpoint; admin ops don't belong there.
- **Independent complexity limits and monitoring** per endpoint.
- **Cleaner module boundaries** — admin is a cross-cutting concern that
doesn't match the "shared-schema configuration" meaning of `/metadata`.
- **Deploy / blast-radius isolation** — a broken admin query can't
affect `/metadata`.

Runtime behavior, auth, and authorization are unchanged — this is a
relocation, not a re-permissioning. All existing guards
(`WorkspaceAuthGuard`, `UserAuthGuard`,
`SettingsPermissionGuard(SECURITY)` at class level; `AdminPanelGuard` /
`ServerLevelImpersonateGuard` at method level) remain on
`AdminPanelResolver`.

## What changed

### Backend
- `@AdminResolver()` decorator with scope `'admin'`, naming parallels
`CoreResolver` / `MetadataResolver`.
- `AdminPanelGraphQLApiModule` + `adminPanelModuleFactory` registered at
`/admin-panel`, same Yoga hook set as the metadata factory (Sentry
tracing, error handler, introspection-disabling in prod, complexity
validation).
- Middleware chain on `/admin-panel` is identical to `/metadata`.
- `@nestjs/graphql` patch extended: `resolverSchemaScope?: 'core' |
'metadata' | 'admin'`.
- `AdminPanelResolver` class decorator swapped from
`@MetadataResolver()` to `@AdminResolver()` — no other changes.

### Frontend
- `codegen-admin.cjs` → `src/generated-admin/graphql.ts` (982 lines).
- `codegen-metadata.cjs` excludes admin paths; metadata file shrinks by
877 lines.
- `ApolloAdminProvider` / `useApolloAdminClient` follow the existing
`ApolloCoreProvider` / `useApolloCoreClient` pattern, wired inside
`AppRouterProviders` alongside the core provider.
- 37 admin consumer files migrated: imports switched to
`~/generated-admin/graphql` and `client: useApolloAdminClient()` is
passed to `useQuery` / `useMutation`.
- Three files intentionally kept on `generated-metadata` because they
consume non-admin Documents: `useHandleImpersonate.ts`,
`SettingsAdminApplicationRegistrationDangerZone.tsx`,
`SettingsAdminApplicationRegistrationGeneralToggles.tsx`.

### CI
- `ci-server.yaml` runs all three `graphql:generate` configurations and
diff-checks all three generated dirs.

## Authorization (unchanged, but audited while reviewing)

Every one of the 38 methods on `AdminPanelResolver` has a method-level
guard:
- `AdminPanelGuard` (32 methods) — requires `canAccessFullAdminPanel ===
true`
- `ServerLevelImpersonateGuard` (6 methods: user/workspace lookup + chat
thread views) — requires `canImpersonate === true`

On top of the class-level guards above. No resolver method is accessible
without these flags + `SECURITY` permission in the workspace.

## Test plan

- [ ] Dev server boots; `/graphql`, `/metadata`, `/admin-panel` all
mapped as separate GraphQL routes (confirmed locally during
development).
- [ ] `nx typecheck twenty-server` passes.
- [ ] `nx typecheck twenty-front` passes.
- [ ] `nx lint:diff-with-main twenty-server` and `twenty-front` both
clean.
- [ ] Manual smoke test: log in with a user who has
`canAccessFullAdminPanel=true`, open the admin panel at
`/settings/admin-panel`, verify each tab loads (General, Health, Config
variables, AI, Apps, Workspace details, User details, chat threads).
- [ ] Manual smoke test: log in with a user who has
`canImpersonate=false` and `canAccessFullAdminPanel=false`, hit
`/admin-panel` directly with a raw GraphQL request, confirm permission
error on every operation.
- [ ] Production deploy note: reverse proxy / ingress must route the new
`/admin-panel` path to the Nest server. If the proxy has an explicit
allowlist, infra change required before cutover.

## Follow-ups (out of scope here)

- Consider cutting over the three
`SettingsAdminApplicationRegistration*` components to admin-scope
versions of the app-registration operations so the admin page is fully
on the admin endpoint.
- The `renderGraphiQL` double-assignment in
`admin-panel.module-factory.ts` is copied from
`metadata.module-factory.ts` — worth cleaning up in both.
2026-04-19 20:55:10 +02:00
Paul Rastoin c99db7d9c6 Fix server-validation ci pending instance command detection (#19558)
https://github.com/twentyhq/twenty/actions/runs/24246052659/job/70792870185
2026-04-10 13:53:56 +00:00
Paul Rastoin e062343802 Refactor typeorm migration lifecycle and generation (#19275)
# Introduction

Typeorm migration are now associated to a given twenty-version from the
`UPGRADE_COMMAND_SUPPORTED_VERSIONS` that the current twenty core engine
handles

This way when we upgrade we retrieve the migrations that need to be run,
this will be useful for the cross-version incremental upgrade so we
preserve sequentiality

## What's new

To generate
```sh
npx nx database:migrate:generate twenty-server -- --name add-index-to-users
```

To apply all
```sh
npx nx database:migrate twenty-server
```

## Next
Introduce slow and fast typeorm migration in order to get rid of the
save point pattern in our code base
Create a clean and dedicated `InstanceUpgradeService` abstraction
2026-04-06 07:11:47 +00:00
Charles Bochet ac8e0d4217 Replace twentycrm/twenty-postgres-spilo with official postgres:16 in CI (#19182)
## Summary
- Replaces `twentycrm/twenty-postgres-spilo` with the official
`postgres:16` image across all 7 CI workflow files
- Removes Docker Hub `credentials` blocks from all service containers
(postgres, redis, clickhouse)
- Removes the `Login to Docker Hub` step from the breaking changes
workflow

## Context
Fork PRs cannot access repository secrets/variables, causing `${{
vars.DOCKERHUB_USERNAME }}` and `${{ secrets.DOCKERHUB_PASSWORD }}` to
resolve to empty strings. GitHub Actions rejects empty credential values
at template validation time, failing the job before any step runs.

The custom spilo image was the original reason credentials were needed
(to avoid Docker Hub rate limits on non-official images). The only
Postgres extensions required in CI (`uuid-ossp`, `unaccent`) are built
into the official `postgres:16` image. Official Docker Hub images have
significantly higher pull rate limits and don't require authentication.
2026-03-31 21:41:42 +02:00
Charles Bochet 191a277ddf fix: invalidate rolesPermissions cache + add Docker Hub auth to CI (#19044)
## Summary

### Cache invalidation fix
- After migrating object/field permissions to syncable entities (#18609,
#18751, #18567), changes to `flatObjectPermissionMaps`,
`flatFieldPermissionMaps`, or `flatPermissionFlagMaps` no longer
triggered `rolesPermissions` cache invalidation
- This caused stale permission data to be served, leading to flaky
`permissions-on-relations` integration tests and potentially incorrect
permission enforcement in production after object permission upserts
- Adds the three permission-related flat map keys to the condition that
triggers `rolesPermissions` cache recomputation in
`WorkspaceMigrationRunnerService.getLegacyCacheInvalidationPromises`
- Clears memoizer after recomputation to prevent concurrent
`getOrRecompute` calls from caching stale data

### Docker Hub rate limit fix
- CI service containers (postgres, redis, clickhouse) and `docker
run`/`docker build` steps were pulling from Docker Hub
**unauthenticated**, hitting the 100-pull-per-6-hour rate limit on
shared GitHub-hosted runner IPs
- Adds `credentials` blocks to all service container definitions and
`docker/login-action` steps before `docker run`/`docker compose`
commands
- Uses `vars.DOCKERHUB_USERNAME` + `secrets.DOCKERHUB_PASSWORD`
(matching the existing twenty-infra convention)
- Affected workflows: ci-server, ci-merge-queue, ci-breaking-changes,
ci-zapier, ci-sdk, ci-create-app-e2e, ci-website,
ci-test-docker-compose, preview-env-keepalive, spawn-twenty-docker-image
action
2026-03-27 17:32:53 +01:00
Paul Rastoin 281bb6d783 Guard yarn database:migrate:prod (#19008)
## Motivations
A lot of self hosters hands up using the `yarn database:migrated:prod`
either manually or through AI assisted debug while they try to upgrade
an instance while their workspace is still blocked in a previous one
Leading to their whole database permanent corruption

## What happened
Replaced the direct call the the typeorm cli to a command calling it
programmatically, adding a layer of security in case a workspace seems
to be blocked in a previous version than the one just before the one
being installed ( e.g 1.0 when you try to upgrade from 1.1 to 1.2 )

For our cloud we still need a way to bypass this security explaining the
-f flag

## Remark
Centralized this logic and refactored creating new services
`WorkspaceVersionService` and `CoreEngineVersionService` that will
become useful for the upcoming upgrade refactor

Related to https://github.com/twentyhq/twenty-infra/pull/529
2026-03-27 14:39:18 +00:00
Paul Rastoin 4ea2e32366 Refactor twenty client sdk provisioning for logic function and front-component (#18544)
## 1. The `twenty-client-sdk` Package (Source of Truth)

The monorepo package at `packages/twenty-client-sdk` ships with:
- A **pre-built metadata client** (static, generated from a fixed
schema)
- A **stub core client** that throws at runtime (`CoreApiClient was not
generated...`)
- Both ESM (`.mjs`) and CJS (`.cjs`) bundles in `dist/`
- A `package.json` with proper `exports` map for
`twenty-client-sdk/core`, `twenty-client-sdk/metadata`, and
`twenty-client-sdk/generate`

## 2. Generation & Upload (Server-Side, at Migration Time)

**When**: `WorkspaceMigrationRunnerService.run()` executes after a
metadata schema change.

**What happens in `SdkClientGenerationService.generateAndStore()`**:
1. Copies the stub `twenty-client-sdk` package from the server's assets
(resolved via `SDK_CLIENT_PACKAGE_DIRNAME` — from
`dist/assets/twenty-client-sdk/` in production, or from `node_modules`
in dev)
2. Filters out `node_modules/` and `src/` during copy — only
`package.json` + `dist/` are kept (like an npm publish)
3. Calls `replaceCoreClient()` which uses `@genql/cli` to introspect the
**application-scoped** GraphQL schema and generates a real
`CoreApiClient`, then compiles it to ESM+CJS and overwrites
`dist/core.mjs` and `dist/core.cjs`
4. Archives the **entire package** (with `package.json` + `dist/`) into
`twenty-client-sdk.zip`
5. Uploads the single archive to S3 under
`FileFolder.GeneratedSdkClient`
6. Sets `isSdkLayerStale = true` on the `ApplicationEntity` in the
database

## 3. Invalidation Signal

The `isSdkLayerStale` boolean column on `ApplicationEntity` is the
invalidation mechanism:
- **Set to `true`** by `generateAndStore()` after uploading a new client
archive
- **Checked** by both logic function drivers before execution — if
`true`, they rebuild their local layer
- **Set back to `false`** by `markSdkLayerFresh()` after the driver has
successfully consumed the new archive

Default is `false` so existing applications without a generated client
aren't affected.

## 4a. Logic Functions — Local Driver

**`ensureSdkLayer()`** is called before every execution:
1. Checks if the local SDK layer directory exists AND `isSdkLayerStale`
is `false` → early return
2. Otherwise, cleans the local layer directory
3. Calls `downloadAndExtractToPackage()` which streams the zip from S3
directly to disk and extracts the full package into
`<tmpdir>/sdk/<workspaceId>-<appId>/node_modules/twenty-client-sdk/`
4. Calls `markSdkLayerFresh()` to set `isSdkLayerStale = false`

**At execution time**, `assembleNodeModules()` symlinks everything from
the deps layer's `node_modules/` **except** `twenty-client-sdk`, which
is symlinked from the SDK layer instead. This ensures the logic
function's `import ... from 'twenty-client-sdk/core'` resolves to the
generated client.

## 4b. Logic Functions — Lambda Driver

**`ensureSdkLayer()`** is called during `build()`:
1. Checks if `isSdkLayerStale` is `false` and an existing Lambda layer
ARN exists → early return
2. Otherwise, deletes all existing layer versions for this SDK layer
name
3. Calls `downloadArchiveBuffer()` to get the raw zip from S3 (no disk
extraction)
4. Calls `reprefixZipEntries()` which streams the zip entries into a
**new zip** with the path prefix
`nodejs/node_modules/twenty-client-sdk/` — this is the Lambda layer
convention path. All done in memory, no disk round-trip
5. Publishes the re-prefixed zip as a new Lambda layer via
`publishLayer()`
6. Calls `markSdkLayerFresh()`

**At function creation**, the Lambda is created with **two layers**:
`[depsLayerArn, sdkLayerArn]`. The SDK layer is listed last so it
overwrites the stub `twenty-client-sdk` from the deps layer (later
layers take precedence in Lambda's `/opt` merge).

## 5. Front Components

Front components are built by `app:build` with `twenty-client-sdk/core`
and `twenty-client-sdk/metadata` as **esbuild externals**. The stored
`.mjs` in S3 has unresolved bare import specifiers like `import {
CoreApiClient } from 'twenty-client-sdk/core'`.

SDK import resolution is split between the **frontend host** (fetching &
caching SDK modules) and the **Web Worker** (rewriting imports):

**Server endpoints**:
- `GET /rest/front-components/:id` —
`FrontComponentService.getBuiltComponentStream()` returns the **raw
`.mjs`** directly from file storage. No bundling, no SDK injection.
- `GET /rest/sdk-client/:applicationId/:moduleName` —
`SdkClientController` reads a single file (e.g. `dist/core.mjs`) from
the generated SDK archive via
`SdkClientGenerationService.readFileFromArchive()` and serves it as
JavaScript.

**Frontend host** (`FrontComponentRenderer` in `twenty-front`):
1. Queries `FindOneFrontComponent` which returns `applicationId`,
`builtComponentChecksum`, `usesSdkClient`, and `applicationTokenPair`
2. If `usesSdkClient` is `true`, renders
`FrontComponentRendererWithSdkClient` which calls the
`useApplicationSdkClient` hook
3. `useApplicationSdkClient({ applicationId, accessToken })` checks the
Jotai atom family cache for existing blob URLs. On cache miss, fetches
both SDK modules from `GET /rest/sdk-client/:applicationId/core` and
`/metadata`, creates **blob URLs** for each, and stores them in the atom
family
4. Once the blob URLs are cached, passes them as `sdkClientUrls`
(already blob URLs, not server URLs) to `SharedFrontComponentRenderer` →
`FrontComponentWorkerEffect` → worker's `render()` call via
`HostToWorkerRenderContext`

**Worker** (`remote-worker.ts` in `twenty-sdk`):
1. Fetches the raw component `.mjs` source as text
2. If `sdkClientUrls` are provided and the source contains SDK import
specifiers (`twenty-client-sdk/core`, `twenty-client-sdk/metadata`),
**rewrites** the bare specifiers to the blob URLs received from the host
(e.g. `'twenty-client-sdk/core'` → `'blob:...'`)
3. Creates a blob URL for the rewritten source and `import()`s it
4. Revokes only the component blob URL after the module is loaded — the
SDK blob URLs are owned and managed by the host's Jotai cache

This approach eliminates server-side esbuild bundling on every request,
caches SDK modules per application in the frontend, and keeps the
worker's job to a simple string rewrite.

## Summary Diagram

```
app:build (SDK)
  └─ twenty-client-sdk stub (metadata=real, core=stub)
       │
       ▼
WorkspaceMigrationRunnerService.run()
  └─ SdkClientGenerationService.generateAndStore()
       ├─ Copy stub package (package.json + dist/)
       ├─ replaceCoreClient() → regenerate core.mjs/core.cjs
       ├─ Zip entire package → upload to S3
       └─ Set isSdkLayerStale = true
              │
     ┌────────┴────────────────────┐
     ▼                             ▼
Logic Functions               Front Components
     │                             │
     ├─ Local Driver               ├─ GET /rest/sdk-client/:appId/core
     │   └─ downloadAndExtract     │    → core.mjs from archive
     │      → symlink into         │
     │        node_modules         ├─ Host (useApplicationSdkClient)
     │                             │    ├─ Fetch SDK modules
     └─ Lambda Driver              │    ├─ Create blob URLs
         └─ downloadArchiveBuffer  │    └─ Cache in Jotai atom family
            → reprefixZipEntries   │
            → publish as Lambda    ├─ GET /rest/front-components/:id
              layer                │    → raw .mjs (no bundling)
                                   │
                                   └─ Worker (browser)
                                        ├─ Fetch component .mjs
                                        ├─ Rewrite imports → blob URLs
                                        └─ import() rewritten source
```

## Next PR
- Estimate perf improvement by implementing a redis caching for front
component client storage ( we don't even cache front comp initially )
- Implem frontent blob invalidation sse event from server

---------

Co-authored-by: Charles Bochet <charlesBochet@users.noreply.github.com>
2026-03-24 18:10:25 +00:00
Paul Rastoin 75bb3a904d [SDK] Refactor clients (#18433)
# Intoduction

Closes https://github.com/twentyhq/core-team-issues/issues/2289

In this PR all the clients becomes available under `twenty-sdk/clients`,
this is a breaking change but generated was too vague and thats still
the now or never best timing to do so

## CoreClient
The core client is now shipped with a default stub empty class for both
the schema and the client
Allowing its import, will still raises typescript errors when consumed
as generated but not generated

## MetadataClient
The metadata client is workspace agnostic, it's now generated and
commited in the repo. added a ci that prevents any schema desync due to
twenty-server additions

Same behavior than for the twenty-front generated graphql schema
2026-03-09 15:32:13 +00:00
Charles Bochet 662de17644 ci: replace 4-core runners with ubuntu-latest (#18503)
## Summary

- Replace all `ubuntu-latest-4-cores` (paid larger runners) with
`ubuntu-latest` across CI workflows
- The free `ubuntu-latest` runner for public repos already provides **4
vCPUs + 16 GB RAM** — identical specs to the paid 4-core larger runner
- Affects 4 workflow files: `ci-server.yaml`, `ci-front.yaml`,
`ci-sdk.yaml`, `ci-zapier.yaml` (8 job definitions total, including the
10-shard integration test matrix)
- The `ubuntu-latest-8-cores` runners are intentionally **kept** for
memory-heavy jobs (frontend build, storybook build, E2E tests) where the
extra capacity (8 vCPUs, 32 GB RAM) is needed
2026-03-09 16:21:03 +01:00
Charles Bochet ef499b6d47 Re-enable disabled lint rules and right-size CI runners (#18461)
## Summary

- Re-enable one lint rule that was temporarily disabled during the
ESLint-to-Oxlint migration:
- **`twenty/sort-css-properties-alphabetically`** in twenty-front — 578
violations auto-fixed across 390 files
- Document why **`typescript/consistent-type-imports`** cannot be
auto-fixed in twenty-server: NestJS relies on `emitDecoratorMetadata`
for DI, so converting constructor parameter imports to `import type`
erases them at compile time and breaks dependency injection at runtime
- Right-size CI runners, reducing 8-core usage from 18 jobs to 3:

| Change | Jobs | Rationale |
|--------|------|-----------|
| **Keep 8-core** | `ci-merge-queue/e2e-test`,
`ci-front/front-sb-build`, `ci-front/front-build` | Heavy builds needing
max CPU + memory (10GB NODE_OPTIONS, full Storybook webpack bundling) |
| **8-core → 4-core** | `ci-server` (build, lint-typecheck, validation,
test, integration-test), `ci-front/front-sb-test`,
`ci-zapier/server-setup`, `ci-sdk/sdk-e2e-test` | Already sharded into
10-12 parallel instances, I/O-bound (DB/Redis), or moderate single
builds |
| **8-core → 2-core** | `ci-emails/emails-test` | Trivially lightweight
(build + curl health check) |
| **Removed** | `ci-front/front-chromatic-deployment` | Dead code —
permanently disabled with `if: false` |

- Fix merge queue CI issues:
- **Concurrency**: Use `merge_group.base_ref` instead of unique merge
group ref so new queue entries cancel previous runs
- **Required status checks**: Add `merge_group` trigger to all 6
required CI workflows (front, server, shared, website, docker-compose,
sdk) with `changed-files-check` auto-skipped for merge_group events —
status check jobs auto-pass without re-running full CI
- **Build caching**: Add Nx build cache restore/save to E2E test job
with fallback to `main` branch cache for faster frontend and server
builds

## Test plan

- [ ] CI passes on this PR (verifies lint rule auto-fix works)
- [ ] Verify 4-core runner jobs complete within their 30-minute timeouts
- [ ] Verify merge queue status checks auto-pass (ci-front-status-check,
ci-server-status-check, etc.)
- [ ] Verify merge queue E2E concurrency cancels previous runs when a
new PR enters the queue
2026-03-06 13:33:02 +00:00
Charles Bochet d37ed7e07c Optimize merge queue to only run E2E and integrate prettier into lint (#18459)
## Summary

- **Merge queue optimization**: Created a dedicated
`ci-merge-queue.yaml` workflow that only runs Playwright E2E tests on
`ubuntu-latest-8-cores`. Removed `merge_group` trigger from all 7
existing CI workflows (front, server, shared, website, sdk, zapier,
docker-compose). The merge queue goes from ~30+ parallel jobs to a
single focused E2E job.
- **Label-based merge queue simulation**: Added `run-merge-queue` label
support so developers can trigger the exact merge queue E2E pipeline on
any open PR before it enters the queue.
- **Prettier in lint**: Chained `prettier --check` into `lint` and
`prettier --write` into `lint --configuration=fix` across `nx.json`
defaults, `twenty-front`, and `twenty-server`. Prettier formatting
errors are now caught by `lint` and fixed by `lint:fix` /
`lint:diff-with-main --configuration=fix`.

## After merge (manual repo settings)

Update GitHub branch protection required status checks:
1. Remove old per-workflow merge queue checks (`ci-front-status-check`,
`ci-e2e-status-check`, `ci-server-status-check`, etc.)
2. Add `ci-merge-queue-status-check` as the required check for the merge
queue
2026-03-06 13:20:57 +01:00
Charles Bochet 364c944ca6 Improve build performance 2x (#18449)
## Summary

Front Before:
<img width="1199" height="670" alt="image"
src="https://github.com/user-attachments/assets/b978f67c-c0a6-49fc-bedd-a443f11c365d"
/>

Front After:
<img width="1199" height="670" alt="image"
src="https://github.com/user-attachments/assets/a4939dbb-a8b4-4c74-978c-daa7f27d00f3"
/>


Server Before:
<img width="1199" height="670" alt="image"
src="https://github.com/user-attachments/assets/da53e97f-ec65-4224-a656-ca41040aef6e"
/>


Server After:
<img width="1199" height="670" alt="image"
src="https://github.com/user-attachments/assets/8cdf3885-f515-4d6c-989f-a421a4e8206c"
/>


### CI Server Pipeline Restructuring
- Split monolithic `server-setup` job into three parallel jobs:
`server-build`, `server-lint-typecheck`, and `server-validation`
- `server-build` only handles build + Nx cache save (~1m vs old 3.5m),
unblocking downstream jobs faster
- `server-lint-typecheck` runs in parallel with no DB dependency
- `server-validation` handles DB setup, migration checks, and GraphQL
generation checks in parallel with tests
- Make `server-test` (unit tests) fully independent — no longer waits
for server-setup, builds its own artifacts
- Increase integration test shards from 8 to 10 for better parallelism
- Expected critical path reduction: ~10m → ~7m (~30% faster)

### CI Front Pipeline Improvements
- Use artifact upload/download for storybook build instead of rebuilding
in test shards
- Serve pre-built storybook via `http-server` in test jobs, with
`STORYBOOK_URL` env var
- Update `vitest.config.ts` to use `storybookUrl` when `STORYBOOK_URL`
is set
- Remove redundant `twenty-shared`, `twenty-ui`, `twenty-sdk` builds
from storybook test shards

### Vite Build Optimizations
- Conditionally enable `rollup-plugin-visualizer` behind `ANALYZE=true`
env var (not loaded by default)
- Broaden Istanbul coverage exclusions to skip test files, stories,
mocks, and decorators
- Remove `@tabler/icons-react` alias from twenty-front and storybook
configs
- Bundle `@tabler/icons-react` into twenty-ui instead of treating it as
an external dependency
- Add lazy loading with `React.lazy` + `Suspense` for all page-level
route components in `useCreateAppRouter`
2026-03-06 11:02:26 +01:00
Charles Bochet 9d57bc39e5 Migrate from ESLint to OxLint (#18443)
## Summary

Fully replaces ESLint with OxLint across the entire monorepo:

- **Replaced all ESLint configs** (`eslint.config.mjs`) with OxLint
configs (`.oxlintrc.json`) for every package: `twenty-front`,
`twenty-server`, `twenty-emails`, `twenty-ui`, `twenty-shared`,
`twenty-sdk`, `twenty-zapier`, `twenty-docs`, `twenty-website`,
`twenty-apps/*`, `create-twenty-app`
- **Migrated custom lint rules** from ESLint plugin format to OxLint JS
plugin system (`@oxlint/plugins`), including
`styled-components-prefixed-with-styled`, `no-hardcoded-colors`,
`sort-css-properties-alphabetically`,
`graphql-resolvers-should-be-guarded`,
`rest-api-methods-should-be-guarded`, `max-consts-per-file`, and
Jotai-related rules
- **Migrated custom rule tests** from ESLint `RuleTester` + Jest to
`oxlint/plugins-dev` `RuleTester` + Vitest
- **Removed all ESLint dependencies** from `package.json` files and
regenerated lockfiles
- **Updated Nx targets** (`lint`, `lint:diff-with-main`, `fmt`) in
`nx.json` and per-project `project.json` to use `oxlint` commands with
proper `dependsOn` for plugin builds
- **Updated CI workflows** (`.github/workflows/ci-*.yaml`) — no more
ESLint executor
- **Updated IDE setup**: replaced `dbaeumer.vscode-eslint` with
`oxc.oxc-vscode` extension, configured `source.fixAll.oxc` and
format-on-save with Prettier
- **Replaced all `eslint-disable` comments** with `oxlint-disable`
equivalents across the codebase
- **Updated docs** (`twenty-docs`) to reference OxLint instead of ESLint
- **Renamed** `twenty-eslint-rules` package to `twenty-oxlint-rules`

### Temporarily disabled rules (tracked in `OXLINT_MIGRATION_TODO.md`)

| Rule | Package | Violations | Auto-fixable |
|------|---------|-----------|-------------|
| `twenty/sort-css-properties-alphabetically` | twenty-front | 578 | Yes
|
| `typescript/consistent-type-imports` | twenty-server | 3814 | Yes |
| `twenty/max-consts-per-file` | twenty-server | 94 | No |

### Dropped plugins (no OxLint equivalent)

`eslint-plugin-project-structure`, `lingui/*`, `@stylistic/*`,
`import/order`, `prefer-arrow/prefer-arrow-functions`,
`eslint-plugin-mdx`, `@next/eslint-plugin-next`,
`eslint-plugin-storybook`, `eslint-plugin-react-refresh`. Partial
coverage for `jsx-a11y` and `unused-imports`.

### Additional fixes (pre-existing issues exposed by merge)

- Fixed `EmailThreadPreview.tsx` broken import from main rename
(`useOpenEmailThreadInSidePanel`)
- Restored truthiness guard in `getActivityTargetObjectRecords.ts`
- Fixed `AgentTurnResolver` return types to match entity (virtual
`fileMediaType`/`fileUrl` are resolved via `@ResolveField()`)

## Test plan

- [x] `npx nx lint twenty-front` passes
- [x] `npx nx lint twenty-server` passes
- [x] `npx nx lint twenty-docs` passes
- [x] Custom oxlint rules validated with Vitest: `npx nx test
twenty-oxlint-rules`
- [x] `npx nx typecheck twenty-front` passes
- [x] `npx nx typecheck twenty-server` passes
- [x] CI workflows trigger correctly with `dependsOn:
["twenty-oxlint-rules:build"]`
- [x] IDE linting works with `oxc.oxc-vscode` extension
2026-03-06 01:03:50 +01:00
Charles Bochet d48c58640c Migrate CI runners from Depot back to GitHub-hosted runners (#18347)
## Summary
- Replaces all `depot-ubuntu-24.04` runners with `ubuntu-latest`
- Replaces all `depot-ubuntu-24.04-8` runners with
`ubuntu-latest-8-cores`
- Updates storybook build cache keys in ci-front.yaml to reflect the
runner name change

Reverts the temporary Depot migration introduced in #18163 / #18179
across all 23 workflow files.
2026-03-03 14:14:27 +01:00
Paul Rastoin 9e8024a07a Restore depot (#18179) 2026-02-24 09:59:54 +01:00
Charles Bochet 129d1ede86 Change runners temp (#18163)
Temporarily moving all ubuntu-latest 1 core to depot except ci-website
2026-02-23 10:53:31 +01:00
Félix Malfait 6faef19f64 chore: replace depot.dev runners with GitHub-hosted runners (#17641)
## Summary
- Replace all `depot-ubuntu-24.04-8` runner references with the
equivalent GitHub-hosted `ubuntu-latest-8-cores` runner
- Updated across 4 workflow files: `ci-front.yaml`, `ci-server.yaml`,
`ci-emails.yaml`, `ci-sdk.yaml`
- Also updated cache key names in `ci-front.yaml` that referenced the
depot runner name

## Test plan
- [ ] Verify CI workflows run successfully on the new GitHub-hosted
larger runners
- [ ] Confirm cache keys work correctly with the updated naming

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
2026-02-02 16:30:13 +01:00
Marie ceba0972cd Remove tests run on push on main (#16971)
Since tests are now run in the pre-merge queue with the latest main
version, they need not to be run again when merged into main, it would
be the exact same thing
2026-01-06 17:53:08 +00:00
Félix Malfait 21ff42074d feat: implement skills system for AI agents (#16865)
## Summary
This PR introduces a Skills system for AI agents, inspired by the [Agent
Skills specification](https://agentskills.io/specification).

## Changes

### Backend
- **SkillEntity**: New database entity with migration for storing skills
- **V2 Sync Mechanism**: Implemented FlatSkill, builders, validators,
and action handlers following the v2 flat entity pattern
- **Standard Skills**: Pre-defined skills (workflow-building,
data-manipulation, dashboard-building, metadata-building, research,
code-interpreter, xlsx, pdf, docx, pptx)
- **GraphQL API**: CRUD operations for skills with proper guards and
permissions
- **Workspace Cache**: Integrated skills into the workspace cache system

### Frontend  
- **Skills Table**: Searchable table in AI settings showing all skills
- **Skill Form**: Create/edit page with Label (primary), Description,
and Content (markdown editor)
- **API Name**: Following existing patterns, name is derived from label
with advanced settings toggle for custom API names
- **Standard vs Custom**: Standard skills are read-only, custom skills
can be edited/deleted

## Key Design Decisions
- Skills are stored in the database (Salesforce-like approach) rather
than files
- Name is derived from Label by default (isLabelSyncedWithName pattern)
- Skills reference functions/files via @ mentions in markdown content
rather than explicit relations
- Standard skills are synced from code, custom skills are created via UI

## Screenshots
Skills table and form UI follow existing settings patterns.

## Testing
- [x] Lint passes
- [x] Typecheck passes
- [ ] CI tests
2026-01-02 14:22:01 +00:00
Félix Malfait e6491d6a80 feat(i18n): fix translation QA issues and add automation (#16756)
## Summary

This PR fixes translation QA issues and adds automation to prevent
future issues.

### Translation Fixes
- Fixed **escaped Unicode sequences** in translations (e.g.,
`\u62db\u5f85` → `招待`)
- Removed **corrupted control characters** from .po files (null bytes,
invalid characters)
- Fixed **missing/incorrect placeholders** in various languages
- Deleted **35 problematic translations** via Crowdin API that had
variable mismatches

### New Scripts (in `packages/twenty-utils/`)
- `fix-crowdin-translations.ts` - Auto-fixes encoding issues and syncs
to Crowdin
- `fix-qa-issues.ts` - Fixes specific QA issues via Crowdin API
- `translation-qa-report.ts` - Generates weekly QA report from Crowdin
API

### New Workflow
- `i18n-qa-report.yaml` - Weekly workflow that creates a PR with
translation QA issues for review

### Other Changes
- Moved GitHub Actions from `.github/workflows/actions/` to
`.github/actions/`
- Fixed `date-utils.ts` to avoid nested `t` macros in plural expressions
(root cause of confusing placeholders)

### QA Status After Fixes
| Category | Count | Status |
|----------|-------|--------|
| variables | 0  | Fixed |
| tags | 1 | Minor |
| empty | 0  | Fixed |
| spaces | 127 | Low priority |
| numbers | 246 | Locale-specific |
| special_symbols | 268 | Locale-specific |
2025-12-22 17:30:46 +01:00
Paul Rastoin 75bba5a8ee Standard Agent, Role, Role target (#16499)
# Introduction
In this pullrequest have been migrated to the flat standard entities:
Role, Agent and RoleTargets.

## What happens
- Removed createStandardMorph tool util in favor of dynamic typing of
`createMorphOrRelationStandardField`
- Implemented a command to remove standard agents and their role that
has been removed in https://github.com/twentyhq/twenty/pull/16513 also
added a default role target to data manipulator role to the only
remaining agent
- Implemented an agent deleteMany service handler
2025-12-16 14:17:31 +01:00
Félix Malfait a2a7ff8b4b Upgrade NestJS from 9.x to 10.x (#15835)
## Overview
This PR upgrades all NestJS dependencies from version 9.x to 10.x,
following the [official migration
guide](https://docs.nestjs.com/v10/migration-guide). This is the first
step before upgrading to NestJS 11.x in a future PR.

## Changes

### Dependencies Updated
- `@nestjs/common`: 9.4.3 → 10.4.16
- `@nestjs/core`: 9.4.3 → 10.4.16
- `@nestjs/passport`: 9.0.3 → 10.0.3
- `@nestjs/platform-express`: 9.4.3 → 10.4.16
- `@nestjs/config`: 2.3.4 → 3.2.3
- `@nestjs/event-emitter`: 2.0.4 → 2.1.0
- `@nestjs/testing`: ^9.0.0 → ^10.4.16
- `@nestjs/schematics`: ^9.0.0 → ^10.1.0

### Code Changes
- Fixed `CacheModuleOptions` import from `@nestjs/common` to
`@nestjs/cache-manager` in cache-storage.module-factory.ts

## Breaking Changes Addressed
 **CacheModule Migration**: Already using `@nestjs/cache-manager`
package
 **TypeScript Version**: Using 5.9.2 (requires 4.8+)  
 **Node.js Version**: Using 24.5.0 (requires 16+)  
 **Deprecated APIs**: All v9 deprecations removed in v10 - standard
patterns in use

## Testing
All tests passing and build successful:
-  Health module: 38 tests passed
-  Auth module: 115 tests passed (critical - uses @nestjs/passport)
-  Admin panel: 30 tests passed
-  REST API: 90 tests passed (uses @nestjs/platform-express)
-  Feature flags: 17 tests passed
-  Workspace: 23 tests passed
-  GraphQL query runner: 17 tests passed
-  **Total: 330+ tests passing**
-  Build: 3,683 files compiled successfully
-  Type checking: Passed
-  Linting: Passed

## Verification
Tested critical NestJS functionality:
- Authentication & Security (JWT, OAuth, guards)
- HTTP Platform (Express integration, REST endpoints)
- Dependency Injection (Services, factories, providers)
- Cache Management (Redis with @nestjs/cache-manager)
- GraphQL (Query runners, resolvers)
- Configuration (Environment config)
- Event Emitters (Event-driven architecture)

## Next Steps
This upgrade positions the codebase for the next upgrade to NestJS 11.x,
which will be handled in a separate PR.

<!-- CURSOR_SUMMARY -->
---

> [!NOTE]
> Upgrades NestJS to v10, fixes cache module import, and narrows CI
GraphQL generation diff to generated directories.
> 
> - **Backend**
> - **Dependencies**: Upgrade `@nestjs/common`, `@nestjs/core`,
`@nestjs/platform-express`, `@nestjs/passport`, `@nestjs/config`,
`@nestjs/event-emitter`, `@nestjs/testing`, and `@nestjs/schematics` to
v10-compatible versions in `packages/twenty-server/package.json`.
> - **Code**: Update `CacheModuleOptions` import to
`@nestjs/cache-manager` in
`src/engine/core-modules/cache-storage/cache-storage.module-factory.ts`.
> - **CI**
> - **GraphQL**: Restrict schema change detection to
`packages/twenty-front/src/generated` and
`packages/twenty-front/src/generated-metadata` in
`.github/workflows/ci-server.yaml`.
> 
> <sup>Written by [Cursor
Bugbot](https://cursor.com/dashboard?tab=bugbot) for commit
f621230fcf06ea86585b58ea9b61d6a5064230db. This will update automatically
on new commits. Configure
[here](https://cursor.com/dashboard?tab=bugbot).</sup>
<!-- /CURSOR_SUMMARY -->
2025-11-15 23:06:13 +01:00
Abdullah. 14c66c942f fix: update tmp to a safer version. (#15554)
@charlesBochet had a conversation with Felix and he said we don't need
to spend time upgrading `zapier-platform-core` and `zapier-platform-cli`
since `twenty-zapier` will be deprecated anyway, making those packages
irrelevant.

I have updated tmp to a safer version elsewhere using `yarn up tmp
--recursive`. Also added `yarn.lock` to both server and front ci.

The massive changes in `yarn.lock` were introduced by
`zapier-platform-cli` version 17x - not sure if they caused those
breaking changes, but if they did and we still want update
zapier-related packages, I will take that up in another PR.
2025-11-04 08:53:33 +01:00
Paul Rastoin 72fd8ae8d2 ci(server): integration server increase shard (#15228) 2025-10-21 16:03:50 +02:00
Félix Malfait 5375a478db Fix: Make CI .env manipulation robust against missing trailing newlines (#15189)
## Problem

CI workflow started timing out on October 14, 2025 after commit
`d750df7fff` removed the trailing newline from `.env.example`.

## Root Cause

When `.env.example` lacks a trailing newline:
```bash
# Last line without newline
# CLICKHOUSE_URL=...twenty
```

And CI runs:
```bash
echo "NODE_PORT=3002" >> .env
```

Result:
```bash
# CLICKHOUSE_URL=...twentyNODE_PORT=3002  ← Commented out!
```

Server starts on default port 3000 instead of 3002, health check fails.

## Fix

1. **Restore trailing newline** to `.env.example`
2. **Make all CI `.env` operations robust** by adding `echo "" >> .env`
before appending
3. **Simplified `set_env_var`** function to always add newline first

Now works regardless of whether template files have trailing newlines.

## Files Changed

- 6 CI workflow files
- 1 .env.example file
2025-10-18 13:46:56 +02:00
Félix Malfait d76abefdee Fix CI concurrency: prevent test cancellation on main branch (#15188)
## Problem
The concurrency rules in CI workflows were cancelling in-progress test
runs even on the main branch. This caused inconsistent check counts when
multiple commits were pushed in quick succession.

## Solution
Updated `cancel-in-progress` in all CI workflows to be conditional:
- **On main branch**: Tests run to completion (no cancellation)
- **On feature branches**: Tests are cancelled when new commits are
pushed (saves CI resources)

## Changes
Modified 11 workflow files to use:
```yaml
cancel-in-progress: ${{ github.ref != 'refs/heads/main' }}
```

This ensures every commit to main gets fully tested while maintaining
efficiency on feature branches.
2025-10-18 10:50:54 +02:00
Etienne c693c4d9cf Common API - create common find one query (#14720)
closes https://github.com/twentyhq/core-team-issues/issues/1418

Tested : 
- findOne on Rest and Gql

### Vision
#### Common
- Common is kind of renamed Gql base resolver
- Common handles args (filter, values, ...) validation
- Common accepts depth or raw gql selected fields to compute
selectedFields
- Common is directly called by each CommonQueries (findOne, ...)
service, which extend CommonBaseQuery service
- Common sequence : 
| - Parse & Validate args (args-handlers, to create)
| - Build query (query-parsers : currently in gql-query-parsers, to
move)
| - Execute query
| - Fetch relation + format
#### Rest
- Simple parsing (without metadata validation)
- Calling Common API
- Simple rest response formatting
#### Gql
- Calling Common API
2025-10-08 10:18:42 +00:00
Antoine Moreaux 93d55d1bc2 chore(workflows): update permissions across GitHub Actions workflows … (#14919)
…for consistency
2025-10-06 17:36:41 +02:00
Charles Bochet a21daa2670 Optimize CI runner cost (#14628) 2025-09-20 12:14:29 +02:00
Félix Malfait 30a2164980 First Application POC (#14382)
Quick proof of concept for twenty-apps + twenty-cli, with local
development / hot reload

Let's discuss it!



https://github.com/user-attachments/assets/c6789936-cd5f-4110-a265-863a6ac1af2d
2025-09-10 15:12:38 +02:00
Charles Bochet 953f1e7207 Fix CI cache for storybook (#14038)
In this PR:
- Migrate twenty-ui to ES2020 (removing undesired type errors during
build)
- Make CI cache work again between `sb-build` and `sb-test` steps

(Note: also fixing a minor bug on Notes)

Total CI time: ~8min
<img width="1088" height="531" alt="image"
src="https://github.com/user-attachments/assets/0bd0a99a-c69e-491d-91b2-9ddf6622464a"
/>

Both node_modules and sb-build caches are being hit now
<img width="1154" height="722" alt="image"
src="https://github.com/user-attachments/assets/a5addcee-51d2-4f51-9ff6-d02cd25da49a"
/>
2025-08-22 09:34:35 +02:00
Félix Malfait d29dbd473b Upgrade SWC Core and Storybook to v8 (#13799)
This is is a blocker for various sub-migrations
2025-08-11 12:02:33 +02:00
Antoine Moreaux 29f7b74756 fix(ci): reorganize workflow steps and move cache saving to correct s… (#13083)
…tage
2025-07-07 16:06:25 +00:00
Antoine Moreaux 0199d5f72a feat(ci): add GraphQL check workflow and update dependencies setup (#13075) 2025-07-07 14:51:18 +02:00
Paul Rastoin 3233734d2c [CI] Jest sharding integration tests (#12400)
# Introduction
Introducing jest sharding along github actions matrix in order to
fluidify the process

Note in case we will compute and guard coverage metrics we will need to
merge coverages reports such as we do for the frontend storybooks
integrations tests, from now this is overkill as unused

Related successful run
https://github.com/twentyhq/twenty/actions/runs/15585113583/job/43889477889
2025-06-11 16:05:40 +02:00
Charles Bochet a68895189c Deprecate old relations completely (#12482)
# What

Fully deprecate old relations because we have one bug tied to it and it
make the codebase complex

# How I've made this PR:
1. remove metadata datasource (we only keep 'core') => this was causing
extra complexity in the refactor + flaky reset
2. merge dev and demo datasets => as I needed to update the tests which
is very painful, I don't want to do it twice
3. remove all code tied to RELATION_METADATA /
relation-metadata.resolver, or anything tied to the old relation system
4. Remove ONE_TO_ONE and MANY_TO_MANY that are not supported
5. fix impacts on the different areas : see functional testing below 

# Functional testing

## Functional testing from the front-end:
1. Database Reset 
2. Sign In 
3. Workspace sign-up 
5. Browsing table / kanban / show 
6. Assigning a record in a one to many / in a many to one 
7. Deleting a record involved in a relation  => broken but not tied to
this PR
8. "Add new" from relation picker  => broken but not tied to this PR
9. Creating a Task / Note, Updating a Task / Note relations, Deleting a
Task / Note (from table, show page, right drawer)  => broken but not
tied to this PR
10. creating a relation from settings (custom / standard x oneToMany /
manyToOne) 
11. updating a relation from settings should not be possible 
12. deleting a relation from settings (custom / standard x oneToMany /
manyToOne) 
13. Make sure timeline activity still work (relation were involved
there), espacially with Task / Note => to be double checked  => Cannot
convert undefined or null to object
14. Workspace deletion / User deletion  
15. CSV Import should keep working  
16. Permissions: I have tested without permissions V2 as it's still hard
to test v2 work and it's not in prod yet 
17. Workflows global test  

## From the API:
1. Review open-api documentation (REST)  
2. Make sure REST Api are still able to fetch relations ==> won't do, we
have a coupling Get/Update/Create there, this requires refactoring
3. Make sure REST Api is still able to update / remove relation => won't
do same

## Automated tests
1. lint + typescript 
2. front unit tests: 
3. server unit tests 2 
4. front stories: 
5. server integration: 
6. chromatic check : expected 0
7. e2e check : expected no more that current failures

## Remove // Todos
1. All are captured by functional tests above, nothing additional to do

## (Un)related regressions
1. Table loading state is not working anymore, we see the empty state
before table content
2. Filtering by Creator Tim Ap return empty results
3. Not possible to add Tasks / Notes / Files from show page

# Result

## New seeds that can be easily extended
<img width="1920" alt="image"
src="https://github.com/user-attachments/assets/d290d130-2a5f-44e6-b419-7e42a89eec4b"
/>

## -5k lines of code
## No more 'metadata' dataSource (we only have 'core)
## No more relationMetadata (I haven't drop the table yet it's not
referenced in the code anymore)
## We are ready to fix the 6 months lag between current API results and
our mocked tests
## No more bug on relation creation / deletion

---------

Co-authored-by: Weiko <corentin@twenty.com>
Co-authored-by: Félix Malfait <felix@twenty.com>
2025-06-10 16:45:27 +02:00
Paul Rastoin f2691e53a0 [CI]: Increase status check timeout (#11896)
# Introduction
Some jobs start to fail because they exceed the timeout
Such as this job
https://github.com/twentyhq/twenty/actions/runs/14862320942/job/41730200276?pr=11881
2025-05-06 17:14:58 +02:00
Antoine Moreaux 587281a541 feat(analytics): add clickhouse (#11174) 2025-04-16 16:33:10 +00:00