houseassassin/twenty
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
13,821 Commits 1 Branch 0 Tags
59eead238ddecd2be34094cd25686a71cbefa0ff
Commit Graph

104 Commits

Author SHA1 Message Date
Thomas Trompette 96a2456367 feat(workflow): periodic core-consistency check for workflows, versions and triggers (#23103) 
Monitoring for the soft-ref migration. The `workflow` /
`workflowVersion` dual-write into core is **best-effort** (async, not
transactional; failures only go to Sentry), so `core.workflow` /
`core.workflowVersion` can silently drift from the workspace source of
truth. This adds a periodic job that detects that drift across **all
three workflow entities** and emits it as metrics.

Supersedes the earlier inline shadow-parity approach that lived on this
branch — that only covered trigger dispatch and added a cache read +
diff to every cron tick and every DB-event batch (too much hot-path
overhead). This is broader and fully off the dispatch path.

## What
A cron (`cron:workflow:core-consistency-check`, every 3 hours, wired
into `cron:register:all`). Per run:
- **Bounded**: `SELECT DISTINCT "workspaceId" FROM core."workflow"` —
only workspaces that actually use workflows (skips the large majority).
- Per such workspace, emit a drift metric per `(entity, driftType)` —
detect-only, plus a triage log:
- **workflow** and **workflowVersion** — `unlinked` / `missingCore` /
`orphanCore` / `fieldMismatch`, via cross-schema `COUNT` aggregates
(core + workspace are the same DB, so indexed joins — no rows pulled
into JS).
- **automated triggers** — the `workflowAutomatedTrigger` table vs the
`workflowAutomatedTriggerMaps` cache: `inTableNotCache` /
`inCacheNotTable` / settings `mismatch`.
- Per-workspace failures are isolated (caught → Sentry) so one bad
workspace does not stop the sweep.

## Why it is efficient
Two central `core.*` queries + a few `COUNT` queries per
*workflow-using* workspace, on a relaxed cadence, off the dispatch path.
Shardable across ticks later if needed.

## Metrics
`workflow-core-consistency/{workflow,version,automated-trigger}/drift`
counters, attribute `driftType`. Dashboards: twentyhq/twenty-infra#800.

## Not in scope
Detect-only — no auto-heal (the existing backfill/rebuild command can
heal). No dispatch or flag changes.

## Test
- The consistency SQL (workflow/version sync counts, orphan counts,
trigger read) validated against a live workspace (it surfaced real drift
there — unlinked versions + an orphan core version). The whole
cron→service→SQL→metric pipeline is proven live: the cron is already
emitting real drift counters on a running server.
- Unit specs for the service (clean → no metric; per-entity drift per
dimension; per-workspace error isolation).
- Command boots and registers via `cron:register:all` (verified).
Typecheck + lint clean.
 2026-07-23 12:30:12 +00:00
Weiko 66df0ac47c Switch application stop/start commands to Redis-backed global kill switch (#23202) 
## Context

[#23183](https://github.com/twentyhq/twenty/pull/23183) introduced the
right enforcement point: every logic-function execution is rejected
centrally before consuming the shared workspace throttle when its
application is stopped.

However, its server-wide path reads PostgreSQL for every execution
attempt. A kill switch is most useful while an application is producing
abnormal load, potentially while PostgreSQL is already under pressure.
The enforcement mechanism should not add more database traffic in that
situation.

This state is also operational and temporary. It is used to troubleshoot
an application, not as durable application configuration.

## What this PR changes

- Uses one global Redis key per application universal identifier:

  ```text
  module:applications:kill-switch:{applicationUniversalIdentifier}
  ```

- Keeps the check in `LogicFunctionExecutorService`, before the
workspace execution throttle.
- Adds a 60-second process-local cache for both present and absent keys.
- Deduplicates concurrent cache refreshes, so an execution burst causes
at most one Redis read per application and process.
- Fails open when Redis cannot be read and caches that result for the
same minute, avoiding a Redis retry storm.
- Removes the database columns, upgrade command, workspace-cache
recomputation, registration lookup, and stop/start CLI commands
introduced by #23183.
- Keeps disabled queued executions non-retriable, without emitting one
warning for every skipped payload.

The switch is operated directly in Redis. For example:

```redis
SET module:applications:kill-switch:{applicationUniversalIdentifier} 1 EX 3600
DEL module:applications:kill-switch:{applicationUniversalIdentifier}
```

Any value means stopped; deleting or expiring the key means enabled.

## Why this is a better fit

| | #23183 | This PR |
|---|---|---|
| State | Durable PostgreSQL fields | Ephemeral Redis key |
| Server-wide hot path | PostgreSQL lookup per execution | At most one
Redis lookup per app/process/minute |
| Scope | Workspace and application registration | Application universal
identifier across all workspaces |
| Operational cleanup | Explicit start command | `DEL`, eviction,
restart, or operator-selected TTL |
| Database dependency during an incident | Required | None |

The trade-off is deliberate: a Redis change can take up to 60 seconds to
reach every process, and the switch is lost when the cache key
disappears. That is acceptable for a temporary troubleshooting control
and keeps the normal execution path inexpensive.

Existing in-flight functions are not interrupted. New direct or queued
executions are rejected when they reach the executor.
 2026-07-23 12:29:22 +00:00
martmull a3f4acadb1 Add workspace and server level stop commands for applications (#23183) 
## Context

When an installed application misbehaves (e.g. a logic function loop
DDoSing the server or the database), we currently have no targeted way
to shut it down in production: the only kill switch is
`LOGIC_FUNCTION_TYPE=DISABLED`, which disables logic functions for the
whole instance. This PR adds an emergency stop mechanism at two levels:

- **Workspace level**: stop one installed application in its workspace.
- **Server level**: stop every application installed from an
`applicationRegistration`, across all workspaces.

## How it works

**New nullable `stoppedAt` columns** on `core.application` and
`core.applicationRegistration` (fast instance command `2.24.0`, with
`up`/`down` and `@WasIntroducedInUpgrade` decorators on the entities).

**Enforcement in a single choke point**:
`LogicFunctionExecutorService.execute()` is the funnel behind every
execution path (public route triggers, server route triggers, cron
triggers, database event triggers, workflow actions, agent tool calls,
manual GraphQL execution, install hooks). A new
`assertApplicationNotStopped` guard runs right after the flat entities
are resolved and throws `LOGIC_FUNCTION_DISABLED` (already mapped to a
403 on route triggers and handled by the GraphQL exception handler)
when:
- `flatApplication.stoppedAt` is set (workspace-level stop, read from
the cached flat application maps: zero extra runtime cost), or
- the linked registration is stopped (one indexed PK lookup, same
pattern as the existing per-execution server-variable query).

**Propagation**: the workspace-level stop invalidates and recomputes
`flatApplicationMaps` for the workspace, so all server instances pick
the flag up within the local cache TTL (100ms). The registration-level
flag is read live, so it is effective immediately.

## Ops commands

```bash
# Workspace level
yarn command:prod application:stop -a <application-id>
yarn command:prod application:start -a <application-id>

# Server level (all applications of the registration, all workspaces)
yarn command:prod application-registration:stop -r <application-registration-id>
yarn command:prod application-registration:start -r <application-registration-id>
```

Each command logs what was stopped/started and, for registrations, how
many installed applications are affected.

## Notes

- Stopped executions fail fast at the guard, so queued trigger jobs
(cron/db-event) burn a negligible amount of work while stopped.
- The two flags are independent: lifting a registration-level stop does
not clear workspace-level stops that were set individually, and vice
versa.
- Unit tests added for `ApplicationStopService`.


---
_Generated by [Claude
Code](https://claude.ai/code/session_01CjEnKUACn89aSgK1wEMH2d)_

<!-- This is an auto-generated description by cubic. -->
<a
href="https://cubic.dev/pr/twentyhq/twenty/pull/23183?utm_source=github"
target="_blank" rel="noopener noreferrer"
data-no-image-dialog="true"><picture><source
media="(prefers-color-scheme: dark)"
srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source
media="(prefers-color-scheme: light)"
srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img
alt="Review in cubic"
src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a>
<!-- End of auto-generated description by cubic. -->
 2026-07-22 18:30:56 +00:00
neo773 c04714b9e0 fix(messaging): register and harden webhook subscription renewal cron (#23006) 
The renewal cron was never wired into cron:register:all, so
Gmail/Calendar/Graph watches were never renewed and went dark ~7 days
after connect (the max watch lifetime all three providers allow).
Register it, fan renewals out as per-channel queue jobs scoped to active
workspaces, retry FAILED channels, and recreate Google Calendar watches
before stopping the old one.

<!-- This is an auto-generated description by cubic. -->
<a
href="https://cubic.dev/pr/twentyhq/twenty/pull/23006?utm_source=github"
target="_blank" rel="noopener noreferrer"
data-no-image-dialog="true"><picture><source
media="(prefers-color-scheme: dark)"
srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source
media="(prefers-color-scheme: light)"
srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img
alt="Review in cubic"
src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a>
<!-- End of auto-generated description by cubic. -->
 2026-07-20 17:26:47 +05:30
Félix Malfait ea9e11581c feat(billing): replace Stripe trial emails with fair, well-timed reminders (#22186) 
## Why

We currently rely on Stripe's automated trial-ending email. It misfires:
the global "remind 7 days before trial ends" setting lands the reminder
on **signup day** for the 7‑day no‑card trial, and the "your card will
be charged" copy makes no sense for a trial with no card. This replaces
it with our own honest, well‑timed, Twenty‑branded emails.

## 🔒 Safety — these emails are OFF by default

Because these reach real customers, the whole feature is gated behind a
kill‑switch that **defaults to `false`**:

- **`BILLING_REMINDER_EMAILS_ENABLED` (default `false`)** — checked
**both** at cron registration **and** on every job run (defense in
depth), so the emails can never be sent inadvertently (not on deploy,
not in staging, not via a stray trigger). They only go out once an
operator explicitly opts in.
- Also gated on `IS_BILLING_ENABLED` (cloud‑only; self‑hosters
unaffected).
- In non‑prod the email driver is typically `logger`, so even if enabled
there, nothing is actually sent.

A unit test asserts that with the switch off, **zero** emails are
produced.

## What it does

A daily cron (`0 8 * * *`) sends three honest, Twenty‑branded emails:

| Plan | Email | When |
|---|---|---|
| No‑card trial (7d) | "Add a card to keep your data" | **1 day before**
trial ends |
| Card‑on‑file trial (30d) | Upcoming‑charge heads‑up (cancel in one
click) | **7 days before** first charge |
| Yearly subscription | Renewal reminder (no surprise) | **7 days
before** each renewal |

- **Monthly renewals get no reminder** (avoids noise) — only the first
charge and annual renewals do.
- Branches no‑card vs with‑card on the customer's payment‑method flag
(with a trial‑duration fallback), so someone who adds a card mid‑trial
correctly gets the charge heads‑up instead of the add‑a‑card one.
- **Idempotent** per `(workspace, boundary date)` via workspace‑level
user vars — yearly reminders re‑fire each period, but the daily cron
never double‑sends.
- Offsets are configurable via new `BILLING_*_REMINDER_DAYS_BEFORE`
variables.

Also **warms up the tone** of the existing suspended / deleted workspace
emails (less robotic, fair, loss‑aversion framing) — these already act
as the "come back or lose your data" win‑back, so no extra win‑back
email was added.

## Rollout

1. Merge.
2. Disable Stripe's automated trial/renewal customer emails in the
Stripe dashboard.
3. Review copy/timing, then set `BILLING_REMINDER_EMAILS_ENABLED=true`
to turn the cron on.

## Notes for reviewers

- **i18n:** new English strings render via Lingui's msgid fallback;
translation catalogs are intentionally **not** included to keep the diff
focused (the repo extracts translations via its standard periodic
`lingui extract` sync — `main` already carries catalog drift). Diff is
18 code files.
- **Recipients:** reminders go to all workspace members, consistent with
the existing suspension emails. Happy to scope the charge‑related ones
to billing admins if preferred.
- **Follow‑ups discussed:** in‑app trial banner, loss‑aversion with real
record counts, and failed‑payment dunning are the higher‑leverage
conversion levers beyond this.

## Test plan

- [x] `typecheck` (twenty-server, twenty-emails)
- [x] oxlint type‑aware + oxfmt
- [x] Unit tests: no‑card path, with‑card path, idempotency, yearly
renewal, billing‑disabled, **kill‑switch off → no send** (6/6 green)
- [ ] Manual: set the flag on a staging instance with `logger` driver
and confirm the right email is logged at each boundary

https://claude.ai/code/session_0147ujzHv1X4vzimf4iGbnT4

---
_Generated by [Claude
Code](https://claude.ai/code/session_0147ujzHv1X4vzimf4iGbnT4)_

<!-- This is an auto-generated description by cubic. -->
<a
href="https://cubic.dev/pr/twentyhq/twenty/pull/22186?utm_source=github"
target="_blank" rel="noopener noreferrer"
data-no-image-dialog="true"><picture><source
media="(prefers-color-scheme: dark)"
srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source
media="(prefers-color-scheme: light)"
srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img
alt="Review in cubic"
src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a>
<!-- End of auto-generated description by cubic. -->
 2026-06-26 08:18:34 +02:00
Félix Malfait bb6da7b7d1 feat(code-interpreter): reuse a warm sandbox per conversation (E2B) (#21664) 
## What

The E2B code-interpreter driver created a **fresh sandbox on every
execution** and killed it in `finally`, so every call in a conversation
paid full cold-start and started blank. This PR keeps **one warm sandbox
per conversation** and, on idle, **pauses** it rather than killing it.

## How

- **Discovery without a registry:** the sandbox is tagged with the chat
`threadId` (scoped `workspaceId:threadId`) via E2B **metadata**, found
with `Sandbox.list({ query: { state: ['running','paused'], metadata }
})` and resumed with `Sandbox.connect()` (which auto-resumes a paused
sandbox). E2B is the source of truth — no Redis/DB mapping.
- **Pause/resume (E2B 2.x):** session sandboxes are created with
`lifecycle: { onTimeout: 'pause', autoResume: true }`. When idle they
**pause** — compute billing stops, filesystem **and** kernel/memory
state are preserved — and resume in ~1s on the next call. This replaces
the earlier keepalive approach.
- **No premature pause mid-run:** the sandbox is kept alive for
`max(execution timeout, idle window)`, so a long execution is never
paused underneath itself.
- **Tenant isolation:** discovery filters by the `twentySessionId` tag
and **re-checks it client-side**, so a loose server-side match can never
hand one conversation's warm sandbox (with its files, kernel state,
token) to another.
- **Concurrency:** executions sharing a session are serialized
in-process (one active stream per thread, run as a single job — the chat
resolver queues concurrent messages), so parallel tool calls can't race
the shared kernel.
- **Output isolation:** `/home/user/output` is reset at the start of
each reused run, so a call only returns the artifacts it actually
produced; durable state lives elsewhere and persists.

## SDK upgrade

`@e2b/code-interpreter` **`^1.0.4` → `^2.6.0`** (pulls `e2b@2.x`). The
typed pause/resume API, `lifecycle`, and the `state`/`metadata` list
filter only exist in the 2.x line; 1.x exposed them only as untyped
OpenAPI internals. `Sandbox.list()` is now a paginator (handled).

## Config

| Var | Default | Purpose |
|---|---|---|
| `CODE_INTERPRETER_TIMEOUT_MS` | `300000` | Max single-execution
duration. |
| `CODE_INTERPRETER_IDLE_TIMEOUT_MS` | `300000` | Idle window before the
warm sandbox auto-pauses. |

Reuse is always-on when a session id is present (chat path). The
workflow-agent path and the dev-only `LocalDriver` are unaffected.

## ⚠️ Open item before merge: paused-sandbox GC

E2B retains paused sandboxes **indefinitely** (no TTL). Unlike the old
keepalive path (which auto-killed on idle), pause means a conversation's
sandbox persists after the chat ends — so without garbage collection,
paused sandboxes accumulate (≈ one per historical conversation) and
consume storage. A GC policy is required; the approach + retention
window are being decided (see PR discussion). Also: the E2B runtime path
can't run in CI, so this still needs a **live smoke test** (reuse hit,
idle→pause, resume) and confirmation of paused-storage pricing before
rollout.

## Tests / checks

- Resolver unit tests (`getOrCreateSessionSandbox`): reuse+extend,
create-when-absent, duplicate reaping, connect-failure fallback,
keep-first-connectable-when-earlier-dead, **ignore cross-tenant
metadata**, and **kill-on-timeout-refresh-failure**.
- `nx typecheck twenty-server` (against e2b 2.x), `oxlint --type-aware`,
`oxfmt --check` all clean.

---------

Co-authored-by: Claude <noreply@anthropic.com>
 2026-06-17 08:12:24 +00:00
Félix Malfait f4ead89956 refactor(twenty-orm): migrate 23 grandfathered entities to WorkspaceScopedRepository (#20987) 
## Summary

Follow-up to #20953. Migrates 23 of the 30 entities that were left in
`WORKSPACE_SCOPED_EXEMPTIONS` last time, so the lint rule's
workspaceId-enforcement default now covers most of the core/metadata
schema.

### Migrated (23 entities, 88 files, 22 commits)

| Family | Entities |
|---|---|
| Trivial caches | `NavigationMenuItem`, `Skill`, `DataSource`,
`Webhook`, `CommandMenuItem`, `IndexMetadata` |
| Views | `View`, `ViewField`, `ViewFieldGroup`, `ViewFilter`,
`ViewFilterGroup`, `ViewGroup`, `ViewSort` |
| Layouts | `PageLayout`, `PageLayoutTab`, `PageLayoutWidget` |
| Roles & permissions | `Role`, `RoleTarget`, `PermissionFlag`,
`ObjectPermission`, `FieldPermission`, `RowLevelPermissionPredicate`,
`RowLevelPermissionPredicateGroup` |

For each entity: swap `@InjectRepository(X)` →
`@InjectWorkspaceScopedRepository(X)` (and the field type →
`WorkspaceScopedRepository<X>`); rewrite every call site to pass
`workspaceId` as the first arg (stripped from `where`/criteria — the
wrapper throws if you include it now); register
`provideWorkspaceScopedRepository(X)` in every owning NestJS module;
update affected spec providers to
`getWorkspaceScopedRepositoryToken(X)`.

### Rule update

- `ApplicationRegistrationVariableEntity` was misclassified — moved to
`STRUCTURAL_EXEMPTIONS` (no `workspaceId` column; it's keyed on
`applicationRegistrationId` at the instance level).
- 22 of the 23 migrated entities removed from
`WORKSPACE_SCOPED_EXEMPTIONS` entirely (zero remaining raw
`@InjectRepository` sites).
- `RoleTargetEntity` also removed; one call site in
`user-workspace.service.ts` keeps a raw injection with an
`eslint-disable` + reason because `softRemove(...)` is not on the
wrapper API yet (the migration would require threading `workspaceId`
through `deleteUserWorkspace`'s three callers).

### Still exempted (7 entities, follow-up PRs)

| Entity | Why deferred |
|---|---|
| `ApplicationEntity` | ~50 sites with several cross-workspace lookups
by id (auth, OAuth, file-storage, cleanup) |
| `CalendarChannelEntity` / `MessageChannelEntity` | Use
`.increment(...)` (not on wrapper) and
`repository.manager.transaction(...)` — wrapper needs to grow
`.increment` + the transaction sites need `withManager` or dual-inject |
| `FieldMetadataEntity` / `ObjectMetadataEntity` | The metadata services
`extends TypeOrmQueryService<X>` and `super(rawRepo)` — requires
dual-inject or reworking the inheritance |
| `KeyValuePairEntity` | Allows `workspaceId: IsNull()` for
instance-level config; wrapper rejects null |
| `UpgradeMigrationEntity` | Same — instance-level + cross-workspace
ledger |

## Test plan

- [x] `npx nx typecheck twenty-server` — clean
- [x] `npx nx lint twenty-server` — clean (0/0)
- [x] All 10 affected unit specs pass (115 tests) — api-key, agent-role,
permissions, workspace-roles-permissions-cache, view-filter-group,
workflow-version-step-operations, two-factor-authentication (service +
resolver), user-workspace, file
- [ ] Server integration tests in CI
 2026-05-28 20:46:21 +02:00
Charles Bochet 9988f98577 feat(server): idempotent CLI to rotate ENCRYPTION_KEY across enc:v2 rows (#20613) 
## Summary
Adds the \`secret-encryption:rotate\` CLI command, which re-encrypts
every at-rest secret stored in an \`enc:v2:\` envelope under the current
\`ENCRYPTION_KEY\`. The command is **online** and **resumable**: a SQL
filter skips rows already on the current keyId, so interrupting it
(Ctrl-C, container restart, …) and re-running picks up where it left off
without re-rotating earlier rows.

### Sites covered (one handler each)
| Site | Table.column | Scope |
| --- | --- | --- |
| \`connected-account-tokens\` | \`connectedAccount.{accessToken,
refreshToken}\` | workspace |
| \`application-variable\` | \`applicationVariable.value\` (isSecret
only) | workspace |
| \`application-registration-variable\` |
\`applicationRegistrationVariable.encryptedValue\` | instance |
| \`signing-key-private-keys\` | \`signingKey.privateKey\` | instance |
| \`sensitive-config-storage\` | \`keyValuePair.value\` (isSensitive +
STRING configs) | instance |
| \`totp-secrets\` | \`twoFactorAuthenticationMethod.secret\` |
workspace |

Each handler:
- Filters at SQL level on \`value LIKE 'enc:v2:%' AND value NOT LIKE
'enc:v2:<primaryKeyId>:%'\` to enforce idempotency without re-decrypting
already-rotated rows.
- Uses cursor-based batching (default **200**, capped **5000**).
- Threads \`workspaceId\` into HKDF for workspace-scoped sites; runs
instance-scoped for the rest.

### CLI flags
| Flag | Description |
| --- | --- |
| \`-s, --site <site>\` | Limit to a single site. |
| \`-b, --batch-size <n>\` | Override per-batch row count. |
| \`-d, --dry-run\` | Decrypt + re-encrypt in memory, skip the
\`UPDATE\`. |

The runner logs progress via Nest \`Logger\` (per-site start,
completion, final summary) and exits non-zero when any site reports
\`errors > 0\`. \`FALLBACK_ENCRYPTION_KEY\` must be set to the previous
\`ENCRYPTION_KEY\` during rotation; the runner warns when it is unset.

Operator documentation lives in #20611 (docs PR).
 2026-05-20 17:51:29 +00:00
Charles Bochet 72ce77864e feat(server): Enterprise cron that rotates the current JWT signing key (#20612) 
## Summary
Adds a daily Enterprise-only cron that rotates the current ES256 JWT
signing key once it has been current for `SIGNING_KEY_ROTATION_DAYS`.
Manual rotation from the admin panel is unaffected.

### Behaviour
- `SIGNING_KEY_ROTATION_DAYS` is **opt-in**: when unset, the cron is a
no-op.
- Rotation flips `isCurrent` and clears the previous key's `privateKey`
in the same transaction, then inserts the new `isCurrent=true` row.
- The previous key's row is kept (`revokedAt` stays `null`) so its
`publicKey` can keep verifying tokens it signed until they expire; only
the encrypted `privateKey` is wiped since it can no longer be used to
sign.
- **No auto-revocation** — revoking a key remains a manual admin action,
reserved for leak / emergency response.
- The cron is also a no-op when `EnterprisePlanService.isValid()` is
`false`.

### Wiring
- `JwtKeyManagerService.rotateCurrent()`
- `SigningKeyRotationService.rotateIfDue()` (reads
`SIGNING_KEY_ROTATION_DAYS`, skips when unset)
- `RotateSigningKeysCronJob` (Enterprise-gated, rethrows on failure)
registered in `JwtModule`
- `RotateSigningKeysCronCommand` registered with `cron:register:all`
- `ROTATE_SIGNING_KEYS_CRON_PATTERN = '15 3 * * *'` (daily, no-op until
threshold)

Operator documentation lives in #20611 (docs PR).
 2026-05-19 10:41:04 +00:00
Etienne fcd2d586ee chore(billing) - remove feature flag (#20531) 
- remove feature flag
- remove old enforce cap usage logic
 2026-05-13 18:52:29 +02:00
Félix Malfait 4f938aa097 feat(app): infrastructure for pre-installed apps (#19973) 
**PR 1 of 2.** Follow-up PR ships the Exa app, sets it as a default
pre-installed app, and removes the current `WebSearchTool` /
`WebSearchService` / `ExaDriver`. This PR adds the plumbing; no
user-visible change yet.

## Summary

- Server admins can declare a list of npm app packages to auto-install
on every new workspace and backfill onto existing workspaces via CLI.
- Server-level secrets (like Exa's API key) live on the
`ApplicationRegistration` (one row per server, encrypted) and are
injected into logic function execution env at runtime. No more
per-workspace storage of global secrets.
- A generic `POST /app/billing/charge` endpoint lets app logic functions
emit workspace usage events for metered features. Exa uses it in PR 2;
future apps (call recorder, etc.) reuse it.
- `LogicFunctionToolProvider` tool name prefix changes `logic_function_`
→ `app_`. Shorter, accurate (they come from installed apps).

## What's in this PR

**Logic function executor — server-level variables**
- `LogicFunctionExecutorService.getExecutionEnvVariables` now resolves
env vars in the order: hardcoded defaults →
`ApplicationRegistrationVariable[]` (server-level) →
`ApplicationVariable[]` (workspace-level override). The manifest
`serverVariables` schema has existed; this closes the loop.

**Config**
- `PRE_INSTALLED_APPS` — comma-separated list of npm packages. Default:
empty.

**\`PreInstalledAppsService\`** (new module)
- \`onApplicationBootstrap()\` — fetches each package's manifest from
the app registry CDN, upserts an \`ApplicationRegistration\`, and seeds
declared \`serverVariables\` from matching env vars (e.g.
\`EXA_API_KEY\` env → encrypted registration variable).
- \`installOnWorkspace(workspaceId)\` — installs all pre-installed apps
on a single workspace. Tolerates per-app failures.

**Auto-install on new workspace activation**
- \`WorkspaceService.prefillCreatedWorkspaceRecords\` invokes
\`installOnWorkspace\` after prefilling standard records. Non-blocking
on failure.

**Backfill CLI command**
- \`install-pre-installed-apps\` — iterates active and suspended
workspaces, installs pre-installed apps that aren't yet installed.
Idempotent. Run after changing \`PRE_INSTALLED_APPS\`.

**App billing endpoint**
- \`POST /app/billing/charge\`. Authenticated via \`APPLICATION_ACCESS\`
token (already injected into logic function execution env as
\`DEFAULT_APP_ACCESS_TOKEN\`). Body: \`{ creditsUsedMicro, quantity,
unit, operationType, resourceContext? }\`. Emits \`USAGE_RECORDED\` with
\`applicationId\` as \`resourceId\`. Generic — reusable by any app.

**Tool name prefix**
- \`LogicFunctionToolProvider.buildLogicFunctionToolName\` now produces
\`app_<name>\` instead of \`logic_function_<name>\`. Only affects tools
sourced from logic functions; other tool providers unchanged.

## Stats

- 16 files, +501 / −2
- 7 new files (1 command, 1 service × 2, 1 controller, 1 DTO, 2 modules)
- Typecheck: 7 pre-existing errors, zero new
- Prettier clean

## Behavior deltas

- **\`PRE_INSTALLED_APPS\` default = empty**: existing servers see no
change on merge.
- **\`ApplicationRegistrationVariable\` is now read by the executor**:
apps that were using manifest \`serverVariables\` but expecting them to
be ignored by the executor will now see them injected. No apps ship with
\`isTool: true\` logic functions today, so this is latent — first
consumer is Exa in PR 2.
- **Tool prefix**: currently no logic-function tools are named
\`logic_function_*\` in any production flow. The prefix change affects
only future tools emitted by \`LogicFunctionToolProvider\`.

## Risks

- **CDN unavailability at startup**: if the app registry CDN is down,
\`ensureRegistrationsExist\` logs warnings but doesn't block server
start. Installation on new workspaces during this window will find no
registrations and log a non-blocking error. Backfill command can retry
after CDN recovers.
- **Cold-start overhead**: \`ensureRegistrationsExist\` is called once
per process on bootstrap. Current configurable default is empty, so zero
overhead. When an admin sets \`PRE_INSTALLED_APPS\`, they accept one
HTTP call per package at boot.
- **Server-level variables flow**:
\`ApplicationRegistrationVariable.encryptedValue\` is shared by all
workspaces of a server. Appropriate for a single-tenant Exa key. Not
appropriate for per-tenant keys — those go in workspace-level
\`ApplicationVariable\` and override.

## Test plan

- [ ] \`npx nx typecheck twenty-server\` passes (verified: 7
pre-existing unrelated errors, zero new)
- [ ] Set \`PRE_INSTALLED_APPS=@twenty-apps/hello-world\` (or any real
npm-published app), \`HELLO_WORLD_API_KEY=xxx\`, restart server:
\`ApplicationRegistration\` row is upserted,
\`ApplicationRegistrationVariable\` for HELLO_WORLD_API_KEY is populated
(encrypted).
- [ ] Create a new workspace: the app is auto-installed,
\`ApplicationEntity\` row created, \`LogicFunctionEntity\` rows created.
- [ ] Existing workspace: run \`yarn nx run twenty-server:command
install-pre-installed-apps\`: apps install across all workspaces,
idempotent on re-run.
- [ ] Trigger a logic function that reads
\`process.env.HELLO_WORLD_API_KEY\`: value resolves from the
server-level \`ApplicationRegistrationVariable\`.
- [ ] Log a charge from the handler: \`POST /app/billing/charge\` with
\`Authorization: Bearer \$DEFAULT_APP_ACCESS_TOKEN\` body
\`{creditsUsedMicro: 1000, quantity: 1, unit: "INVOCATION",
operationType: "WEB_SEARCH"}\` → returns \`{success: true}\`,
\`USAGE_RECORDED\` event emitted with correct
\`resourceId=applicationId\`.
- [ ] Tool name generated by \`LogicFunctionToolProvider\` starts with
\`app_\`.

## What's NOT in this PR (PR 2 scope)

- The Exa app itself (\`packages/twenty-apps/...\` directory)
- Removing \`WebSearchTool\`, \`WebSearchService\`, \`ExaDriver\`,
\`web-search\` module
- Removing \`WEB_SEARCH_DRIVER\` config var
- Removing the current \`exa_web_search\` entry in
\`ActionToolProvider\`
- Chat preload list updated to \`app_exa_web_search\`
- Frontend \`getToolDisplayMessage\` branch for \`app_exa_web_search\`
- Setting \`PRE_INSTALLED_APPS\` default to include \`@twenty-apps/exa\`

---------

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-04-23 21:59:00 +02:00
Paul Rastoin 61fdb613e6 Reset default app packages command (#19931) 
# Introduction
If a self host creates its twenty instance using storage type local, and
then edit through the admin panel the storage type, the apps default
deps file won't be swapped to the new storage location

This command allow to manually rebuild them

## What could be done in addition
- We could display a modal in the admin panel when the user is editing
env variable that might have a side effect
- We might wanna rebuild the deps by default if we detect such a change
through the UI though we can't really if it's through the `.env` so I'm
not sure we wanna prio such logic
 2026-04-21 13:32:59 +00:00
Paul Rastoin bf410ae438 upgrade:status command (#19584) 
## Introduction
Introducing a new command in order to determine the curent twenty
instance and workspaces status as it's not stored in database but a
derivation of each current curors


## `upgrade:status` all healthy
<img width="1376" height="1202" alt="image"
src="https://github.com/user-attachments/assets/e90d6987-07d2-4b6b-b573-105249aca325"
/>

## `upgrade:status` Nearly use cases
<img width="1442" height="1304" alt="image"
src="https://github.com/user-attachments/assets/c336cb9d-eb9d-4c7d-9392-ec1ef54a7326"
/>

## `upgrade:status --failed-only`
<img width="1442" height="940" alt="image"
src="https://github.com/user-attachments/assets/93a3dfdb-0d2f-4a01-b185-118e5cf0a078"
/>

## `upgrade:status -w aa8fdcb1-8ee1-4012-98af-44a97caa7411 -w
20202020-1c25-4d02-bf25-6aeccf7ea419 -w
20202020-1c25-4d02-bf25-6aeccf7ea412`
<img width="1486" height="928" alt="image"
src="https://github.com/user-attachments/assets/ec1b1abc-46e8-4e36-9799-ab3a4b85e410"
/>
 2026-04-16 16:05:02 +00:00
Paul Rastoin 3e699c4458 Fix upgrade commands discovery outside of cli (#19671) 
# Introduction
We were allowing the sequence to be empty in the worker context that was
facing an edge case importing the UpgradeModule through the
WorkspaceModule god module, no commands were discovered and it was
throwing as the sequence must have at least one workspace commands to
allow a workspace creation

Though the issue was also applicable to the twenty-server `AppModule`
too that was not discovering any commands

## Integration tests were passing
The integration test were importing the `CommandModule` at the nest
testing app creating leading to asymmetric testing context
It was a requirement for a legacy commands import and global assignation

## Fix
The `UpgradeModule` now import both `WorkspaceCommandsProviderModule`
and `InstanceCommandProviderModule` which ships the commands directly in
the module
We could consider moving the commands into the `engine/upgrade` folder

## Concern
Bootstrap could become more and more long to load at both server and
worker start
When this becomes a problem we will have to only import the latest
workspace command or whatever
For the moment this is not worth it the risk to import not the latest
workspace command
 2026-04-14 09:20:33 +00:00
Félix Malfait 455022f652 Add ClickHouse-backed metered credit cap enforcement (#19586) 
## Summary
Implements a ClickHouse-backed polling system to enforce metered-credit
caps for workflow executions, replacing reliance on Stripe billing
alerts. The system re-evaluates tier caps against live pricing on every
poll cycle, allowing price/tier changes to propagate immediately without
recreating Stripe alert objects.

## Key Changes

- **BillingUsageCapService**: New service that queries ClickHouse for
current-period credit usage and evaluates whether a subscription has
reached its metered-credit allowance (tier cap + credit balance)
  - `isClickHouseEnabled()`: Checks if ClickHouse is configured
- `getCurrentPeriodCreditsUsed()`: Sums creditsUsedMicro from usageEvent
table for a workspace within a billing period
- `evaluateCap()`: Determines if usage has reached the allowance by
reading live pricing from the subscription

- **EnforceUsageCapJob**: Cron job that polls all active subscriptions
and updates `hasReachedCurrentPeriodCap` on metered items
  - Runs every 2 minutes to keep cap enforcement in sync with live usage
- Supports shadow mode (log-only) via
`BILLING_USAGE_CAP_CLICKHOUSE_ENABLED` flag for safe rollout
- Continues processing after per-subscription errors with detailed
logging

- **EnforceUsageCapCronCommand**: CLI command to register the
enforcement cron job

- **MeteredCreditService**: Extracted
`extractMeteredPricingInfoFromSubscription()` as a pure function for
callers that already hold the subscription with pricing loaded, avoiding
redundant DB queries

- **Configuration**: Added `BILLING_USAGE_CAP_CLICKHOUSE_ENABLED` flag
to control enforcement mode (active vs. shadow)

- **Constants**: Added `METERED_OPERATION_TYPES` to define which
operation types count toward the metered product's credit cap

## Implementation Details

- The service queries ClickHouse for the sum of `creditsUsedMicro` in
the current billing period, matching Stripe meter semantics
- Pricing is re-read on every evaluation, so tier changes propagate
within one poll cycle without Stripe alert recreation
- The cron job only updates the database when the cap state actually
changes (no-op if already in the correct state)
- Shadow mode allows safe validation before enabling enforcement;
transitions are logged but not persisted
- Comprehensive test coverage for both the service and cron job,
including error handling and state transitions

https://claude.ai/code/session_01VksTSrYLXJVCPVBQhQdBTe

---------

Co-authored-by: Claude <noreply@anthropic.com>
 2026-04-13 21:17:41 +02:00
Paul Rastoin 21142d98fe Implement cross version upgrade (#19559) 
# Introduction
Refactoring the upgrade engine to handle cross version upgrade,
completely getting rid of the semver `version` at db and runtime level
It remains a visual a listing indicator for or CD process but also
during devenv in order to prepare next release
Will write a release process runbook documentation on how to handle
upgrade step patch, command insertion etc as it needs to be cascaded
across all the involved supported version

**The upgrade sequence model:**

The sequence is a flat, ordered array of upgrade steps
(`UpgradeStep[]`), built from the registry by chaining all versions in
order, each version contributing its fast-instance → slow-instance →
workspace commands sorted by timestamp. Version is metadata for logging,
not used in the algorithm.

**Segments:**

The sequence naturally splits into alternating segments of contiguous
instance steps and contiguous workspace steps. The runner processes
segments in order:

- **Instance segment:** Run sequentially from the instance cursor. Each
step runs once globally.
- **Workspace segment:** Each workspace independently walks from its own
cursor through the end of the segment. Workspaces are independent within
a segment — they can be at different positions.
- **Synchronization (workspace → instance):** The runner blocks before
entering an instance segment. All active/suspended workspaces must have
completed the last workspace step of the preceding workspace segment. If
any workspace failed, abort. This is the only explicit synchronization
point.
- Instance → workspace ordering is implicit — the runner processes
segments sequentially, so the instance segment naturally completes
before the workspace segment begins.


full docs
https://gist.github.com/prastoin/e62106d455fd72d6b6ebada8351e5492

## Version constants & type-level deprecation

Version management is split into three atomic constants:
`TWENTY_PREVIOUS_VERSIONS`, `TWENTY_CURRENT_VERSION`, and
`TWENTY_NEXT_VERSIONS`. Two derived constants compose them:
`CROSS_UPGRADE_SUPPORTED_VERSIONS` (previous + current — what the engine
runs) and `ALL_TWENTY_VERSIONS` (the full ordered tuple including next).
The registry service validates at module init that no version is
duplicated across constants and that at least one previous version
exists.

A `DeprecatedSinceVersion<RemoveAtVersion, T>` type utility resolves to
`T` while `TWENTY_CURRENT_VERSION` is below `RemoveAtVersion`, and to
`never` once it reaches it — turning deprecation into a compile-time
guarantee via `IndexOf` and `IsGreaterOrEqual` generics in
`twenty-shared`.

### `workspace.version` column deprecation

The column is replaced by cursor-based state inference from
`UpgradeMigration` records, but cannot be dropped in 1.22: workspaces
activated during 1.21 predate the cursor system and need their initial
cursor backfilled first (`backfillWorkspaceCreatedIn1_21_0Cursors`).
This backfill itself depends on a new `isInitial` column on
`UpgradeMigration`, bootstrapped via a targeted TypeORM migration before
the upgrade sequence runs.

Both functions and the entity field are typed with
`DeprecatedSinceVersion<'1.23.0', ...>`. When `TWENTY_CURRENT_VERSION`
reaches `1.23.0`, compile errors force their removal — and the
pre-declared `DropWorkspaceVersionColumnFastInstanceCommand` takes over
to drop the column.

## What's next
- ci cross version upgrade ( wip )
- banner asking to contact twenty administrator if workspace is outdated
- upgrade healthcheck cli 

## New unit/integ test pattern
Create a dedicated `createNestApp` that consumes a real database in
order not to have to mack any database interaction to the
`upgradeMigrations` allowing full coverage of the whole
`upgradeRunnerService.run` core logic
 2026-04-13 11:42:27 +02:00
Charles Bochet f6423f5925 Remove DataSourceService and clean up datasource migration logic (#19532) 
## Summary

- **Drop the `objectMetadata.dataSourceId` foreign key and index** via a
1-22 fast instance command — column kept nullable for data preservation
- **Delete `DataSourceService`, `DataSourceModule`, and
`DataSourceException`** — all code now uses `workspace.databaseSchema`
directly
- **Remove `IS_DATASOURCE_MIGRATED` feature flag** from default flags
and all branching logic
- **Simplify workspace/object creation pipelines** —
`WorkspaceManagerService`, `DevSeederService`, and the object creation
action handler no longer route through `DataSourceService`
- **Keep `DataSourceEntity` and the `dataSource` table** for historical
data — entity stripped of all ORM relations
 2026-04-10 07:34:05 +00:00
Paul Rastoin 23874848a4 Instance commands and upgrade_migrations table (#19356) 
# Introduction
Now only using typeorm to generate migrations up and down statement
We handle and maintain our own migration table history

## What's new
Now all the instance commands will live within the same module and
folder than the upgrade commands
Sequentiality comes from the timestamp located in the filename
Same sequentiality also applies to the workspace commands in the future,
for the moment still expected a as code explicit declaration

( below screen is an example see below section )
<img width="1382" height="634" alt="image"
src="https://github.com/user-attachments/assets/5610a246-4eae-485e-99f4-98fb89ad5ac8"
/>

## Existing 1.21 migrations
We won't start following this pattern in 1.21 yet at least not with the
migration that has already been released as typeorm migrations in cloud
production as they would rerun


## Small duplication
Duplicating the legacy typeorm and instance commands run in the
`run-instance-commands` to avoid any merge of interest for the moment

## Concurrency
Not handling any run in parrallel of the upgrade for the moment
 2026-04-07 08:55:17 +00:00
Paul Rastoin e062343802 Refactor typeorm migration lifecycle and generation (#19275) 
# Introduction

Typeorm migration are now associated to a given twenty-version from the
`UPGRADE_COMMAND_SUPPORTED_VERSIONS` that the current twenty core engine
handles

This way when we upgrade we retrieve the migrations that need to be run,
this will be useful for the cross-version incremental upgrade so we
preserve sequentiality

## What's new

To generate
```sh
npx nx database:migrate:generate twenty-server -- --name add-index-to-users
```

To apply all
```sh
npx nx database:migrate twenty-server
```

## Next
Introduce slow and fast typeorm migration in order to get rid of the
save point pattern in our code base
Create a clean and dedicated `InstanceUpgradeService` abstraction
 2026-04-06 07:11:47 +00:00
Paul Rastoin 281bb6d783 Guard yarn database:migrate:prod (#19008) 
## Motivations
A lot of self hosters hands up using the `yarn database:migrated:prod`
either manually or through AI assisted debug while they try to upgrade
an instance while their workspace is still blocked in a previous one
Leading to their whole database permanent corruption

## What happened
Replaced the direct call the the typeorm cli to a command calling it
programmatically, adding a layer of security in case a workspace seems
to be blocked in a previous version than the one just before the one
being installed ( e.g 1.0 when you try to upgrade from 1.1 to 1.2 )

For our cloud we still need a way to bypass this security explaining the
-f flag

## Remark
Centralized this logic and refactored creating new services
`WorkspaceVersionService` and `CoreEngineVersionService` that will
become useful for the upcoming upgrade refactor

Related to https://github.com/twentyhq/twenty-infra/pull/529
 2026-03-27 14:39:18 +00:00
Félix Malfait c9ca4dfa14 fix: run AI catalog sync as standalone script to avoid DB dependency (#18853) 
## Summary
- The `ci-ai-catalog-sync` cron workflow was failing because the
`ai:sync-models-dev` NestJS command bootstraps the full app, which tries
to connect to PostgreSQL — unavailable in CI.
- Converted the sync logic to a standalone `ts-node` script
(`scripts/ai-sync-models-dev.ts`) that runs without NestJS, eliminating
the database dependency.
- Removed the `Build twenty-server` step from the workflow since it's no
longer needed, making the job faster.

## Test plan
- [x] Verified the standalone script runs successfully locally via `npx
nx run twenty-server:ts-node-no-deps-transpile-only --
./scripts/ai-sync-models-dev.ts`
- [x] Verified `--dry-run` flag works correctly
- [x] Verified the output `ai-providers.json` is correctly written with
valid JSON (135 models across 5 providers)
- [x] Verified the script passes linting with zero errors
- [ ] CI should pass without requiring a database service

Fixes:
https://github.com/twentyhq/twenty/actions/runs/23424202182/job/68135439740

Made with [Cursor](https://cursor.com)
 2026-03-23 13:07:53 +01:00
Félix Malfait 908aefe7c1 feat: replace hardcoded AI model constants with JSON seed catalog (#18818) 
## Summary

- Replaces per-provider TypeScript constant files
(`openai-models.const.ts`, `anthropic-models.const.ts`, etc.) with a
single `ai-providers.json` catalog as the source of truth
- Adds runtime model discovery via AI SDK for self-hosted providers,
with `models.dev` enrichment for pricing/capabilities
- Introduces composite model IDs (`provider/modelId`) for canonical,
conflict-free identification
- Simplifies provider configuration: API keys are injected from
environment variables (e.g., `OPENAI_API_KEY`)
- Adds admin panel UI for provider management (add/remove/test), model
discovery, recommended model configuration, and default fast/smart model
selection per workspace
- Removes deprecated config variables (`AI_DISABLED_MODEL_IDS`,
`AUTO_ENABLE_NEW_AI_MODELS`, etc.)
- Adds database migration for composite model ID format

## Test plan

- [ ] Server typecheck passes
- [ ] Frontend typecheck passes
- [ ] Server unit tests pass
- [ ] Frontend unit tests pass
- [ ] CI pipeline green
- [ ] Admin panel AI tab loads correctly
- [ ] Provider discovery works for configured providers
- [ ] Model recommendation toggles persist
- [ ] Default fast/smart model selection works


Made with [Cursor](https://cursor.com)
 2026-03-21 16:03:58 +01:00
neo773 87efaf2ff8 workspace:export command (#18695) 
This PR adds a `workspace:export` server command for ongoing debug
tooling/administrative work

Demo Video shows
1. Exporting a sample workspace YC
2. Restoring Local DB Docker volume snapshot to Dropped YC Workspace
3. Importing exported SQL
4. Booting and navigating the imported Workspace



https://github.com/user-attachments/assets/0e1ac6cb-8ce1-440b-8b56-f81dcb27a9c8
 2026-03-18 10:40:12 +00:00
Félix Malfait 95a35f8a1d Implement OAuth 2.0 Dynamic Client Registration (RFC 7591) (#18608) 
## Summary
This PR implements OAuth 2.0 Dynamic Client Registration (RFC 7591) and
OAuth 2.0 Protected Resource Metadata (RFC 9728) support, enabling
third-party applications to dynamically register as OAuth clients
without manual configuration.

## Key Changes

### OAuth Dynamic Client Registration
- **New Controller**: `OAuthRegistrationController` at `POST
/oauth/register` endpoint
  - Validates client metadata according to RFC 7591 specifications
  - Enforces PKCE-only public client model (no client secrets)
- Supports only `authorization_code` grant type and `code` response type
  - Rate limits registrations to 10 per hour per IP address
  - Returns `client_id` and registration metadata in response

- **Input Validation**: `OAuthRegisterInput` DTO with constraints on:
  - Client name (max 256 chars)
  - Redirect URIs (max 20, validated for security)
  - Grant types, response types, scopes, and auth methods
  - Logo and client URIs (max 2048 chars)

- **Discovery Endpoint Update**: Added `registration_endpoint` to OAuth
discovery metadata

### Stale Registration Cleanup
- **Cleanup Service**: Automatically removes OAuth-only registrations
older than 30 days that have no active installations
- **Cron Job**: Runs daily at 02:30 AM UTC with batch processing (100
records per batch)
- **CLI Command**: `cron:stale-registration-cleanup` to manually trigger
cleanup

### MCP (Model Context Protocol) Authentication
- **New Guard**: `McpAuthGuard` implements RFC 9728 compliance
  - Wraps JWT authentication with proper error responses
- Returns `WWW-Authenticate` header with protected resource metadata URL
on 401
  - Enables OAuth-protected MCP endpoints

### Protected Resource Metadata
- **New Endpoint**: `GET /.well-known/oauth-protected-resource` (RFC
9728)
  - Advertises MCP resource as OAuth-protected
  - Lists supported scopes and bearer token methods
  - Enables OAuth clients to discover authorization requirements

### Application Registration Updates
- **New Source Type**: `OAUTH_ONLY` enum value for OAuth-only
registrations
- **Install Service**: Skips artifact installation for OAuth-only apps
(no code artifacts)

### Frontend Updates
- **Authorization Page**: Support both snake_case (standard OAuth) and
camelCase (legacy) query parameters
  - `client_id` / `clientId`
  - `code_challenge` / `codeChallenge`
  - `redirect_uri` / `redirectUrl`

## Implementation Details

- **Rate Limiting**: Uses token bucket algorithm with 10 registrations
per 3,600,000ms window per IP
- **Scope Validation**: Requested scopes are capped to allowed OAuth
scopes; defaults to all scopes if not specified
- **Redirect URI Validation**: Uses existing `validateRedirectUri`
utility for security
- **Cache Headers**: Registration responses include `Cache-Control:
no-store` and `Pragma: no-cache`
- **Batch Processing**: Cleanup operations process 100 records at a time
to avoid memory issues
- **Grace Period**: 30-day grace period before cleanup to allow time for
client activation

https://claude.ai/code/session_01PxcuWFFRuXMASMaMGTLYk2

---------

Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: claude[bot] <41898282+claude[bot]@users.noreply.github.com>
Co-authored-by: github-actions <github-actions@twenty.com>
Co-authored-by: cubic-dev-ai[bot] <191113872+cubic-dev-ai[bot]@users.noreply.github.com>
 2026-03-16 09:42:28 +01:00
Marie c1da7be6d7 Billing for self-hosts (#18075) 
## Summary

Implements enterprise licensing and per-seat billing for self-hosted
environments, with Stripe as the single source of truth for subscription
data.

### Components

- **twenty-website** hosts the private key to sign `ENTERPRISE_KEY` and
`ENTERPRISE_VALIDITY_TOKEN`. It communicates with Stripe to emit the
daily `ENTERPRISE_VALIDITY_TOKEN` if the subscription is active, based
on the user's Stripe subscription ID stored in `ENTERPRISE_KEY`.
- **Stripe** is the single source of truth for subscription data
(status, seats, billing).
- **The client** (twenty-server + DB + workers) saves `ENTERPRISE_KEY`
in the `keyValuePair` table (or `.env` if
`IS_CONFIG_VARIABLES_IN_DB_ENABLED` is false) and the daily-renewed
`ENTERPRISE_VALIDITY_TOKEN` in the `appToken` table.
`ENTERPRISE_VALIDITY_TOKEN` is verified client-side using a public key
to grant access to enterprise features (RLS, SSO, audit logs, etc.).

### Flow

1. When requesting an upgrade to an enterprise plan (from **Enterprise**
in settings), the user is shown a modal to choose monthly/yearly
billing, then redirected to Stripe to enter payment details. After
checkout, they land on twenty-website where they are exposed to their
`ENTERPRISE_KEY`, which they paste in the UI. It is saved in the
`keyValuePair` table. On activation, a first `ENTERPRISE_VALIDITY_TOKEN`
with 30-day validity is stored in the `appToken` table.

2. **Every day**, a cron job runs and does two things:
- **Refreshes the validity token**: communicates with twenty-website to
get a new `ENTERPRISE_VALIDITY_TOKEN` with 30-day validity if the Stripe
subscription is still active. If the subscription is in cancellation,
the emitted token has a validity equal to the cancellation date. If it's
no longer valid, the token is not replaced. The cron only needs to run
every 30 days in practice, but runs daily so it's resilient to
occasional failures.
- **Reports seat count**: counts active (non-soft-deleted)
`UserWorkspace` entries and sends the count to twenty-website, which
updates the Stripe subscription quantity with proration. Seats are also
reported on first activation. If the subscription is canceled or
scheduled for cancellation, the seat update is skipped.

3. `ENTERPRISE_VALIDITY_TOKEN` is verified server-side via a public key
to grant access to enterprise features.

### Key concepts

Three distinct checks are exposed as GraphQL fields on `Workspace`:

| Field | Meaning |
|---|---|
| `hasValidEnterpriseKey` | Has any valid enterprise key (signed JWT
**or** legacy plain string) |
| `hasValidSignedEnterpriseKey` | `ENTERPRISE_KEY` is a properly signed
JWT (billing portal makes sense) |
| `hasValidEnterpriseValidityToken` | `ENTERPRISE_VALIDITY_TOKEN` is
present and not expired (expiration depends on signed token payload, not
on "expiresAt" on appToken table which is only indicative) |

Feature access is gated by `isValid()` =
`hasValidEnterpriseValidityToken || hasValidEnterpriseKey` (to support
both new and legacy keys during transition). After transition isValid()
= hasValidEnterpriseValidityToken

### Frontend states

The Enterprise settings page handles multiple states:
- **No key**: show "Get Enterprise" with checkout modal
- **Orphaned validity token** (token valid but no signed key): prompt
user to set a valid enterprise key
- **Active/trialing but no validity token**: show subscription status
with a "Reload validity token" action
- **Active/trialing**: show full subscription info, billing portal
access, cancel option
- **Cancellation scheduled**: show cancellation date, billing portal
- **Canceled**: show billing history link and option to start a new
subscription
- **Past due / Incomplete**: prompt to update payment or restart

### Temporary retro-compatibility: legacy plain-text keys

Previously, enterprise features were gated by a simple check: any
non-empty string in `ENTERPRISE_KEY` granted access. With this PR, we
transition to a controlled system relying on signed JWTs.

To avoid breaking existing self-hosted users:
- **Legacy plain-text keys still grant access** to enterprise features.
`hasValidEnterpriseKey` returns `true` for both signed JWTs and plain
strings, and `isValid()` checks `hasValidEnterpriseKey` as a fallback
when no validity token is present.
- **A deprecation banner** is shown at the top of the app when
`hasValidEnterpriseKey` is `true` but `hasValidSignedEnterpriseKey` is
`false`, informing the user that their key format is deprecated and they
should activate a new signed key.
- **No billing portal or subscription management** is available for
legacy keys since there is no Stripe subscription to manage.

This retro-compatibility will be removed in a future version. At that
point, `isValid()` will only check `hasValidEnterpriseValidityToken`.

### Edge cases

- **Air-gapped / production environments**: for self-hosted clients that
block external traffic (or for our own production), provide a long-lived
`ENTERPRISE_VALIDITY_TOKEN` (e.g. 99 years) directly in the `appToken`
table, with no `ENTERPRISE_KEY`. The daily cron will skip the refresh
(no enterprise key to authenticate with), but the pre-seeded validity
token will be used to grant feature access. No billing or seat reporting
occurs in this mode.
- **`IS_CONFIG_VARIABLES_IN_DB_ENABLED` is false**: if the user tries to
activate an enterprise key but DB config writes are disabled, the
backend returns a clear error asking them to add `ENTERPRISE_KEY` to
their `.env` file manually.
- **Canceled subscriptions**: the `/seats` endpoint skips Stripe updates
for canceled or cancellation-scheduled subscriptions to avoid Stripe API
errors.

### How to test
- launch twenty-website on a different url (eg localhost:1002)
- add ENTERPRISE_API_URL=http://localhost:3002/api/enterprise (or else)
in your server .env
- ask me for twenty-website's .env file content (STRIPE_SECRET_KEY;
STRIPE_ENTERPRISE_MONTHLY_PRICE_ID;STRIPE_ENTERPRISE_YEARLY_PRICE_ID;
ENTERPRISE_JWT_PRIVATE_KEY; ENTERPRISE_JWT_PUBLIC_KEY;
NEXT_PUBLIC_WEBSITE_URL)
- visit Admin panel / enterprise
 2026-03-12 15:07:53 +01:00
Félix Malfait 514d0017ea Refactor application module architecture for clarity and explicitness (#18432) 
## Summary

- **Module reorganization**: Moved `ApplicationUpgradeService` and cron
jobs to `application-upgrade/`, `ApplicationSyncService` to
`application-manifest/`, and
`runWorkspaceMigration`/`uninstallApplication` mutations to the manifest
resolver — each module now has a single clear responsibility.
- **Explicit install flow**: Removed implicit `ApplicationEntity`
creation from `ApplicationSyncService`. The install service and dev
resolver now explicitly create the `ApplicationEntity` before syncing.
npm packages are resolved at registration time to extract manifest
metadata (universalIdentifier, name, description, etc.), eliminating the
`reconcileUniversalIdentifier` hack.
- **Better error handling**: Frontend hooks now surface actual server
error messages in snackbars instead of swallowing them. Replaced the
ugly `ConfirmationModal` for transfer ownership with a proper form
modal. Fixed `SettingsAdminTableCard` row height overflow and corrected
the `yarn-engine` asset path.

## Test plan
- [ ] Register an npm package — verify manifest metadata (name,
description, universalIdentifier) is extracted correctly
- [ ] Install a registered npm app on a workspace — verify
ApplicationEntity is created and sync succeeds
- [ ] Test `app:dev` CLI flow — verify local app registration and sync
work
- [ ] Upload a tarball — verify registration and install flow
- [ ] Transfer ownership — verify the new modal UX works
- [ ] Verify error messages appear correctly in snackbars when
operations fail


Made with [Cursor](https://cursor.com)
 2026-03-06 08:45:08 +01:00
Félix Malfait 0e89c96170 feat: add npm and tarball app distribution with upgrade mechanism (#18358) 
## Summary

- **npm + tarball app distribution**: Apps can be installed from the npm
registry (public or private) or uploaded as `.tar.gz` tarballs, with
`AppRegistrationSourceType` tracking the origin
- **Upgrade mechanism**: `AppUpgradeService` checks for newer versions,
supports rollback for npm-sourced apps, and a cron job runs every 6
hours to update `latestAvailableVersion` on registrations
- **Security hardening**: Tarball extraction uses path traversal
protection, and `enableScripts: false` in `.yarnrc.yml` disables all
lifecycle scripts during `yarn install` to prevent RCE
- **Frontend**: "Install from npm" and "Upload tarball" modals, upgrade
button on app detail page, blue "Update" badge on installed apps table
when a newer version is available
- **Marketplace catalog sync**: Hourly cron job syncs a hardcoded
catalog index into `ApplicationRegistration` entities
- **Integration tests**: Coverage for install, upgrade, tarball upload,
and catalog sync flows

## Backend changes

| Area | Files |
|------|-------|
| Entity & migration | `ApplicationRegistrationEntity` (sourceType,
sourcePackage, latestAvailableVersion), `ApplicationEntity`
(applicationRegistrationId), migration |
| Services | `AppPackageResolverService`, `ApplicationInstallService`,
`AppUpgradeService`, `MarketplaceCatalogSyncService` |
| Cron jobs | `MarketplaceCatalogSyncCronJob` (hourly),
`AppVersionCheckCronJob` (every 6h) |
| REST endpoint | `AppRegistrationUploadController` — tarball upload
with secure extraction |
| Resolver | `MarketplaceResolver` — simplified `installMarketplaceApp`
(removed redundant `sourcePackage` arg) |
| Security | `.yarnrc.yml` — `enableScripts: false` to block postinstall
RCE |

## Frontend changes

| Area | Files |
|------|-------|
| Modals | `SettingsInstallNpmAppModal`, `SettingsUploadTarballModal`,
`SettingsAppModalLayout` |
| Hooks | `useUploadAppTarball`, `useInstallMarketplaceApp` (cleaned up)
|
| Upgrade UI | `SettingsApplicationVersionContainer`,
`SettingsApplicationDetailAboutTab` |
| Badge | `SettingsApplicationTableRow` — blue "Update" tag,
`SettingsApplicationsInstalledTab` — fetches registrations for version
comparison |
| Styling | Migrated to Linaria (matching main) |

## Test plan

- [ ] Install an app from npm via the "Install from npm" modal
- [ ] Upload a `.tar.gz` tarball via the "Upload tarball" modal
- [ ] Verify upgrade badge appears when `latestAvailableVersion >
version`
- [ ] Verify upgrade flow from app detail page
- [ ] Run integration tests: `app-distribution.integration-spec.ts`,
`marketplace-catalog-sync.integration-spec.ts`
- [ ] Verify `enableScripts: false` blocks postinstall scripts during
yarn install


Made with [Cursor](https://cursor.com)
 2026-03-05 10:34:08 +01:00
Thomas Trompette ccddd105d8 Add generate key command (#18171) 
Will be useful to auto log users on app creation
 2026-02-23 19:56:44 +01:00
Félix Malfait 177cae8c53 feat: audit Logs (#17660) 2026-02-04 13:30:55 +01:00
Paul Rastoin bd9688421f ObjectMetadata and FieldMetadata agnostic workspace migration runner (#17572) 
# Introduction
Important note: This PR officially deprecates the `standardId`, about to
drop col and entity property after this has been merged

Important note2: Haven't updated the optimistic tool to also update the
universal identifier aggregators only the ids one, they should not be
consumed in the runner context -> need to improve typing or either the
optimistic tooling

In this PR we're introducing all the devxp allowing future metadata
incremental universal migration -> this has an impact on all existing
metadata actions handler ( explaining its size )
This PR also introduce workspace agnostic create update actions runner
for both field and object metadata in order to battle test the described
above devxp

Noting that these two metadata are the most complex to handle

Notes:
- A workspace migration is now highly bind to a
`applicationUniversalIdentifier`. Though we don't strictly validate
application scope for the moment

## Next
Migrate both object and field builder to universal comparison

## Universal Actions vs Flat Actions Architecture

### Concept

The migration system uses a two-phase action model:

1. **Universal Actions** - Actions defined using `universalIdentifier`
(stable, portable identifiers like `standardId` + `applicationId`)
2. **Flat Actions** - Actions defined using database `entityId` (UUIDs
specific to a workspace)

### Why This Separation?

- **Universal actions are portable**: They can be serialized, stored,
and replayed across different workspaces
- **Flat actions are executable**: They contain the actual database IDs
needed to perform operations
- **Decoupling**: The builder produces universal actions; the runner
transpiles them to flat actions at execution time

### Transpiler Pattern

Each action handler must implement
`transpileUniversalActionToFlatAction()`:

```typescript
@Injectable()
export class CreateFieldActionHandlerService extends WorkspaceMigrationRunnerActionHandler(
  'create',
  'fieldMetadata',
) {
  override async transpileUniversalActionToFlatAction(
    context: WorkspaceMigrationActionRunnerArgs<UniversalCreateFieldAction>,
  ): Promise<FlatCreateFieldAction> {
    // Resolve universal identifiers to database IDs
    const flatObjectMetadata = findFlatEntityByUniversalIdentifierOrThrow({
      flatEntityMaps: allFlatEntityMaps.flatObjectMetadataMaps,
      universalIdentifier: action.objectMetadataUniversalIdentifier,
    });
    
    return {
      type: action.type,
      metadataName: action.metadataName,
      objectMetadataId: flatObjectMetadata.id, // Resolved ID
      flatFieldMetadatas: /* ... transpiled entities ... */,
    };
  }
}
```

### Action Handler Base Class

`BaseWorkspaceMigrationRunnerActionHandlerService<TActionType,
TMetadataName>` provides:

- **`transpileUniversalActionToFlatAction()`** - Abstract method each
handler must implement
- **`transpileUniversalDeleteActionToFlatDeleteAction()`** - Shared
helper for delete actions

## FlatEntityMaps custom properties
Introduced a `TWithCustomMapsProperties` generic parameter to control
whether custom indexing structures are included:

- **`false` (default)**: Returns `FlatEntityMaps<MetadataFlatEntity<T>>`
- used in builder/runner contexts
- **`true`**: Returns the full maps type with custom properties (e.g.,
`byUserWorkspaceIdAndFolderId`) - used in cache contexts

## Create Field Actions Refactor

Refactored create-field actions to support relation field pairs
bundling.

**Problem:** Relation fields (e.g., `Attachment.targetTask` ↔
`Task.attachments`) couldn't resolve each other's IDs during
transpilation because they were in separate actions with independent
`fieldIdByUniversalIdentifier` maps.

**Solution:** 
- Removed `objectMetadataUniversalIdentifier` from
`UniversalCreateFieldAction` and `objectMetadataId` from
`FlatCreateFieldAction` - each field now carries its own
- Runner groups fields by object internally and processes each table
separately
- Split aggregator into two focused utilities:
- `aggregateNonRelationFieldsIntoObjectActions` - merges non-relation
fields into object actions
- `aggregateRelationFieldPairs` - bundles relation pairs with shared
`fieldIdByUniversalIdentifier`
 2026-02-02 12:22:38 +00:00
Charles Bochet 4a770eafa1 Rework logic function module (#17588) 
core-modules/logic-function/
├── logic-function.module.ts
├── logic-function-executor/
│   ├── logic-function-executor.module.ts
│   ├── commands/
│   │   └── add-packages.command.ts
│   ├── constants/
│   │   └── logic-function-executor.constants.ts
│   ├── factories/
│   │   └── logic-function-module.factory.ts
│   ├── interfaces/
│   │   └── logic-function-executor.interface.ts
│   └── services/
│       └── logic-function-executor.service.ts
├── logic-function-build/
│   ├── logic-function-build.module.ts
│   ├── services/
│   │   └── logic-function-build.service.ts
│   └── utils/
│       └── get-logic-function-base-folder-path.util.ts
├── logic-function-drivers/
│   ├── logic-function-drivers.module.ts
│   ├── constants/
│   │   └── ...
│   ├── drivers/
│   │   ├── disabled.driver.ts
│   │   ├── lambda.driver.ts
│   │   └── local.driver.ts
│   ├── interfaces/
│   │   └── logic-function-executor-driver.interface.ts
│   ├── layers/
│   │   └── ...
│   └── utils/
│       └── ...
├── logic-function-layer/
│   ├── logic-function-layer.module.ts
│   └── services/
│       └── logic-function-layer.service.ts
└── logic-function-trigger/
    ├── logic-function-trigger.module.ts
    ├── jobs/
    │   └── logic-function-trigger.job.ts
    └── triggers/
        ├── cron/
        ├── database-event/
        └── route/
            ├── exceptions/
            ├── services/
            │   └── route-trigger.service.ts
            └── utils/
 2026-01-30 19:28:20 +01:00
Paul Rastoin 942d2fef83 Remove sync-metadata and IS_WORKSPACE_CREATION_V2_ENABLED feature flag (#16997) 
# Introduction
Followup of
https://github.com/twentyhq/twenty/pull/17001#pullrequestreview-3638508738
close https://github.com/twentyhq/core-team-issues/issues/1910

We've completely decom the `sync-metadata` in production. We're now then
removing its implementation in favor of the v2.

## TODO:
- [x] Remove sync-metadata implem and commands
- [x] Remove workspace decorators 
- [x] Type each deprecated field to deprecated on their workspaceEntity
- [x] Remove the `workspace-sync-metadata` folder entirely
- [x] remove workspace migration
- [x] workspace migration removal migration
- [x] remove the `v2` references from workspace manager file names
- [x] remove the `v2` references from workspace manager modules
- [ ] Double check impact on translation file path updates


## Note
- Removed the gate logic
- Remains some service v2 naming, serverless needs to be migrated on v2
fully
- Removed workspaceMigration service app health consumption, making it
always returning up ( no more down ) cc @FelixMalfait ( quite obsolete
health check now, will require complete refactor once we introduce inter
app dependency etc )
 2026-01-08 15:45:12 +00:00
Paul Rastoin 96f7f1cb0e Fix 1-15 upgrade commands bundle (#16957) 2026-01-06 14:13:40 +00:00
Paul Rastoin bd9e5986d2 Clean orphan metadata (#16914) 
# Introduction
Followup https://github.com/twentyhq/twenty/pull/16863

Important note: This is not an upgrade command and will have to be
manually run

In this pull request we're introducing a coding that will allow this
[migration](https://github.com/twentyhq/twenty/blob/clean-orphan-metadata/packages/twenty-server/src/database/typeorm/core/migrations/utils/1767002571103-addWorkspaceForeignKeys.util.ts#L3)
to pass, it enforces the `workspaceId` foreignKey on all metadata
entities. Allowing workspace deletion cascading of all its related
entities and avoiding orphan metadata entities to reoccur in the future

Also introduced a small migration that will set the workspaceId col type
to `uuid`, as it has been historically `varchar`
This migration is a requirement for the above command to work
successfully


## Note
Chunking by relations fields the orphan field deletion as would take way
too much time within a transac,

## Test
Tested on a prod extract locally ( both dry and not dry )
 2026-01-06 13:36:53 +01:00
Paul Rastoin 28cdb02fbb Twenty standard application Objects and fields as allFlatEntityMaps ID non-agnostic (#16298) 
# Introduction

Related to https://github.com/twentyhq/core-team-issues/issues/1995
This PR introduces the basis of the `twentyStandard` application as code
on demand, it's highly tied to `ids` where it will becomes workspace
agnostic following the builder and runner `universalIdentifier` refactor
later.

The goal here to allow computing the `allFlatEntityMaps` `to` of the
`twentyStandard` application on a empty workspace ( workspace creation
). Allowing installing the twenty standard app through a workspace
migration instead of passing by the sync metadata

Nothing done will be run in production for the moment if it's not the
small validation refactor we've introduced

Please note that everything introduced here will be replaced at some
point by a twenty app instance when the twenty sdk is mature enough to
handle of the edge cases we need here

## How we've proceeded
We've been iterating over every workspace entity both objects and their
fields, and transpiled them to flatEntity.
Being sure we migrate the defaultValue, settings and so on accordingly.
We've also compute all the ids in prior of the whole entities
computation so we don't face any hoisting issue.

## Current state
At the moment only handling all of the 29 standard objects and their
fields
Settings a unique universalIdentifier for all of them

Will come views, agent role targets and so on later

## `workspace:compute-twenty-standard-migration` command
This command allow generating a workspace migration that will result in
installing the twenty standard app in an empty workspace
It's temporary and aims to allow debugging for the moment we might not
keep it in the future as it is right now
It contains debug writeFileSync which is expected no worries greptile

## `LabelFieldMetadataIdentifierId`
Small refactor allowing defining the label identifier field metadata id
of a uuid field metadata type for system object, as some of our standard
object don't have a name field and don't aim to
Also please note that we might remove this build options later in the
sake of the currently installed universal identifier application that we
could compare with the deterministic twenty standard one

## `runFlatFieldMetadataValidators`
Deprecated this pattern which was redundant and not v2 friendly pattern

## Current errors that will address in upcoming PR
Current standard objects and fields metadata does not pass the
validation that we have in place, as historically the sync metadata
would directly consume the repositories and would just ignore the
validation. This is about to change.
Will handle the below errors in dedicated PRs as they will required
upgrade commands in order to migrate the data, or will handle that from
the sync metadata instead still to be determined but nothing critical
here

- camel case field metadata name
- options label invalid format

```json
{
  "status": "fail",
  "report": {
    "fieldMetadata": [
      {
        "status": "fail",
        "errors": [
          {
            "code": "INVALID_FIELD_INPUT",
            "message": "Name should be in camelCase",
            "userFriendlyMessage": {
              "id": "P+jdmX",
              "message": "Name should be in camelCase"
            },
            "value": "iCalUID"
          }
        ],
        "flatEntityMinimalInformation": {
          "id": "68dd83cd-92c8-4233-bb28-47939bab6124",
          "name": "iCalUID",
          "objectMetadataId": "11c16ab6-9176-439e-a2db-a12c5a58a524"
        },
        "type": "create_field"
      },
      {
        "status": "fail",
        "errors": [
          {
            "code": "INVALID_FIELD_INPUT",
            "message": "Value must be in UPPER_CASE and follow snake_case \"email\"",
            "userFriendlyMessage": {
              "id": "UBPzFQ",
              "message": "Value must be in UPPER_CASE and follow snake_case \"{sanitizedValue}\"",
              "values": {
                "sanitizedValue": "email"
              }
            },
            "value": "email"
          },
          {
            "code": "INVALID_FIELD_INPUT",
            "message": "Value must be in UPPER_CASE and follow snake_case \"sms\"",
            "userFriendlyMessage": {
              "id": "UBPzFQ",
              "message": "Value must be in UPPER_CASE and follow snake_case \"{sanitizedValue}\"",
              "values": {
                "sanitizedValue": "sms"
              }
            },
            "value": "sms"
          }
        ],
        "flatEntityMinimalInformation": {
          "id": "e3caaf2a-e07d-4146-8dfc-9eef904e82c9",
          "name": "type",
          "objectMetadataId": "4b777de5-4c7b-4af4-9b92-655c0f87512b"
        },
        "type": "create_field"
      },
      {
        "status": "fail",
        "errors": [
          {
            "code": "INVALID_FIELD_INPUT",
            "message": "Value must be in UPPER_CASE and follow snake_case \"incoming\"",
            "userFriendlyMessage": {
              "id": "UBPzFQ",
              "message": "Value must be in UPPER_CASE and follow snake_case \"{sanitizedValue}\"",
              "values": {
                "sanitizedValue": "incoming"
              }
            },
            "value": "incoming"
          },
          {
            "code": "INVALID_FIELD_INPUT",
            "message": "Value must be in UPPER_CASE and follow snake_case \"outgoing\"",
            "userFriendlyMessage": {
              "id": "UBPzFQ",
              "message": "Value must be in UPPER_CASE and follow snake_case \"{sanitizedValue}\"",
              "values": {
                "sanitizedValue": "outgoing"
              }
            },
            "value": "outgoing"
          }
        ],
        "flatEntityMinimalInformation": {
          "id": "d96233a4-93be-45ea-9548-3b50f3c700cf",
          "name": "direction",
          "objectMetadataId": "480a648a-d2e5-482a-992f-ef053e1b4bb0"
        },
        "type": "create_field"
      },
      {
        "status": "fail",
        "errors": [
          {
            "code": "INVALID_FIELD_INPUT",
            "message": "Value must be in UPPER_CASE and follow snake_case \"from\"",
            "userFriendlyMessage": {
              "id": "UBPzFQ",
              "message": "Value must be in UPPER_CASE and follow snake_case \"{sanitizedValue}\"",
              "values": {
                "sanitizedValue": "from"
              }
            },
            "value": "from"
          },
          {
            "code": "INVALID_FIELD_INPUT",
            "message": "Value must be in UPPER_CASE and follow snake_case \"to\"",
            "userFriendlyMessage": {
              "id": "UBPzFQ",
              "message": "Value must be in UPPER_CASE and follow snake_case \"{sanitizedValue}\"",
              "values": {
                "sanitizedValue": "to"
              }
            },
            "value": "to"
          },
          {
            "code": "INVALID_FIELD_INPUT",
            "message": "Value must be in UPPER_CASE and follow snake_case \"cc\"",
            "userFriendlyMessage": {
              "id": "UBPzFQ",
              "message": "Value must be in UPPER_CASE and follow snake_case \"{sanitizedValue}\"",
              "values": {
                "sanitizedValue": "cc"
              }
            },
            "value": "cc"
          },
          {
            "code": "INVALID_FIELD_INPUT",
            "message": "Value must be in UPPER_CASE and follow snake_case \"bcc\"",
            "userFriendlyMessage": {
              "id": "UBPzFQ",
              "message": "Value must be in UPPER_CASE and follow snake_case \"{sanitizedValue}\"",
              "values": {
                "sanitizedValue": "bcc"
              }
            },
            "value": "bcc"
          }
        ],
        "flatEntityMinimalInformation": {
          "id": "961c598e-67c3-452d-8bb2-b92c0bc64404",
          "name": "role",
          "objectMetadataId": "8af8a13c-ff97-4cd3-b70d-52a7dc2924b4"
        },
        "type": "create_field"
      },
      {
        "status": "fail",
        "errors": [
          {
            "code": "INVALID_FIELD_INPUT",
            "message": "Label must not contain a comma",
            "userFriendlyMessage": {
              "id": "k731jp",
              "message": "Label must not contain a comma"
            },
            "value": "Commas and dot (1,234.56)"
          },
          {
            "code": "INVALID_FIELD_INPUT",
            "message": "Label must not contain a comma",
            "userFriendlyMessage": {
              "id": "k731jp",
              "message": "Label must not contain a comma"
            },
            "value": "Spaces and comma (1 234,56)"
          },
          {
            "code": "INVALID_FIELD_INPUT",
            "message": "Label must not contain a comma",
            "userFriendlyMessage": {
              "id": "k731jp",
              "message": "Label must not contain a comma"
            },
            "value": "Dots and comma (1.234,56)"
          }
        ],
        "flatEntityMinimalInformation": {
          "id": "7fa20caf-2597-42e3-84e5-15a91b125b9b",
          "name": "numberFormat",
          "objectMetadataId": "a6974302-9e72-461c-aa09-9390f4ff16fc"
        },
        "type": "create_field"
      }
    ],
    "objectMetadata": [],
    "view": [],
    "viewField": [],
    "viewGroup": [],
    "index": [],
    "serverlessFunction": [],
    "cronTrigger": [],
    "databaseEventTrigger": [],
    "routeTrigger": [],
    "viewFilter": [],
    "role": [],
    "roleTarget": [],
    "agent": []
  }
}
```
 2025-12-04 17:39:12 +01:00
Marie 4ce93aee52 Fix user deletion flows (#15614) 
**Before**
- any user with workpace_members permission was able to remove a user
from their workspace. This triggered the deletion of workspaceMember +
of userWorkspace, but did not delete the user (even if they had no
workspace left) nor the roleTarget (acts as junction between role and
userWorkspace) which was left with a userWorkspaceId pointing to
nothing. This is because roleTarget points to userWorkspaceId but the
foreign key constraint was not implemented
- any user could delete their own account. This triggered the deletion
of all their workspaceMembers, but not of their userWorkspace nor their
user nor the roleTarget --> we have orphaned userWorkspace, not
technically but product wise - a userWorkspace without a workspaceMember
does not make sense

So the problems are
- we have some roleTargets pointing to non-existing userWorkspaceId
(which caused https://github.com/twentyhq/twenty/issues/14608 )
- we have userWorkspaces that should not exist and that have no
workspaceMember counterpart
- it is not possible for a user to leave a workspace by themselves, they
can only leave all workspaces at once, except if they are being removed
from the workspace by another user

**Now**
- if a user has multiple workspaces, they are given the possibility to
leave one workspace while remaining in the others (we show two buttons:
Leave workspace and Delete account buttons). if a user has just one
workspace, they only see Delete account
- when a user leaves a workspace, we delete their workspaceMember,
userWorkspace and roleTarget. If they don't belong to any other
workspace we also soft-delete their user
- soft-deleted users get hard deleted after 30 days thanks to a cron
- we have two commands to clean the orphans roleTarget and userWorkspace
(TODO: query db to see how many must be run)

**Next**
- once the commands have been run, we can implement and introduce the
foreign key constraint on roleTarget


Fixes https://github.com/twentyhq/twenty/issues/14608
 2025-11-06 18:29:12 +00:00
Félix Malfait c5564d9bd0 [BREAKING CHANGE] refactor: Add Entity suffix to TypeORM entity classes (#15239) 
## Summary

This PR refactors all TypeORM entity classes in the Twenty codebase to
include an 'Entity' suffix (e.g., User → UserEntity, Workspace →
WorkspaceEntity) to improve code clarity and follow TypeORM naming
conventions.

## Changes

### Entity Renaming
- ✅ Renamed **57 core TypeORM entities** with 'Entity' suffix
- ✅ Updated all related imports, decorators, and type references
- ✅ Fixed Repository<T>, @InjectRepository(), and
TypeOrmModule.forFeature() patterns
- ✅ Fixed @ManyToOne/@OneToMany/@OneToOne decorator references

### Backward Compatibility
- ✅ Preserved GraphQL schema names using @ObjectType('OriginalName')
decorators
- ✅ **No breaking changes** to GraphQL API
- ✅ **No database migrations** required
- ✅ File names unchanged (user.entity.ts remains as-is)

### Code Quality
- ✅ Fixed **497 TypeScript errors** (82% reduction from 606 to 109)
- ✅ **All linter checks passing**
- ✅ Improved type safety across the codebase

## Entities Renamed

```
User → UserEntity
Workspace → WorkspaceEntity
ApiKey → ApiKeyEntity
AppToken → AppTokenEntity
UserWorkspace → UserWorkspaceEntity
Webhook → WebhookEntity
FeatureFlag → FeatureFlagEntity
ApprovedAccessDomain → ApprovedAccessDomainEntity
TwoFactorAuthenticationMethod → TwoFactorAuthenticationMethodEntity
WorkspaceSSOIdentityProvider → WorkspaceSSOIdentityProviderEntity
EmailingDomain → EmailingDomainEntity
KeyValuePair → KeyValuePairEntity
PublicDomain → PublicDomainEntity
PostgresCredentials → PostgresCredentialsEntity
...and 43 more entities
```

## Impact

### Files Changed
- **400 files** modified
- **2,575 insertions**, **2,191 deletions**

### Progress
- ✅ **82% complete** (497/606 errors fixed)
- ⚠️ **109 TypeScript errors** remain (18% of original)

## Remaining Work

The 109 remaining TypeScript errors are primarily:

1. **Function signature mismatches** (~15 errors) - Test mocks with
incorrect parameter counts
2. **Entity type mismatches** (~25 errors) - UserEntity vs
UserWorkspaceEntity confusion
3. **Pre-existing issues** (~50 errors) - Null safety and DTO
compatibility (unrelated to refactoring)
4. **Import type issues** (~10 errors) - Entities imported with 'import
type' but used as values
5. **Minor decorator issues** (~9 errors) - onDelete property
configurations

These can be addressed in follow-up PRs without blocking this
refactoring.

## Testing Checklist

- [x] Linter passing
- [ ] Unit tests should be run (CI will verify)
- [ ] Integration tests should be run (CI will verify)
- [ ] Manual testing recommended for critical user flows

## Breaking Changes

**None** - This is a pure refactoring with full backward compatibility:
- GraphQL API unchanged (uses original entity names)
- Database schema unchanged
- External APIs unchanged

## Notes

- Created comprehensive `REFACTORING_STATUS.md` documenting the entire
process
- All temporary scripts have been cleaned up
- Branch: `refactor/add-entity-suffix-to-typeorm-entities`

## Reviewers

Please review especially:
- Entity renaming patterns
- GraphQL backward compatibility
- Any areas where entity types are confused (UserEntity vs
UserWorkspaceEntity)

---------

Co-authored-by: Charles Bochet <charles@twenty.com>
 2025-10-22 09:55:20 +02:00
Abdullah. d750df7fff Automatically clean up soft-deleted records after X days. (#14862) 
Closes #14726

### Added
- `trashRetentionDays` field to workspace entity (default: 14 days)  
- Automated trash cleanup using BullMQ jobs  
- Daily cron (00:10 UTC) that enqueues cleanup jobs for all active
workspaces
- Per-workspace limit: 100k records deleted per day  
- Calendar-based retention: records deleted on day X are cleaned up X+14
days later (at midnight UTC boundaries)

### Architecture
- **Cron (WorkspaceTrashCleanupCronJob):** Runs daily, enqueues jobs in
parallel for all workspaces
- **Job (WorkspaceTrashCleanupJob):** Processes individual workspace
cleanup
- **Service (WorkspaceTrashCleanupService):** Discovers tables with
`deletedAt`, deletes old records with quota enforcement
- **Command:** `npx nx run twenty-server:command
cron:workspace:cleanup-trash` to register the cron

### Testing
- Unit tests for service with 100% coverage of public API  
- Tested quota enforcement, error handling, and edge cases

---------

Co-authored-by: Félix Malfait <felix.malfait@gmail.com>
 2025-10-14 18:49:40 +02:00
Weiko 68b21d2942 Add db event trigger and cron trigger to migration v2 (#14772) 
## Context
Add DB events triggers and Cron triggers to migration v2 builder/runner
(and adding corresponding services/resolvers)
 2025-09-30 11:14:18 +02:00
martmull 57ff06f47c 1487 extensibility look into public domains to identify workspace (#14456) 
- identify workspace based on public domains
- add cron job to validate public domains
- add endpoint to validate a public domain
 2025-09-15 15:14:11 +02:00
martmull b84f4075e5 14240 extensibility ability to create multiple custom domains for each workspace 2 (#14307) 
Adds a public-domain core-module. 
Reorganize custom-domain files properly
 2025-09-11 12:24:57 +02:00
Charles Bochet a758154690 Add clean workspace cron command (#14353) 2025-09-08 16:46:31 +02:00
Charles Bochet f802294c84 Improve upgrade command and prepare 1.5 release (#14325) 
In this PR:
- refactor the upgrade command / upgrade command runner to keep upgrade
command as light as possible (all wrapping logic should go to upgrade
command runner)
- prevent any upgrade if there is at least one workspace.version <
previsousVersion ==> this leads to corrupted state where only core
migrations are run if the self-hoster is skipping a version
 2025-09-05 15:58:17 +02:00
neo773 8ab71fef1f feat: message folders control (#14144) 
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
 2025-08-30 17:39:10 +02:00
martmull 9b41a3be54 Add cron trigger table (#14110) 
This Pr begins the extensibility journey
- adds a `core.cronTrigger` table
- add a oneToMany relation between core.serverlessFunction and
`core.cronTrigger` (one serverlessFunction can be triggered by multiple
cronTriggers)
- add a job to trigger a serverless function
- adds a cron to trigger serverlessFunction (via the trigger job) based
on the core.cronTrigger.setting.pattern
- adds a command to register the cron
- add the command in `cron-register-all.command.ts`
 2025-08-28 14:47:48 +02:00
Weiko 533d7fe49a Deprecate legacy core datasource token (#14096) 2025-08-27 20:09:54 +02:00
Thomas Trompette cee647435c Improve workflow queue cron reliability (#13818) 
Improve workflow enqueue cron : instead of relying on the cache to know
how many workflows we can enqueue, query the DB. Then set the cache and
process the not started workflows.

Also adding a second cron that will look for workflows enqueued one hour
ago or more and put these back in the not started status. This will
allow the first cron to start these again.
 2025-08-12 15:54:58 +02:00
Antoine Moreaux 23353e31e6 feat(domain-manager): refactor custom domain validation and improve c… (#13388) 2025-08-01 09:01:27 +02:00
Raphaël Bosi 7653be8fde Improve view migration command (#13403) 
Remove unused imports.
Improve typing.
 2025-07-24 11:50:25 +02:00
Raphaël Bosi abc3969b41 Create view migration script (#13356) 
Create view migration command to copy views from the workspace schema to
the core schema.
Closes https://github.com/twentyhq/core-team-issues/issues/1247
 2025-07-23 12:57:16 +00:00