## Summary
Follow-up to #20953. Migrates 23 of the 30 entities that were left in
`WORKSPACE_SCOPED_EXEMPTIONS` last time, so the lint rule's
workspaceId-enforcement default now covers most of the core/metadata
schema.
### Migrated (23 entities, 88 files, 22 commits)
| Family | Entities |
|---|---|
| Trivial caches | `NavigationMenuItem`, `Skill`, `DataSource`,
`Webhook`, `CommandMenuItem`, `IndexMetadata` |
| Views | `View`, `ViewField`, `ViewFieldGroup`, `ViewFilter`,
`ViewFilterGroup`, `ViewGroup`, `ViewSort` |
| Layouts | `PageLayout`, `PageLayoutTab`, `PageLayoutWidget` |
| Roles & permissions | `Role`, `RoleTarget`, `PermissionFlag`,
`ObjectPermission`, `FieldPermission`, `RowLevelPermissionPredicate`,
`RowLevelPermissionPredicateGroup` |
For each entity: swap `@InjectRepository(X)` →
`@InjectWorkspaceScopedRepository(X)` (and the field type →
`WorkspaceScopedRepository<X>`); rewrite every call site to pass
`workspaceId` as the first arg (stripped from `where`/criteria — the
wrapper throws if you include it now); register
`provideWorkspaceScopedRepository(X)` in every owning NestJS module;
update affected spec providers to
`getWorkspaceScopedRepositoryToken(X)`.
### Rule update
- `ApplicationRegistrationVariableEntity` was misclassified — moved to
`STRUCTURAL_EXEMPTIONS` (no `workspaceId` column; it's keyed on
`applicationRegistrationId` at the instance level).
- 22 of the 23 migrated entities removed from
`WORKSPACE_SCOPED_EXEMPTIONS` entirely (zero remaining raw
`@InjectRepository` sites).
- `RoleTargetEntity` also removed; one call site in
`user-workspace.service.ts` keeps a raw injection with an
`eslint-disable` + reason because `softRemove(...)` is not on the
wrapper API yet (the migration would require threading `workspaceId`
through `deleteUserWorkspace`'s three callers).
### Still exempted (7 entities, follow-up PRs)
| Entity | Why deferred |
|---|---|
| `ApplicationEntity` | ~50 sites with several cross-workspace lookups
by id (auth, OAuth, file-storage, cleanup) |
| `CalendarChannelEntity` / `MessageChannelEntity` | Use
`.increment(...)` (not on wrapper) and
`repository.manager.transaction(...)` — wrapper needs to grow
`.increment` + the transaction sites need `withManager` or dual-inject |
| `FieldMetadataEntity` / `ObjectMetadataEntity` | The metadata services
`extends TypeOrmQueryService<X>` and `super(rawRepo)` — requires
dual-inject or reworking the inheritance |
| `KeyValuePairEntity` | Allows `workspaceId: IsNull()` for
instance-level config; wrapper rejects null |
| `UpgradeMigrationEntity` | Same — instance-level + cross-workspace
ledger |
## Test plan
- [x] `npx nx typecheck twenty-server` — clean
- [x] `npx nx lint twenty-server` — clean (0/0)
- [x] All 10 affected unit specs pass (115 tests) — api-key, agent-role,
permissions, workspace-roles-permissions-cache, view-filter-group,
workflow-version-step-operations, two-factor-authentication (service +
resolver), user-workspace, file
- [ ] Server integration tests in CI
## Summary
Adds the \`secret-encryption:rotate\` CLI command, which re-encrypts
every at-rest secret stored in an \`enc:v2:\` envelope under the current
\`ENCRYPTION_KEY\`. The command is **online** and **resumable**: a SQL
filter skips rows already on the current keyId, so interrupting it
(Ctrl-C, container restart, …) and re-running picks up where it left off
without re-rotating earlier rows.
### Sites covered (one handler each)
| Site | Table.column | Scope |
| --- | --- | --- |
| \`connected-account-tokens\` | \`connectedAccount.{accessToken,
refreshToken}\` | workspace |
| \`application-variable\` | \`applicationVariable.value\` (isSecret
only) | workspace |
| \`application-registration-variable\` |
\`applicationRegistrationVariable.encryptedValue\` | instance |
| \`signing-key-private-keys\` | \`signingKey.privateKey\` | instance |
| \`sensitive-config-storage\` | \`keyValuePair.value\` (isSensitive +
STRING configs) | instance |
| \`totp-secrets\` | \`twoFactorAuthenticationMethod.secret\` |
workspace |
Each handler:
- Filters at SQL level on \`value LIKE 'enc:v2:%' AND value NOT LIKE
'enc:v2:<primaryKeyId>:%'\` to enforce idempotency without re-decrypting
already-rotated rows.
- Uses cursor-based batching (default **200**, capped **5000**).
- Threads \`workspaceId\` into HKDF for workspace-scoped sites; runs
instance-scoped for the rest.
### CLI flags
| Flag | Description |
| --- | --- |
| \`-s, --site <site>\` | Limit to a single site. |
| \`-b, --batch-size <n>\` | Override per-batch row count. |
| \`-d, --dry-run\` | Decrypt + re-encrypt in memory, skip the
\`UPDATE\`. |
The runner logs progress via Nest \`Logger\` (per-site start,
completion, final summary) and exits non-zero when any site reports
\`errors > 0\`. \`FALLBACK_ENCRYPTION_KEY\` must be set to the previous
\`ENCRYPTION_KEY\` during rotation; the runner warns when it is unset.
Operator documentation lives in #20611 (docs PR).
## Summary
Adds a daily Enterprise-only cron that rotates the current ES256 JWT
signing key once it has been current for `SIGNING_KEY_ROTATION_DAYS`.
Manual rotation from the admin panel is unaffected.
### Behaviour
- `SIGNING_KEY_ROTATION_DAYS` is **opt-in**: when unset, the cron is a
no-op.
- Rotation flips `isCurrent` and clears the previous key's `privateKey`
in the same transaction, then inserts the new `isCurrent=true` row.
- The previous key's row is kept (`revokedAt` stays `null`) so its
`publicKey` can keep verifying tokens it signed until they expire; only
the encrypted `privateKey` is wiped since it can no longer be used to
sign.
- **No auto-revocation** — revoking a key remains a manual admin action,
reserved for leak / emergency response.
- The cron is also a no-op when `EnterprisePlanService.isValid()` is
`false`.
### Wiring
- `JwtKeyManagerService.rotateCurrent()`
- `SigningKeyRotationService.rotateIfDue()` (reads
`SIGNING_KEY_ROTATION_DAYS`, skips when unset)
- `RotateSigningKeysCronJob` (Enterprise-gated, rethrows on failure)
registered in `JwtModule`
- `RotateSigningKeysCronCommand` registered with `cron:register:all`
- `ROTATE_SIGNING_KEYS_CRON_PATTERN = '15 3 * * *'` (daily, no-op until
threshold)
Operator documentation lives in #20611 (docs PR).
**PR 1 of 2.** Follow-up PR ships the Exa app, sets it as a default
pre-installed app, and removes the current `WebSearchTool` /
`WebSearchService` / `ExaDriver`. This PR adds the plumbing; no
user-visible change yet.
## Summary
- Server admins can declare a list of npm app packages to auto-install
on every new workspace and backfill onto existing workspaces via CLI.
- Server-level secrets (like Exa's API key) live on the
`ApplicationRegistration` (one row per server, encrypted) and are
injected into logic function execution env at runtime. No more
per-workspace storage of global secrets.
- A generic `POST /app/billing/charge` endpoint lets app logic functions
emit workspace usage events for metered features. Exa uses it in PR 2;
future apps (call recorder, etc.) reuse it.
- `LogicFunctionToolProvider` tool name prefix changes `logic_function_`
→ `app_`. Shorter, accurate (they come from installed apps).
## What's in this PR
**Logic function executor — server-level variables**
- `LogicFunctionExecutorService.getExecutionEnvVariables` now resolves
env vars in the order: hardcoded defaults →
`ApplicationRegistrationVariable[]` (server-level) →
`ApplicationVariable[]` (workspace-level override). The manifest
`serverVariables` schema has existed; this closes the loop.
**Config**
- `PRE_INSTALLED_APPS` — comma-separated list of npm packages. Default:
empty.
**\`PreInstalledAppsService\`** (new module)
- \`onApplicationBootstrap()\` — fetches each package's manifest from
the app registry CDN, upserts an \`ApplicationRegistration\`, and seeds
declared \`serverVariables\` from matching env vars (e.g.
\`EXA_API_KEY\` env → encrypted registration variable).
- \`installOnWorkspace(workspaceId)\` — installs all pre-installed apps
on a single workspace. Tolerates per-app failures.
**Auto-install on new workspace activation**
- \`WorkspaceService.prefillCreatedWorkspaceRecords\` invokes
\`installOnWorkspace\` after prefilling standard records. Non-blocking
on failure.
**Backfill CLI command**
- \`install-pre-installed-apps\` — iterates active and suspended
workspaces, installs pre-installed apps that aren't yet installed.
Idempotent. Run after changing \`PRE_INSTALLED_APPS\`.
**App billing endpoint**
- \`POST /app/billing/charge\`. Authenticated via \`APPLICATION_ACCESS\`
token (already injected into logic function execution env as
\`DEFAULT_APP_ACCESS_TOKEN\`). Body: \`{ creditsUsedMicro, quantity,
unit, operationType, resourceContext? }\`. Emits \`USAGE_RECORDED\` with
\`applicationId\` as \`resourceId\`. Generic — reusable by any app.
**Tool name prefix**
- \`LogicFunctionToolProvider.buildLogicFunctionToolName\` now produces
\`app_<name>\` instead of \`logic_function_<name>\`. Only affects tools
sourced from logic functions; other tool providers unchanged.
## Stats
- 16 files, +501 / −2
- 7 new files (1 command, 1 service × 2, 1 controller, 1 DTO, 2 modules)
- Typecheck: 7 pre-existing errors, zero new
- Prettier clean
## Behavior deltas
- **\`PRE_INSTALLED_APPS\` default = empty**: existing servers see no
change on merge.
- **\`ApplicationRegistrationVariable\` is now read by the executor**:
apps that were using manifest \`serverVariables\` but expecting them to
be ignored by the executor will now see them injected. No apps ship with
\`isTool: true\` logic functions today, so this is latent — first
consumer is Exa in PR 2.
- **Tool prefix**: currently no logic-function tools are named
\`logic_function_*\` in any production flow. The prefix change affects
only future tools emitted by \`LogicFunctionToolProvider\`.
## Risks
- **CDN unavailability at startup**: if the app registry CDN is down,
\`ensureRegistrationsExist\` logs warnings but doesn't block server
start. Installation on new workspaces during this window will find no
registrations and log a non-blocking error. Backfill command can retry
after CDN recovers.
- **Cold-start overhead**: \`ensureRegistrationsExist\` is called once
per process on bootstrap. Current configurable default is empty, so zero
overhead. When an admin sets \`PRE_INSTALLED_APPS\`, they accept one
HTTP call per package at boot.
- **Server-level variables flow**:
\`ApplicationRegistrationVariable.encryptedValue\` is shared by all
workspaces of a server. Appropriate for a single-tenant Exa key. Not
appropriate for per-tenant keys — those go in workspace-level
\`ApplicationVariable\` and override.
## Test plan
- [ ] \`npx nx typecheck twenty-server\` passes (verified: 7
pre-existing unrelated errors, zero new)
- [ ] Set \`PRE_INSTALLED_APPS=@twenty-apps/hello-world\` (or any real
npm-published app), \`HELLO_WORLD_API_KEY=xxx\`, restart server:
\`ApplicationRegistration\` row is upserted,
\`ApplicationRegistrationVariable\` for HELLO_WORLD_API_KEY is populated
(encrypted).
- [ ] Create a new workspace: the app is auto-installed,
\`ApplicationEntity\` row created, \`LogicFunctionEntity\` rows created.
- [ ] Existing workspace: run \`yarn nx run twenty-server:command
install-pre-installed-apps\`: apps install across all workspaces,
idempotent on re-run.
- [ ] Trigger a logic function that reads
\`process.env.HELLO_WORLD_API_KEY\`: value resolves from the
server-level \`ApplicationRegistrationVariable\`.
- [ ] Log a charge from the handler: \`POST /app/billing/charge\` with
\`Authorization: Bearer \$DEFAULT_APP_ACCESS_TOKEN\` body
\`{creditsUsedMicro: 1000, quantity: 1, unit: "INVOCATION",
operationType: "WEB_SEARCH"}\` → returns \`{success: true}\`,
\`USAGE_RECORDED\` event emitted with correct
\`resourceId=applicationId\`.
- [ ] Tool name generated by \`LogicFunctionToolProvider\` starts with
\`app_\`.
## What's NOT in this PR (PR 2 scope)
- The Exa app itself (\`packages/twenty-apps/...\` directory)
- Removing \`WebSearchTool\`, \`WebSearchService\`, \`ExaDriver\`,
\`web-search\` module
- Removing \`WEB_SEARCH_DRIVER\` config var
- Removing the current \`exa_web_search\` entry in
\`ActionToolProvider\`
- Chat preload list updated to \`app_exa_web_search\`
- Frontend \`getToolDisplayMessage\` branch for \`app_exa_web_search\`
- Setting \`PRE_INSTALLED_APPS\` default to include \`@twenty-apps/exa\`
---------
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
# Introduction
If a self host creates its twenty instance using storage type local, and
then edit through the admin panel the storage type, the apps default
deps file won't be swapped to the new storage location
This command allow to manually rebuild them
## What could be done in addition
- We could display a modal in the admin panel when the user is editing
env variable that might have a side effect
- We might wanna rebuild the deps by default if we detect such a change
through the UI though we can't really if it's through the `.env` so I'm
not sure we wanna prio such logic
# Introduction
We were allowing the sequence to be empty in the worker context that was
facing an edge case importing the UpgradeModule through the
WorkspaceModule god module, no commands were discovered and it was
throwing as the sequence must have at least one workspace commands to
allow a workspace creation
Though the issue was also applicable to the twenty-server `AppModule`
too that was not discovering any commands
## Integration tests were passing
The integration test were importing the `CommandModule` at the nest
testing app creating leading to asymmetric testing context
It was a requirement for a legacy commands import and global assignation
## Fix
The `UpgradeModule` now import both `WorkspaceCommandsProviderModule`
and `InstanceCommandProviderModule` which ships the commands directly in
the module
We could consider moving the commands into the `engine/upgrade` folder
## Concern
Bootstrap could become more and more long to load at both server and
worker start
When this becomes a problem we will have to only import the latest
workspace command or whatever
For the moment this is not worth it the risk to import not the latest
workspace command
## Summary
Implements a ClickHouse-backed polling system to enforce metered-credit
caps for workflow executions, replacing reliance on Stripe billing
alerts. The system re-evaluates tier caps against live pricing on every
poll cycle, allowing price/tier changes to propagate immediately without
recreating Stripe alert objects.
## Key Changes
- **BillingUsageCapService**: New service that queries ClickHouse for
current-period credit usage and evaluates whether a subscription has
reached its metered-credit allowance (tier cap + credit balance)
- `isClickHouseEnabled()`: Checks if ClickHouse is configured
- `getCurrentPeriodCreditsUsed()`: Sums creditsUsedMicro from usageEvent
table for a workspace within a billing period
- `evaluateCap()`: Determines if usage has reached the allowance by
reading live pricing from the subscription
- **EnforceUsageCapJob**: Cron job that polls all active subscriptions
and updates `hasReachedCurrentPeriodCap` on metered items
- Runs every 2 minutes to keep cap enforcement in sync with live usage
- Supports shadow mode (log-only) via
`BILLING_USAGE_CAP_CLICKHOUSE_ENABLED` flag for safe rollout
- Continues processing after per-subscription errors with detailed
logging
- **EnforceUsageCapCronCommand**: CLI command to register the
enforcement cron job
- **MeteredCreditService**: Extracted
`extractMeteredPricingInfoFromSubscription()` as a pure function for
callers that already hold the subscription with pricing loaded, avoiding
redundant DB queries
- **Configuration**: Added `BILLING_USAGE_CAP_CLICKHOUSE_ENABLED` flag
to control enforcement mode (active vs. shadow)
- **Constants**: Added `METERED_OPERATION_TYPES` to define which
operation types count toward the metered product's credit cap
## Implementation Details
- The service queries ClickHouse for the sum of `creditsUsedMicro` in
the current billing period, matching Stripe meter semantics
- Pricing is re-read on every evaluation, so tier changes propagate
within one poll cycle without Stripe alert recreation
- The cron job only updates the database when the cap state actually
changes (no-op if already in the correct state)
- Shadow mode allows safe validation before enabling enforcement;
transitions are logged but not persisted
- Comprehensive test coverage for both the service and cron job,
including error handling and state transitions
https://claude.ai/code/session_01VksTSrYLXJVCPVBQhQdBTe
---------
Co-authored-by: Claude <noreply@anthropic.com>
# Introduction
Refactoring the upgrade engine to handle cross version upgrade,
completely getting rid of the semver `version` at db and runtime level
It remains a visual a listing indicator for or CD process but also
during devenv in order to prepare next release
Will write a release process runbook documentation on how to handle
upgrade step patch, command insertion etc as it needs to be cascaded
across all the involved supported version
**The upgrade sequence model:**
The sequence is a flat, ordered array of upgrade steps
(`UpgradeStep[]`), built from the registry by chaining all versions in
order, each version contributing its fast-instance → slow-instance →
workspace commands sorted by timestamp. Version is metadata for logging,
not used in the algorithm.
**Segments:**
The sequence naturally splits into alternating segments of contiguous
instance steps and contiguous workspace steps. The runner processes
segments in order:
- **Instance segment:** Run sequentially from the instance cursor. Each
step runs once globally.
- **Workspace segment:** Each workspace independently walks from its own
cursor through the end of the segment. Workspaces are independent within
a segment — they can be at different positions.
- **Synchronization (workspace → instance):** The runner blocks before
entering an instance segment. All active/suspended workspaces must have
completed the last workspace step of the preceding workspace segment. If
any workspace failed, abort. This is the only explicit synchronization
point.
- Instance → workspace ordering is implicit — the runner processes
segments sequentially, so the instance segment naturally completes
before the workspace segment begins.
full docs
https://gist.github.com/prastoin/e62106d455fd72d6b6ebada8351e5492
## Version constants & type-level deprecation
Version management is split into three atomic constants:
`TWENTY_PREVIOUS_VERSIONS`, `TWENTY_CURRENT_VERSION`, and
`TWENTY_NEXT_VERSIONS`. Two derived constants compose them:
`CROSS_UPGRADE_SUPPORTED_VERSIONS` (previous + current — what the engine
runs) and `ALL_TWENTY_VERSIONS` (the full ordered tuple including next).
The registry service validates at module init that no version is
duplicated across constants and that at least one previous version
exists.
A `DeprecatedSinceVersion<RemoveAtVersion, T>` type utility resolves to
`T` while `TWENTY_CURRENT_VERSION` is below `RemoveAtVersion`, and to
`never` once it reaches it — turning deprecation into a compile-time
guarantee via `IndexOf` and `IsGreaterOrEqual` generics in
`twenty-shared`.
### `workspace.version` column deprecation
The column is replaced by cursor-based state inference from
`UpgradeMigration` records, but cannot be dropped in 1.22: workspaces
activated during 1.21 predate the cursor system and need their initial
cursor backfilled first (`backfillWorkspaceCreatedIn1_21_0Cursors`).
This backfill itself depends on a new `isInitial` column on
`UpgradeMigration`, bootstrapped via a targeted TypeORM migration before
the upgrade sequence runs.
Both functions and the entity field are typed with
`DeprecatedSinceVersion<'1.23.0', ...>`. When `TWENTY_CURRENT_VERSION`
reaches `1.23.0`, compile errors force their removal — and the
pre-declared `DropWorkspaceVersionColumnFastInstanceCommand` takes over
to drop the column.
## What's next
- ci cross version upgrade ( wip )
- banner asking to contact twenty administrator if workspace is outdated
- upgrade healthcheck cli
## New unit/integ test pattern
Create a dedicated `createNestApp` that consumes a real database in
order not to have to mack any database interaction to the
`upgradeMigrations` allowing full coverage of the whole
`upgradeRunnerService.run` core logic
## Summary
- **Drop the `objectMetadata.dataSourceId` foreign key and index** via a
1-22 fast instance command — column kept nullable for data preservation
- **Delete `DataSourceService`, `DataSourceModule`, and
`DataSourceException`** — all code now uses `workspace.databaseSchema`
directly
- **Remove `IS_DATASOURCE_MIGRATED` feature flag** from default flags
and all branching logic
- **Simplify workspace/object creation pipelines** —
`WorkspaceManagerService`, `DevSeederService`, and the object creation
action handler no longer route through `DataSourceService`
- **Keep `DataSourceEntity` and the `dataSource` table** for historical
data — entity stripped of all ORM relations
# Introduction
Now only using typeorm to generate migrations up and down statement
We handle and maintain our own migration table history
## What's new
Now all the instance commands will live within the same module and
folder than the upgrade commands
Sequentiality comes from the timestamp located in the filename
Same sequentiality also applies to the workspace commands in the future,
for the moment still expected a as code explicit declaration
( below screen is an example see below section )
<img width="1382" height="634" alt="image"
src="https://github.com/user-attachments/assets/5610a246-4eae-485e-99f4-98fb89ad5ac8"
/>
## Existing 1.21 migrations
We won't start following this pattern in 1.21 yet at least not with the
migration that has already been released as typeorm migrations in cloud
production as they would rerun
## Small duplication
Duplicating the legacy typeorm and instance commands run in the
`run-instance-commands` to avoid any merge of interest for the moment
## Concurrency
Not handling any run in parrallel of the upgrade for the moment
# Introduction
Typeorm migration are now associated to a given twenty-version from the
`UPGRADE_COMMAND_SUPPORTED_VERSIONS` that the current twenty core engine
handles
This way when we upgrade we retrieve the migrations that need to be run,
this will be useful for the cross-version incremental upgrade so we
preserve sequentiality
## What's new
To generate
```sh
npx nx database:migrate:generate twenty-server -- --name add-index-to-users
```
To apply all
```sh
npx nx database:migrate twenty-server
```
## Next
Introduce slow and fast typeorm migration in order to get rid of the
save point pattern in our code base
Create a clean and dedicated `InstanceUpgradeService` abstraction
## Motivations
A lot of self hosters hands up using the `yarn database:migrated:prod`
either manually or through AI assisted debug while they try to upgrade
an instance while their workspace is still blocked in a previous one
Leading to their whole database permanent corruption
## What happened
Replaced the direct call the the typeorm cli to a command calling it
programmatically, adding a layer of security in case a workspace seems
to be blocked in a previous version than the one just before the one
being installed ( e.g 1.0 when you try to upgrade from 1.1 to 1.2 )
For our cloud we still need a way to bypass this security explaining the
-f flag
## Remark
Centralized this logic and refactored creating new services
`WorkspaceVersionService` and `CoreEngineVersionService` that will
become useful for the upcoming upgrade refactor
Related to https://github.com/twentyhq/twenty-infra/pull/529
## Summary
- The `ci-ai-catalog-sync` cron workflow was failing because the
`ai:sync-models-dev` NestJS command bootstraps the full app, which tries
to connect to PostgreSQL — unavailable in CI.
- Converted the sync logic to a standalone `ts-node` script
(`scripts/ai-sync-models-dev.ts`) that runs without NestJS, eliminating
the database dependency.
- Removed the `Build twenty-server` step from the workflow since it's no
longer needed, making the job faster.
## Test plan
- [x] Verified the standalone script runs successfully locally via `npx
nx run twenty-server:ts-node-no-deps-transpile-only --
./scripts/ai-sync-models-dev.ts`
- [x] Verified `--dry-run` flag works correctly
- [x] Verified the output `ai-providers.json` is correctly written with
valid JSON (135 models across 5 providers)
- [x] Verified the script passes linting with zero errors
- [ ] CI should pass without requiring a database service
Fixes:
https://github.com/twentyhq/twenty/actions/runs/23424202182/job/68135439740
Made with [Cursor](https://cursor.com)
## Summary
- Replaces per-provider TypeScript constant files
(`openai-models.const.ts`, `anthropic-models.const.ts`, etc.) with a
single `ai-providers.json` catalog as the source of truth
- Adds runtime model discovery via AI SDK for self-hosted providers,
with `models.dev` enrichment for pricing/capabilities
- Introduces composite model IDs (`provider/modelId`) for canonical,
conflict-free identification
- Simplifies provider configuration: API keys are injected from
environment variables (e.g., `OPENAI_API_KEY`)
- Adds admin panel UI for provider management (add/remove/test), model
discovery, recommended model configuration, and default fast/smart model
selection per workspace
- Removes deprecated config variables (`AI_DISABLED_MODEL_IDS`,
`AUTO_ENABLE_NEW_AI_MODELS`, etc.)
- Adds database migration for composite model ID format
## Test plan
- [ ] Server typecheck passes
- [ ] Frontend typecheck passes
- [ ] Server unit tests pass
- [ ] Frontend unit tests pass
- [ ] CI pipeline green
- [ ] Admin panel AI tab loads correctly
- [ ] Provider discovery works for configured providers
- [ ] Model recommendation toggles persist
- [ ] Default fast/smart model selection works
Made with [Cursor](https://cursor.com)
This PR adds a `workspace:export` server command for ongoing debug
tooling/administrative work
Demo Video shows
1. Exporting a sample workspace YC
2. Restoring Local DB Docker volume snapshot to Dropped YC Workspace
3. Importing exported SQL
4. Booting and navigating the imported Workspace
https://github.com/user-attachments/assets/0e1ac6cb-8ce1-440b-8b56-f81dcb27a9c8
## Summary
This PR implements OAuth 2.0 Dynamic Client Registration (RFC 7591) and
OAuth 2.0 Protected Resource Metadata (RFC 9728) support, enabling
third-party applications to dynamically register as OAuth clients
without manual configuration.
## Key Changes
### OAuth Dynamic Client Registration
- **New Controller**: `OAuthRegistrationController` at `POST
/oauth/register` endpoint
- Validates client metadata according to RFC 7591 specifications
- Enforces PKCE-only public client model (no client secrets)
- Supports only `authorization_code` grant type and `code` response type
- Rate limits registrations to 10 per hour per IP address
- Returns `client_id` and registration metadata in response
- **Input Validation**: `OAuthRegisterInput` DTO with constraints on:
- Client name (max 256 chars)
- Redirect URIs (max 20, validated for security)
- Grant types, response types, scopes, and auth methods
- Logo and client URIs (max 2048 chars)
- **Discovery Endpoint Update**: Added `registration_endpoint` to OAuth
discovery metadata
### Stale Registration Cleanup
- **Cleanup Service**: Automatically removes OAuth-only registrations
older than 30 days that have no active installations
- **Cron Job**: Runs daily at 02:30 AM UTC with batch processing (100
records per batch)
- **CLI Command**: `cron:stale-registration-cleanup` to manually trigger
cleanup
### MCP (Model Context Protocol) Authentication
- **New Guard**: `McpAuthGuard` implements RFC 9728 compliance
- Wraps JWT authentication with proper error responses
- Returns `WWW-Authenticate` header with protected resource metadata URL
on 401
- Enables OAuth-protected MCP endpoints
### Protected Resource Metadata
- **New Endpoint**: `GET /.well-known/oauth-protected-resource` (RFC
9728)
- Advertises MCP resource as OAuth-protected
- Lists supported scopes and bearer token methods
- Enables OAuth clients to discover authorization requirements
### Application Registration Updates
- **New Source Type**: `OAUTH_ONLY` enum value for OAuth-only
registrations
- **Install Service**: Skips artifact installation for OAuth-only apps
(no code artifacts)
### Frontend Updates
- **Authorization Page**: Support both snake_case (standard OAuth) and
camelCase (legacy) query parameters
- `client_id` / `clientId`
- `code_challenge` / `codeChallenge`
- `redirect_uri` / `redirectUrl`
## Implementation Details
- **Rate Limiting**: Uses token bucket algorithm with 10 registrations
per 3,600,000ms window per IP
- **Scope Validation**: Requested scopes are capped to allowed OAuth
scopes; defaults to all scopes if not specified
- **Redirect URI Validation**: Uses existing `validateRedirectUri`
utility for security
- **Cache Headers**: Registration responses include `Cache-Control:
no-store` and `Pragma: no-cache`
- **Batch Processing**: Cleanup operations process 100 records at a time
to avoid memory issues
- **Grace Period**: 30-day grace period before cleanup to allow time for
client activation
https://claude.ai/code/session_01PxcuWFFRuXMASMaMGTLYk2
---------
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: claude[bot] <41898282+claude[bot]@users.noreply.github.com>
Co-authored-by: github-actions <github-actions@twenty.com>
Co-authored-by: cubic-dev-ai[bot] <191113872+cubic-dev-ai[bot]@users.noreply.github.com>
## Summary
Implements enterprise licensing and per-seat billing for self-hosted
environments, with Stripe as the single source of truth for subscription
data.
### Components
- **twenty-website** hosts the private key to sign `ENTERPRISE_KEY` and
`ENTERPRISE_VALIDITY_TOKEN`. It communicates with Stripe to emit the
daily `ENTERPRISE_VALIDITY_TOKEN` if the subscription is active, based
on the user's Stripe subscription ID stored in `ENTERPRISE_KEY`.
- **Stripe** is the single source of truth for subscription data
(status, seats, billing).
- **The client** (twenty-server + DB + workers) saves `ENTERPRISE_KEY`
in the `keyValuePair` table (or `.env` if
`IS_CONFIG_VARIABLES_IN_DB_ENABLED` is false) and the daily-renewed
`ENTERPRISE_VALIDITY_TOKEN` in the `appToken` table.
`ENTERPRISE_VALIDITY_TOKEN` is verified client-side using a public key
to grant access to enterprise features (RLS, SSO, audit logs, etc.).
### Flow
1. When requesting an upgrade to an enterprise plan (from **Enterprise**
in settings), the user is shown a modal to choose monthly/yearly
billing, then redirected to Stripe to enter payment details. After
checkout, they land on twenty-website where they are exposed to their
`ENTERPRISE_KEY`, which they paste in the UI. It is saved in the
`keyValuePair` table. On activation, a first `ENTERPRISE_VALIDITY_TOKEN`
with 30-day validity is stored in the `appToken` table.
2. **Every day**, a cron job runs and does two things:
- **Refreshes the validity token**: communicates with twenty-website to
get a new `ENTERPRISE_VALIDITY_TOKEN` with 30-day validity if the Stripe
subscription is still active. If the subscription is in cancellation,
the emitted token has a validity equal to the cancellation date. If it's
no longer valid, the token is not replaced. The cron only needs to run
every 30 days in practice, but runs daily so it's resilient to
occasional failures.
- **Reports seat count**: counts active (non-soft-deleted)
`UserWorkspace` entries and sends the count to twenty-website, which
updates the Stripe subscription quantity with proration. Seats are also
reported on first activation. If the subscription is canceled or
scheduled for cancellation, the seat update is skipped.
3. `ENTERPRISE_VALIDITY_TOKEN` is verified server-side via a public key
to grant access to enterprise features.
### Key concepts
Three distinct checks are exposed as GraphQL fields on `Workspace`:
| Field | Meaning |
|---|---|
| `hasValidEnterpriseKey` | Has any valid enterprise key (signed JWT
**or** legacy plain string) |
| `hasValidSignedEnterpriseKey` | `ENTERPRISE_KEY` is a properly signed
JWT (billing portal makes sense) |
| `hasValidEnterpriseValidityToken` | `ENTERPRISE_VALIDITY_TOKEN` is
present and not expired (expiration depends on signed token payload, not
on "expiresAt" on appToken table which is only indicative) |
Feature access is gated by `isValid()` =
`hasValidEnterpriseValidityToken || hasValidEnterpriseKey` (to support
both new and legacy keys during transition). After transition isValid()
= hasValidEnterpriseValidityToken
### Frontend states
The Enterprise settings page handles multiple states:
- **No key**: show "Get Enterprise" with checkout modal
- **Orphaned validity token** (token valid but no signed key): prompt
user to set a valid enterprise key
- **Active/trialing but no validity token**: show subscription status
with a "Reload validity token" action
- **Active/trialing**: show full subscription info, billing portal
access, cancel option
- **Cancellation scheduled**: show cancellation date, billing portal
- **Canceled**: show billing history link and option to start a new
subscription
- **Past due / Incomplete**: prompt to update payment or restart
### Temporary retro-compatibility: legacy plain-text keys
Previously, enterprise features were gated by a simple check: any
non-empty string in `ENTERPRISE_KEY` granted access. With this PR, we
transition to a controlled system relying on signed JWTs.
To avoid breaking existing self-hosted users:
- **Legacy plain-text keys still grant access** to enterprise features.
`hasValidEnterpriseKey` returns `true` for both signed JWTs and plain
strings, and `isValid()` checks `hasValidEnterpriseKey` as a fallback
when no validity token is present.
- **A deprecation banner** is shown at the top of the app when
`hasValidEnterpriseKey` is `true` but `hasValidSignedEnterpriseKey` is
`false`, informing the user that their key format is deprecated and they
should activate a new signed key.
- **No billing portal or subscription management** is available for
legacy keys since there is no Stripe subscription to manage.
This retro-compatibility will be removed in a future version. At that
point, `isValid()` will only check `hasValidEnterpriseValidityToken`.
### Edge cases
- **Air-gapped / production environments**: for self-hosted clients that
block external traffic (or for our own production), provide a long-lived
`ENTERPRISE_VALIDITY_TOKEN` (e.g. 99 years) directly in the `appToken`
table, with no `ENTERPRISE_KEY`. The daily cron will skip the refresh
(no enterprise key to authenticate with), but the pre-seeded validity
token will be used to grant feature access. No billing or seat reporting
occurs in this mode.
- **`IS_CONFIG_VARIABLES_IN_DB_ENABLED` is false**: if the user tries to
activate an enterprise key but DB config writes are disabled, the
backend returns a clear error asking them to add `ENTERPRISE_KEY` to
their `.env` file manually.
- **Canceled subscriptions**: the `/seats` endpoint skips Stripe updates
for canceled or cancellation-scheduled subscriptions to avoid Stripe API
errors.
### How to test
- launch twenty-website on a different url (eg localhost:1002)
- add ENTERPRISE_API_URL=http://localhost:3002/api/enterprise (or else)
in your server .env
- ask me for twenty-website's .env file content (STRIPE_SECRET_KEY;
STRIPE_ENTERPRISE_MONTHLY_PRICE_ID;STRIPE_ENTERPRISE_YEARLY_PRICE_ID;
ENTERPRISE_JWT_PRIVATE_KEY; ENTERPRISE_JWT_PUBLIC_KEY;
NEXT_PUBLIC_WEBSITE_URL)
- visit Admin panel / enterprise
## Summary
- **Module reorganization**: Moved `ApplicationUpgradeService` and cron
jobs to `application-upgrade/`, `ApplicationSyncService` to
`application-manifest/`, and
`runWorkspaceMigration`/`uninstallApplication` mutations to the manifest
resolver — each module now has a single clear responsibility.
- **Explicit install flow**: Removed implicit `ApplicationEntity`
creation from `ApplicationSyncService`. The install service and dev
resolver now explicitly create the `ApplicationEntity` before syncing.
npm packages are resolved at registration time to extract manifest
metadata (universalIdentifier, name, description, etc.), eliminating the
`reconcileUniversalIdentifier` hack.
- **Better error handling**: Frontend hooks now surface actual server
error messages in snackbars instead of swallowing them. Replaced the
ugly `ConfirmationModal` for transfer ownership with a proper form
modal. Fixed `SettingsAdminTableCard` row height overflow and corrected
the `yarn-engine` asset path.
## Test plan
- [ ] Register an npm package — verify manifest metadata (name,
description, universalIdentifier) is extracted correctly
- [ ] Install a registered npm app on a workspace — verify
ApplicationEntity is created and sync succeeds
- [ ] Test `app:dev` CLI flow — verify local app registration and sync
work
- [ ] Upload a tarball — verify registration and install flow
- [ ] Transfer ownership — verify the new modal UX works
- [ ] Verify error messages appear correctly in snackbars when
operations fail
Made with [Cursor](https://cursor.com)
# Introduction
Important note: This PR officially deprecates the `standardId`, about to
drop col and entity property after this has been merged
Important note2: Haven't updated the optimistic tool to also update the
universal identifier aggregators only the ids one, they should not be
consumed in the runner context -> need to improve typing or either the
optimistic tooling
In this PR we're introducing all the devxp allowing future metadata
incremental universal migration -> this has an impact on all existing
metadata actions handler ( explaining its size )
This PR also introduce workspace agnostic create update actions runner
for both field and object metadata in order to battle test the described
above devxp
Noting that these two metadata are the most complex to handle
Notes:
- A workspace migration is now highly bind to a
`applicationUniversalIdentifier`. Though we don't strictly validate
application scope for the moment
## Next
Migrate both object and field builder to universal comparison
## Universal Actions vs Flat Actions Architecture
### Concept
The migration system uses a two-phase action model:
1. **Universal Actions** - Actions defined using `universalIdentifier`
(stable, portable identifiers like `standardId` + `applicationId`)
2. **Flat Actions** - Actions defined using database `entityId` (UUIDs
specific to a workspace)
### Why This Separation?
- **Universal actions are portable**: They can be serialized, stored,
and replayed across different workspaces
- **Flat actions are executable**: They contain the actual database IDs
needed to perform operations
- **Decoupling**: The builder produces universal actions; the runner
transpiles them to flat actions at execution time
### Transpiler Pattern
Each action handler must implement
`transpileUniversalActionToFlatAction()`:
```typescript
@Injectable()
export class CreateFieldActionHandlerService extends WorkspaceMigrationRunnerActionHandler(
'create',
'fieldMetadata',
) {
override async transpileUniversalActionToFlatAction(
context: WorkspaceMigrationActionRunnerArgs<UniversalCreateFieldAction>,
): Promise<FlatCreateFieldAction> {
// Resolve universal identifiers to database IDs
const flatObjectMetadata = findFlatEntityByUniversalIdentifierOrThrow({
flatEntityMaps: allFlatEntityMaps.flatObjectMetadataMaps,
universalIdentifier: action.objectMetadataUniversalIdentifier,
});
return {
type: action.type,
metadataName: action.metadataName,
objectMetadataId: flatObjectMetadata.id, // Resolved ID
flatFieldMetadatas: /* ... transpiled entities ... */,
};
}
}
```
### Action Handler Base Class
`BaseWorkspaceMigrationRunnerActionHandlerService<TActionType,
TMetadataName>` provides:
- **`transpileUniversalActionToFlatAction()`** - Abstract method each
handler must implement
- **`transpileUniversalDeleteActionToFlatDeleteAction()`** - Shared
helper for delete actions
## FlatEntityMaps custom properties
Introduced a `TWithCustomMapsProperties` generic parameter to control
whether custom indexing structures are included:
- **`false` (default)**: Returns `FlatEntityMaps<MetadataFlatEntity<T>>`
- used in builder/runner contexts
- **`true`**: Returns the full maps type with custom properties (e.g.,
`byUserWorkspaceIdAndFolderId`) - used in cache contexts
## Create Field Actions Refactor
Refactored create-field actions to support relation field pairs
bundling.
**Problem:** Relation fields (e.g., `Attachment.targetTask` ↔
`Task.attachments`) couldn't resolve each other's IDs during
transpilation because they were in separate actions with independent
`fieldIdByUniversalIdentifier` maps.
**Solution:**
- Removed `objectMetadataUniversalIdentifier` from
`UniversalCreateFieldAction` and `objectMetadataId` from
`FlatCreateFieldAction` - each field now carries its own
- Runner groups fields by object internally and processes each table
separately
- Split aggregator into two focused utilities:
- `aggregateNonRelationFieldsIntoObjectActions` - merges non-relation
fields into object actions
- `aggregateRelationFieldPairs` - bundles relation pairs with shared
`fieldIdByUniversalIdentifier`
# Introduction
Followup of
https://github.com/twentyhq/twenty/pull/17001#pullrequestreview-3638508738
close https://github.com/twentyhq/core-team-issues/issues/1910
We've completely decom the `sync-metadata` in production. We're now then
removing its implementation in favor of the v2.
## TODO:
- [x] Remove sync-metadata implem and commands
- [x] Remove workspace decorators
- [x] Type each deprecated field to deprecated on their workspaceEntity
- [x] Remove the `workspace-sync-metadata` folder entirely
- [x] remove workspace migration
- [x] workspace migration removal migration
- [x] remove the `v2` references from workspace manager file names
- [x] remove the `v2` references from workspace manager modules
- [ ] Double check impact on translation file path updates
## Note
- Removed the gate logic
- Remains some service v2 naming, serverless needs to be migrated on v2
fully
- Removed workspaceMigration service app health consumption, making it
always returning up ( no more down ) cc @FelixMalfait ( quite obsolete
health check now, will require complete refactor once we introduce inter
app dependency etc )
# Introduction
Followup https://github.com/twentyhq/twenty/pull/16863
Important note: This is not an upgrade command and will have to be
manually run
In this pull request we're introducing a coding that will allow this
[migration](https://github.com/twentyhq/twenty/blob/clean-orphan-metadata/packages/twenty-server/src/database/typeorm/core/migrations/utils/1767002571103-addWorkspaceForeignKeys.util.ts#L3)
to pass, it enforces the `workspaceId` foreignKey on all metadata
entities. Allowing workspace deletion cascading of all its related
entities and avoiding orphan metadata entities to reoccur in the future
Also introduced a small migration that will set the workspaceId col type
to `uuid`, as it has been historically `varchar`
This migration is a requirement for the above command to work
successfully
## Note
Chunking by relations fields the orphan field deletion as would take way
too much time within a transac,
## Test
Tested on a prod extract locally ( both dry and not dry )
# Introduction
Related to https://github.com/twentyhq/core-team-issues/issues/1995
This PR introduces the basis of the `twentyStandard` application as code
on demand, it's highly tied to `ids` where it will becomes workspace
agnostic following the builder and runner `universalIdentifier` refactor
later.
The goal here to allow computing the `allFlatEntityMaps` `to` of the
`twentyStandard` application on a empty workspace ( workspace creation
). Allowing installing the twenty standard app through a workspace
migration instead of passing by the sync metadata
Nothing done will be run in production for the moment if it's not the
small validation refactor we've introduced
Please note that everything introduced here will be replaced at some
point by a twenty app instance when the twenty sdk is mature enough to
handle of the edge cases we need here
## How we've proceeded
We've been iterating over every workspace entity both objects and their
fields, and transpiled them to flatEntity.
Being sure we migrate the defaultValue, settings and so on accordingly.
We've also compute all the ids in prior of the whole entities
computation so we don't face any hoisting issue.
## Current state
At the moment only handling all of the 29 standard objects and their
fields
Settings a unique universalIdentifier for all of them
Will come views, agent role targets and so on later
## `workspace:compute-twenty-standard-migration` command
This command allow generating a workspace migration that will result in
installing the twenty standard app in an empty workspace
It's temporary and aims to allow debugging for the moment we might not
keep it in the future as it is right now
It contains debug writeFileSync which is expected no worries greptile
## `LabelFieldMetadataIdentifierId`
Small refactor allowing defining the label identifier field metadata id
of a uuid field metadata type for system object, as some of our standard
object don't have a name field and don't aim to
Also please note that we might remove this build options later in the
sake of the currently installed universal identifier application that we
could compare with the deterministic twenty standard one
## `runFlatFieldMetadataValidators`
Deprecated this pattern which was redundant and not v2 friendly pattern
## Current errors that will address in upcoming PR
Current standard objects and fields metadata does not pass the
validation that we have in place, as historically the sync metadata
would directly consume the repositories and would just ignore the
validation. This is about to change.
Will handle the below errors in dedicated PRs as they will required
upgrade commands in order to migrate the data, or will handle that from
the sync metadata instead still to be determined but nothing critical
here
- camel case field metadata name
- options label invalid format
```json
{
"status": "fail",
"report": {
"fieldMetadata": [
{
"status": "fail",
"errors": [
{
"code": "INVALID_FIELD_INPUT",
"message": "Name should be in camelCase",
"userFriendlyMessage": {
"id": "P+jdmX",
"message": "Name should be in camelCase"
},
"value": "iCalUID"
}
],
"flatEntityMinimalInformation": {
"id": "68dd83cd-92c8-4233-bb28-47939bab6124",
"name": "iCalUID",
"objectMetadataId": "11c16ab6-9176-439e-a2db-a12c5a58a524"
},
"type": "create_field"
},
{
"status": "fail",
"errors": [
{
"code": "INVALID_FIELD_INPUT",
"message": "Value must be in UPPER_CASE and follow snake_case \"email\"",
"userFriendlyMessage": {
"id": "UBPzFQ",
"message": "Value must be in UPPER_CASE and follow snake_case \"{sanitizedValue}\"",
"values": {
"sanitizedValue": "email"
}
},
"value": "email"
},
{
"code": "INVALID_FIELD_INPUT",
"message": "Value must be in UPPER_CASE and follow snake_case \"sms\"",
"userFriendlyMessage": {
"id": "UBPzFQ",
"message": "Value must be in UPPER_CASE and follow snake_case \"{sanitizedValue}\"",
"values": {
"sanitizedValue": "sms"
}
},
"value": "sms"
}
],
"flatEntityMinimalInformation": {
"id": "e3caaf2a-e07d-4146-8dfc-9eef904e82c9",
"name": "type",
"objectMetadataId": "4b777de5-4c7b-4af4-9b92-655c0f87512b"
},
"type": "create_field"
},
{
"status": "fail",
"errors": [
{
"code": "INVALID_FIELD_INPUT",
"message": "Value must be in UPPER_CASE and follow snake_case \"incoming\"",
"userFriendlyMessage": {
"id": "UBPzFQ",
"message": "Value must be in UPPER_CASE and follow snake_case \"{sanitizedValue}\"",
"values": {
"sanitizedValue": "incoming"
}
},
"value": "incoming"
},
{
"code": "INVALID_FIELD_INPUT",
"message": "Value must be in UPPER_CASE and follow snake_case \"outgoing\"",
"userFriendlyMessage": {
"id": "UBPzFQ",
"message": "Value must be in UPPER_CASE and follow snake_case \"{sanitizedValue}\"",
"values": {
"sanitizedValue": "outgoing"
}
},
"value": "outgoing"
}
],
"flatEntityMinimalInformation": {
"id": "d96233a4-93be-45ea-9548-3b50f3c700cf",
"name": "direction",
"objectMetadataId": "480a648a-d2e5-482a-992f-ef053e1b4bb0"
},
"type": "create_field"
},
{
"status": "fail",
"errors": [
{
"code": "INVALID_FIELD_INPUT",
"message": "Value must be in UPPER_CASE and follow snake_case \"from\"",
"userFriendlyMessage": {
"id": "UBPzFQ",
"message": "Value must be in UPPER_CASE and follow snake_case \"{sanitizedValue}\"",
"values": {
"sanitizedValue": "from"
}
},
"value": "from"
},
{
"code": "INVALID_FIELD_INPUT",
"message": "Value must be in UPPER_CASE and follow snake_case \"to\"",
"userFriendlyMessage": {
"id": "UBPzFQ",
"message": "Value must be in UPPER_CASE and follow snake_case \"{sanitizedValue}\"",
"values": {
"sanitizedValue": "to"
}
},
"value": "to"
},
{
"code": "INVALID_FIELD_INPUT",
"message": "Value must be in UPPER_CASE and follow snake_case \"cc\"",
"userFriendlyMessage": {
"id": "UBPzFQ",
"message": "Value must be in UPPER_CASE and follow snake_case \"{sanitizedValue}\"",
"values": {
"sanitizedValue": "cc"
}
},
"value": "cc"
},
{
"code": "INVALID_FIELD_INPUT",
"message": "Value must be in UPPER_CASE and follow snake_case \"bcc\"",
"userFriendlyMessage": {
"id": "UBPzFQ",
"message": "Value must be in UPPER_CASE and follow snake_case \"{sanitizedValue}\"",
"values": {
"sanitizedValue": "bcc"
}
},
"value": "bcc"
}
],
"flatEntityMinimalInformation": {
"id": "961c598e-67c3-452d-8bb2-b92c0bc64404",
"name": "role",
"objectMetadataId": "8af8a13c-ff97-4cd3-b70d-52a7dc2924b4"
},
"type": "create_field"
},
{
"status": "fail",
"errors": [
{
"code": "INVALID_FIELD_INPUT",
"message": "Label must not contain a comma",
"userFriendlyMessage": {
"id": "k731jp",
"message": "Label must not contain a comma"
},
"value": "Commas and dot (1,234.56)"
},
{
"code": "INVALID_FIELD_INPUT",
"message": "Label must not contain a comma",
"userFriendlyMessage": {
"id": "k731jp",
"message": "Label must not contain a comma"
},
"value": "Spaces and comma (1 234,56)"
},
{
"code": "INVALID_FIELD_INPUT",
"message": "Label must not contain a comma",
"userFriendlyMessage": {
"id": "k731jp",
"message": "Label must not contain a comma"
},
"value": "Dots and comma (1.234,56)"
}
],
"flatEntityMinimalInformation": {
"id": "7fa20caf-2597-42e3-84e5-15a91b125b9b",
"name": "numberFormat",
"objectMetadataId": "a6974302-9e72-461c-aa09-9390f4ff16fc"
},
"type": "create_field"
}
],
"objectMetadata": [],
"view": [],
"viewField": [],
"viewGroup": [],
"index": [],
"serverlessFunction": [],
"cronTrigger": [],
"databaseEventTrigger": [],
"routeTrigger": [],
"viewFilter": [],
"role": [],
"roleTarget": [],
"agent": []
}
}
```
**Before**
- any user with workpace_members permission was able to remove a user
from their workspace. This triggered the deletion of workspaceMember +
of userWorkspace, but did not delete the user (even if they had no
workspace left) nor the roleTarget (acts as junction between role and
userWorkspace) which was left with a userWorkspaceId pointing to
nothing. This is because roleTarget points to userWorkspaceId but the
foreign key constraint was not implemented
- any user could delete their own account. This triggered the deletion
of all their workspaceMembers, but not of their userWorkspace nor their
user nor the roleTarget --> we have orphaned userWorkspace, not
technically but product wise - a userWorkspace without a workspaceMember
does not make sense
So the problems are
- we have some roleTargets pointing to non-existing userWorkspaceId
(which caused https://github.com/twentyhq/twenty/issues/14608 )
- we have userWorkspaces that should not exist and that have no
workspaceMember counterpart
- it is not possible for a user to leave a workspace by themselves, they
can only leave all workspaces at once, except if they are being removed
from the workspace by another user
**Now**
- if a user has multiple workspaces, they are given the possibility to
leave one workspace while remaining in the others (we show two buttons:
Leave workspace and Delete account buttons). if a user has just one
workspace, they only see Delete account
- when a user leaves a workspace, we delete their workspaceMember,
userWorkspace and roleTarget. If they don't belong to any other
workspace we also soft-delete their user
- soft-deleted users get hard deleted after 30 days thanks to a cron
- we have two commands to clean the orphans roleTarget and userWorkspace
(TODO: query db to see how many must be run)
**Next**
- once the commands have been run, we can implement and introduce the
foreign key constraint on roleTarget
Fixes https://github.com/twentyhq/twenty/issues/14608
Closes#14726
### Added
- `trashRetentionDays` field to workspace entity (default: 14 days)
- Automated trash cleanup using BullMQ jobs
- Daily cron (00:10 UTC) that enqueues cleanup jobs for all active
workspaces
- Per-workspace limit: 100k records deleted per day
- Calendar-based retention: records deleted on day X are cleaned up X+14
days later (at midnight UTC boundaries)
### Architecture
- **Cron (WorkspaceTrashCleanupCronJob):** Runs daily, enqueues jobs in
parallel for all workspaces
- **Job (WorkspaceTrashCleanupJob):** Processes individual workspace
cleanup
- **Service (WorkspaceTrashCleanupService):** Discovers tables with
`deletedAt`, deletes old records with quota enforcement
- **Command:** `npx nx run twenty-server:command
cron:workspace:cleanup-trash` to register the cron
### Testing
- Unit tests for service with 100% coverage of public API
- Tested quota enforcement, error handling, and edge cases
---------
Co-authored-by: Félix Malfait <felix.malfait@gmail.com>
In this PR:
- refactor the upgrade command / upgrade command runner to keep upgrade
command as light as possible (all wrapping logic should go to upgrade
command runner)
- prevent any upgrade if there is at least one workspace.version <
previsousVersion ==> this leads to corrupted state where only core
migrations are run if the self-hoster is skipping a version
This Pr begins the extensibility journey
- adds a `core.cronTrigger` table
- add a oneToMany relation between core.serverlessFunction and
`core.cronTrigger` (one serverlessFunction can be triggered by multiple
cronTriggers)
- add a job to trigger a serverless function
- adds a cron to trigger serverlessFunction (via the trigger job) based
on the core.cronTrigger.setting.pattern
- adds a command to register the cron
- add the command in `cron-register-all.command.ts`
Improve workflow enqueue cron : instead of relying on the cache to know
how many workflows we can enqueue, query the DB. Then set the cache and
process the not started workflows.
Also adding a second cron that will look for workflows enqueued one hour
ago or more and put these back in the not started status. This will
allow the first cron to start these again.
# What
Fully deprecate old relations because we have one bug tied to it and it
make the codebase complex
# How I've made this PR:
1. remove metadata datasource (we only keep 'core') => this was causing
extra complexity in the refactor + flaky reset
2. merge dev and demo datasets => as I needed to update the tests which
is very painful, I don't want to do it twice
3. remove all code tied to RELATION_METADATA /
relation-metadata.resolver, or anything tied to the old relation system
4. Remove ONE_TO_ONE and MANY_TO_MANY that are not supported
5. fix impacts on the different areas : see functional testing below
# Functional testing
## Functional testing from the front-end:
1. Database Reset ✅
2. Sign In ✅
3. Workspace sign-up ✅
5. Browsing table / kanban / show ✅
6. Assigning a record in a one to many / in a many to one ✅
7. Deleting a record involved in a relation ✅ => broken but not tied to
this PR
8. "Add new" from relation picker ✅ => broken but not tied to this PR
9. Creating a Task / Note, Updating a Task / Note relations, Deleting a
Task / Note (from table, show page, right drawer) ✅ => broken but not
tied to this PR
10. creating a relation from settings (custom / standard x oneToMany /
manyToOne) ✅
11. updating a relation from settings should not be possible ✅
12. deleting a relation from settings (custom / standard x oneToMany /
manyToOne) ✅
13. Make sure timeline activity still work (relation were involved
there), espacially with Task / Note => to be double checked ✅ => Cannot
convert undefined or null to object
14. Workspace deletion / User deletion ✅
15. CSV Import should keep working ✅
16. Permissions: I have tested without permissions V2 as it's still hard
to test v2 work and it's not in prod yet ✅
17. Workflows global test ✅
## From the API:
1. Review open-api documentation (REST) ✅
2. Make sure REST Api are still able to fetch relations ==> won't do, we
have a coupling Get/Update/Create there, this requires refactoring
3. Make sure REST Api is still able to update / remove relation => won't
do same
## Automated tests
1. lint + typescript ✅
2. front unit tests: ✅
3. server unit tests 2 ✅
4. front stories: ✅
5. server integration: ✅
6. chromatic check : expected 0
7. e2e check : expected no more that current failures
## Remove // Todos
1. All are captured by functional tests above, nothing additional to do
## (Un)related regressions
1. Table loading state is not working anymore, we see the empty state
before table content
2. Filtering by Creator Tim Ap return empty results
3. Not possible to add Tasks / Notes / Files from show page
# Result
## New seeds that can be easily extended
<img width="1920" alt="image"
src="https://github.com/user-attachments/assets/d290d130-2a5f-44e6-b419-7e42a89eec4b"
/>
## -5k lines of code
## No more 'metadata' dataSource (we only have 'core)
## No more relationMetadata (I haven't drop the table yet it's not
referenced in the code anymore)
## We are ready to fix the 6 months lag between current API results and
our mocked tests
## No more bug on relation creation / deletion
---------
Co-authored-by: Weiko <corentin@twenty.com>
Co-authored-by: Félix Malfait <felix@twenty.com>
In this PR we are
1. cleaning typeORM service by removing connectToDataSource method
2. using workspaceDataSource instead of mainDataSource when possible,
and replacing raw SQL with workspaceRepository methods to use
Simplifying a lot the upgrade system.
New way to upgrade:
`yarn command:prod upgrade`
New way to write upgrade commands (all wrapping is done for you)
```
override async runOnWorkspace({
index,
total,
workspaceId,
options,
}: RunOnWorkspaceArgs): Promise<void> {}
```
Also cleaning CommandModule imports to make it lighter
Félix Malfait