fcd2d586eec8979d2663cbcff8be4b9f07a9add0
9 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Charles Bochet
|
e0b4c9918b |
feat(twenty-server): introduce ENCRYPTION_KEY env var with versioned envelope (#20528)
## Summary
- Adds `ENCRYPTION_KEY` (primary) and `FALLBACK_ENCRYPTION_KEY`
(decrypt-only fallback for rotation) env vars to twenty-server, with
backward-compatible fallback to `APP_SECRET` when `ENCRYPTION_KEY` is
unset.
- Introduces a versioned ciphertext envelope `enc:v2:<keyId>:<base64>`
using AES-256-GCM with HKDF-SHA256 derived per-context keys. The 8-hex
`keyId` fingerprint lets every row identify which physical key encrypted
it, so rotation routes directly to primary or fallback without trial
decryption; GCM's auth tag gives true integrity (legacy CTR has none).
- Migrates `ConnectedAccountTokenEncryptionService` to the new envelope
and plumbs `workspaceId` through every caller, so per-workspace HKDF
context binds each row to its tenant.
The remaining encryption sites (`jwt-key-manager`, `config-storage`,
`postgres-credentials`, `application-variable`, TOTP) stay on the legacy
unprefixed CTR path and will be migrated in follow-up PRs. The
operator-facing rotation runbook is out of scope here.
### Format details
`enc:v{N}:{keyId}:{base64}` — `N=2` is the only version produced by new
writes (`v1` exists for backward-compatible decryption of existing
connected-account rows). `keyId =
sha256(rawKey).slice(0,4).toString('hex')`. The CHECK constraint on
`core.connectedAccount.{accessToken,refreshToken}` is relaxed from `LIKE
'enc:v1:%'` to `LIKE 'enc:v_:%'` so both versions pass.
### Key resolution
| `ENCRYPTION_KEY` | `FALLBACK_ENCRYPTION_KEY` | `APP_SECRET` | Encrypt
with | Decrypt try order |
|---|---|---|---|---|
| set | set | (any) | `ENCRYPTION_KEY` | match `keyId` → primary →
fallback |
| set | unset | (any) | `ENCRYPTION_KEY` | match `keyId` → primary |
| unset | set | set | `APP_SECRET` | match `keyId` → `APP_SECRET` →
fallback |
| unset | unset | set | `APP_SECRET` | match `keyId` → `APP_SECRET` |
| unset | unset | unset | startup error | n/a |
## Test plan
- [x] `npx nx typecheck twenty-server` — clean
- [x] `npx jest
'secret-encryption|connected-account-token-encryption|connected-account-refresh-tokens|encrypt-connected-account-tokens|connection-provider-oauth-flow'`
— 87 tests pass
- [x] New `secret-encryption.service.versioned.spec.ts` covers: key
resolution table (no-key error, APP_SECRET fallback, ENCRYPTION_KEY
precedence), v2 round-trip with/without workspaceId, GCM tamper
rejection, workspaceId-mismatch rejection, keyId-based primary→fallback
routing, missing-key error names the fingerprint, v1 legacy decryption,
no-prefix legacy decryption, malformed envelope rejection.
- [x] Updated `connected-account-token-encryption.service.spec.ts`
covers workspaceId binding and HKDF context isolation.
- [x] Updated slow instance command spec verifies workspaceId is
threaded through encryption and the relaxed `enc:v_:%` LIKE pattern
matches both v1 and v2.
- [ ] Manual E2E: connect a Gmail account on a freshly deployed instance
with `APP_SECRET` only → confirm `core.connectedAccount.accessToken` is
`enc:v2:<keyId>:<base64>`.
- [ ] Manual E2E: rotate — set `ENCRYPTION_KEY=<new>` and
`FALLBACK_ENCRYPTION_KEY=<old APP_SECRET>`, restart, confirm
pre-rotation rows still decrypt and new rows carry the new `keyId`.
- [ ] Manual E2E: missing key — set `ENCRYPTION_KEY=<new>` without the
fallback, confirm decrypt error names the old `keyId` so the operator
can identify the missing key.
|
||
|
Marie
|
4852ac401a |
Add server upgrade status on admin panel (#20107)
## Summary Adds an admin upgrade-status panel that surfaces per-instance and per-workspace migration health, backed by a Redis-cached aggregate to keep the page snappy on large fleets. <img width="827" height="880" alt="Screenshot 2026-04-28 at 10 21 03" src="https://github.com/user-attachments/assets/8f88baa9-7268-4eff-bf6a-906a7f06ca91" /> <img width="804" height="892" alt="Screenshot 2026-04-28 at 10 21 11" src="https://github.com/user-attachments/assets/1e6decf8-766a-4d0e-96b1-03a9962bba3c" /> ## Computed metrics **Instance** (`InstanceUpgradeStatus`) - `inferredVersion` — version derived from the latest non-initial instance command name - `health` — `upToDate` | `behind` | `failed`, derived from the latest attempt vs. the last expected instance step in the upgrade sequence - `latestCommand` — `{ name, status, executedByVersion, errorMessage, createdAt }` from the most recent attempt **Per-workspace** (`WorkspaceUpgradeStatus`) - `workspaceId`, `displayName` - `inferredVersion`, `health`, `latestCommand` (same shape as instance), computed against the latest expected step in the sequence **Aggregate** (`AllWorkspacesUpgradeStatus`, only across `ACTIVE` / `SUSPENDED` workspaces) - `instanceUpgradeStatus` - `totalCount`, `upToDateCount`, `behindCount`, `failedCount` - `workspacesBehindIds[]`, `workspacesFailedIds[]` - `computedAt` ## Fetching strategy All reads go through `UpgradeStatusCacheService` (cache namespace: `EngineHealth`). - **Aggregate read** (`getAllWorkspacesStatus` → `getAllWorkspacesUpgradeStatus` query): reads summary + behind-ids + failed-ids in parallel; if any of the three keys is missing, full recompute (`recomputeAllWorkspaces`) is triggered, which also primes per-workspace entries. - **Per-workspace read** (`getWorkspacesStatus(ids)` → `getUpgradeStatus(ids)` query): `mget` on workspace keys; misses are recomputed individually (`recomputeWorkspace`), and aggregates are reconciled in place (count + id list deltas) without a full recompute. - **Recompute on demand**: `refreshUpgradeStatus` mutation calls `recomputeAllWorkspaces` to bypass cache and rewrite all keys. - **Auto-invalidation**: `InstanceCommandRunnerService` (fast + slow paths) and `WorkspaceCommandRunnerService` invalidate after every run via `safeInvalidateUpgradeStatusCache()` (`flushByPattern('upgrade-status:*')`). Failures in cache invalidation are swallowed and logged so they never break the migration runner. - **TTL**: `60 * 60 * 1000` ms (1 hour) on every key — protects against stale data even if a runner crashes before invalidating. ## Introduced cache keys All under the `EngineHealth` cache-storage namespace: | Key | Type | Purpose | | --- | --- | --- | | `upgrade-status:all-workspaces:summary` | `CachedAllWorkspacesStatusSummary` | Counts + instance status + `computedAt` | | `upgrade-status:all-workspaces:behind-ids` | `string[]` | Workspace ids in `behind` state | | `upgrade-status:all-workspaces:failed-ids` | `string[]` | Workspace ids in `failed` state | | `upgrade-status:workspace:<workspaceId>` | `CachedWorkspaceUpgradeStatus` | Per-workspace status (one key per workspace) | Full invalidation uses the pattern `upgrade-status:*`. ## Index added on `upgradeMigration` (already added on prod) Migration `2-2-instance-command-fast-1777308014234-addUpgradeMigrationWorkspaceIdIndex.ts`: ```sql CREATE INDEX "IDX_upgradeMigration_workspaceId_name_attempt" ON "core"."upgradeMigration" ("workspaceId", "name", "attempt") WHERE "workspaceId" IS NOT NULL; |
||
|
Félix Malfait
|
5bf7e8b101 |
test(upgrade): assert sequence-runner error structurally instead of snapshot (#20213)
## Summary The integration test `should throw when cursor command is not found in the sequence` in `failing-sequence-runner.integration-spec.ts` used `toThrowErrorMatchingSnapshot()`. The captured snapshot included the literal `TWENTY_CROSS_UPGRADE_SUPPORTED_VERSIONS` list from `upgrade-sequence-reader.service.ts`, which grows by one entry on every Twenty release. As a result, the snapshot drifted and broke whenever a new instance command landed (noticed during PR #20181), creating recurring "snapshot needs updating" churn with no real signal value. This PR replaces the snapshot assertion with a regex match on the structural part of the error message: ```ts ).rejects.toThrow(/Step "RemovedCommand" not found in upgrade sequence/); ``` The regex still catches the same class of regressions (the runner failing to surface a missing-step error) without pinning the version list. The now-empty snap file is removed (it had only this one entry). ## Test plan - [ ] CI integration tests pass on this branch - [ ] No remaining references to the deleted snapshot 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
Paul Rastoin
|
d2dda67596 |
Fix upgrade --start-from-workspace-id (#20116)
# Introduction Prevent using both `--start-from-workspace-id` and `--workspace` When any of the two are being passed we prevent passing to the next instance segment, it would require an upgrade re run even if legit When `--start-from-workspace-id` is passed we filter from all the fetched active or suspended workspace ids and apply equivalent filter as before |
||
|
Paul Rastoin
|
f8c1ad5b3c |
Filtered upgrade logs when stopping before starting next instance segment (#20078)
# Introduction In a nutshell, added a more readable logs that relates that when performing a workspace fitlered workspace you can only browser the workspace commands sequence This PR is not a fix, but a log improvement As before when cross-upgrading a single workspace you would be facing a `instance` sync barrier error as not all your workspaces would have been updated Now logging and early returning instead of letting the guard hard throw ## Tests Created a dedicated test for instance prevention on filtered upgrade Standardized `migrationRecordToKey` usage across all upgrade integration tests suites |
||
|
Charles Bochet
|
d74e3fa3b5 |
chore(server): bump current version to 2.2.0 (#20040)
## Summary We are releasing Twenty v2.2.0. This PR sets up the upgrade-version-command machinery for the new release line: - Promote `2.1.0` into `TWENTY_PREVIOUS_VERSIONS` (it just shipped) - Set `TWENTY_CURRENT_VERSION` to `2.2.0` - Reset `TWENTY_NEXT_VERSIONS` to `[]` - Refresh the `InstanceCommandGenerationService` snapshots to reflect the new current version (`2.2.0` / `2-2-` slug) - Update the failing-sequence-runner snapshot to include `2.2.0` in the covered versions list The `2-2/` upgrade-version-command module is already in place and wired into `WorkspaceCommandProviderModule`, so future upgrade commands targeting `2.2.0` can land directly under `2-2/` (or be generated against `--version 2.2.0`). |
||
|
Félix Malfait
|
251f5deab6 |
[breaking, deploy server first] fix(ai-chat): persist providerExecuted flag on tool parts (#20030)
## Summary Fixes Sentry errors of the form: > \`messages.3: \`tool_use\` ids were found without \`tool_result\` blocks immediately after: srvtoolu_…. Each \`tool_use\` block must have a corresponding \`tool_result\` block in the next message.\` ### Root cause When the model invokes a **provider-hosted tool** (e.g. Anthropic's native \`web_search\` — note the \`srvtoolu_\` ID prefix), the AI SDK marks the resulting \`UIMessagePart\` with \`providerExecuted: true\`. \`convertToModelMessages\` uses that flag to emit the tool_use/tool_result pair *inside the same assistant message* — the format Anthropic requires for server-side tools. Our \`AgentMessagePart\` persistence was dropping \`providerExecuted\` on the way to the DB (and re-hydration didn't know to set it). On the next turn, \`convertToModelMessages\` treated the rehydrated part as a client-side tool call, splitting it into \`assistant(tool_use)\` + \`user(tool_result)\` — which Anthropic then rejects with the error above. ### Fix - Add nullable \`providerExecuted BOOLEAN\` column on \`core.agentMessagePart\` via a fast instance command. - Surface the field on \`AgentMessagePartDTO\` (GraphQL). - Preserve it through \`mapUIMessagePartsToDBParts\` (server) and both \`mapDBPartToUIMessagePart\` mappers (server + frontend). - Include it in \`GET_CHAT_MESSAGES\` and \`GET_AGENT_TURNS\` selections. - Regenerate \`generated-metadata/graphql.ts\`. ### Backwards compatibility Existing rows have \`NULL providerExecuted\` and round-trip as the omitted flag — which is exactly the pre-fix behaviour for tool parts that were never provider-executed. Only *new* assistant messages using \`web_search\` (or other provider-hosted tools) will write \`true\`, and those are the only ones that were breaking. ## Test plan - [x] \`npx tsgo\` typecheck — server + front clean - [x] \`oxlint\` + \`prettier --check\` on all touched files — clean - [x] \`npx nx run twenty-server:database:migrate:prod\` runs the new instance command locally; \`providerExecuted\` column present on \`core.agentMessagePart\` - [x] Regenerated \`generated-metadata/graphql.ts\` — \`providerExecuted\` wired into both queries and \`AgentMessagePart\` type - [ ] Manual: start a chat with Anthropic web_search enabled, invoke the tool in turn 1, reply in turn 2 — should not throw the srvtoolu error 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
Charles Bochet
|
41ee6eac7a |
chore(server): bump current version to 2.0.0 and add 2.1.0 as next (#19907)
## Summary We are releasing Twenty v2.0. This PR sets up the upgrade-version-command machinery for the new release line: - Move `1.23.0` into `TWENTY_PREVIOUS_VERSIONS` (it just shipped) - Set `TWENTY_CURRENT_VERSION` to `2.0.0` (no specific upgrade commands — this is just the major version cut) - Set `TWENTY_NEXT_VERSIONS` to `['2.1.0']` so future PRs that previously would have targeted `1.24.0` now target `2.1.0` - Add empty `V2_0_UpgradeVersionCommandModule` and `V2_1_UpgradeVersionCommandModule` and wire them into `WorkspaceCommandProviderModule` - Refresh the `InstanceCommandGenerationService` snapshots to reflect the new current version (`2.0.0` / `2-0-` slug) The `2-0/` directory is intentionally empty — there are no specific upgrade commands for the v2.0 cut. New upgrade commands authored after this merges should land in `2-1/` (or be generated against `--version 2.1.0`). ## Test plan - [x] `npx jest` on the impacted upgrade test files (`upgrade-sequence-reader`, `upgrade-command-registry`, `instance-command-generation`) passes (41 tests, 8 snapshots) - [x] `prettier --check` and `oxlint` clean on touched files - [ ] Manual: open `nx run twenty-server:command -- upgrade --dry-run` against a local stack with workspaces still on `1.23.0` and confirm the sequence is computed without errors Made with [Cursor](https://cursor.com) |
||
|
Paul Rastoin
|
a4cc7fb9c5 |
[Upgrade] Fix workspace creation cursor (#19701)
## Summary ### Problem The upgrade migration system required new workspaces to always start from a workspace command, which was too rigid. When the system was mid-upgrade within an instance command (IC) segment, workspace creation would fail or produce inconsistent state. ### Solution #### Workspace-scoped instance command rows Instance commands now write upgrade migration rows for **all active/suspended workspaces** alongside the global row. This means every workspace has a complete migration history, including instance command records. - `InstanceCommandRunnerService` reloads `activeOrSuspendedWorkspaceIds` immediately before writing records (both success and failure paths) to mitigate race conditions with concurrent workspace creation. - `recordUpgradeMigration` in `UpgradeMigrationService` accepts a discriminated union over `status`, handles `error: unknown` formatting internally, and writes global + workspace rows in batch. #### Flexible initial cursor for new workspaces `getInitialCursorForNewWorkspace` now accepts the last **attempted** (not just completed) instance command with its status: - If the IC is `completed` and the next step is a workspace segment → cursor is set to the last WC of that segment (existing behavior). - If the IC is `failed` or not the last of its segment → cursor is set to that IC itself, preserving its status. This allows workspaces to be created at any point during the upgrade lifecycle, including mid-IC-segment and after IC failure. #### Relaxed workspace segment validation `validateWorkspaceCursorsAreInWorkspaceSegment` accepts workspaces whose cursor is: 1. Within the current workspace segment, OR 2. At the immediately preceding instance command with `completed` status (handles the `-w` single-workspace upgrade scenario). Workspaces with cursors in a previous segment, ahead of the current segment, or at a preceding IC with `failed` status are rejected. ### Test plan created empty workspaces to allow testing upgrade with several active workspaces |