houseassassin/twenty
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
12,117 Commits 1 Branch 0 Tags
45ac3e8218e6839ff73e109679f6dd2fa01ccb85
Commit Graph

4 Commits

Author SHA1 Message Date
Charles Bochet 7fa136f305 feat(twenty-server): migrate remaining at-rest encryption sites to versioned envelope (#20550) 
## Summary

Second PR in the encryption key rotation series. The previous PR
(#20528) introduced `ENCRYPTION_KEY` + the versioned
`enc:v2:<keyId>:<base64>` envelope inside `SecretEncryptionService` and
migrated `ConnectedAccountTokenEncryptionService` as the first consumer.
This PR routes every remaining at-rest encryption site through the
versioned envelope so that `ENCRYPTION_KEY` (and the future
`FALLBACK_ENCRYPTION_KEY`) actually covers them. The legacy unprefixed
CTR ciphertext remains readable as a fallback during the rollout window
— every migrated read site uses `decryptVersioned`, which transparently
delegates to the legacy CTR decrypt when it sees an unprefixed payload.

### Service migrations
- **`ApplicationVariableEntityService` (#8)** — workspace-scoped. HKDF
info is bound to each row's `workspaceId`. A new
`decryptAndMaskVersioned` helper lands on `SecretEncryptionService` for
the resolver display path.
- **`ApplicationRegistrationVariableService` (#7)** + consumers —
**instance-scoped**. Registration variables are server-level config
readable by every workspace that installs the application, so HKDF info
is `instance`. Updated consumers:
  - `LogicFunctionExecutorService.buildServerVariableEnvMap`
  - `ConnectionProviderService.getClientCredentials`
- **`LogicFunctionExecutorService.buildEnvVar` (#9)** —
workspace-scoped. Each variable's `workspaceId` is threaded into
`decryptVersioned`, so per-workspace HKDF contexts are honoured at
execution time.
- **`UpdateApplicationVariableActionHandlerService`**
(workspace-migration runner) — threads `workspaceId` through the
secret/non-secret toggle.
- **`JwtKeyManagerService` (#3)** — instance-scoped. Signing keys are
shared across the JWKS.
- **`ConfigStorageService` (#6)** — instance-scoped sensitive STRING
config variables.

### Slow instance commands (2.5.0)

Each migrated site has a paired backfill that re-encrypts existing rows
into the v2 envelope before the column is constrained:

| timestamp | command | scope | CHECK constraint |
|---|---|---|---|
| `1798000005000` | encrypt-application-variable | workspaceId |
`"isSecret" = false OR value = '' OR value LIKE 'enc:v2:%'` |
| `1798000006000` | encrypt-application-registration-variable | instance
| `"encryptedValue" = '' OR value LIKE 'enc:v2:%'` |
| `1798000007000` | encrypt-signing-key-private-keys | instance |
`"privateKey" IS NULL OR value LIKE 'enc:v2:%'` |
| `1798000008000` | encrypt-sensitive-config-storage | instance | _none_
— heterogeneous jsonb column |

All backfills are idempotent (the SELECT filter skips rows already in v2
form) and run before their respective `up()` adds the CHECK constraint.
Every `down()` deliberately stops at dropping the CHECK constraint —
they intentionally do not re-introduce plaintext on rollback.

### Tests

- Unit specs for each new slow command cover the v2 upgrade path, the
idempotency invariant, and the instance vs workspace HKDF scope.
- New `JwtKeyManagerService` spec asserts
`decryptVersioned`/`encryptVersioned` are called without `workspaceId`
(instance scope).
- Updated existing specs for `ApplicationVariableEntityService`,
`ConfigStorageService`, and `buildEnvVar` to assert the versioned API
and the workspace HKDF context plumbing.
- New `SecretEncryptionService.decryptAndMaskVersioned` cases in the
service spec.
- Updated the `applicationRegistrationVariable` integration spec to
assert the column now stores `enc:v2:<keyId>:<base64>` instead of raw
legacy CTR.

### Out of scope (future PRs)
- `PostgresCredentialsService` — bespoke
`jwtWrapperService.generateAppSecret`–derived key +
`encryptText`/`decryptText` from `auth.util.ts`; deserves its own
migration.
- `SimpleSecretEncryptionUtil` (TOTP) — entirely different `aes-256-cbc`
`iv:enc` format; deserves its own migration.

## Test plan

- [x] `npx nx typecheck twenty-server`
- [x] `npx nx lint:diff-with-main twenty-server` (oxlint + prettier)
- [x] Local jest run for `secret-encryption | connected-account-token |
application-variable | application-registration-variable | build-env-var
| jwt-key-manager | config-storage | encrypt-application-variable |
encrypt-application-registration-variable | encrypt-signing-key |
encrypt-sensitive-config-storage` — 17 suites, 106 tests pass.
- [x] Local jest run for `upgrade | instance-command` — 12 suites, 86
tests pass.
- [ ] CI green
- [ ] Manual review of CHECK constraint shapes by a server reviewer
(each one matches `enc:v2:%` rather than `enc:v_:%` since none of the
migrated columns can legitimately hold `enc:v1:` ciphertext).
 2026-05-14 06:10:52 +00:00
Marie 4852ac401a Add server upgrade status on admin panel (#20107) 
## Summary

Adds an admin upgrade-status panel that surfaces per-instance and
per-workspace migration health, backed by a Redis-cached aggregate to
keep the page snappy on large fleets.

<img width="827" height="880" alt="Screenshot 2026-04-28 at 10 21 03"
src="https://github.com/user-attachments/assets/8f88baa9-7268-4eff-bf6a-906a7f06ca91"
/>
<img width="804" height="892" alt="Screenshot 2026-04-28 at 10 21 11"
src="https://github.com/user-attachments/assets/1e6decf8-766a-4d0e-96b1-03a9962bba3c"
/>


## Computed metrics

**Instance** (`InstanceUpgradeStatus`)
- `inferredVersion` — version derived from the latest non-initial
instance command name
- `health` — `upToDate` | `behind` | `failed`, derived from the latest
attempt vs. the last expected instance step in the upgrade sequence
- `latestCommand` — `{ name, status, executedByVersion, errorMessage,
createdAt }` from the most recent attempt

**Per-workspace** (`WorkspaceUpgradeStatus`)
- `workspaceId`, `displayName`
- `inferredVersion`, `health`, `latestCommand` (same shape as instance),
computed against the latest expected step in the sequence

**Aggregate** (`AllWorkspacesUpgradeStatus`, only across `ACTIVE` /
`SUSPENDED` workspaces)
- `instanceUpgradeStatus`
- `totalCount`, `upToDateCount`, `behindCount`, `failedCount`
- `workspacesBehindIds[]`, `workspacesFailedIds[]`
- `computedAt`

## Fetching strategy

All reads go through `UpgradeStatusCacheService` (cache namespace:
`EngineHealth`).

- **Aggregate read** (`getAllWorkspacesStatus` →
`getAllWorkspacesUpgradeStatus` query):
reads summary + behind-ids + failed-ids in parallel; if any of the three
keys is missing, full recompute (`recomputeAllWorkspaces`) is triggered,
which also primes per-workspace entries.
- **Per-workspace read** (`getWorkspacesStatus(ids)` →
`getUpgradeStatus(ids)` query):
`mget` on workspace keys; misses are recomputed individually
(`recomputeWorkspace`), and aggregates are reconciled in place (count +
id list deltas) without a full recompute.
- **Recompute on demand**: `refreshUpgradeStatus` mutation calls
`recomputeAllWorkspaces` to bypass cache and rewrite all keys.
- **Auto-invalidation**: `InstanceCommandRunnerService` (fast + slow
paths) and `WorkspaceCommandRunnerService` invalidate after every run
via `safeInvalidateUpgradeStatusCache()`
(`flushByPattern('upgrade-status:*')`). Failures in cache invalidation
are swallowed and logged so they never break the migration runner.
- **TTL**: `60 * 60 * 1000` ms (1 hour) on every key — protects against
stale data even if a runner crashes before invalidating.

## Introduced cache keys

All under the `EngineHealth` cache-storage namespace:

| Key | Type | Purpose |
| --- | --- | --- |
| `upgrade-status:all-workspaces:summary` |
`CachedAllWorkspacesStatusSummary` | Counts + instance status +
`computedAt` |
| `upgrade-status:all-workspaces:behind-ids` | `string[]` | Workspace
ids in `behind` state |
| `upgrade-status:all-workspaces:failed-ids` | `string[]` | Workspace
ids in `failed` state |
| `upgrade-status:workspace:<workspaceId>` |
`CachedWorkspaceUpgradeStatus` | Per-workspace status (one key per
workspace) |

Full invalidation uses the pattern `upgrade-status:*`.

## Index added on `upgradeMigration` (already added on prod)

Migration
`2-2-instance-command-fast-1777308014234-addUpgradeMigrationWorkspaceIdIndex.ts`:

```sql
CREATE INDEX "IDX_upgradeMigration_workspaceId_name_attempt"
  ON "core"."upgradeMigration" ("workspaceId", "name", "attempt")
  WHERE "workspaceId" IS NOT NULL;
 2026-05-04 15:06:07 +00:00
Paul Rastoin f8c1ad5b3c Filtered upgrade logs when stopping before starting next instance segment (#20078) 
# Introduction
In a nutshell, added a more readable logs that relates that when
performing a workspace fitlered workspace you can only browser the
workspace commands sequence

This PR is not a fix, but a log improvement
As before when cross-upgrading a single workspace you would be facing a
`instance` sync barrier error as not all your workspaces would have been
updated

Now logging and early returning instead of letting the guard hard throw

## Tests
Created a dedicated test for instance prevention on filtered upgrade
Standardized `migrationRecordToKey` usage across all upgrade integration
tests suites
 2026-04-27 12:12:04 +00:00
Paul Rastoin a4cc7fb9c5 [Upgrade] Fix workspace creation cursor (#19701) 
## Summary

### Problem

The upgrade migration system required new workspaces to always start
from a workspace command, which was too rigid. When the system was
mid-upgrade within an instance command (IC) segment, workspace creation
would fail or produce inconsistent state.

### Solution

#### Workspace-scoped instance command rows

Instance commands now write upgrade migration rows for **all
active/suspended workspaces** alongside the global row. This means every
workspace has a complete migration history, including instance command
records.

- `InstanceCommandRunnerService` reloads `activeOrSuspendedWorkspaceIds`
immediately before writing records (both success and failure paths) to
mitigate race conditions with concurrent workspace creation.
- `recordUpgradeMigration` in `UpgradeMigrationService` accepts a
discriminated union over `status`, handles `error: unknown` formatting
internally, and writes global + workspace rows in batch.

#### Flexible initial cursor for new workspaces

`getInitialCursorForNewWorkspace` now accepts the last **attempted**
(not just completed) instance command with its status:

- If the IC is `completed` and the next step is a workspace segment →
cursor is set to the last WC of that segment (existing behavior).
- If the IC is `failed` or not the last of its segment → cursor is set
to that IC itself, preserving its status.

This allows workspaces to be created at any point during the upgrade
lifecycle, including mid-IC-segment and after IC failure.

#### Relaxed workspace segment validation

`validateWorkspaceCursorsAreInWorkspaceSegment` accepts workspaces whose
cursor is:
1. Within the current workspace segment, OR
2. At the immediately preceding instance command with `completed` status
(handles the `-w` single-workspace upgrade scenario).

Workspaces with cursors in a previous segment, ahead of the current
segment, or at a preceding IC with `failed` status are rejected.

### Test plan
created empty workspaces to allow testing upgrade with several active
workspaces
 2026-04-15 15:41:10 +00:00