feat(twenty-server): migrate remaining at-rest encryption sites to versioned envelope (#20550)

## Summary

Second PR in the encryption key rotation series. The previous PR
(#20528) introduced `ENCRYPTION_KEY` + the versioned
`enc:v2:<keyId>:<base64>` envelope inside `SecretEncryptionService` and
migrated `ConnectedAccountTokenEncryptionService` as the first consumer.
This PR routes every remaining at-rest encryption site through the
versioned envelope so that `ENCRYPTION_KEY` (and the future
`FALLBACK_ENCRYPTION_KEY`) actually covers them. The legacy unprefixed
CTR ciphertext remains readable as a fallback during the rollout window
— every migrated read site uses `decryptVersioned`, which transparently
delegates to the legacy CTR decrypt when it sees an unprefixed payload.

### Service migrations
- **`ApplicationVariableEntityService` (#8)** — workspace-scoped. HKDF
info is bound to each row's `workspaceId`. A new
`decryptAndMaskVersioned` helper lands on `SecretEncryptionService` for
the resolver display path.
- **`ApplicationRegistrationVariableService` (#7)** + consumers —
**instance-scoped**. Registration variables are server-level config
readable by every workspace that installs the application, so HKDF info
is `instance`. Updated consumers:
  - `LogicFunctionExecutorService.buildServerVariableEnvMap`
  - `ConnectionProviderService.getClientCredentials`
- **`LogicFunctionExecutorService.buildEnvVar` (#9)** —
workspace-scoped. Each variable's `workspaceId` is threaded into
`decryptVersioned`, so per-workspace HKDF contexts are honoured at
execution time.
- **`UpdateApplicationVariableActionHandlerService`**
(workspace-migration runner) — threads `workspaceId` through the
secret/non-secret toggle.
- **`JwtKeyManagerService` (#3)** — instance-scoped. Signing keys are
shared across the JWKS.
- **`ConfigStorageService` (#6)** — instance-scoped sensitive STRING
config variables.

### Slow instance commands (2.5.0)

Each migrated site has a paired backfill that re-encrypts existing rows
into the v2 envelope before the column is constrained:

| timestamp | command | scope | CHECK constraint |
|---|---|---|---|
| `1798000005000` | encrypt-application-variable | workspaceId |
`"isSecret" = false OR value = '' OR value LIKE 'enc:v2:%'` |
| `1798000006000` | encrypt-application-registration-variable | instance
| `"encryptedValue" = '' OR value LIKE 'enc:v2:%'` |
| `1798000007000` | encrypt-signing-key-private-keys | instance |
`"privateKey" IS NULL OR value LIKE 'enc:v2:%'` |
| `1798000008000` | encrypt-sensitive-config-storage | instance | _none_
— heterogeneous jsonb column |

All backfills are idempotent (the SELECT filter skips rows already in v2
form) and run before their respective `up()` adds the CHECK constraint.
Every `down()` deliberately stops at dropping the CHECK constraint —
they intentionally do not re-introduce plaintext on rollback.

### Tests

- Unit specs for each new slow command cover the v2 upgrade path, the
idempotency invariant, and the instance vs workspace HKDF scope.
- New `JwtKeyManagerService` spec asserts
`decryptVersioned`/`encryptVersioned` are called without `workspaceId`
(instance scope).
- Updated existing specs for `ApplicationVariableEntityService`,
`ConfigStorageService`, and `buildEnvVar` to assert the versioned API
and the workspace HKDF context plumbing.
- New `SecretEncryptionService.decryptAndMaskVersioned` cases in the
service spec.
- Updated the `applicationRegistrationVariable` integration spec to
assert the column now stores `enc:v2:<keyId>:<base64>` instead of raw
legacy CTR.

### Out of scope (future PRs)
- `PostgresCredentialsService` — bespoke
`jwtWrapperService.generateAppSecret`–derived key +
`encryptText`/`decryptText` from `auth.util.ts`; deserves its own
migration.
- `SimpleSecretEncryptionUtil` (TOTP) — entirely different `aes-256-cbc`
`iv:enc` format; deserves its own migration.

## Test plan

- [x] `npx nx typecheck twenty-server`
- [x] `npx nx lint:diff-with-main twenty-server` (oxlint + prettier)
- [x] Local jest run for `secret-encryption | connected-account-token |
application-variable | application-registration-variable | build-env-var
| jwt-key-manager | config-storage | encrypt-application-variable |
encrypt-application-registration-variable | encrypt-signing-key |
encrypt-sensitive-config-storage` — 17 suites, 106 tests pass.
- [x] Local jest run for `upgrade | instance-command` — 12 suites, 86
tests pass.
- [ ] CI green
- [ ] Manual review of CHECK constraint shapes by a server reviewer
(each one matches `enc:v2:%` rather than `enc:v_:%` since none of the
migrated columns can legitimately hold `enc:v1:` ciphertext).
This commit is contained in:
Charles Bochet
2026-05-14 08:10:52 +02:00
committed by GitHub
parent 04eb913551
commit 7fa136f305
30 changed files with 1815 additions and 63 deletions
@@ -0,0 +1,108 @@
import { isDefined } from 'twenty-shared/utils';
import { DataSource, QueryRunner } from 'typeorm';
import { SECRET_ENCRYPTION_ENVELOPE_V2_PREFIX } from 'src/engine/core-modules/secret-encryption/constants/secret-encryption.constant';
import { SecretEncryptionService } from 'src/engine/core-modules/secret-encryption/secret-encryption.service';
import { RegisteredInstanceCommand } from 'src/engine/core-modules/upgrade/decorators/registered-instance-command.decorator';
import { SlowInstanceCommand } from 'src/engine/core-modules/upgrade/interfaces/slow-instance-command.interface';
const BACKFILL_BATCH_SIZE = 500;
const VALUE_CHECK_CONSTRAINT_NAME = 'CHK_applicationVariable_value_encrypted';
const V2_ENCRYPTED_LIKE_PATTERN = `${SECRET_ENCRYPTION_ENVELOPE_V2_PREFIX}%`;
type ApplicationVariableRow = {
id: string;
workspaceId: string;
value: string;
};
@RegisteredInstanceCommand('2.5.0', 1798000005000, { type: 'slow' })
export class EncryptApplicationVariableSlowInstanceCommand
implements SlowInstanceCommand
{
constructor(
private readonly secretEncryptionService: SecretEncryptionService,
) {}
// Re-encrypts every secret application variable into the versioned envelope
// bound to its row's workspaceId. Non-secret rows are left untouched —
// their `value` is plaintext by design. Idempotent: the SELECT filter
// skips rows already in v2 form.
async runDataMigration(dataSource: DataSource): Promise<void> {
let cursor = '00000000-0000-0000-0000-000000000000';
while (true) {
const rows: ApplicationVariableRow[] = await dataSource.query(
`SELECT id, "workspaceId", "value"
FROM "core"."applicationVariable"
WHERE id > $1
AND "isSecret" = true
AND "value" <> ''
AND "value" NOT LIKE $2
ORDER BY id
LIMIT $3`,
[cursor, V2_ENCRYPTED_LIKE_PATTERN, BACKFILL_BATCH_SIZE],
);
if (rows.length === 0) {
break;
}
for (const row of rows) {
// decryptVersioned handles legacy unprefixed CTR ciphertext by
// falling through to the raw-key decrypt path — exactly what we
// need to read the pre-migration rows.
const plaintext = this.secretEncryptionService.decryptVersioned(
row.value,
{ workspaceId: row.workspaceId },
);
if (!isDefined(plaintext)) {
continue;
}
const encryptedValue = this.secretEncryptionService.encryptVersioned(
plaintext,
{ workspaceId: row.workspaceId },
);
await dataSource.query(
`UPDATE "core"."applicationVariable"
SET "value" = $2
WHERE id = $1`,
[row.id, encryptedValue],
);
}
cursor = rows[rows.length - 1].id;
}
}
// The CHECK constraint accepts three cases:
// 1. Non-secret rows (plaintext value, possibly empty)
// 2. Empty secret rows (uninitialised — value defaults to '')
// 3. Secret rows in the versioned envelope
// It is intentionally not strict on the keyId so future key rotations,
// which change the keyId but keep the envelope shape, do not require a
// schema migration.
public async up(queryRunner: QueryRunner): Promise<void> {
await queryRunner.query(
`ALTER TABLE "core"."applicationVariable"
ADD CONSTRAINT "${VALUE_CHECK_CONSTRAINT_NAME}"
CHECK ("isSecret" = false OR "value" = '' OR "value" LIKE '${V2_ENCRYPTED_LIKE_PATTERN}')`,
);
}
// Deliberately do NOT decrypt rows on rollback — re-introducing plaintext
// secrets to the database would be a security regression. Dropping the
// CHECK constraint is enough; ApplicationVariableEntityService can still
// read the encrypted column whether or not the constraint exists.
public async down(queryRunner: QueryRunner): Promise<void> {
await queryRunner.query(
`ALTER TABLE "core"."applicationVariable"
DROP CONSTRAINT IF EXISTS "${VALUE_CHECK_CONSTRAINT_NAME}"`,
);
}
}
@@ -0,0 +1,98 @@
import { isDefined } from 'twenty-shared/utils';
import { DataSource, QueryRunner } from 'typeorm';
import { SECRET_ENCRYPTION_ENVELOPE_V2_PREFIX } from 'src/engine/core-modules/secret-encryption/constants/secret-encryption.constant';
import { SecretEncryptionService } from 'src/engine/core-modules/secret-encryption/secret-encryption.service';
import { RegisteredInstanceCommand } from 'src/engine/core-modules/upgrade/decorators/registered-instance-command.decorator';
import { SlowInstanceCommand } from 'src/engine/core-modules/upgrade/interfaces/slow-instance-command.interface';
const BACKFILL_BATCH_SIZE = 500;
const ENCRYPTED_VALUE_CHECK_CONSTRAINT_NAME =
'CHK_applicationRegistrationVariable_encryptedValue_encrypted';
const V2_ENCRYPTED_LIKE_PATTERN = `${SECRET_ENCRYPTION_ENVELOPE_V2_PREFIX}%`;
type ApplicationRegistrationVariableRow = {
id: string;
encryptedValue: string;
};
@RegisteredInstanceCommand('2.5.0', 1798000006000, { type: 'slow' })
export class EncryptApplicationRegistrationVariableSlowInstanceCommand
implements SlowInstanceCommand
{
constructor(
private readonly secretEncryptionService: SecretEncryptionService,
) {}
// Registration variables are server-level config — readable by any
// workspace that installs the parent registration — so they use the
// instance-scoped versioned envelope (no workspaceId in the HKDF info).
// Idempotent: the SELECT filter skips rows already in v2 form and rows
// still in their default '' (unfilled) state.
async runDataMigration(dataSource: DataSource): Promise<void> {
let cursor = '00000000-0000-0000-0000-000000000000';
while (true) {
const rows: ApplicationRegistrationVariableRow[] = await dataSource.query(
`SELECT id, "encryptedValue"
FROM "core"."applicationRegistrationVariable"
WHERE id > $1
AND "encryptedValue" <> ''
AND "encryptedValue" NOT LIKE $2
ORDER BY id
LIMIT $3`,
[cursor, V2_ENCRYPTED_LIKE_PATTERN, BACKFILL_BATCH_SIZE],
);
if (rows.length === 0) {
break;
}
for (const row of rows) {
const plaintext = this.secretEncryptionService.decryptVersioned(
row.encryptedValue,
);
if (!isDefined(plaintext)) {
continue;
}
const encryptedValue =
this.secretEncryptionService.encryptVersioned(plaintext);
await dataSource.query(
`UPDATE "core"."applicationRegistrationVariable"
SET "encryptedValue" = $2
WHERE id = $1`,
[row.id, encryptedValue],
);
}
cursor = rows[rows.length - 1].id;
}
}
// The CHECK constraint accepts unfilled rows ('') and rows in the
// versioned envelope. The keyId portion is left unconstrained so future
// ENCRYPTION_KEY rotations do not require a schema migration.
public async up(queryRunner: QueryRunner): Promise<void> {
await queryRunner.query(
`ALTER TABLE "core"."applicationRegistrationVariable"
ADD CONSTRAINT "${ENCRYPTED_VALUE_CHECK_CONSTRAINT_NAME}"
CHECK ("encryptedValue" = '' OR "encryptedValue" LIKE '${V2_ENCRYPTED_LIKE_PATTERN}')`,
);
}
// Deliberately do NOT decrypt rows on rollback — re-introducing plaintext
// secrets to the database would be a security regression. Dropping the
// CHECK constraint is enough; the service can still read the encrypted
// column whether or not the constraint exists.
public async down(queryRunner: QueryRunner): Promise<void> {
await queryRunner.query(
`ALTER TABLE "core"."applicationRegistrationVariable"
DROP CONSTRAINT IF EXISTS "${ENCRYPTED_VALUE_CHECK_CONSTRAINT_NAME}"`,
);
}
}
@@ -0,0 +1,95 @@
import { isDefined } from 'twenty-shared/utils';
import { DataSource, QueryRunner } from 'typeorm';
import { SECRET_ENCRYPTION_ENVELOPE_V2_PREFIX } from 'src/engine/core-modules/secret-encryption/constants/secret-encryption.constant';
import { SecretEncryptionService } from 'src/engine/core-modules/secret-encryption/secret-encryption.service';
import { RegisteredInstanceCommand } from 'src/engine/core-modules/upgrade/decorators/registered-instance-command.decorator';
import { SlowInstanceCommand } from 'src/engine/core-modules/upgrade/interfaces/slow-instance-command.interface';
const BACKFILL_BATCH_SIZE = 200;
const PRIVATE_KEY_CHECK_CONSTRAINT_NAME = 'CHK_signingKey_privateKey_encrypted';
const V2_ENCRYPTED_LIKE_PATTERN = `${SECRET_ENCRYPTION_ENVELOPE_V2_PREFIX}%`;
type SigningKeyRow = {
id: string;
privateKey: string;
};
@RegisteredInstanceCommand('2.5.0', 1798000007000, { type: 'slow' })
export class EncryptSigningKeyPrivateKeysSlowInstanceCommand
implements SlowInstanceCommand
{
constructor(
private readonly secretEncryptionService: SecretEncryptionService,
) {}
// Signing keys are instance-scoped — every workspace shares the JWKS — so
// the versioned envelope uses no workspaceId in its HKDF info. The
// SELECT filter skips already-migrated rows (idempotent re-runs) and
// NULL privateKey rows (typically revoked or rotated keys).
async runDataMigration(dataSource: DataSource): Promise<void> {
let cursor = '00000000-0000-0000-0000-000000000000';
while (true) {
const rows: SigningKeyRow[] = await dataSource.query(
`SELECT id, "privateKey"
FROM "core"."signingKey"
WHERE id > $1
AND "privateKey" IS NOT NULL
AND "privateKey" NOT LIKE $2
ORDER BY id
LIMIT $3`,
[cursor, V2_ENCRYPTED_LIKE_PATTERN, BACKFILL_BATCH_SIZE],
);
if (rows.length === 0) {
break;
}
for (const row of rows) {
const plaintext = this.secretEncryptionService.decryptVersioned(
row.privateKey,
);
if (!isDefined(plaintext)) {
continue;
}
const encryptedPrivateKey =
this.secretEncryptionService.encryptVersioned(plaintext);
await dataSource.query(
`UPDATE "core"."signingKey"
SET "privateKey" = $2
WHERE id = $1`,
[row.id, encryptedPrivateKey],
);
}
cursor = rows[rows.length - 1].id;
}
}
// The CHECK constraint admits two states: NULL (revoked keys whose
// private material has been purged) or the versioned envelope.
public async up(queryRunner: QueryRunner): Promise<void> {
await queryRunner.query(
`ALTER TABLE "core"."signingKey"
ADD CONSTRAINT "${PRIVATE_KEY_CHECK_CONSTRAINT_NAME}"
CHECK ("privateKey" IS NULL OR "privateKey" LIKE '${V2_ENCRYPTED_LIKE_PATTERN}')`,
);
}
// Deliberately do NOT decrypt rows on rollback — re-introducing
// plaintext private keys would be a severe security regression.
// Dropping the CHECK constraint is enough; JwtKeyManagerService can
// still read the encrypted column whether or not the constraint exists.
public async down(queryRunner: QueryRunner): Promise<void> {
await queryRunner.query(
`ALTER TABLE "core"."signingKey"
DROP CONSTRAINT IF EXISTS "${PRIVATE_KEY_CHECK_CONSTRAINT_NAME}"`,
);
}
}
@@ -0,0 +1,113 @@
import { isDefined } from 'twenty-shared/utils';
import { DataSource, QueryRunner } from 'typeorm';
import { KeyValuePairType } from 'src/engine/core-modules/key-value-pair/key-value-pair.entity';
import { SECRET_ENCRYPTION_ENVELOPE_V2_PREFIX } from 'src/engine/core-modules/secret-encryption/constants/secret-encryption.constant';
import { SecretEncryptionService } from 'src/engine/core-modules/secret-encryption/secret-encryption.service';
import { ConfigVariables } from 'src/engine/core-modules/twenty-config/config-variables';
import { type ConfigVariablesMetadataMap } from 'src/engine/core-modules/twenty-config/decorators/config-variables-metadata.decorator';
import { ConfigVariableType } from 'src/engine/core-modules/twenty-config/enums/config-variable-type.enum';
import { RegisteredInstanceCommand } from 'src/engine/core-modules/upgrade/decorators/registered-instance-command.decorator';
import { SlowInstanceCommand } from 'src/engine/core-modules/upgrade/interfaces/slow-instance-command.interface';
import { TypedReflect } from 'src/utils/typed-reflect';
type SensitiveConfigRow = { id: string; value: unknown };
@RegisteredInstanceCommand('2.5.0', 1798000008000, { type: 'slow' })
export class EncryptSensitiveConfigStorageSlowInstanceCommand
implements SlowInstanceCommand
{
constructor(
private readonly secretEncryptionService: SecretEncryptionService,
) {}
// ConfigStorage shares the `keyValuePair.value` (jsonb) column with
// user/feature-flag entries and with non-sensitive config — so a CHECK
// constraint cannot be added column-wide. The backfill walks only the
// CONFIG_VARIABLE rows whose key is declared `isSensitive` + STRING in
// the ConfigVariables metadata, decrypts the legacy CTR ciphertext, and
// re-encrypts it into the instance-scoped versioned envelope. Idempotent:
// already-v2 rows are left untouched.
async runDataMigration(dataSource: DataSource): Promise<void> {
const sensitiveStringKeys = this.collectSensitiveStringConfigKeys();
if (sensitiveStringKeys.length === 0) {
return;
}
for (const key of sensitiveStringKeys) {
const rows: SensitiveConfigRow[] = await dataSource.query(
`SELECT id, value
FROM "core"."keyValuePair"
WHERE type = $1
AND "userId" IS NULL
AND "workspaceId" IS NULL
AND key = $2`,
[KeyValuePairType.CONFIG_VARIABLE, key],
);
for (const row of rows) {
const rawValue = row.value;
if (typeof rawValue !== 'string') {
continue;
}
if (
rawValue === '' ||
rawValue.startsWith(SECRET_ENCRYPTION_ENVELOPE_V2_PREFIX)
) {
continue;
}
const plaintext =
this.secretEncryptionService.decryptVersioned(rawValue);
if (!isDefined(plaintext)) {
continue;
}
const encrypted =
this.secretEncryptionService.encryptVersioned(plaintext);
await dataSource.query(
`UPDATE "core"."keyValuePair"
SET value = to_jsonb($1::text)
WHERE id = $2`,
[encrypted, row.id],
);
}
}
}
// No CHECK constraint: the jsonb `value` column is heterogeneous (it
// stores booleans, numbers, strings, JSON for both sensitive and
// non-sensitive config plus unrelated user/feature-flag rows), so no
// single CHECK can usefully constrain it.
public async up(_queryRunner: QueryRunner): Promise<void> {
return;
}
public async down(_queryRunner: QueryRunner): Promise<void> {
return;
}
private collectSensitiveStringConfigKeys(): string[] {
const metadata = TypedReflect.getMetadata(
'config-variables',
ConfigVariables.prototype.constructor,
) as ConfigVariablesMetadataMap | undefined;
if (!isDefined(metadata)) {
return [];
}
return Object.entries(metadata)
.filter(
([, descriptor]) =>
descriptor?.isSensitive === true &&
descriptor?.type === ConfigVariableType.STRING,
)
.map(([key]) => key);
}
}
@@ -1,10 +1,11 @@
import { Module } from '@nestjs/common';
import { INSTANCE_COMMANDS } from 'src/database/commands/upgrade-version-command/instance-commands.constant';
import { SecretEncryptionModule } from 'src/engine/core-modules/secret-encryption/secret-encryption.module';
import { ConnectedAccountTokenEncryptionModule } from 'src/engine/metadata-modules/connected-account/services/connected-account-token-encryption.module';
@Module({
imports: [ConnectedAccountTokenEncryptionModule],
imports: [ConnectedAccountTokenEncryptionModule, SecretEncryptionModule],
providers: [...INSTANCE_COMMANDS],
})
export class InstanceCommandProviderModule {}
@@ -34,6 +34,10 @@ import { AddApplicationIdToPublicDomainFastInstanceCommand } from 'src/database/
import { AddIsInternalMessagesImportEnabledFastInstanceCommand } from 'src/database/commands/upgrade-version-command/2-5/2-5-instance-command-fast-1778525104406-add-is-internal-messages-import-enabled';
import { CreateSigningKeyTableFastInstanceCommand } from 'src/database/commands/upgrade-version-command/2-5/2-5-instance-command-fast-1778550000000-create-signing-key-table';
import { EncryptConnectedAccountTokensSlowInstanceCommand } from 'src/database/commands/upgrade-version-command/2-5/2-5-instance-command-slow-1798000004000-encrypt-connected-account-tokens';
import { EncryptApplicationVariableSlowInstanceCommand } from 'src/database/commands/upgrade-version-command/2-5/2-5-instance-command-slow-1798000005000-encrypt-application-variable';
import { EncryptApplicationRegistrationVariableSlowInstanceCommand } from 'src/database/commands/upgrade-version-command/2-5/2-5-instance-command-slow-1798000006000-encrypt-application-registration-variable';
import { EncryptSigningKeyPrivateKeysSlowInstanceCommand } from 'src/database/commands/upgrade-version-command/2-5/2-5-instance-command-slow-1798000007000-encrypt-signing-key-private-keys';
import { EncryptSensitiveConfigStorageSlowInstanceCommand } from 'src/database/commands/upgrade-version-command/2-5/2-5-instance-command-slow-1798000008000-encrypt-sensitive-config-storage';
import { AddSubFieldNameToViewSortFastInstanceCommand } from 'src/database/commands/upgrade-version-command/2-5/2-5-instance-command-fast-1778502963794-add-sub-field-name-to-view-sort';
export const INSTANCE_COMMANDS = [
@@ -71,5 +75,9 @@ export const INSTANCE_COMMANDS = [
AddIsInternalMessagesImportEnabledFastInstanceCommand,
CreateSigningKeyTableFastInstanceCommand,
EncryptConnectedAccountTokensSlowInstanceCommand,
EncryptApplicationVariableSlowInstanceCommand,
EncryptApplicationRegistrationVariableSlowInstanceCommand,
EncryptSigningKeyPrivateKeysSlowInstanceCommand,
EncryptSensitiveConfigStorageSlowInstanceCommand,
AddSubFieldNameToViewSortFastInstanceCommand,
];