houseassassin/twenty
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

feat(twenty-server): introduce ENCRYPTION_KEY env var with versioned envelope (#20528)

Browse Source 
## Summary

- Adds `ENCRYPTION_KEY` (primary) and `FALLBACK_ENCRYPTION_KEY`
(decrypt-only fallback for rotation) env vars to twenty-server, with
backward-compatible fallback to `APP_SECRET` when `ENCRYPTION_KEY` is
unset.
- Introduces a versioned ciphertext envelope `enc:v2:<keyId>:<base64>`
using AES-256-GCM with HKDF-SHA256 derived per-context keys. The 8-hex
`keyId` fingerprint lets every row identify which physical key encrypted
it, so rotation routes directly to primary or fallback without trial
decryption; GCM's auth tag gives true integrity (legacy CTR has none).
- Migrates `ConnectedAccountTokenEncryptionService` to the new envelope
and plumbs `workspaceId` through every caller, so per-workspace HKDF
context binds each row to its tenant.

The remaining encryption sites (`jwt-key-manager`, `config-storage`,
`postgres-credentials`, `application-variable`, TOTP) stay on the legacy
unprefixed CTR path and will be migrated in follow-up PRs. The
operator-facing rotation runbook is out of scope here.

### Format details

`enc:v{N}:{keyId}:{base64}` — `N=2` is the only version produced by new
writes (`v1` exists for backward-compatible decryption of existing
connected-account rows). `keyId =
sha256(rawKey).slice(0,4).toString('hex')`. The CHECK constraint on
`core.connectedAccount.{accessToken,refreshToken}` is relaxed from `LIKE
'enc:v1:%'` to `LIKE 'enc:v_:%'` so both versions pass.

### Key resolution

| `ENCRYPTION_KEY` | `FALLBACK_ENCRYPTION_KEY` | `APP_SECRET` | Encrypt
with | Decrypt try order |
|---|---|---|---|---|
| set | set | (any) | `ENCRYPTION_KEY` | match `keyId` → primary →
fallback |
| set | unset | (any) | `ENCRYPTION_KEY` | match `keyId` → primary |
| unset | set | set | `APP_SECRET` | match `keyId` → `APP_SECRET` →
fallback |
| unset | unset | set | `APP_SECRET` | match `keyId` → `APP_SECRET` |
| unset | unset | unset | startup error | n/a |

## Test plan

- [x] `npx nx typecheck twenty-server` — clean
- [x] `npx jest
'secret-encryption|connected-account-token-encryption|connected-account-refresh-tokens|encrypt-connected-account-tokens|connection-provider-oauth-flow'`
— 87 tests pass
- [x] New `secret-encryption.service.versioned.spec.ts` covers: key
resolution table (no-key error, APP_SECRET fallback, ENCRYPTION_KEY
precedence), v2 round-trip with/without workspaceId, GCM tamper
rejection, workspaceId-mismatch rejection, keyId-based primary→fallback
routing, missing-key error names the fingerprint, v1 legacy decryption,
no-prefix legacy decryption, malformed envelope rejection.
- [x] Updated `connected-account-token-encryption.service.spec.ts`
covers workspaceId binding and HKDF context isolation.
- [x] Updated slow instance command spec verifies workspaceId is
threaded through encryption and the relaxed `enc:v_:%` LIKE pattern
matches both v1 and v2.
- [ ] Manual E2E: connect a Gmail account on a freshly deployed instance
with `APP_SECRET` only → confirm `core.connectedAccount.accessToken` is
`enc:v2:<keyId>:<base64>`.
- [ ] Manual E2E: rotate — set `ENCRYPTION_KEY=<new>` and
`FALLBACK_ENCRYPTION_KEY=<old APP_SECRET>`, restart, confirm
pre-rotation rows still decrypt and new rows carry the new `keyId`.
- [ ] Manual E2E: missing key — set `ENCRYPTION_KEY=<new>` without the
fallback, confirm decrypt error names the old `keyId` so the operator
can identify the missing key.
This commit is contained in:
Charles Bochet
2026-05-13 18:15:54 +02:00
committed by GitHub
parent aec2e01662
commit e0b4c9918b
60 changed files with 1687 additions and 394 deletions
Download Patch File Download Diff File Expand all files Collapse all files
packages/twenty-server/src/database/commands/upgrade-version-command/2-5/2-5-instance-command-slow-1798000004000-encrypt-connected-account-tokens.ts
+28 -35
View File
@@ -1,11 +1,10 @@
import { isDefined } from 'twenty-shared/utils';
import { DataSource, QueryRunner } from 'typeorm';
import { SECRET_ENCRYPTION_ENVELOPE_V2_PREFIX } from 'src/engine/core-modules/secret-encryption/constants/secret-encryption.constant';
import { RegisteredInstanceCommand } from 'src/engine/core-modules/upgrade/decorators/registered-instance-command.decorator';
import { SlowInstanceCommand } from 'src/engine/core-modules/upgrade/interfaces/slow-instance-command.interface';
import {
CONNECTED_ACCOUNT_TOKEN_ENCRYPTION_PREFIX,
ConnectedAccountTokenEncryptionService,
} from 'src/engine/metadata-modules/connected-account/services/connected-account-token-encryption.service';
import { ConnectedAccountTokenEncryptionService } from 'src/engine/metadata-modules/connected-account/services/connected-account-token-encryption.service';
const BACKFILL_BATCH_SIZE = 500;
@@ -14,12 +13,18 @@ const ACCESS_TOKEN_CHECK_CONSTRAINT_NAME =
const REFRESH_TOKEN_CHECK_CONSTRAINT_NAME =
'CHK_connectedAccount_refreshToken_encrypted';
const V2_ENCRYPTED_LIKE_PATTERN = `${SECRET_ENCRYPTION_ENVELOPE_V2_PREFIX}%`;
type ConnectedAccountTokenRow = {
id: string;
workspaceId: string;
accessToken: string | null;
refreshToken: string | null;
};
const isPlaintext = (value: string | null): value is string =>
isDefined(value) && !value.startsWith(SECRET_ENCRYPTION_ENVELOPE_V2_PREFIX);
@RegisteredInstanceCommand('2.5.0', 1798000004000, { type: 'slow' })
export class EncryptConnectedAccountTokensSlowInstanceCommand
implements SlowInstanceCommand
@@ -29,14 +34,11 @@ export class EncryptConnectedAccountTokensSlowInstanceCommand
) {}
async runDataMigration(dataSource: DataSource): Promise<void> {
// Cursor + prefix-filter on the SELECT makes the loop both bounded in
// memory and idempotent: re-runs after a partial failure skip rows that
// were already encrypted on a prior pass.
let cursor = '00000000-0000-0000-0000-000000000000';
while (true) {
const rows: ConnectedAccountTokenRow[] = await dataSource.query(
`SELECT id, "accessToken", "refreshToken"
`SELECT id, "workspaceId", "accessToken", "refreshToken"
FROM "core"."connectedAccount"
WHERE id > $1
AND (
@@ -45,11 +47,7 @@ export class EncryptConnectedAccountTokensSlowInstanceCommand
)
ORDER BY id
LIMIT $3`,
[
cursor,
`${CONNECTED_ACCOUNT_TOKEN_ENCRYPTION_PREFIX}%`,
BACKFILL_BATCH_SIZE,
],
[cursor, V2_ENCRYPTED_LIKE_PATTERN, BACKFILL_BATCH_SIZE],
);
if (rows.length === 0) {
@@ -59,28 +57,23 @@ export class EncryptConnectedAccountTokensSlowInstanceCommand
for (const row of rows) {
const sets: string[] = [];
const params: unknown[] = [row.id];
if (
row.accessToken !== null &&
!row.accessToken.startsWith(CONNECTED_ACCOUNT_TOKEN_ENCRYPTION_PREFIX)
) {
if (isPlaintext(row.accessToken)) {
params.push(
this.connectedAccountTokenEncryptionService.encrypt(
row.accessToken,
),
this.connectedAccountTokenEncryptionService.encrypt({
plaintext: row.accessToken,
workspaceId: row.workspaceId,
}),
);
sets.push(`"accessToken" = $${params.length}`);
}
if (
row.refreshToken !== null &&
!row.refreshToken.startsWith(
CONNECTED_ACCOUNT_TOKEN_ENCRYPTION_PREFIX,
)
) {
if (isPlaintext(row.refreshToken)) {
params.push(
this.connectedAccountTokenEncryptionService.encrypt(
row.refreshToken,
),
this.connectedAccountTokenEncryptionService.encrypt({
plaintext: row.refreshToken,
workspaceId: row.workspaceId,
}),
);
sets.push(`"refreshToken" = $${params.length}`);
}
@@ -105,20 +98,20 @@ export class EncryptConnectedAccountTokensSlowInstanceCommand
await queryRunner.query(
`ALTER TABLE "core"."connectedAccount"
ADD CONSTRAINT "${ACCESS_TOKEN_CHECK_CONSTRAINT_NAME}"
CHECK ("accessToken" IS NULL OR "accessToken" LIKE '${CONNECTED_ACCOUNT_TOKEN_ENCRYPTION_PREFIX}%')`,
CHECK ("accessToken" IS NULL OR "accessToken" LIKE '${V2_ENCRYPTED_LIKE_PATTERN}')`,
);
await queryRunner.query(
`ALTER TABLE "core"."connectedAccount"
ADD CONSTRAINT "${REFRESH_TOKEN_CHECK_CONSTRAINT_NAME}"
CHECK ("refreshToken" IS NULL OR "refreshToken" LIKE '${CONNECTED_ACCOUNT_TOKEN_ENCRYPTION_PREFIX}%')`,
CHECK ("refreshToken" IS NULL OR "refreshToken" LIKE '${V2_ENCRYPTED_LIKE_PATTERN}')`,
);
}
// Deliberately do NOT decrypt rows on rollback — re-introducing plaintext
// tokens to the database would be a security regression. Dropping the
// CHECK constraints is enough; ConnectedAccountTokenEncryptionService can
// still read the encrypted columns whether or not the constraints exist.
public async down(queryRunner: QueryRunner): Promise<void> {
// Deliberately do NOT decrypt rows on rollback — re-introducing plaintext
// tokens to the database would be a security regression. Dropping the
// CHECK constraints is enough; ConnectedAccountTokenEncryptionService can
// still read the encrypted columns whether or not the constraints exist.
await queryRunner.query(
`ALTER TABLE "core"."connectedAccount"
DROP CONSTRAINT IF EXISTS "${REFRESH_TOKEN_CHECK_CONSTRAINT_NAME}"`,
packages/twenty-server/src/database/commands/upgrade-version-command/2-5/__tests__/2-5-instance-command-slow-1798000004000-encrypt-connected-account-tokens.spec.ts
+92 -81
View File
@@ -1,20 +1,22 @@
import { isDefined } from 'twenty-shared/utils';
import { type DataSource } from 'typeorm';
import { EncryptConnectedAccountTokensSlowInstanceCommand } from 'src/database/commands/upgrade-version-command/2-5/2-5-instance-command-slow-1798000004000-encrypt-connected-account-tokens';
import {
CONNECTED_ACCOUNT_TOKEN_ENCRYPTION_PREFIX,
type ConnectedAccountTokenEncryptionService,
} from 'src/engine/metadata-modules/connected-account/services/connected-account-token-encryption.service';
import { SECRET_ENCRYPTION_ENVELOPE_V2_PREFIX } from 'src/engine/core-modules/secret-encryption/constants/secret-encryption.constant';
import { type ConnectedAccountTokenEncryptionService } from 'src/engine/metadata-modules/connected-account/services/connected-account-token-encryption.service';
type FakeRow = {
id: string;
workspaceId: string;
accessToken: string | null;
refreshToken: string | null;
};
// In-memory stand-in that mimics the slow command's exact SELECT / UPDATE
// shape (LIKE filter, cursor, batch) — anything looser would let regressions
// in the SQL slip past these tests.
const FAKE_V2_KEY_ID = 'deadbeef';
const wrapAsV2 = (plaintext: string, workspaceId: string): string =>
`${SECRET_ENCRYPTION_ENVELOPE_V2_PREFIX}${FAKE_V2_KEY_ID}:CIPHER(${plaintext}|${workspaceId})`;
const buildFakeDataSource = (
initialRows: FakeRow[],
{ batchSize }: { batchSize: number } = { batchSize: 500 },
@@ -26,23 +28,29 @@ const buildFakeDataSource = (
const rows = [...initialRows].sort((a, b) => a.id.localeCompare(b.id));
let queryCallCount = 0;
const matchesLikePattern = (value: string, pattern: string): boolean => {
const escaped = pattern.replace(/[-/\\^$+?.()|[\]{}]/g, '\\$&');
const expr = escaped.replace(/%/g, '.*').replace(/_/g, '.');
return new RegExp(`^${expr}$`).test(value);
};
const fakeDataSource = {
query: jest.fn(async (sql: string, params?: unknown[]) => {
queryCallCount++;
if (sql.includes('SELECT id')) {
const cursor = params?.[0] as string;
const prefixPattern = params?.[1] as string;
const prefix = prefixPattern.replace(/%$/, '');
const likePattern = params?.[1] as string;
return rows
.filter((row) => row.id > cursor)
.filter(
(row) =>
(row.accessToken !== null &&
!row.accessToken.startsWith(prefix)) ||
(row.refreshToken !== null &&
!row.refreshToken.startsWith(prefix)),
(isDefined(row.accessToken) &&
!matchesLikePattern(row.accessToken, likePattern)) ||
(isDefined(row.refreshToken) &&
!matchesLikePattern(row.refreshToken, likePattern)),
)
.slice(0, batchSize);
}
@@ -51,20 +59,19 @@ const buildFakeDataSource = (
const id = params?.[0] as string;
const target = rows.find((row) => row.id === id);
if (!target) {
if (!isDefined(target)) {
return;
}
// Mirror the SQL: SET "accessToken" = $N, "refreshToken" = $M WHERE id = $1
const accessTokenMatch = sql.match(/"accessToken" = \$(\d+)/);
const refreshTokenMatch = sql.match(/"refreshToken" = \$(\d+)/);
if (accessTokenMatch) {
if (isDefined(accessTokenMatch)) {
target.accessToken = params?.[
Number(accessTokenMatch[1]) - 1
] as string;
}
if (refreshTokenMatch) {
if (isDefined(refreshTokenMatch)) {
target.refreshToken = params?.[
Number(refreshTokenMatch[1]) - 1
] as string;
@@ -84,109 +91,118 @@ const buildFakeDataSource = (
};
};
const buildFakeTokenEncryptionService =
(): ConnectedAccountTokenEncryptionService =>
({
encrypt: jest.fn(
({
plaintext,
workspaceId,
}: {
plaintext: string;
workspaceId: string;
}): string => wrapAsV2(plaintext, workspaceId),
),
}) as unknown as ConnectedAccountTokenEncryptionService;
const buildCommand = (): {
command: EncryptConnectedAccountTokensSlowInstanceCommand;
} => {
const command = new EncryptConnectedAccountTokensSlowInstanceCommand(
buildFakeTokenEncryptionService(),
);
return { command };
};
describe('EncryptConnectedAccountTokensSlowInstanceCommand', () => {
// Real AES round-trip is asserted in ConnectedAccountTokenEncryptionService's
// own spec; here we use a CIPHER(...) wrapper so assertions match exact strings.
const buildFakeTokenEncryptionService =
(): ConnectedAccountTokenEncryptionService =>
({
encrypt: jest.fn(
(plaintext: string): string =>
`${CONNECTED_ACCOUNT_TOKEN_ENCRYPTION_PREFIX}CIPHER(${plaintext})`,
),
}) as unknown as ConnectedAccountTokenEncryptionService;
const buildCommand = (): {
command: EncryptConnectedAccountTokensSlowInstanceCommand;
connectedAccountTokenEncryptionService: ConnectedAccountTokenEncryptionService;
} => {
const connectedAccountTokenEncryptionService =
buildFakeTokenEncryptionService();
const command = new EncryptConnectedAccountTokensSlowInstanceCommand(
connectedAccountTokenEncryptionService,
);
return { command, connectedAccountTokenEncryptionService };
};
describe('runDataMigration', () => {
it('should encrypt every legacy plaintext row and leave already-prefixed rows untouched', async () => {
const alreadyEncrypted = `${CONNECTED_ACCOUNT_TOKEN_ENCRYPTION_PREFIX}preexisting-ciphertext`;
it('upgrades plaintext rows to v2 with workspaceId threaded through, and leaves v2 rows untouched', async () => {
const wsA = '11111111-1111-1111-1111-111111111111';
const wsB = '22222222-2222-2222-2222-222222222222';
const alreadyV2 = `${SECRET_ENCRYPTION_ENVELOPE_V2_PREFIX}cafebabe:preexisting-v2-ciphertext`;
const { dataSource, rows } = buildFakeDataSource([
{
id: 'aaaaaaaa-0000-0000-0000-000000000001',
workspaceId: wsA,
accessToken: 'plaintext-access-1',
refreshToken: 'plaintext-refresh-1',
},
{
id: 'bbbbbbbb-0000-0000-0000-000000000002',
accessToken: alreadyEncrypted,
refreshToken: alreadyEncrypted,
workspaceId: wsB,
accessToken: alreadyV2,
refreshToken: alreadyV2,
},
{
id: 'cccccccc-0000-0000-0000-000000000003',
workspaceId: wsB,
accessToken: 'plaintext-access-3',
refreshToken: null,
},
]);
const { command } = buildCommand();
await command.runDataMigration(dataSource);
await buildCommand().command.runDataMigration(dataSource);
expect(rows()).toEqual([
{
id: 'aaaaaaaa-0000-0000-0000-000000000001',
accessToken: `${CONNECTED_ACCOUNT_TOKEN_ENCRYPTION_PREFIX}CIPHER(plaintext-access-1)`,
refreshToken: `${CONNECTED_ACCOUNT_TOKEN_ENCRYPTION_PREFIX}CIPHER(plaintext-refresh-1)`,
workspaceId: wsA,
accessToken: wrapAsV2('plaintext-access-1', wsA),
refreshToken: wrapAsV2('plaintext-refresh-1', wsA),
},
{
id: 'bbbbbbbb-0000-0000-0000-000000000002',
accessToken: alreadyEncrypted,
refreshToken: alreadyEncrypted,
workspaceId: wsB,
accessToken: alreadyV2,
refreshToken: alreadyV2,
},
{
id: 'cccccccc-0000-0000-0000-000000000003',
accessToken: `${CONNECTED_ACCOUNT_TOKEN_ENCRYPTION_PREFIX}CIPHER(plaintext-access-3)`,
workspaceId: wsB,
accessToken: wrapAsV2('plaintext-access-3', wsB),
refreshToken: null,
},
]);
});
// Regression guard: the SELECT filter is per-row (one column unencrypted is
// enough to fetch the row), so the loop body sees rows where one column is
// already prefixed and the other isn't. The per-cell prefix check inside
// the loop is what prevents the prefixed column from being double-encrypted
// into `enc:v1:CIPHER(enc:v1:...)`. If that check ever regresses, this is
// the test that should fail.
it('should only encrypt the plaintext column when a row mixes encrypted and plaintext tokens', async () => {
const alreadyEncryptedAccess = `${CONNECTED_ACCOUNT_TOKEN_ENCRYPTION_PREFIX}preexisting-access-cipher`;
// Regression guard: the SELECT filter is per-row (one non-v2 column is
// enough to fetch the row), so the loop sees rows where one column is
// already v2 and the other is plaintext. The per-cell guard inside the
// loop is what prevents the v2 column from being double-encrypted.
it('only rewrites the non-v2 column when a row mixes v2 and plaintext', async () => {
const ws = '33333333-3333-3333-3333-333333333333';
const alreadyV2 = `${SECRET_ENCRYPTION_ENVELOPE_V2_PREFIX}cafebabe:preexisting-v2-access`;
const { dataSource, rows } = buildFakeDataSource([
{
id: 'aaaaaaaa-0000-0000-0000-000000000001',
accessToken: alreadyEncryptedAccess,
workspaceId: ws,
accessToken: alreadyV2,
refreshToken: 'plaintext-refresh-mixed',
},
]);
const { command } = buildCommand();
await command.runDataMigration(dataSource);
await buildCommand().command.runDataMigration(dataSource);
expect(rows()).toEqual([
{
id: 'aaaaaaaa-0000-0000-0000-000000000001',
accessToken: alreadyEncryptedAccess,
refreshToken: `${CONNECTED_ACCOUNT_TOKEN_ENCRYPTION_PREFIX}CIPHER(plaintext-refresh-mixed)`,
workspaceId: ws,
accessToken: alreadyV2,
refreshToken: wrapAsV2('plaintext-refresh-mixed', ws),
},
]);
});
it('should be idempotent — re-running on already-migrated data leaves it unchanged', async () => {
it('is idempotent — a second run leaves already-migrated data unchanged', async () => {
const ws = '44444444-4444-4444-4444-444444444444';
const { dataSource, rows } = buildFakeDataSource([
{
id: 'aaaaaaaa-0000-0000-0000-000000000001',
workspaceId: ws,
accessToken: 'plaintext-token',
refreshToken: null,
},
@@ -197,7 +213,8 @@ describe('EncryptConnectedAccountTokensSlowInstanceCommand', () => {
const expectedFinalState = [
{
id: 'aaaaaaaa-0000-0000-0000-000000000001',
accessToken: `${CONNECTED_ACCOUNT_TOKEN_ENCRYPTION_PREFIX}CIPHER(plaintext-token)`,
workspaceId: ws,
accessToken: wrapAsV2('plaintext-token', ws),
refreshToken: null,
},
];
@@ -209,11 +226,11 @@ describe('EncryptConnectedAccountTokensSlowInstanceCommand', () => {
expect(rows()).toEqual(expectedFinalState);
});
it('should paginate through more rows than the batch size', async () => {
// 1100 rows + batch size 500 → at least 3 SELECT batches.
it('paginates through more rows than the batch size', async () => {
const ws = '55555555-5555-5555-5555-555555555555';
const initialRows: FakeRow[] = Array.from({ length: 1100 }, (_, idx) => ({
// Lex-sortable hex IDs so the cursor advance works the way the SQL does.
id: `${idx.toString(16).padStart(12, '0')}-0000-0000-0000-000000000000`,
workspaceId: ws,
accessToken: `plaintext-${idx}`,
refreshToken: null,
}));
@@ -223,21 +240,15 @@ describe('EncryptConnectedAccountTokensSlowInstanceCommand', () => {
{ batchSize: 500 },
);
const { command } = buildCommand();
await buildCommand().command.runDataMigration(dataSource);
await command.runDataMigration(dataSource);
// Every row got encrypted
expect(
rows().every((row) =>
row.accessToken!.startsWith(
CONNECTED_ACCOUNT_TOKEN_ENCRYPTION_PREFIX,
),
row.accessToken!.startsWith(SECRET_ENCRYPTION_ENVELOPE_V2_PREFIX),
),
).toBe(true);
// Sanity check: at least the expected number of SELECT batches happened
// (3 SELECTs for 500/500/100 + 1 final empty SELECT + 1100 UPDATEs)
// 3 SELECT batches (500/500/100) + 1 final empty SELECT + 1100 UPDATEs.
expect(queryCallCount()).toBeGreaterThanOrEqual(1100 + 4);
});
});