b9e5ff206512e08b321ecb2be3cee58b36269ce6
636 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Félix Malfait
|
b338a7a1d2 |
feat(settings): discovery hero rollout + ephemeral playground token (#21072)
## Summary
Two intertwined streams of work:
### UI — discovery hero pattern, settings shell, AI/API redesign
- **Generalize `SettingsDiscoveryHeroCard`** and use it on Layout, Data
Model, Apps, AI, API/Webhooks, Members. Drops 4 per-page wrapper files
(`SettingsObjectCoverImage`, `SettingsLayoutCoverImage`,
`SettingsLayoutCustomizeVideoModal`,
`SettingsDataModelVisualizeVideoModal`). Each page now supplies cover
src, modal id, and tab list.
- **Modal**: swap `<video>` placeholder for the Vimeo iframe pattern
from `twenty-docs`, per-tab `vimeoId`. Drop the parallel border-bottom
on the header (TabList draws its own baseline) and the grey background
behind the video. Note: Vimeo's embed allowlist applies — the iframes
load with the correct URL on `localhost` but the player itself requires
the video owner to allow the dev/staging domains in Vimeo settings.
- **AI page** rebuilt into a Cockpit pattern (Overview / Models / Skills
/ Tools / Usage). New `SettingsAiOverviewTab` with default Smart/Fast
pickers, at-a-glance stats, and an MCP signpost that deep-links to
`/settings/api-webhooks#mcp`. System Prompt link moved under Models.
Advanced tab removed.
- **API & Webhooks** now has 4 tabs (Playground / MCP / API Keys /
Webhooks). Hero card above tabs. Playground tab inverted to "Core API" /
"Metadata API" sections, each containing REST + GraphQL cards — schema
is the meaningful axis, protocol is secondary. Hash deep-link sync
delegated to the shared `TabListFromUrlOptionalEffect`.
- **Settings shell**: unified drawer outer padding (kill `isSettings`
branch), extract `CollapsibleNavigationDrawerSection`, add `iconColor`
on settings nav items, fix Exit Settings button alignment, 880px content
cap.
### Backend — strategy C: ephemeral playground token
The legacy paste-your-API-key flow is replaced by an on-demand
short-lived token scoped to the calling user's permissions. No shared
"Playground" API key to manage or revoke.
- New `JwtTokenTypeEnum.PLAYGROUND`. `PlaygroundTokenJwtPayload =
Omit<AccessTokenJwtPayload, 'type' | impersonation fields>` so any
future ACCESS claim flows through automatically.
- `AccessTokenService.generatePlaygroundToken` signs an access-shaped
JWT with `type: PLAYGROUND` and a configurable short TTL. A shared
private `resolveTokenSubject` helper parallelizes the user / workspace /
userWorkspace lookups for both generators.
- `JwtAuthStrategy.validateAccessToken` widened to accept
`AccessTokenJwtPayload | PlaygroundTokenJwtPayload`; impersonation gated
on `payload.type === ACCESS` so the union narrows without `as unknown
as` casts. The two branches in `validate()` collapse into one.
- New `PLAYGROUND_TOKEN_EXPIRES_IN` config var (default `2h`).
- New `generatePlaygroundToken` mutation (`WorkspaceAuthGuard`, no args,
returns `AuthToken`).
- Frontend `useOpenPlayground` hook centralizes mint → atom write →
navigate, with Apollo `onError` snackbar and a "use cached PLAYGROUND
token if still fresh" short-circuit (decodes via `jwt-decode`, checks
both `type` AND `exp`). Old API_KEY tokens left in localStorage from the
prior paste-form flow are rejected on `type` alone and force a re-mint —
this is what was causing the "This API Key is revoked" symptom on stale
browsers.
### Drive-by cleanups
- `PlaygroundToken` DTO removed (identical shape to `AuthToken` already
in use).
- 5 `customize-sidebar.webm` imports and the dead placeholder pipeline
removed.
## Test plan
### Discovery hero
- [ ] `/settings/layout`, `/settings/data-model`,
`/settings/applications`, `/settings/ai`, `/settings/api-webhooks`,
`/settings/members` each render the discovery hero card with its
illustration + play button + tabbed modal
- [ ] Modal tabs show the correct Vimeo embed URL per tab; aspect ratio
stays at 1440/900; no parallel border-bottom jog at the tab baseline
- [ ] AI Overview tab shows Smart/Fast model pickers + stats grid + MCP
signpost card; the MCP card lands on `/settings/api-webhooks#mcp` with
the MCP tab active
### API playground (ephemeral token)
- [ ] With an empty `playgroundApiKeyState` in localStorage, clicking
REST or GraphQL playground card opens the playground and the cached
token has `type: "PLAYGROUND"` with ~2h exp
- [ ] Clicking the card again within the freshness window does **not**
re-mint (`iat` / fingerprint stable across visits)
- [ ] Planting a fake API_KEY-shaped JWT in localStorage and clicking
the card forces a fresh mint (old token rejected on `type`)
- [ ] `GET /rest/companies?limit=1` with the cached token returns 200 +
real data
- [ ] `POST /graphql { __typename }` returns 200
### Settings shell
- [ ] Settings nav matches main app drawer padding; sections collapse;
Exit Settings button aligns with the workspace links above
- [ ] Active nav items have a right-gap (cleaner active state)
- [ ] Content area capped at 880px
### Verify
- [ ] `npx nx typecheck twenty-front` passes
- [ ] `npx nx typecheck twenty-server` passes
- [ ] `npx nx lint:diff-with-main twenty-front` passes
- [ ] `npx nx lint:diff-with-main twenty-server` passes
|
||
|
nitin
|
13f09d8946 |
[Dashboards] Remove gauge chart types and code (#20410)
Follow-up cleanup to #20172. |
||
|
nitin
|
996cdaf3ff |
refactor(agents): split tool resolution into native and action rails (#20331)
## Summary
Splits AI agent tool resolution into two independent rails:
- **Native tools** — capabilities baked into the model SDK
(Anthropic/OpenAI `web_search`, xAI `web`/`x` provider options). Bound
by `NativeToolBinderService`, controlled by per-agent
`modelConfiguration` toggles. Opaque to Twenty — executed on the model
provider's servers.
- **Action tools** — registry-scoped tools from `ToolRegistryService`
(code interpreter, send email, record CRUD, etc.). Permission-gated via
the agent's role. Executed on Twenty's server.
Both rails merge into a single `ToolSet` at call time. When both
surfaces expose a search tool the model picks at runtime — coexistence
is intentional (relevant once Exa returns as an action, see below).
## Notable changes worth calling out
**Contract change: `AgentAsyncExecutorService.executeAgent` no longer
accepts `rolePermissionConfig`.** Workflow agents now scope exclusively
by the agent's own permission-tab role (`unionOf: [agentRoleId]`). The
previous role-merging path (caller role intersected with agent role) is
removed. No agent role → no registry tools (fail-closed by design).
**`NativeToolBinderService` relocated** from
`core-modules/tool-provider/native/` →
`metadata-modules/ai/ai-models/services/`. The binder needs SDK-package
knowledge, which lives in `ai-models`. Old location created a backwards
module dependency.
**`NATIVE_MODEL_TOOLS_BY_SDK_PACKAGE` is exhaustive over
`AiSdkPackage`** (`Record<>`, not `Partial<Record<>>`). Adding a new SDK
without thinking about native tools now fails the build. SDKs without
native tools (Bedrock, Google, Mistral, Azure, OpenAI-compatible) get
explicit `{}` entries.
**Discriminated union `kind: 'sdk-tool' | 'provider-option'`** lets one
registry describe both function tools (Anthropic/OpenAI) and runtime
sources (xAI). Follows the local `tool-provider` convention from #19321.
## Deferred to follow-ups
- **Exa web search is dropped from this PR** (along with its
`WEB_SEARCH_TOOL` permission flag and the Exa-specific gating). Exa
comes back as an **action/app tool** once apps can define permission
flags through the SDK — ongoing work in #20481.
- **xAI native search currently errors.** xAI deprecated its Live Search
API (the `web`/`x` provider-option sources this rail maps to), so xAI
returns `410` when native search is actually exercised. The code path
itself is clear — it's only hit if you test xAI native tools. Fixed
separately alongside the broader xAI model fixes.
## Conscious non-decisions
- **No "twenty-native" category.** `native` is reserved for
model/provider SDK features; everything Twenty-owned is just a
tool/action.
- **Coexistence over precedence.** No rule forcing an action search tool
to override native search (or vice-versa) — when both exist, it's the
user's choice in workflow agents and the model's choice in chat.
---------
Co-authored-by: Félix Malfait <felix.malfait@gmail.com>
Co-authored-by: Félix Malfait <felix@twenty.com>
|
||
|
Paul Rastoin
|
ebfaca5b3d |
EncryptedString PlaintextString branded string types (#21001)
## Summary closes https://github.com/twentyhq/core-team-issues/issues/2464 Introduces compile-time branded types to distinguish encrypted ciphertext from plaintext strings, preventing mix-ups like the one fixed in #20819 — but at the type level rather in addition to the one existing at runtime. ### Branded string primitives - Created `EncryptedString` and `PlaintextString` as hard nominal brands using `z.string().brand(...)`, making them non-assignable to each other or to raw `string` - Created `isEncryptedString` type predicate to narrow `string` to `EncryptedString` based on the `enc:v2:` envelope prefix - Retyped `SecretEncryptionService`: `encryptVersioned` accepts `PlaintextString`, `decryptVersioned` returns `PlaintextString` ### Entity typing - Typed encrypted columns across entities: `SigningKeyEntity.privateKey`, `TwoFactorAuthenticationMethodEntity.secret`, `ApplicationRegistrationVariableEntity.encryptedValue`, `ApplicationVariableEntity.value` - Parameterized JSONB types for connected account connection parameters (`ImapSmtpCaldavParams<Pwd>`) with reusable aliases `EncryptedImapSmtpCaldavParams` / `DecryptedImapSmtpCaldavParams` - Typed DTOs (`CreateApplicationRegistrationVariableInput`, `UpdateApplicationRegistrationVariablePayload`, `UpdateApplicationVariableEntityInput`) with `PlaintextString` ### ApplicationVariable always-encrypt uniformization - Retyped `ApplicationVariableEntity.value` to `EncryptedString | ''` — all values are now encrypted regardless of `isSecret` - Updated `ApplicationVariableEntityService` to always encrypt on write and always decrypt on read - Simplified `UpdateApplicationVariableActionHandlerService` by removing conditional encrypt/decrypt-on-isSecret-toggle logic - Added slow instance command (`2.9.0`) to backfill-encrypt existing `isSecret=false` plaintext rows and tighten the `CHECK` constraint ### ConfigStorageService refactor - Split `convertAndSecureValue` (which used `any`) into two well-typed methods: `convertAndDecrypt` and `convertAndEncrypt` - Introduced `isSensitiveStringValue` type predicate to narrow values before encryption/decryption ### What's next - Typeorm entity derivation to strictly type sitemap configuration as code + handler logic for encryption rotation - https://github.com/twentyhq/core-team-issues/issues/2465 |
||
|
Félix Malfait
|
531410f64a |
fix(ai): expose MORPH_RELATION join columns in AI/MCP tool schemas (#21012)
## Summary
- Fixes a bug where `noteTarget` (and any other morph-relation join
object) created via AI/MCP would land with `targetCompanyId` /
`targetPersonId` / `targetOpportunityId` left null, even though the tool
reported success.
- Root cause: the Zod schema generators for the AI tools only branched
on `FieldMetadataType.RELATION`. MORPH_RELATION fields fell through to
the default case — for `create_*` they were exposed as `targetCompany:
string` instead of `targetCompanyId: uuid`, and for `group_by_*` they
were silently skipped entirely. Downstream
(`data-arg-processor.service.ts` and the group-by arg processor) already
accept the join-column form for both kinds of relations via
`computeMorphOrRelationFieldJoinColumnName` and
`isMorphOrRelationFlatFieldMetadata`, so the fix is purely in the schema
generators.
## Changes
- `record-properties.zod-schema.ts` — extend the existing RELATION
MANY_TO_ONE / ONE_TO_MANY branches to also match MORPH_RELATION.
- `group-by-tool.zod-schema.ts` — replace the silent MORPH_RELATION skip
with the same treatment as RELATION MANY_TO_ONE (exposes `${name}Id` as
a groupBy option).
- `test/integration/ai/suites/mcp-tool-execution.integration-spec.ts` —
new file. First integration test for tool execution end-to-end. Drives
the real MCP JSON-RPC endpoint with the seeded API key (`learn_tools`
for schema introspection, `execute_tool` for invocation):
- asserts `create_note_target`'s schema exposes `targetCompanyId` /
`targetPersonId` / `targetOpportunityId` as UUIDs and does **not**
expose `targetCompany` / `targetPerson` / `targetOpportunity`.
- creates a company + note + noteTarget via MCP, then queries the
workspace schema to confirm `targetCompanyId` is actually persisted in
the FK column.
- asserts `group_by_note_targets` schema accepts `targetCompanyId` as a
groupBy key.
- sets up 3 noteTargets (2 → company A, 1 → company B), calls
`group_by_note_targets` by `targetCompanyId`, and asserts the counts.
Out of scope: `record-filter.zod-schema.ts` has the same pattern (only
RELATION) — left for a follow-up so this PR stays focused on what was
reported.
## Test plan
- [x] `npx nx typecheck twenty-server`
- [x] `npx oxlint --type-aware` on changed files — clean
- [x] `npx oxfmt --check` on changed files — clean
- [x] Integration tests pass (4/4) after `database:reset`:
- `should expose the morph-relation join columns as \`${name}Id\` UUID
parameters`
- `should persist targetCompanyId when create_note_target is invoked via
MCP`
- `should expose targetCompanyId as a valid groupBy option`
- `should group noteTargets by targetCompanyId via MCP`
|
||
|
neo773
|
c5606212f2 |
Ses outbound followup (#20610)
This pull request unifies outbound with inbound under the new feature and the new email groups feature. These are workspace level shared inboxes that are shared between all workspace members. outbound sending with SES works, we only listen for tenant status events, rest is managed by AWS PR refactors old code and webhook to be split for outbound and inbound for proper separation | Area | Change | |---|---| | AWS SES driver | Split into `AwsSesRegisterDomainService` (tenant + identity + DKIM + MAIL FROM + configuration-set + EventBridge dest + contact list) and `AwsSesSendEmailService` (SendEmail). | | Reputation webhook | New `/webhooks/messaging/ses/outbound` route. SES → EventBridge (`Sending Status Enabled/Disabled` on default bus) → SNS → router → `SesOutboundSendingStateHandlerService` updates `emailing_domain.tenantStatus`. | | Inbound webhook | Refactored into `SesInboundWebhookRouterService` + `SesInboundMailHandlerService`. Shared `SnsSignatureVerifierService` + `SnsSubscriptionConfirmerService` across both routes. | | Global uniqueness | New migration + instance command: `emailing_domain.domain` is now globally unique (one tenant per domain across workspaces). | | Tenant status | New `emailing_domain.tenantStatus` column (`ACTIVE` / `PAUSED`) + `EmailingDomainTenantStatusService`. | | Send-email mutation | New `sendEmailViaDomain` GraphQL mutation + DTOs. | | Cleanup | `EmailingDomainWorkspaceCleanupJob` wired into `WorkspaceService.deleteWorkspace` — tears down SES tenant association + identity on workspace delete. | | Settings UI | Rewritten around reusable `SettingsTableListSection`. "Email Group" → "Email Handle" rename. New cells for status/source/forwarding. Outbound domains surfaced on workspace settings page. | ### Env vars (new) All in `config-variables.ts`, group `AWS_SES_SETTINGS`, all optional: - `AWS_SES_REGION` — `@IsAWSRegion`, consumed by `AwsSesClientProvider` + driver factory - `AWS_SES_ACCOUNT_ID` — used for ARN construction in driver factory - `SES_SNS_TOPIC_ARN_ALLOWLIST` — **shared** by inbound + outbound webhook routers, comma-separated list of accepted SNS topic ARNs (verified via `sns-payload-validator`) ### Migrations - `1778862608620-add-emailing-domain-tenant-status` (fast) — adds `tenantStatus` column. - `1778865501791-unique-emailing-domain-globally` (slow, idempotent) — enforces global uniqueness on `domain`. - Instance commands bumped to `2.5`. ### Infra dependency Two coupled twenty-infra PRs: - `ses-inbound-email` — receipt-rule + inbound SNS topic + S3 bucket policy + KMS grant + `email_group_*` outputs. - `ses-outbound-tf` — EventBridge rule + outbound SNS topic + SES IAM policy + outbound `webhook_url` subscription. **Based on `ses-inbound-email`.** Merge order: inbound first, then outbound. Outbound PR's chart edit owns the comma-joined `SES_SNS_TOPIC_ARN_ALLOWLIST` value (both ARNs). Features lives under `/settings/general` <img width="1496" height="845" alt="SCR-20260519-ofhi-2" src="https://github.com/user-attachments/assets/a025485a-09f7-4131-91cd-0067690ff18d" /> --------- Co-authored-by: Félix Malfait <felix.malfait@gmail.com> Co-authored-by: claude[bot] <41898282+claude[bot]@users.noreply.github.com> Co-authored-by: Félix Malfait <FelixMalfait@users.noreply.github.com> |
||
|
martmull
|
fbae66de8a |
Fix error when token invalid (#20972)
## Before <img width="914" height="519" alt="image" src="https://github.com/user-attachments/assets/8933bf9e-d8db-4670-ad08-69e900083fc1" /> ## After <img width="1105" height="523" alt="image" src="https://github.com/user-attachments/assets/d8638dbb-adee-4b7d-9d29-1a2a0185ab4f" /> |
||
|
nitin
|
863f3f29a2 |
Fix page layout widget tab moves (#20915)
Fixes widget moves between page layout tabs by making pageLayoutTabId part of the flat-entity diff, so the save mutation no longer silently drops the new tab assignment. https://discord.com/channels/1130383047699738754/1508737039128985680 |
||
|
Félix Malfait
|
8cb88cabee |
fix(role): rebind API keys + agents before deleting their role (#20935)
## Customer-reported bug
A customer hit this when using the AI chat:
```json
{
"message": "API key 760d4822-da40-4b3f-9031-40563d7ed6c9 has no role assigned",
"extensions": {
"code": "INTERNAL_SERVER_ERROR",
"userFriendlyMessage": "This API key has no role assigned."
}
}
```
Their integration authenticates via API key. Somewhere along the way,
the role bound to that API key was deleted, leaving the API key
authenticated but role-less. Any request that hits a permission check
(`getRoleIdForApiKeyId`) blows up.
## Root cause
In `RoleService.deleteManyRoles`, the pre-deletion cleanup
(`assignDefaultRoleToMembersWithRoleToDelete`) only rebinds **user
workspaces** to the workspace default role. API keys and agents pointing
at the role are ignored. Because `RoleTargetEntity.role` declares
`onDelete: 'CASCADE'`, the FK then drops the role_target rows for those
API keys / agents — but the API keys themselves stay in `api_key`, now
orphaned in `apiKeyRoleMap`.
A previous read-side workaround
([2767ddac44](https://github.com/twentyhq/twenty/commit/2767ddac44) —
make the `role` ResolveField nullable) handled the API-key-details page,
but did not address the write paths (`getRoleIdForApiKeyId`).
## Fix
- Rename `assignDefaultRoleToMembersWithRoleToDelete` →
`rebindTargetsOfRoleToDeleteToDefaultRole` and extend it to rebind API
keys (via `ApiKeyRoleService.assignRoleToApiKey`) and agents (via
`AiAgentRoleService.assignRoleToAgent`) in the same step, before the
role is deleted.
- If the workspace default role doesn't satisfy `canBeAssignedToApiKeys`
/ `canBeAssignedToAgents`, the inner `assignRoleTo*` validation throws.
We catch that and rethrow as a `PermissionsException` with a
role-deletion-context message and two new codes —
`ROLE_CANNOT_BE_ASSIGNED_TO_API_KEYS` /
`ROLE_CANNOT_BE_ASSIGNED_TO_AGENTS` — so the admin sees a clear
"reassign these first" prompt rather than a confusing inner error.
## Scope / non-goals
- **Already-orphaned API keys are not auto-healed.** The customer still
needs to reassign a role to their existing orphan API key via the UI
(Settings > API Keys > [the key] > role). A separate cleanup command for
existing orphans is a follow-up.
- I did not investigate *why* the customer's session was authenticated
via API key in the AI chat — that may be their integration setup. Worth
confirming with them separately.
## Test plan
- [ ] Workspace with default role `Admin` (which has
`canBeAssignedToApiKeys: true`): create an API key with a custom role,
delete the custom role → API key is rebound to Admin, requests keep
working.
- [ ] Workspace with default role `Member` (default, has
`canBeAssignedToApiKeys: false`): create an API key with a custom role,
delete the custom role → role deletion fails with the new
`ROLE_CANNOT_BE_ASSIGNED_TO_API_KEYS` error explaining the admin must
reassign first. API key + custom role are both unchanged.
- [ ] Same two scenarios for agents (`canBeAssignedToAgents`).
- [ ] Existing user-workspace rebind behavior is unchanged.
- [ ] Role deletion with no dependent API keys / agents still works.
|
||
|
Félix Malfait
|
ac89d2ff56 |
feat: raise FILES field max number of values from 10 to 60 (#20950)
## Summary Raises the artificial hardcoded ceiling on `maxNumberOfValues` for custom FILES fields from `10` to `60` so users can attach more files per record. - Bumped `FILES_FIELD_MAX_NUMBER_OF_VALUES` constant in `twenty-shared` from `10` to `60` - Updated validator unit test (inline snapshots + "exceeds max" case) - Updated create/update files-field metadata integration tests and Jest snapshots The frontend Zod schema only enforces a `min`, so no frontend changes are required — the backend constant is the single source of truth for the upper bound. Refs #20942 🤖 Generated with [Claude Code](https://claude.ai/code) Co-authored-by: claude[bot] <41898282+claude[bot]@users.noreply.github.com> |
||
|
Etienne
|
5eb79e7797 |
fix(billing) - fix orphaned stripe subs 2/2 (#20916)
Fixes https://sonarly.com/issue/40688 Should have been included in https://github.com/twentyhq/twenty/commit/0edd8d400c646cd6a40ff0fea5342a0f61645d5e |
||
|
Paul Rastoin
|
b8de469f37 |
Refactor and centralize file mimeType integrity check and sanitization (#20889)
# Introduction closes https://github.com/twentyhq/private-issues/issues/484 This PR refactors the writeFile API to never expect to be passed a mimetype, its extract is done programmatically low level so any callers will pass through Same for the file sanitization ## IANA override Disclaimer for consistency we existing behavior we wanted to always have `application/typescript` - should we rather consider fallbacking to octect-steam instead ? - Any pulbic assets that has .ts will now also fallback to `application/typescript` instead of the official IANA ## Integration Added coverage |
||
|
Weiko
|
08682fb3f5 |
Various fixes - some AI findings (#20921)
App permissions tab: - The fallback uuidv4() for a marketplace field was generated twice, so id and universalIdentifier could diverge; it's now computed once and reused as it seemed to be the intention (even though I don't really think it's a good idea) - Renamed buildobjectMetadataItemsFromMarketplaceApp → buildObjectMetadataItemsFromMarketplaceApp to follow camelCase. Morph relation validation: - Fixed the user-facing message "At least one relation is require" → "...is required" - Typos in the related test descriptions (Morh → Morph, samefield → same field) and their snapshots. Docs - The UUID field-type row in views.mdx only listed IS; updated to the full set supported by FILTER_OPERANDS_MAP (IS, IS_NOT, IS_EMPTY, IS_NOT_EMPTY). |
||
|
Paul Rastoin
|
076c05cbd0 |
File service uniformize not found behavior and stream management (#20891)
# Introduction closes https://github.com/twentyhq/private-issues/issues/485 |
||
|
Weiko
|
90f711361c |
Add definePermissionFlag for app-defined permission flags (#20887)
## Context
Adds the SDK plumbing for apps to declare custom permission flags and
the server-side manifest pipeline to persist them.
```typescript
import { definePermissionFlag } from 'twenty-sdk/define';
export const MANAGE_INVOICES_PERMISSION_FLAG_UNIVERSAL_IDENTIFIER = '…';
export default definePermissionFlag({
universalIdentifier: MANAGE_INVOICES_PERMISSION_FLAG_UNIVERSAL_IDENTIFIER,
key: 'MANAGE_INVOICES',
label: 'Manage Invoices',
description: 'Create, edit, and delete invoices',
icon: 'IconReceipt',
});
```
```typescript
import { defineApplicationRole, SystemPermissionFlag } from 'twenty-sdk/define';
import { MANAGE_INVOICES_PERMISSION_FLAG_UNIVERSAL_IDENTIFIER } from './permission-flags/manage-invoices';
export default defineApplicationRole({
universalIdentifier: DEFAULT_ROLE_UNIVERSAL_IDENTIFIER,
label: `${APP_DISPLAY_NAME} default function role`,
// ...
permissionFlagUniversalIdentifiers: [
SystemPermissionFlag.UPLOAD_FILE,
MANAGE_INVOICES_PERMISSION_FLAG_UNIVERSAL_IDENTIFIER,
],
});
```
The flag can then be referenced by UUID in a role's
permissionFlagUniversalIdentifiers. On sync, the catalog row lands in
core.permissionFlag and the link in core.rolePermissionFlag.
## Not in this PR
- Runtime permission checks.
PermissionsService.getUserWorkspacePermissions still builds its result
from Object.values(PermissionFlagType), so custom flags are stored but
not yet enforced, code asking "does this role have MANAGE_INVOICES?"
won't get a meaningful answer. Widening PermissionsService and
UserWorkspacePermissions.permissionFlags to support arbitrary flag keys
is the next PR.
- PermissionFlag from apps can only define "tool" permissions and not
"settings" as a permissionType, this parameter is not mutable. This is
because "settings" are for settings page (until we might decide to
separate both type of permissions into 2 different entities) and apps
can't declare settings page or interact with them so this parameter
would be unnecessary.
|
||
|
Paul Rastoin
|
69d89f8cfc |
Early return in public assets (#20881)
# Introduction Related https://github.com/twentyhq/twenty/issues/20879 More abstracted response error and cleaner integrity check before performing any in database search Nothing critical patched here Also added integration coverage to the related endpoint Fixed the stream on error throw that would have been bubbling up into node process ## Next Once this has been approved will re-apply to all the existing prone file.getBy* methods and controllers endpoints |
||
|
Paul Rastoin
|
b8b115f4e3 |
FileStorageService Dedicated file and folder code flow + integrity check (#20831)
# Introduction
Next handling mimetype integrity check and checksum integrity check for
s3 storage type
Always expecting a trailing end slash when deleting a folder etc
## Application
Uninstalling an application now deletes all its related files
## File storage service
Making a distincton between folder path and file path
## Validation Pipeline
Every file operation in `FileStorageService.buildOnStoragePath` runs
through `validateResourcePath`, which chains three validators in order:
**1. `validateSafeRelativePath`** -- rejects path traversal attacks
| Input | Result | Error |
|---|---|---|
| `../../../etc/passwd` | Rejected | `Resource path must not contain
path traversal (..)` |
| `/etc/passwd` | Rejected | `Resource path must be relative, not
absolute` |
| `file\0.txt` | Rejected | `Resource path contains null bytes` |
| `..\\..\\etc\\passwd` | Rejected | `Resource path must not contain
backslashes` |
| _(empty)_ | Rejected | `Resource path must not be empty` |
**2. `validateFilenameIntegrity`** -- enforces safe characters, length
limits, extension required
| Input | Result | Error |
|---|---|---|
| `my folder/file.mjs` | Rejected | `A path segment contains invalid
characters...` |
| `Makefile` | Rejected | `Filename must have an extension` |
| `aaa...(256 chars).mjs` | Rejected | `A path segment exceeds the
maximum length of 255 characters` |
| `a/b/.../file.mjs` (1025+ chars) | Rejected | `Resource path exceeds
maximum length of 1024 characters` |
| `src/handlers/index.mjs` | Accepted | -- |
| `my-app/my_file.tsx` | Accepted | -- |
| `v1.0/module.config.mjs` | Accepted | -- |
Allowed characters per segment: `a-z`, `A-Z`, `0-9`, `.`, `-`, `_`
**3. `validateResourceExtension`** -- checks extension against the
`FileFolder` allowlist
| Input | FileFolder | Result | Error |
|---|---|---|---|
| `handler.js` | `BuiltLogicFunction` | Rejected | `Invalid file
extension. Allowed extensions: .mjs` |
| `card.tsx` | `BuiltFrontComponent` | Rejected | `Invalid file
extension. Allowed extensions: .mjs` |
| `script.js` | `PublicAsset` | Rejected | `Invalid file extension.
Allowed extensions: .png, .jpg, ...` |
| `index.mjs` | `BuiltLogicFunction` | Accepted | -- |
| `app.tsx` | `Source` | Accepted | -- |
| `photo.png` | `CorePicture` | Accepted | -- (unconfigured folder,
passes through) |
## Consumers
- **`FileStorageService`** -- calls `validateResourcePath`, throws
`FileStorageException` on failure (last-resort defense)
- **Resolver (`uploadApplicationFile`)** -- calls
`validateResourcePath`, throws `ApplicationException` on failure
(user-facing)
- **Flat validators** -- call `validateResourcePath`, push the error to
`validationResult.errors` (non-throwing, collects all errors)
All error messages are translated via Lingui `t` and returned in a
discriminated union `{ isValid: true } | { isValid: false, error: string
}`, letting each consumer decide how to handle failures.
|
||
|
Weiko
|
3bda05ea57 |
[Breaking change] Prepare non-system permission flags (#20847)
# Summary Replaces the enum-keyed `permissionFlags: PermissionFlag[]` on roles with `permissionFlagUniversalIdentifiers: string[]` This unlocks mixing system flags (`SystemPermissionFlag.*`) with app-defined flags in a role config. This is a breaking change. Existing app source must switch to the new field. # Breaking changes - `RoleManifest.permissionFlags` removed. Use `RoleManifest.permissionFlagUniversalIdentifiers: string[]`. - `RoleConfig.permissionFlags` removed (was `PermissionFlagType[]`). Use `RoleConfig.permissionFlagUniversalIdentifiers: string[]`. - `PermissionFlagManifest` type removed from `twenty-shared/application`. - `PermissionFlag` re-export removed from `twenty-sdk/define`. `SystemPermissionFlag` is re-exported in its place. - Retargeting a permission flag between roles is now classified as delete + create instead of update ### Not in this PR - definePermissionFlag SDK function and top-level Manifest.permissionFlags catalog (apps defining their own custom flags). Until those land, permissionFlagUniversalIdentifiers only accepts SystemPermissionFlag.* UUIDs; arbitrary UUIDs fail validation. |
||
|
Paul Rastoin
|
76e144e85a |
Deprecate messageChannel messageFolder calendarChannel standard objects (#20836)
# Introduction Removing old standard objects `messageChannel` and `messageFolder` and `calendarChannel` --------- Co-authored-by: Cursor <cursoragent@cursor.com> |
||
|
Félix Malfait
|
084fa8eaba |
fix(server): auto-index target<X>Id join columns on polymorphic standard objects (#20820)
Closes #20726 ## The bug `timelineActivity` (and the three other polymorphic standard objects — `attachment`, `noteTarget`, `taskTarget`) store relations as N nullable `target<X>Id` columns, one per related object. Each one is a join key queried as `WHERE target<X>Id IN (...) AND deletedAt IS NULL`. For **built-in** related objects (Person, Company, Opportunity, …), each `target<X>Id` column gets a BTREE index, declared statically in `compute-{timelineActivity,attachment,noteTarget,taskTarget}-standard-flat-index-metadata.util.ts`. For **custom** related objects, the same `target<CustomObject>Id` column was added — **without an index**. On a `timelineActivity` table at issue-reporter scale (~21.9M rows, 7.1 GB), this turned record loads into 20–40s sequential scans and produced `QueryFailedError: Query read timeout` for end users. ## Diagnosis The morph/relation field generator (`generateMorphOrRelationFlatFieldMetadataPair`) already creates a BTREE index for the field that owns the join column and returns it alongside the field metadata pair. The two user-driven entry points (`fromRelationCreateFieldInput…`, `fromMorphRelationCreateFieldInput…`) correctly destructure and propagate that index. But the **custom-object creation path** — `buildDefaultRelationFlatFieldMetadatasForCustomObject`, called when a user creates a new custom object — destructured only `{ flatFieldMetadatas }` and threw away `indexMetadatas`. So every `target<CustomObject>Id` column added to the four polymorphic standard objects has been shipping unindexed since custom morph relations went in. ## The fix Three commits. ### 1. `fix(server): index target<CustomObject>Id columns on standard polymorphic objects` 13 lines across 2 files. - `build-default-relation-flat-field-metadatas-for-custom-object.util.ts` — also destructure `indexMetadatas` from the pair generator and accumulate them into the returned record (new field `standardTargetFlatIndexMetadatas`). - `from-create-object-input-to-flat-object-metadata-and-flat-field-metadatas-to-create.util.ts` — append the accumulated indexes to `flatIndexMetadataToCreate`. The migration pipeline at `object-metadata.service.ts:559–562` already passes `flatIndexMetadataToCreate` to the migration runner, so no further wiring is needed. From now on, creating a custom object also creates the four BTREE indexes — one per polymorphic standard object's new `target<CustomObject>Id` column — atomically with the rest of the migration. ### 2. `feat(server): backfill workspace command for relation join column indexes` For existing workspaces whose custom objects were created before the forward-fix. `upgrade:2-8:backfill-relation-join-column-indexes` is a `@RegisteredWorkspaceCommand('2.8.0', 1798100000000)` matching the pattern from `2-7-workspace-command-…-drop-connected-account-standard-object.command.ts`. Per workspace: 1. Load `flatObjectMetadataMaps`, `flatFieldMetadataMaps`, `flatIndexMaps` from the workspace cache. 2. Resolve the four polymorphic standard object IDs by `nameSingular` against `DEFAULT_RELATIONS_OBJECTS_STANDARD_IDS`. 3. Collect every field ID that's already covered by any existing index. 4. Filter `flatFieldMetadataMaps` to MORPH_RELATION fields on those four objects whose `settings.relationType === MANY_TO_ONE` (i.e. owns a join column) and whose ID isn't in the indexed set. 5. Generate a BTREE `UniversalFlatIndexMetadata` for each via `generateIndexForFlatFieldMetadata` (same helper the forward-fix uses). 6. Create the indexes in the workspace schema with **CONCURRENTLY** (see commit 3). 7. Submit the metadata through `WorkspaceMigrationValidateBuildAndRunService` so it lands in `indexMetadata` and the cache — same pipeline as a normal metadata change. The pipeline's own `CREATE INDEX IF NOT EXISTS` no-ops because the index already exists. Properties: - **Idempotent.** Re-running is a no-op once indexes exist. - **Scoped.** Only the four polymorphic standard objects, only their MANY_TO_ONE morph relation fields, only those with no covering index. - **Same code path as the forward-fix.** The backfill produces exactly the indexes the forward-fix would have created at custom-object creation time. - **`--dry-run` supported** via the base `ActiveOrSuspendedWorkspaceCommandRunner`. ### 3. `feat(server): create index CONCURRENTLY in relation join column backfill` Adds an opt-in `concurrently` flag to `WorkspaceSchemaIndexManagerService.createIndex` (threaded through `createIndexInWorkspaceSchema`). When `true`, emits `CREATE INDEX CONCURRENTLY IF NOT EXISTS …`. Defaults to `false` — every existing caller keeps the current transactional `CREATE INDEX` behavior. The backfill command opts in. It creates a QueryRunner **without** `startTransaction()`, issues the CONCURRENTLY indexes one-by-one (each waits for the previous to finish), then submits the metadata through the normal migration pipeline whose own `CREATE INDEX IF NOT EXISTS` is now a no-op. Why not flip the default for the helper: - `CREATE INDEX CONCURRENTLY` cannot run inside a transaction — Postgres errors out. The migration pipeline calls `createIndex` from inside a transactional schema migration. - CONCURRENTLY doesn't roll back with the transaction. If the surrounding migration fails, the index remains and you end up with metadata/schema drift. - Failed CONCURRENTLY builds leave an INVALID index behind that needs manual `DROP`. - UNIQUE indexes have different failure semantics under CONCURRENTLY (deferred, not immediate). So CONCURRENTLY is opt-in, used only where it's the right tool (post-hoc backfills on populated tables). ## Decisions / tradeoffs - **Single-column BTREE vs partial `WHERE deletedAt IS NULL` vs composite.** Twenty's queries always include `deletedAt IS NULL`. A partial index would be slightly better than a plain BTREE (smaller, no wasted seeks on soft-deleted rows). This PR ships single-column to match the existing built-in target index pattern, which already covers >95% of the available speedup (the 20s→4ms drop the reporter saw comes from having any index — composite/partial is a second-order effect). Switching all relation indexes to partial is a separate, broader change. - **CONCURRENTLY operator caveat.** If a CONCURRENTLY build is interrupted (kill, connection drop, OOM), Postgres leaves the index as INVALID. We deliberately don't probe `pg_index` for invalid leftovers on every create — catalog-table queries can be slow at multi-tenant scale and the failure mode is rare. Recovery is manual: `DROP INDEX <name>` and re-run the backfill. - **Forward-fix is not gated** behind a feature flag. The change is metadata-pipeline-internal; before, custom-object creation silently produced a degraded state. After, it produces the correct state. No new public API, no behavioural change for end users besides the indexes existing. ## Risk - Forward-fix: changes only the metadata produced during custom-object creation. New objects get four extra `FlatIndexMetadata` rows and four extra `CREATE INDEX` statements during their creation migration. Tables are empty at that point so the index builds in microseconds. - Helper change: API-compatible, default behavior unchanged. The new `concurrently` parameter is optional. - Backfill: read-only state probe → CONCURRENTLY index creation (no write blocking) → metadata insert via the normal migration pipeline. Idempotent. Reverting is `DROP INDEX`. ## Test plan - [ ] Verify forward-fix: create a custom object, confirm four new BTREE indexes appear on `timelineActivity`, `attachment`, `noteTarget`, `taskTarget` for the new `target<CustomObject>Id` columns, and that `flatIndexMaps` has matching entries. - [ ] Verify backfill on a workspace that had custom objects created before the fix: run `--dry-run` first, confirm the expected indexes are listed; then run for real, confirm the indexes appear in pg (and as `indisvalid = true` in `pg_index`) and in `flatIndexMaps`. Re-run; confirm no-op. - [ ] Verify backfill on a clean workspace: should log "no missing indexes" and exit. - [ ] Verify CONCURRENTLY behavior under load: run backfill against a workspace with active writes on `timelineActivity`; confirm inserts/updates keep working during index build (no `ShareLock` waits in `pg_stat_activity`). - [ ] On the affected reporter-scale workspace, confirm `EXPLAIN ANALYZE` switches from sequential scan to index scan and timeline activity timeouts go away. |
||
|
nitin
|
068d365731 |
feat(sdk): error on incompatible view filter operand at sync time (#20763)
view filters with mismatched operand + field type now error at sync -- was silently failing before |
||
|
neo773
|
323e66433e |
lint: migrate prettier to oxfmt (#20783)
Most changes are `implements` being unwrapped this is not a oxfmt regression Prettier in 3.7 (we're on 3.1) changed this behaviour prettier blog [post](https://prettier.io/blog/2025/11/27/3.7.0#change-18094) This unifies our linting tooling --------- Co-authored-by: github-actions <github-actions@twenty.com> Co-authored-by: Charles Bochet <charles@twenty.com> |
||
|
martmull
|
9a7f50fcb7 |
Fix breaking change in install app command (#20825)
add backward compatibility for twenty-sdk install command |
||
|
Charles Bochet
|
edf2bfbf76 |
feat(server): rotate connectedAccount.connectionParameters via secret-encryption:rotate (#20807)
## Summary Adds a new `connected-account-connection-parameters` site to the `secret-encryption:rotate` CLI introduced in #20613, so the nested password envelopes inside `connectedAccount.connectionParameters` (IMAP / SMTP / CALDAV — encrypted at-rest in #20673) are re-encrypted under the current `ENCRYPTION_KEY` alongside every other at-rest secret site. Without this, rotating `ENCRYPTION_KEY` on a 2.7+ instance would silently leave IMAP / SMTP / CalDav passwords on the old key id. ### Why a new handler A dedicated handler is required (rather than reusing `ColumnRotationSiteHandler`) because the envelope lives at `connectionParameters->'<PROTOCOL>'->>'password'`, not in the whole column, and up to three independent envelopes may need rotating per row. The handler: - Uses the same cursor-based, idempotent, online pattern as the existing handlers, with a SQL predicate that skips rows where every non-null protocol password is already on the current key id. - Threads \`workspaceId\` into HKDF, matching how \`EncryptConnectionParametersSlowInstanceCommand\` backfilled. - Rebuilds only the protocols whose passwords are not yet current, so a partial mid-row failure cannot cause unnecessary re-encryption on resume. - Guards the UPDATE with jsonb-level deep equality (\`IS NOT DISTINCT FROM CAST(:json AS jsonb)\`) so optimistic concurrency is unaffected by Postgres's internal jsonb key ordering vs. JSON.stringify ordering. - Refuses to rotate plaintext passwords (counted as \`errors\`) — operators must finish the 2.7 slow instance command (\`EncryptConnectionParametersSlowInstanceCommand\`) before running rotation. ### Sites covered (now) | Site | Location | Scope | | --- | --- | --- | | \`connected-account-access-token\` | \`connectedAccount.accessToken\` | workspace | | \`connected-account-refresh-token\` | \`connectedAccount.refreshToken\` | workspace | | **\`connected-account-connection-parameters\`** (new) | \`connectedAccount.connectionParameters.{IMAP,SMTP,CALDAV}.password\` | workspace | | \`application-variable\` | \`applicationVariable.value\` (isSecret) | workspace | | \`application-registration-variable\` | \`applicationRegistrationVariable.encryptedValue\` | instance | | \`signing-key-private-key\` | \`signingKey.privateKey\` | instance | | \`totp-secret\` | \`twoFactorAuthenticationMethod.secret\` | workspace | | \`sensitive-config-storage\` | \`keyValuePair.value\` (sensitive STRING configs) | instance | |
||
|
Paul Rastoin
|
570c57563a |
Upload application file resolver exception management and integration coverage (#20803)
# Introduction Earlier and better exception handling of the upload application file resolver + coverage |
||
|
martmull
|
2963fa9324 |
Navigate to installed page after app install (#20797)
as title |
||
|
Paul Rastoin
|
a3c92311e3 |
Application file storage service (#20793)
# Introduction Fix unsafe resource path join with expected prefix at file storage directly Add early paths transversal detections in metadata validators |
||
|
Félix Malfait
|
923c3beead |
fix(auth): clarify error when joining a non-active workspace (#20769)
## Summary When an existing user accepts an invite into a workspace whose `activationStatus` is not `ACTIVE` (e.g. `SUSPENDED`, `INACTIVE`, `PENDING_CREATION`), the throw in `throwIfWorkspaceIsNotReadyForSignInUp` returns: > User is not part of the workspace The message describes the symptom (they aren't a member yet) instead of the cause (the workspace can't accept new members), which makes invitees assume their invite is broken when the real issue is the target workspace's state. The sibling branch a few lines above — for brand-new users hitting the same non-ACTIVE workspace — already returns `"Workspace is not ready to welcome new members"`. This PR reuses the same message in the existing-user branch so both paths give a consistent, accurate explanation. Single file, two string literals. ## Test plan - [ ] Sign in via Google with an existing Twenty account, accepting an invite to a `SUSPENDED` workspace → confirm the new message is shown instead of "User is not part of the workspace". - [ ] Confirm the happy path (sign-in to an `ACTIVE` workspace via invite) is unchanged — early-return on `ACTIVE` is untouched. --------- Co-authored-by: Charles Bochet <charles@twenty.com> |
||
|
Charles Bochet
|
9988f98577 |
feat(server): idempotent CLI to rotate ENCRYPTION_KEY across enc:v2 rows (#20613)
## Summary
Adds the \`secret-encryption:rotate\` CLI command, which re-encrypts
every at-rest secret stored in an \`enc:v2:\` envelope under the current
\`ENCRYPTION_KEY\`. The command is **online** and **resumable**: a SQL
filter skips rows already on the current keyId, so interrupting it
(Ctrl-C, container restart, …) and re-running picks up where it left off
without re-rotating earlier rows.
### Sites covered (one handler each)
| Site | Table.column | Scope |
| --- | --- | --- |
| \`connected-account-tokens\` | \`connectedAccount.{accessToken,
refreshToken}\` | workspace |
| \`application-variable\` | \`applicationVariable.value\` (isSecret
only) | workspace |
| \`application-registration-variable\` |
\`applicationRegistrationVariable.encryptedValue\` | instance |
| \`signing-key-private-keys\` | \`signingKey.privateKey\` | instance |
| \`sensitive-config-storage\` | \`keyValuePair.value\` (isSensitive +
STRING configs) | instance |
| \`totp-secrets\` | \`twoFactorAuthenticationMethod.secret\` |
workspace |
Each handler:
- Filters at SQL level on \`value LIKE 'enc:v2:%' AND value NOT LIKE
'enc:v2:<primaryKeyId>:%'\` to enforce idempotency without re-decrypting
already-rotated rows.
- Uses cursor-based batching (default **200**, capped **5000**).
- Threads \`workspaceId\` into HKDF for workspace-scoped sites; runs
instance-scoped for the rest.
### CLI flags
| Flag | Description |
| --- | --- |
| \`-s, --site <site>\` | Limit to a single site. |
| \`-b, --batch-size <n>\` | Override per-batch row count. |
| \`-d, --dry-run\` | Decrypt + re-encrypt in memory, skip the
\`UPDATE\`. |
The runner logs progress via Nest \`Logger\` (per-site start,
completion, final summary) and exits non-zero when any site reports
\`errors > 0\`. \`FALLBACK_ENCRYPTION_KEY\` must be set to the previous
\`ENCRYPTION_KEY\` during rotation; the runner warns when it is unset.
Operator documentation lives in #20611 (docs PR).
|
||
|
Paul Rastoin
|
3d49c17e34 |
[CONNECTED_ACCOUNT_BREAKING_CHANGE] Unify connected account permissions (#20732)
# Introduction This PR is a followup of https://github.com/twentyhq/twenty/pull/20673 It aims to unify the authentication/permissions layer with all the connectedAccount interactions across the application ## Deprecate - findAll - findById ## Email sync An user can only sync the message of his own connected account ## Workflow email - Related https://github.com/twentyhq/private-issues/issues/478 - Only reauthorize owned account |
||
|
Paul Rastoin
|
57f13c9b92 |
[CONNECTED_ACCOUNT_BREAKING_CHANGE] Encrypt ConnectedAccount connectionParameters (#20673)
# Introduction Prevent any cross user `connectedAccount` `connectionParamaters` leak Also encrypt in db all `connectionParameters` password Never return any password through `DTO` anymore The settings now allow update mutation without providing the password in edition mode Verified all `connectionParameters.password` interaction ## Integration tests - Added more coverage for both failing and successful paths - Introduced a new env var that allow bypass the provider connection test ## Legacy connected Account decryption support Stop allowing non encrypted decryption on `accessToken` and `refreshToken`, only allow legacy decryption on refactored `connectionParameters` ## Upsert ownership Completely got rid of the connected workspace schema context which is legacy Also now a user can only upsert a connected account for him only.. ## New UI <img width="1770" height="1852" alt="image" src="https://github.com/user-attachments/assets/55c1dc89-42ff-4084-95e2-cc5f9e23753b" /> If in edition the password is by default disabled It needs to be selected as being edited to be enabled ## Next - Refactor tool permissions flag not to include connected accounts - Remove the legacy connected standard object - Refactor and improve connected account resolver auth |
||
|
Charles Bochet
|
72ce77864e |
feat(server): Enterprise cron that rotates the current JWT signing key (#20612)
## Summary Adds a daily Enterprise-only cron that rotates the current ES256 JWT signing key once it has been current for `SIGNING_KEY_ROTATION_DAYS`. Manual rotation from the admin panel is unaffected. ### Behaviour - `SIGNING_KEY_ROTATION_DAYS` is **opt-in**: when unset, the cron is a no-op. - Rotation flips `isCurrent` and clears the previous key's `privateKey` in the same transaction, then inserts the new `isCurrent=true` row. - The previous key's row is kept (`revokedAt` stays `null`) so its `publicKey` can keep verifying tokens it signed until they expire; only the encrypted `privateKey` is wiped since it can no longer be used to sign. - **No auto-revocation** — revoking a key remains a manual admin action, reserved for leak / emergency response. - The cron is also a no-op when `EnterprisePlanService.isValid()` is `false`. ### Wiring - `JwtKeyManagerService.rotateCurrent()` - `SigningKeyRotationService.rotateIfDue()` (reads `SIGNING_KEY_ROTATION_DAYS`, skips when unset) - `RotateSigningKeysCronJob` (Enterprise-gated, rethrows on failure) registered in `JwtModule` - `RotateSigningKeysCronCommand` registered with `cron:register:all` - `ROTATE_SIGNING_KEYS_CRON_PATTERN = '15 3 * * *'` (daily, no-op until threshold) Operator documentation lives in #20611 (docs PR). |
||
|
Félix Malfait
|
291ce5ccdb |
fix(filters): make filter dispatcher own relation-target resolution (#20670)
## Summary Two relation-traversal bugs surfaced post-merge of #20533, both rooted in the same architectural smell: the GraphQL filter dispatcher took a flat `fields: FieldShared[]` array and silently dropped any filter whose `relationTargetFieldMetadataId` wasn't in that array. Callers had to remember to pre-augment the list with relation targets — and 16+ call sites did not all know this. This PR fixes both bugs and removes the smell. ### Bug 1 — Save as new view loses the relation target `useCreateViewFromCurrentView` built the create-filter input without `relationTargetFieldMetadataId`. The saved view's filter persisted without the traversal — on reload the chip showed "Company contains 'air'" instead of "Company → Name contains 'air'". Discarded at save time, not at read time. Fix: include `relationTargetFieldMetadataId` in the create input. (Commit 1.) ### Bug 2 — Workflow Search Records drops one-hop traversals `FindRecordsWorkflowAction` built its fields list from `flatObjectMetadata.fieldIds` only (source object's fields). The shared dispatcher then couldn't resolve the relation target field on the related object and silently dropped the filter — a configured "People where Company → Name Contains 'Airbnb'" came through as `{ and: [] }`. This was the same shape as bugs already fixed in 5 other call sites (chart filters, view filters, record table, etc.). The pattern was: caller forgets to augment fields → dispatcher silently drops the filter. Fix (commit 2): change the dispatcher to take a `findFieldMetadataItemById: (id) => FieldShared | undefined` resolver callback. Both source-field and relation-target-field lookups go through the same resolver, so callers no longer need to know about the augmentation requirement. Frontend callers pass a workspace-wide resolver built from `flattenedFieldMetadataItemsSelector`; server callers wrap `findFlatEntityByIdInFlatEntityMaps` on `flatFieldMetadataMaps`. In both cases relation-target lookups just work, because the resolver can see fields on related objects. ## Why this matters Before: "if you call the dispatcher, pre-augment your fields list with relation targets, or filters get silently dropped." An invariant only enforceable by code review, broken often enough to ship two user-visible bugs in one week. After: the dispatcher resolves field ids itself. There's no list to forget to augment. The failure mode (filter silently dropped) becomes structurally impossible at the dispatcher boundary. Net diff: 240 insertions, 319 deletions. Removed `augmentFieldsWithRelationTargets` (frontend) and the workflow whack-a-mole code (server). ## Test plan - [ ] Save view: create an advanced filter using a one-hop relation traversal, click "Save as new view", reload, confirm the chip still reads "Source → Target operator value" - [ ] Workflow: configure a Search Records action with a relation-traversal filter, run the workflow, confirm the filter is actually applied - [ ] Dashboard chart: configure a chart with a relation-traversal filter, confirm the chart data respects it - [ ] Record table, group-by, calendar, total count, footer aggregates: all continue to work with both plain and relation-traversal filters |
||
|
Charles Bochet
|
1d3d3999e2 |
feat(server): upgrade-aware entity decorators for cross-version upgrades (#20686)
## What When the same PR introduces a new core entity *and* adds a cache provider that queries it, every workspace step from older versions that runs before the introducing instance step hits `relation … does not exist` — the cause of the failed [v2.6.0 staging-ci run](https://github.com/twentyhq/twenty-infra/actions/runs/26042742000). Same class of failure for renamed core entities and for new FK columns hidden inside relation loads. This PR adds **upgrade-aware entity decorators** + a runtime that adapts TypeORM's view of the schema to the current `core.upgradeMigration` cursor. ## Strategy ``` ┌────────────────────────────────┐ │ @Entity classes (final shape) │ │ + @WasIntroducedInUpgrade │ │ + @WasRenamedInUpgrade │ └───────────────┬────────────────┘ │ UpgradeSequenceRunner.run() ┌─────────────────────┴─────────────────────┐ ▼ ▼ step N+1 begins step N just completed │ │ └────────► adapter.refresh() ◄──────────────┘ │ reads core.upgradeMigration via UpgradeMigrationService.getLastAttemptedInstanceCommand │ ▼ ┌────────────────────────────────────────────────────────┐ │ UpgradeAwareEntityMetadataAdapter │ │ • mutates EntityMetadata.tableName / tablePath │ │ -> historical name for renames not yet applied │ │ • flips column.isSelect = false for not-yet-introduced│ │ columns │ │ • tracks per-entity availability sidecar │ └─────────────────┬──────────────────────────────────────┘ │ ▼ DataSource.getRepository wrapped at TypeOrmModule.forRoot: repo.find() / findOne() / count() / … ┌─────────────────────────────────────────┐ │ wrapRepositoryWithUpgradeAwareProxy │ │ • entity unavailable -> short-circuit │ │ (find -> [], count -> 0, │ │ findOneOrFail -> EntityNotFound) │ │ • write -> Promise.reject( │ │ UpgradeUnavailableEntityWriteEx) │ │ • find({ relations: ['X'] }) with X │ │ unavailable -> X stripped │ └─────────────────────────────────────────┘ ``` The decorator strings reference real `core.upgradeMigration.name` values (`${version}_${className}_${timestamp}`). A boot-time validator walks the actual `UpgradeSequenceReaderService.getUpgradeSequence()` and fails fast on typos. ## Files - New decorators: `engine/core-modules/upgrade/decorators/was-introduced-in-upgrade.decorator.ts`, `was-renamed-in-upgrade.decorator.ts` - Runtime: `engine/twenty-orm/upgrade-aware/` (adapter, proxy, install hook, state singleton, exceptions) - Wired into `UpgradeSequenceRunnerService` (`refresh()` between steps) and `TypeOrmModule.forRoot` (proxy install) - 2-6 entity decorations: `RolePermissionFlagEntity` (rename history + new `permissionFlagId` column), `PermissionFlagEntity` (new catalog) ## Validation End-to-end local cross-version upgrade (v1.22 → HEAD): `28 workspace(s) succeeded, 0 failed`; `upgrade:status → Instance: Up to date, 4 up to date, 0 behind, 0 failed`. Full log excerpts and the second-failure-found-and-fixed (`WorkspaceRolesPermissionsCacheService` relation load) in [this comment](https://github.com/twentyhq/twenty/pull/20686#issuecomment-4480036816). ## Test plan - [x] Adapter spec covers rename mutation; proxy spec covers `find()` short-circuit on unavailable entity. Resolver + validator + decorators are covered by `resolve-entity-shape-at-upgrade-cursor.util.spec.ts` (integration-level via real decorator application). - [x] `nx lint:diff-with-main twenty-server` + `nx typecheck twenty-server` clean - [x] All 82 affected tests passing - [ ] Cross-version-upgrade CI re-runs after this lands; v2.6.0 retag once green ## Follow-ups deferred - v2.7 `connectionProvider` rename repro as a permanent end-to-end test artifact - Extending the proxy to also cover `EntityManager.getRepository` and `createQueryBuilder` if a non-`find()` upgrade-time consumer surfaces |
||
|
nitin
|
db0547f503 |
[1/3] Rename permissionFlag to rolePermissionFlag + add permissionFlag catalog/backfill (#20481)
Split of #20377. ## Summary This PR separates available permission flags from per-role permission flag grants. Previously, `core.permissionFlag` stored the role assignment directly: `roleId + flag`. This PR renames that legacy grant table to `core.rolePermissionFlag`, then recreates `core.permissionFlag` as the catalog of available permission flags. ## What changed - Rename the existing `core.permissionFlag` grant table to `core.rolePermissionFlag`. - Add the new syncable `core.permissionFlag` catalog entity with key, label, description, icon, permission type, relevance flags, and custom/standard metadata. - Add stable `SystemPermissionFlag` universal identifiers for the built-in `PermissionFlagType` values. - Seed the standard permission flags for every workspace under the Twenty standard application. - Backfill existing role grants: - create missing catalog rows for existing grant keys, - add `rolePermissionFlag.permissionFlagId`, - migrate grants from the old string `flag` column to the new catalog FK, - replace the old `(flag, roleId)` uniqueness with `(permissionFlagId, roleId)`. - Rewire role permission flag caches, permission checks, role DTO mapping, and `upsertPermissionFlags` to resolve through the catalog. - Keep the existing public role permission API shape: product/app surfaces still talk about `permissionFlags` and return `{ id, roleId, flag }`. - Update metadata flat-entity machinery, migration builders, validators, action handlers, snapshots, generated schemas, docs, and app fixtures for the new `permissionFlag` / `rolePermissionFlag` split. ## Behavior after this PR - Existing permission flag grants keep working. - Existing GraphQL role permission flows keep the same public naming. - Standard permission flags are represented as catalog rows. - Permission checks now compare grants through catalog universal identifiers instead of the legacy `flag` column. - Workspace deletion cleanup now verifies both `permissionFlag` and `rolePermissionFlag`. ## What is not in this PR - Public GraphQL CRUD for custom permission flags. - App manifest support for declaring new custom permission flags. - Frontend UI for creating or assigning custom permission flags beyond the existing role permission flow. --------- Co-authored-by: Weiko <corentin@twenty.com> |
||
|
Charles Bochet
|
cf4b4455d3 |
fix(server): normalize composite defaultValues in manifest converter (unblock app re-install on 2.5-normalized workspaces) (#20615)
## Context The runtime create-field path and the v2.5 `NormalizeCompositeFieldDefaultsCommand` workspace upgrade both run composite `defaultValue`s through `nullifyEmptyCompositeDefaultValue`. The manifest install/sync path was the only write path that skipped it: [`fromFieldManifestToUniversalFlatFieldMetadata`](https://github.com/twentyhq/twenty/blob/main/packages/twenty-server/src/engine/core-modules/application/application-manifest/converters/from-field-manifest-to-universal-flat-field-metadata.util.ts) passed `fieldManifest.defaultValue` through verbatim. For the SDK-emitted ACTOR system fields (`createdBy` / `updatedBy`), `twenty-sdk` ships `{ name: "''", source: "'MANUAL'" }`. After the runtime or the 2.5 normalize command stores them, the workspace row holds the canonical four-key form `{ context: null, name: null, source: "'MANUAL'", workspaceMemberId: null }`. The next install computes its TO map from the manifest, still gets the raw two-key shape, and diffs it against the normalized FROM. The dispatcher emits a `defaultValue` update on each system actor field; the flat-field-metadata validator rejects it with `FIELD_MUTATION_NOT_ALLOWED`, blocking every re-install of any application that defines a custom object on a v2.5-normalized workspace. ## Fix Normalize composite `defaultValue`s inside the converter, reusing the same `nullifyEmptyCompositeDefaultValue` helper the three other write paths already share: - [`get-default-flat-field-metadata-from-create-field-input.util.ts`](https://github.com/twentyhq/twenty/blob/main/packages/twenty-server/src/engine/metadata-modules/flat-field-metadata/utils/get-default-flat-field-metadata-from-create-field-input.util.ts) — `createOneObject` and `createOneField` GraphQL paths. - [`sanitize-raw-update-field-input.ts`](https://github.com/twentyhq/twenty/blob/main/packages/twenty-server/src/engine/metadata-modules/flat-field-metadata/utils/sanitize-raw-update-field-input.ts) — `updateOneField` GraphQL path. - [`2-5-workspace-command-1778000001000-normalize-composite-field-defaults.command.ts`](https://github.com/twentyhq/twenty/blob/main/packages/twenty-server/src/database/commands/upgrade-version-command/2-5/2-5-workspace-command-1778000001000-normalize-composite-field-defaults.command.ts) — the upgrade backfill that introduced the divergence. After the fix, the four write paths agree on the canonical shape, so re-installs are no-ops on system actor fields regardless of when the 2.5 normalize command ran. Non-composite types pass through unchanged. ## Test New spec `from-field-manifest-to-universal-flat-field-metadata.util.spec.ts` covers: - Empty-name actor defaults are normalized to the four-key canonical shape. - The converter is idempotent: feeding its own output back in produces the same result (so two consecutive syncs of the same manifest never emit a `defaultValue` update). - When the manifest omits `defaultValue`, the converter falls back to `generateDefaultValue` and normalizes the result. - Non-composite defaults pass through unchanged. ``` PASS src/engine/core-modules/application/application-manifest/converters/__tests__/from-field-manifest-to-universal-flat-field-metadata.util.spec.ts fromFieldManifestToUniversalFlatFieldMetadata composite defaultValue normalization ✓ normalizes empty-name actor defaults to the canonical four-key shape ✓ is idempotent: re-running the converter on its own output yields the same defaultValue ✓ falls back to the generated default and normalizes it when defaultValue is omitted ✓ leaves non-composite defaults untouched Tests: 4 passed ``` ## CI gap that let this through The integration suites covering manifest install (`appDevOnce` against the test workspace) never re-installed an existing app on a workspace whose composite fields had already been put through the 2.5 normalize command. They synced once, then ran assertions on the resulting state; the second sync that would have re-triggered the `defaultValue` diff was never exercised. If we want to catch this class of regression at the integration level too, we'd add a test that (1) syncs an app whose manifest includes an ACTOR system field with the raw SDK shape, (2) invokes `NormalizeCompositeFieldDefaultsCommand` directly on the test workspace, (3) re-syncs the same manifest, and (4) asserts no `FIELD_MUTATION_NOT_ALLOWED` errors. The unit-level idempotency check in this PR is the minimal version of that same coverage. Happy to ship that integration spec in a follow-up if it'd help. |
||
|
Félix Malfait
|
c938fbf4d6 |
feat(twenty-front): relation traversal in filter dropdown (stacked) (#20533)
**Stacked on #20527** https://github.com/user-attachments/assets/48995655-401a-4c35-8094-e88da8408bdd ## Summary Surfaces the one-hop relation traversal added in #20527 through the existing **composite sub-field dropdown pattern**. Clicking a MANY_TO_ONE relation field in the "+ Filter" picker now opens the same second-level dropdown that composite fields (FULL_NAME, ADDRESS, CURRENCY, etc.) already use — populated with the target object's filterable fields. Picking one (e.g. `Company → Name`) builds a filter that serializes to the nested GraphQL filter the backend now accepts: `{ company: { name: { ilike: "%X%" } } }`. No new components. The whole feature reuses `AdvancedFilterSubFieldSelectMenu` + the existing `subFieldNameUsedInDropdownComponentState` + the existing `MenuItem hasSubMenu` indicator. Only the conditions that gate the sub-menu (and the sub-menu's content for relations) were broadened. ## What landed | File | Change | |---|---| | `ObjectFilterDropdownFilterSelectMenuItem` | Sub-menu chevron now shows on MANY_TO_ONE relations (`isManyToOneRelationField` util). | | `AdvancedFilterFieldSelectMenu` | Relation clicks open the sub-menu alongside composite clicks. | | `AdvancedFilterSubFieldSelectMenu` | New branch: when the sub-menu type is `'RELATION'`, render the target object's filterable fields via `useFilterableFieldMetadataItems(targetObjectMetadataId)`. Composite logic untouched. | | `objectFilterDropdownSubMenuFieldType` state | Widened to accept a `'RELATION'` sentinel. Role-permissions sub-field menu narrows it back out (it doesn't traverse relations). | | `useSelectFieldUsedInAdvancedFilterDropdown` | New optional `targetFieldMetadataItem` arg. When present, the stored RecordFilter's `type` is the target field's type so the operand picker and value input render the target's operands (`'TEXT'` operators when filtering `company.name`, etc.). | | `turnRecordFilterIntoGqlOperationFilter` (shared) | When the filter targets a `RELATION` field with a `subFieldName`, synthesize a field-metadata for the target, recurse to build the inner filter, then wrap it under the relation field's name → `{ relationName: { targetFieldName: { ...operator } } }`. | `RecordFilter.subFieldName` stays narrowly typed as `CompositeFieldSubFieldName` so the wide downstream consumers (`shouldShowFilterTextInput`, composite handlers in the serializer, etc.) don't change. The relation target field's name is stored through a narrowly-scoped cast at the dropdown's storage point — the serializer checks `filter.type === 'RELATION'` before interpreting it as a target field name, so the cast can't be mis-read by composite-only code paths. ## Test plan - [ ] Open a table view on People, click "+ Filter", click "Company" → sub-menu opens with Company's filterable fields - [ ] Pick "Name" → operand picker shows TEXT operators (Contains, Equals, …) - [ ] Type "Airbnb" → filter applies, table shows people whose company name contains "Airbnb" - [ ] Verify network tab: the GraphQL filter variable is `{ company: { name: { ilike: "%Airbnb%" } } }` - [ ] Same flow with a composite target field (e.g. `Company → annualRecurringRevenue → amountMicros`) — should work end-to-end (backend supports composite-within-relation; #20527 has an integration test covering this) - [ ] Composite fields (FULL_NAME, ADDRESS) still open their normal sub-menu and filter correctly — no regression - [ ] Role-permissions field-select sub-field menu is unaffected (it bails out early on the RELATION sentinel) ## Out of scope - ONE_TO_MANY traversal (no backend support yet) - Aggregates (`people.count > 5`) - Persisting relation-traversal filters into a saved view (ViewFilter has no `relationPath` column yet; that's a separate slice) - REST API DSL changes - AI Tools 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> |
||
|
Charles Bochet
|
75b9b2fe5d |
feat(admin-panel): signing keys management tab with usage tracking (#20586)
## Summary - Adds a new admin-only **Security** tab to the Admin Panel (alongside General/Apps/AI/Config/Health) containing a **Signing Keys** section. The tab is intentionally introduced now so the upcoming **Encryption rotation** work can land as a sibling section. - Lists every JWT signing key with key id, `createdAt`, `revokedAt`, current/active/revoked status, and a **7-day verification count** read from Redis. A trailing row aggregates **legacy HS256** verifications so it is clear when the deprecated path is still in use. - Lets an admin **revoke** a public key. Revoking the current key drops `isCurrent`, sets `revokedAt`, nulls the encrypted `privateKey` and clears the in-process cached current key; the existing lazy path in `JwtKeyManagerService.getCurrentSigningKey()` then mints a fresh current key on the next sign. ## Backend - `SigningKeyVerifyCounterService` — bucketed Redis counter under the existing `EngineMetrics` namespace. 1-day UTC-aligned buckets, 8-day TTL refreshed on every increment, batched read via `mget`. Failures are swallowed and logged at `warn` so a Redis hiccup cannot break auth. - `JwtWrapperService.verifyJwtToken` records verifies **after success** for both ES256 (`kid` as identifier) and HS256 (the literal `legacy` identifier). - `JwtKeyManagerService.listSigningKeys()` and `revokeSigningKey(id)`: list ordered by `isCurrent DESC, createdAt DESC`; revoke is idempotent, validates the UUID, invalidates the public-key cache, and resets the cached current-key promise. - `AdminPanelResolver.getSigningKeys` (query) and `revokeSigningKey` (mutation) are both decorated with `@UseGuards(AdminPanelGuard)` so they are admin-only, like the 35 existing admin-only methods on this resolver. `privateKey` is never returned over GraphQL. ## Frontend - New `SECURITY` tab id wired into `SettingsAdminContent` and `SettingsAdminTabContent` (gated by `canAccessFullAdminPanel`). - `SettingsAdminSecurity` / `SettingsAdminSigningKeysTable` strictly reuse existing admin-panel components: `Section`, `H2Title`, `Table`/`TableRow`/`TableCell`/`TableHeader` from `@/ui/layout/table`, `Tag`/`Button` from `twenty-ui`, and `ConfirmationModal` mirroring the queue retry/delete modals. Only one minimal styled helper for the monospaced UUID rendering. - `useRevokeSigningKey` uses `useApolloAdminClient`, refetches `GetSigningKeys`, shows success/error snackbars (same pattern as `useRetryJobs`/`useDeleteJobs`). <img width="1293" height="881" alt="image" src="https://github.com/user-attachments/assets/7cf98664-950b-4451-af85-27781a8e9a9c" /> |
||
|
Charles Bochet
|
ca1571676c |
fix(server): treat plaintext-under-isSecret rows as plaintext in app variable encryption migration (#20590)
## Summary Prod 2.5 upgrade failed on the slow instance command `EncryptApplicationVariableSlowInstanceCommand`: ``` [Nest] LOG [InstanceCommandRunnerService] 2.5.0_EncryptApplicationVariableSlowInstanceCommand_1798000005000 starting data migration... [Nest] WARN [SecretEncryptionService] Decrypted a legacy unprefixed AES-CTR ciphertext... [Nest] ERROR [InstanceCommandRunnerService] data migration failed TypeError: Invalid initialization vector ``` ### Root cause The migration assumes every row matching `isSecret = true AND value <> '' AND value NOT LIKE 'enc:v2:%'` is legacy AES-CTR ciphertext. In prod we found multiple `isSecret = true` rows whose `value` is plaintext (e.g. `SLACK_HOOK_URL = 'https://hooks.slack.com/services/...'`) — most likely the result of `isSecret` being flipped to true on a row that already held a plaintext value, or a write path that bypassed `ApplicationVariableEntityService.update`. Those values can't decode into the 16-byte IV that AES-CTR needs, so `Buffer.from(value, 'base64')` truncates at the first non-base64 char (`:`), the buffer is < 16 bytes, and `createDecipheriv` throws. ### Fix Follow the same policy as `EncryptConnectedAccountTokensSlowInstanceCommand`: anything that isn't already in the `enc:v2:` envelope is plaintext. Concretely: 1. Try `decryptVersioned` — legacy CTR rows decrypt fine. 2. If it throws (mis-classified plaintext), log a warning naming the row id and fall back to treating `row.value` as plaintext. 3. Encrypt the resulting plaintext into the `enc:v2:` envelope and update the row. In-loop `isSecret` guard is kept (alongside the SQL filter) so non-secret rows are never touched even if the SQL filter is ever loosened. ### Integration test coverage Added one new case alongside the existing ones in `…encrypt-application-variable.integration-spec.ts`: - `treats plaintext-under-isSecret=true as plaintext and re-encrypts as v2` — seeds a row with `isSecret = true` and a URL value (`:` and `/` are not base64, so this is the exact failure shape from prod), runs the migration, and asserts the value is now `enc:v2:...` and decrypts back to the original URL. Existing cases unchanged: legacy CTR happy path, non-secret rows untouched, idempotent across re-runs, `up()` adds the CHECK constraint, `down()` removes it. ### Why this is a 2-5 edit `TWENTY_CURRENT_VERSION` is now 2.6.0, so editing a 2-5 file trips the `server-previous-version-upgrade-mutation-guard` — `ci:allow-previous-version-upgrade-mutation` label is on the PR. `up()` and `down()` are unchanged; only `runDataMigration` is modified. ## Test plan - [ ] Re-deploy 2.5 to prod and confirm `EncryptApplicationVariableSlowInstanceCommand` completes - [ ] Inspect warning log to count rows that went through the plaintext fallback - [ ] Verify resulting secret rows all satisfy `value = '' OR value LIKE 'enc:v2:%'` and the CHECK constraint is in place |
||
|
Félix Malfait
|
af4765effe |
feat(twenty-server): one-hop relation filters in GraphQL API (#20527)
## Summary
Adds support for filtering records by fields on a related MANY_TO_ONE
object via the GraphQL API. Backend only — no frontend, no REST, no
view-filter persistence yet.
```graphql
{
people(filter: { company: { name: { like: "%Airbnb%" } } }) {
edges { node { id } }
}
}
```
### Where the work lands
- **Schema** — `relation-field-metadata-gql-type.generator.ts` now emits
`{relationName}: TargetFilterInput` alongside the existing
`{joinColumnName}: UUIDFilter` for MANY_TO_ONE relations. Mirrors the
order-by generator that already does this for sort. Lazy thunks in
`object-metadata-filter-gql-input-type.generator.ts` handle the cycle
between filter inputs.
- **Arg processor** — `FilterArgProcessorService` no longer hard-rejects
accessing a relation by its name. When the value is a nested object on a
MANY_TO_ONE field, it recurses into the target object's metadata so each
leaf still gets validated and coerced. Depth-capped at 1.
- **Query parser** — new `parseRelationSubFilter` branch in
`graphql-query-filter-field.parser.ts`. When triggered: looks up the
target object metadata, calls `ensureRelationJoin` against the outer
query builder, and recurses via a child
`GraphqlQueryFilterConditionParser` scoped to the target.
`and`/`or`/`not` inside the relation filter keep working because the
child dispatches through the same `parseKeyFilter`.
- **Shared join utility** — `ensureRelationJoin.util.ts` is a single
function that inspects `queryBuilder.expressionMap.joinAttributes` for
the alias before adding a `LEFT JOIN`. Rewired the existing inline
`qb.leftJoin` calls in the order parser and group-by service to use it,
so filter-driven joins no longer collide with sort-driven joins on the
same relation.
### Out of scope (explicit)
- ONE_TO_MANY reverse traversal (needs EXISTS subqueries)
- Aggregates (`company.people.count > 5` — needs HAVING)
- View-filter storage (no `relationPath` column on `ViewFilterEntity`)
- REST DSL changes
- Frontend filter-picker UX
- Nesting deeper than one hop (parser and arg-processor both reject)
### Open question for review
Permissions. The order-by-on-relation code path already lets users sort
People by Company.name without a Company read-permission check, and this
PR matches that behavior for filters — felt wrong to add a stricter gate
only on the filter side. If we want object-permission gating on the
relation target, it should be a follow-up that covers both paths
consistently. The only attack surface today is existence inference via
timing, identical to what sort already exposes.
## Test plan
- [x] `tsc --noEmit` — clean for changed files (5 unrelated pre-existing
errors on main untouched)
- [x] `oxlint --type-aware` + `prettier --check` — 0 errors on all 17
changed/new files
- [x] `jest filter-arg-processor.service.spec` — 229 tests pass (the new
optional `flatObjectMetadataMaps` arg is backwards-compatible)
- [x] Integration test (`filter-by-relation-field.integration-spec.ts`,
6 cases) — needs to be verified against a seeded test DB. Could not
exercise the happy path in my isolated worktree; depth-2 rejection
passed there.
- [ ] EXPLAIN ANALYZE on the integration test query to confirm the FK on
`person.companyId` is indexed for both standard and custom MANY_TO_ONE
relations.
### Integration test cases
1. Filter People by `company.name = "Airbnb"` (exact match)
2. Filter People by `company.name like "%irbnb%"`
3. Non-matching filter returns empty
4. Combined with a scalar filter at root via `and`
5. **Combined with `orderBy` on the same relation** — proves the
join-dedupe works (without `ensureRelationJoin`, TypeORM throws
"duplicate alias")
6. Depth-2 nesting (`company.accountOwner.name`) returns
`INVALID_ARGS_FILTER`
🤖 Generated with [Claude Code](https://claude.com/claude-code)
---------
Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
|
||
|
Charles Bochet
|
a941f6fe01 |
feat(server): migrate TOTP secret encryption to SecretEncryptionService (#20577)
## Summary
Removes the last `APP_SECRET`-derived at-rest encryption site by
migrating `core.twoFactorAuthenticationMethod.secret` from
`SimpleSecretEncryptionUtil` (AES-256-CBC with key derived from
`sha256(APP_SECRET + userId + workspaceId + 'otp-secret' +
'KEY_ENCRYPTION_KEY')`) to the versioned `enc:v2:` envelope
(ENCRYPTION_KEY → HKDF-SHA256 bound to `workspaceId` → AES-256-GCM).
- New `decrypt-legacy-aes-cbc.util.ts` faithfully reproduces the
pre-migration CBC derivation byte-for-byte;
`SecretEncryptionService.decryptVersioned` dispatches to it when callers
pass `legacyAesCbcPurpose`, with a dedicated one-shot WARN log family.
- `TwoFactorAuthenticationService` now uses `encryptVersioned` /
`decryptVersioned` (passing the legacy purpose so existing rows still
decrypt). `SimpleSecretEncryptionUtil` and its spec are deleted;
`TwoFactorAuthenticationModule` imports `SecretEncryptionModule` in
their place.
- `TwoFactorAuthenticationMethodEntity` gets a `@Check` decorator
(`CHK_twoFactorAuthenticationMethod_secret_encrypted`) restricting
`secret` to the `enc:v2:` envelope; the matching 2.5 slow instance
command (`1798000009000-encrypt-totp-secrets`) cursor-paginates `JOIN`ed
`userWorkspace` rows to recover the legacy `userId`, re-encrypts to
`enc:v2`, and applies the CHECK constraint in `up()`.
### Deviation note
The plan suggested wiring a workspace-only legacy derivation directly
into `decryptVersioned`. In practice the production rows are
user-and-workspace-scoped (the legacy purpose is
`\${userId}\${workspaceId}otp-secret`), so a workspace-only derivation
could not recover them. The PR keeps the public `decryptVersioned` API
intact and adds an optional `legacyAesCbcPurpose` so callers that can
reconstruct the legacy context (the 2FA service and the slow command)
opt in.
### Final state of remaining `APP_SECRET` usages
- HS256 JWT verify (read-only, self-retiring once asymmetric migration
completes).
- Express-session cookie signing.
- Approved-access-domain HMAC (signing root, not at-rest).
- Zero-friction fallback in `resolveEncryptionKeysOrThrow`
(intentional).
No production at-rest data is encrypted with `APP_SECRET`-derived keys
anymore.
## Test plan
- [x] `npx jest src/engine/core-modules/secret-encryption
src/engine/core-modules/two-factor-authentication` — 170 unit tests
pass, including new unit tests for the legacy CBC util and the new
`SecretEncryptionService` fallback branch.
- [x] `npx jest --config ./jest-integration.config.ts
test/integration/upgrade/suites/2-5-instance-command-slow-1798000009000-encrypt-totp-secrets.integration-spec.ts`
— 4 integration tests cover legacy-CBC seed → slow command → `enc:v2`
round-trip, idempotency, CHECK constraint enforcement on `up()`, and
rollback via `down()`.
- [x] `npx oxlint --type-aware` and `npx prettier --check` clean on all
touched files.
- [ ] CI on this PR (server validation, tests, lint, typecheck).
|
||
|
Charles Bochet
|
7fa136f305 |
feat(twenty-server): migrate remaining at-rest encryption sites to versioned envelope (#20550)
## Summary Second PR in the encryption key rotation series. The previous PR (#20528) introduced `ENCRYPTION_KEY` + the versioned `enc:v2:<keyId>:<base64>` envelope inside `SecretEncryptionService` and migrated `ConnectedAccountTokenEncryptionService` as the first consumer. This PR routes every remaining at-rest encryption site through the versioned envelope so that `ENCRYPTION_KEY` (and the future `FALLBACK_ENCRYPTION_KEY`) actually covers them. The legacy unprefixed CTR ciphertext remains readable as a fallback during the rollout window — every migrated read site uses `decryptVersioned`, which transparently delegates to the legacy CTR decrypt when it sees an unprefixed payload. ### Service migrations - **`ApplicationVariableEntityService` (#8)** — workspace-scoped. HKDF info is bound to each row's `workspaceId`. A new `decryptAndMaskVersioned` helper lands on `SecretEncryptionService` for the resolver display path. - **`ApplicationRegistrationVariableService` (#7)** + consumers — **instance-scoped**. Registration variables are server-level config readable by every workspace that installs the application, so HKDF info is `instance`. Updated consumers: - `LogicFunctionExecutorService.buildServerVariableEnvMap` - `ConnectionProviderService.getClientCredentials` - **`LogicFunctionExecutorService.buildEnvVar` (#9)** — workspace-scoped. Each variable's `workspaceId` is threaded into `decryptVersioned`, so per-workspace HKDF contexts are honoured at execution time. - **`UpdateApplicationVariableActionHandlerService`** (workspace-migration runner) — threads `workspaceId` through the secret/non-secret toggle. - **`JwtKeyManagerService` (#3)** — instance-scoped. Signing keys are shared across the JWKS. - **`ConfigStorageService` (#6)** — instance-scoped sensitive STRING config variables. ### Slow instance commands (2.5.0) Each migrated site has a paired backfill that re-encrypts existing rows into the v2 envelope before the column is constrained: | timestamp | command | scope | CHECK constraint | |---|---|---|---| | `1798000005000` | encrypt-application-variable | workspaceId | `"isSecret" = false OR value = '' OR value LIKE 'enc:v2:%'` | | `1798000006000` | encrypt-application-registration-variable | instance | `"encryptedValue" = '' OR value LIKE 'enc:v2:%'` | | `1798000007000` | encrypt-signing-key-private-keys | instance | `"privateKey" IS NULL OR value LIKE 'enc:v2:%'` | | `1798000008000` | encrypt-sensitive-config-storage | instance | _none_ — heterogeneous jsonb column | All backfills are idempotent (the SELECT filter skips rows already in v2 form) and run before their respective `up()` adds the CHECK constraint. Every `down()` deliberately stops at dropping the CHECK constraint — they intentionally do not re-introduce plaintext on rollback. ### Tests - Unit specs for each new slow command cover the v2 upgrade path, the idempotency invariant, and the instance vs workspace HKDF scope. - New `JwtKeyManagerService` spec asserts `decryptVersioned`/`encryptVersioned` are called without `workspaceId` (instance scope). - Updated existing specs for `ApplicationVariableEntityService`, `ConfigStorageService`, and `buildEnvVar` to assert the versioned API and the workspace HKDF context plumbing. - New `SecretEncryptionService.decryptAndMaskVersioned` cases in the service spec. - Updated the `applicationRegistrationVariable` integration spec to assert the column now stores `enc:v2:<keyId>:<base64>` instead of raw legacy CTR. ### Out of scope (future PRs) - `PostgresCredentialsService` — bespoke `jwtWrapperService.generateAppSecret`–derived key + `encryptText`/`decryptText` from `auth.util.ts`; deserves its own migration. - `SimpleSecretEncryptionUtil` (TOTP) — entirely different `aes-256-cbc` `iv:enc` format; deserves its own migration. ## Test plan - [x] `npx nx typecheck twenty-server` - [x] `npx nx lint:diff-with-main twenty-server` (oxlint + prettier) - [x] Local jest run for `secret-encryption | connected-account-token | application-variable | application-registration-variable | build-env-var | jwt-key-manager | config-storage | encrypt-application-variable | encrypt-application-registration-variable | encrypt-signing-key | encrypt-sensitive-config-storage` — 17 suites, 106 tests pass. - [x] Local jest run for `upgrade | instance-command` — 12 suites, 86 tests pass. - [ ] CI green - [ ] Manual review of CHECK constraint shapes by a server reviewer (each one matches `enc:v2:%` rather than `enc:v_:%` since none of the migrated columns can legitimately hold `enc:v1:` ciphertext). |
||
|
Félix Malfait
|
e16977f97b |
[breaking: deploy server before front] feat(view-sort): pick sort sub-field inline on the chip (#20445)
## Summary Lets users choose which sub-field of a composite column to sort by — directly from the sort chip — by clicking the sub-field label and picking from a dropdown. Persists per view via a new nullable \`subFieldName\` column on \`ViewSort\`. Replaces #20438, which proposed a field-settings (admin) configuration for the same problem. The chip-level approach is more discoverable (the option lives where the user is looking) and per-view, so different views on the same object can sort by different sub-fields. ### What changes for users - **FullName columns**: previously sorted by \`firstName\` and \`lastName\` together as a stable dual-key sort. Now the user can pick which sub-field is primary (the other is the tie-breaker). Default remains \`firstName\` primary, \`lastName\` tie-breaker. - **Address columns**: previously not sortable at all (not in \`SORTABLE_FIELD_METADATA_TYPES\`). Now sortable, with a chip dropdown listing each enabled sub-field. Default is \`addressCity\` if enabled, else the first enabled sub-field. Disabling a sub-field at the field-metadata level (existing setting) removes it from the dropdown. - **Other composite types** (Currency, Phones, Emails, Links, Actor) and scalar fields keep their existing single-key sort behavior. ### UX ``` ┌─────────────────────────┐ ┌─────────────────────────┐ │ ↑ Name · Last name ✕ │ │ ↑ Address · City ✕ │ └────────┬────────────────┘ └────────┬────────────────┘ ▼ (click sub-field) ▼ ┌────────────┐ ┌────────────┐ │ First name │ │ Address 1 │ │ Last name ✓│ │ Address 2 │ └────────────┘ │ City ✓│ │ State │ │ Postcode │ │ Country │ └────────────┘ ``` The chip body still toggles direction on click — the \`Dropdown\`'s internal wrapper calls \`stopPropagation\` so the sub-field click doesn't bubble to the chip's onClick. ## What changed **Backend:** - \`ViewSortEntity\` — new nullable \`subFieldName: varchar\` column - \`ViewSortDTO\`, \`CreateViewSortInput\`, \`UpdateViewSortInputUpdates\` — new \`@Field(() => String, { nullable: true })\` - \`FLAT_VIEW_SORT_EDITABLE_PROPERTIES\` — \`'subFieldName'\` added so the property flows through the update merge path - \`ALL_ENTITY_PROPERTIES_CONFIGURATION_BY_METADATA_NAME.viewSort\` — new \`subFieldName\` entry with \`toCompare: true\` so cache diffs notice it - \`fromCreateViewSortInputToFlatViewSortToCreate\` — threads \`subFieldName\` through - Instance command migration (\`add-sub-field-name-to-view-sort\`) — single \`ALTER TABLE core.viewSort ADD subFieldName varchar\` / \`DROP\` **Frontend:** - \`RecordSort\` and \`ViewSort\` types — \`subFieldName?: string | null\` - \`VIEW_SORT_FRAGMENT\` — adds \`subFieldName\` so the field round-trips - \`mapRecordSortToViewSort\` + \`areViewSortsEqual\` — carry the new field through, include it in the diff so the usual \`useSaveRecordSortsToViewSorts\` create/update flow fires when it changes - \`useSaveRecordSortsToViewSorts\` — passes \`subFieldName\` in both \`CreateViewSortInput\` and \`UpdateViewSortInputUpdates\` - \`getOrderByForFieldMetadataType(field, direction, subFieldName?)\` — new optional third arg. \`turnSortsIntoOrderBy\` threads \`sort.subFieldName\` into it. - \`Address\` added to \`SORTABLE_FIELD_METADATA_TYPES\` - New helpers: \`getEnabledAddressSubFields\` (filters by the field's \`subFields\` setting, falls back to the 6 default visible address sub-fields), \`getDefaultSortSubFieldForAddress\`, \`getDefaultSortSubFieldForFullName\` - New shared types/constants: \`AllowedFullNameSubField\`, \`ALLOWED_FULL_NAME_SUBFIELDS\`, \`DEFAULT_VISIBLE_ADDRESS_SUBFIELDS\` - \`SortOrFilterChip\` — new \`labelSubField?: ReactNode\` slot; renders as \` · {sub-field}\` with subdued weight after the main label - \`EditableSortChip\` — builds options from field metadata (\`ALLOWED_FULL_NAME_SUBFIELDS\` for FullName, \`getEnabledAddressSubFields\` for Address), uses i18n-wrapped labels, persists picks via \`upsertRecordSort\` ## Test plan - [x] \`npx nx typecheck\` passes for twenty-shared, twenty-front, twenty-server - [x] \`oxlint --type-aware\` on all 19 frontend + 9 server changed files: 0 errors - [x] \`prettier --check\`: clean - [x] 16 unit tests pass — \`getOrderByForFieldMetadataType\` covers the new \`subFieldName\` override branch for FULL_NAME and ADDRESS; \`getDefaultSortSubFieldForAddress\` covers the city/first-enabled fallback path; \`getDefaultSortSubFieldForFullName\` exercises its constant - [ ] Manual: sort a People view by Full Name → click the chip's sub-field label → switch between First name and Last name → reload page → choice is preserved - [ ] Manual: sort a Company view by Address → confirm dropdown lists only enabled sub-fields → disable Address \`addressCity\` in field settings → confirm dropdown options update and runtime falls back to the first enabled sub-field 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> Co-authored-by: claude[bot] <41898282+claude[bot]@users.noreply.github.com> |
||
|
Charles Bochet
|
e0b4c9918b |
feat(twenty-server): introduce ENCRYPTION_KEY env var with versioned envelope (#20528)
## Summary
- Adds `ENCRYPTION_KEY` (primary) and `FALLBACK_ENCRYPTION_KEY`
(decrypt-only fallback for rotation) env vars to twenty-server, with
backward-compatible fallback to `APP_SECRET` when `ENCRYPTION_KEY` is
unset.
- Introduces a versioned ciphertext envelope `enc:v2:<keyId>:<base64>`
using AES-256-GCM with HKDF-SHA256 derived per-context keys. The 8-hex
`keyId` fingerprint lets every row identify which physical key encrypted
it, so rotation routes directly to primary or fallback without trial
decryption; GCM's auth tag gives true integrity (legacy CTR has none).
- Migrates `ConnectedAccountTokenEncryptionService` to the new envelope
and plumbs `workspaceId` through every caller, so per-workspace HKDF
context binds each row to its tenant.
The remaining encryption sites (`jwt-key-manager`, `config-storage`,
`postgres-credentials`, `application-variable`, TOTP) stay on the legacy
unprefixed CTR path and will be migrated in follow-up PRs. The
operator-facing rotation runbook is out of scope here.
### Format details
`enc:v{N}:{keyId}:{base64}` — `N=2` is the only version produced by new
writes (`v1` exists for backward-compatible decryption of existing
connected-account rows). `keyId =
sha256(rawKey).slice(0,4).toString('hex')`. The CHECK constraint on
`core.connectedAccount.{accessToken,refreshToken}` is relaxed from `LIKE
'enc:v1:%'` to `LIKE 'enc:v_:%'` so both versions pass.
### Key resolution
| `ENCRYPTION_KEY` | `FALLBACK_ENCRYPTION_KEY` | `APP_SECRET` | Encrypt
with | Decrypt try order |
|---|---|---|---|---|
| set | set | (any) | `ENCRYPTION_KEY` | match `keyId` → primary →
fallback |
| set | unset | (any) | `ENCRYPTION_KEY` | match `keyId` → primary |
| unset | set | set | `APP_SECRET` | match `keyId` → `APP_SECRET` →
fallback |
| unset | unset | set | `APP_SECRET` | match `keyId` → `APP_SECRET` |
| unset | unset | unset | startup error | n/a |
## Test plan
- [x] `npx nx typecheck twenty-server` — clean
- [x] `npx jest
'secret-encryption|connected-account-token-encryption|connected-account-refresh-tokens|encrypt-connected-account-tokens|connection-provider-oauth-flow'`
— 87 tests pass
- [x] New `secret-encryption.service.versioned.spec.ts` covers: key
resolution table (no-key error, APP_SECRET fallback, ENCRYPTION_KEY
precedence), v2 round-trip with/without workspaceId, GCM tamper
rejection, workspaceId-mismatch rejection, keyId-based primary→fallback
routing, missing-key error names the fingerprint, v1 legacy decryption,
no-prefix legacy decryption, malformed envelope rejection.
- [x] Updated `connected-account-token-encryption.service.spec.ts`
covers workspaceId binding and HKDF context isolation.
- [x] Updated slow instance command spec verifies workspaceId is
threaded through encryption and the relaxed `enc:v_:%` LIKE pattern
matches both v1 and v2.
- [ ] Manual E2E: connect a Gmail account on a freshly deployed instance
with `APP_SECRET` only → confirm `core.connectedAccount.accessToken` is
`enc:v2:<keyId>:<base64>`.
- [ ] Manual E2E: rotate — set `ENCRYPTION_KEY=<new>` and
`FALLBACK_ENCRYPTION_KEY=<old APP_SECRET>`, restart, confirm
pre-rotation rows still decrypt and new rows carry the new `keyId`.
- [ ] Manual E2E: missing key — set `ENCRYPTION_KEY=<new>` without the
fallback, confirm decrypt error names the old `keyId` so the operator
can identify the missing key.
|
||
|
Weiko
|
27fd124c2e |
Dedicated REST controllers for object & field metadata (#20364)
## Summary
- Replace the dynamic `RestApiMetadataController` (which parsed
`/rest/metadata/*path` and proxied to internal GraphQL) with two
dedicated controllers: `ObjectMetadataController` and
`FieldMetadataController`.
- Drop the GraphQL hop: reads hit Postgres directly via TypeORM
repositories; writes call the existing
`{create,update,delete}One{Object,Field}` service methods.
- Introduce a new clean response shape behind a workspace feature flag
(`IS_REST_METADATA_API_NEW_FORMAT_DIRECT`) — see grace period below.
- Update the OpenAPI spec so the REST playground reflects the (default)
legacy shape during the grace period.
## Why
The legacy metadata controller was over-complex: it routed every method
through a path parser, a set of GraphQL query-builder factories, an
internal GraphQL call, and a
`cleanGraphQLResponse` post-processor. Operation names from GraphQL
(`createOneObject`, `updateOneField`, …) leaked straight into REST
responses. The internal-GraphQL hop also gave us
nothing on metadata reads — pagination, filtering, and serialization all
happen against the same Postgres tables either way.
## Feature flag & grace period
`IS_REST_METADATA_API_NEW_FORMAT_DIRECT` (workspace-scoped):
- **Existing workspaces:** flag absent → resolves to `false` → **legacy
response shape** (no behavior change).
- **Newly created workspaces:** flag seeded to `true` via
`DEFAULT_FEATURE_FLAGS` → **new response shape** from day one.
- **Toggle:** support-assisted (no frontend); customers contact us to
opt into the new shape early.
- **Removal:** the flag, the legacy adapter utils
(`to-legacy-{object,field}-metadata-response.util.ts`), and the
parametrized test wrapper get deleted after the grace window. New shape
becomes the only shape; OpenAPI flips to new shape; POST loses the
conditional and reverts to a declarative response.
## Response shapes
| Operation | Legacy (flag OFF, default for existing) | New (flag ON) |
|-----------|-----------------------------------------|---------------|
| `GET /rest/metadata/objects` | `{ data: { objects: [...] }, pageInfo,
totalCount }` | `{ data: [...], pageInfo, totalCount }` |
| `GET /rest/metadata/objects/:id` | `{ data: { object: {...} } }` | `{
... }` |
| `POST /rest/metadata/objects` | `201 { data: { createOneObject: {...}
} }` | `201 { ... }` |
| `PATCH/PUT /rest/metadata/objects/:id` | `{ data: { updateOneObject:
{...} } }` | `{ ... }` |
| `DELETE /rest/metadata/objects/:id` | `{ data: { deleteOneObject: {
... } } }` | `{ ... }` |
Same matrix for `/rest/metadata/fields`. Cursor params
(`starting_after`, `ending_before`, `limit`) and `totalCount` are
preserved across both shapes. POST returns `201` in both (old
controller already did — the doc on main saying `200` was wrong).
## Implementation notes
- Reads go straight to Postgres with TypeORM cursor pagination
(`paginateByIdCursor` util, mutually-exclusive `starting_after` /
`ending_before`). No cache on this path — caching +
filterable pagination didn't combine cleanly.
- Object endpoints inline `fields[]` via a single follow-up `WHERE
objectMetadataId IN (...)` query.
- Controllers read the flag via `FeatureFlagService.isFeatureEnabled`
and conditionally pass the result through a legacy-shape adapter util
before returning.
- Per-domain REST exception filters
(`{Object,Field}MetadataRestApiExceptionFilter`); the `exceptionCode →
httpStatus` switch is extracted to a util so it can be merged with the
existing GraphQL handler later.
- New controllers live inside the metadata domain modules
(`metadata-modules/{object,field}-metadata/controllers/`) to match
existing precedent (view-field, view, page-layout, …).
- Removes: `RestApiMetadataController`, `RestApiMetadataService`,
`metadata/query-builder/`, `clean-graphql-response.utils.ts`.
- Integration tests are parametrized over both flag values via
`describe.each` — both shapes are asserted in CI.
- OpenAPI fixes inherited from the migration (kept as-is): documents
flat `fields: [...]` rather than the obsolete `{edges:{node:[...]}}`
wrapping; always emits `totalCount`; POST
status `201`. These match what customers actually receive on both
shapes.
Note: Next goal is to implement something similar for graphql and remove
nestjs-query dependency for those 2 entities, then generalise it.
Note2: We have the same issue with Core Rest API such as
```json
{
"data": {
"createCompany": {
"id": "123e4567-e89b-12d3-a456-426614174000",
"createdAt": "2026-05-07T12:14:52.769Z",
"updatedAt": "2026-05-07T12:14:52.769Z",
"deletedAt": "2026-05-07T12:14:52.769Z",
...
```
with "createCompany" here which is odd compared to REST standards (FYI
@etiennejouan @charlesBochet)
## Before (Without feature flag)
<img width="1346" height="712" alt="Screenshot 2026-05-12 at 20 50 38"
src="https://github.com/user-attachments/assets/316ce225-1045-4aac-97a9-60fd537eb1ec"
/>
<img width="1378" height="729" alt="Screenshot 2026-05-12 at 20 52 24"
src="https://github.com/user-attachments/assets/a621ab6f-e4f8-44d5-817c-1efd25d33c30"
/>
## After (With feature flag)
<img width="1376" height="728" alt="Screenshot 2026-05-12 at 20 50 46"
src="https://github.com/user-attachments/assets/2424d9c5-e4ed-497c-8e5c-6b54d78675e4"
/>
<img width="1375" height="727" alt="Screenshot 2026-05-12 at 20 51 47"
src="https://github.com/user-attachments/assets/101d957f-38ed-45d9-ab7b-f4f4eb983397"
/>
---------
Co-authored-by: prastoin <paul@twenty.com>
|
||
|
MkDev11
|
bbc55193f5 |
Fix phone unique constraints (#20261)
## Summary Closes #20195 Fix phone field unique constraints so phone numbers are considered unique by both `primaryPhoneNumber` and `primaryPhoneCallingCode`. - Include `primaryPhoneCallingCode` in the shared phone composite unique constraint metadata - Align the frontend settings composite field config with the backend metadata - Return all included unique composite subfields when building create-many conflict fields - Match composite unique conflict fields as a group during create-many upserts ## Root Cause Phone composite metadata only marked `primaryPhoneNumber` as part of the unique constraint. That made different international phone numbers with the same national number conflict, for example `+1 123456789` and `+32 123456789`. ## Test Plan - `yarn workspace twenty-shared build` - `jest --runTestsByPath <index action handler and create-many utility specs>` - `prettier --check <touched files>` - `oxlint --type-aware <touched files>` - `nx run twenty-shared:typecheck` - `nx run twenty-server:typecheck` - `nx run twenty-front:typecheck` --------- Co-authored-by: mkdev11 <MkDev11@users.noreply.github.com> Co-authored-by: Etienne <45695613+etiennejouan@users.noreply.github.com> Co-authored-by: prastoin <paul@twenty.com> |
||
|
Charles Bochet
|
ac653182b2 |
feat(server): migrate all remaining JWT token types to ES256 (#20513)
## Summary Extends the asymmetric signing work from #20467 to cover **every remaining `JwtTokenTypeEnum` value**: `LOGIN`, `WORKSPACE_AGNOSTIC`, `FILE`, `API_KEY`, `APPLICATION_ACCESS`, `APPLICATION_REFRESH`, `APP_OAUTH_STATE`, plus the ACCESS-shaped session token issued by the code interpreter tool. After this PR, every JWT the server signs is ES256 with a `kid` pointing at the current `core."signingKey"` row, while legacy HS256 tokens (no `kid` header) remain verifiable indefinitely through the existing fallback in `JwtWrapperService.resolveVerificationKey`. No new entity / migration / config: this is a pure routing change on top of the infrastructure that already shipped. ## Why `#20467` only flipped `ACCESS` and `REFRESH` to ES256. Every other JWT type was still HS256-signed against the global `APP_SECRET`, which kept the original blast radius (a leaked `APP_SECRET` invalidates *every* JWT type forever). Migrating the rest unifies the sign path on rotatable per-server private keys without forcing any token reissue. ## Mechanical changes ### Sign side (8 services) - `LoginTokenService.generateLoginToken` - `TransientTokenService.generateTransientToken` - `WorkspaceAgnosticTokenService.generateWorkspaceAgnosticToken` - `ApplicationTokenService.signApplicationToken` (`APPLICATION_ACCESS` + `APPLICATION_REFRESH`) - `ApiKeyService.generateApiKeyToken` - `FileUrlService.signFileByIdUrl` / `signWorkspaceLogoUrl` - `ConnectionProviderOAuthFlowService.signState` (`APP_OAUTH_STATE`) - `CodeInterpreterTool.generateSessionToken` Each call site swaps `jwtWrapperService.sign(payload, { secret: generateAppSecret(...), ... })` for `await jwtWrapperService.signAsync(payload, { expiresIn, [jwtid] })`. The `generateAppSecret` calls on the sign side are dropped (verifier-side `generateAppSecret` stays in `resolveVerificationKey` for the HS256 fallback). ### Verifier side - `WorkspaceAgnosticTokenService.validateToken` now goes through `verifyJwtToken` instead of the bespoke `verify({ secret })` path, so new ES256 tokens are accepted while the legacy HS256 fallback inside `resolveVerificationKey` still serves the old shape. - `JwtWrapperService.sign()` is kept (legacy compat / tests) but is now strictly deprecated — there are no remaining production callers. ### Async ripple (`signFileByIdUrl` was synchronous) - `FileUrlService.signFileByIdUrl` and `signWorkspaceLogoUrl` are now `async`; the `signUrl` callback used by `getRecordImageIdentifier` is widened to accept `Promise<string | null>`. - Every direct/indirect caller is updated: admin panel (user lookup + statistics + top workspaces), search service (`computeSearchObjectResults`, `getImageIdentifierValue`), workspace resolver (`logo` resolver, public workspace by domain/id), `WorkspaceMemberTranspiler` (now `async toWorkspaceMemberDto[s]` / `toDeletedWorkspaceMemberDto[s]` / `generateSignedAvatarUrl`), `UserService.loadSignedAvatarUrlsByUserId`, `UserWorkspaceService.castWorkspaceToAvailableWorkspace`, workspace-invitation, approved-access-domain, agent-chat-streaming, agent-message-part resolver, navigation-menu-item record identifier, file-ai-chat / file-core-picture / file-email-attachment / file-workflow / files-field services, rich-text & files-field query result getters, and the code-interpreter tool. ## Backward compatibility - **Legacy HS256 tokens (no `kid`)** keep verifying via `resolveVerificationKey` → `extractAppSecretBody` → `generateAppSecret` for both `workspaceId`-bearing and `userId`-bearing payloads. - The `API_KEY` HS256-via-ACCESS-secret fallback (#16504) still kicks in inside `verifyJwtToken` for pre-2025-12-12 API keys. - No payload shape changes, no DB writes, no env var changes — old tokens issued by `main` continue to authenticate. ## Tests ### Unit (all green locally — 63/63) Updated specs for every migrated service to mock `signAsync` instead of `sign` and assert the new option shape: - `login-token.service.spec.ts`, `transient-token.service.spec.ts`, `workspace-agnostic-token.service.spec.ts`, `application-token.service.spec.ts`, `api-key.service.spec.ts`, `connection-provider-oauth-flow.service.spec.ts`. ### Integration (`jwt-key-rotation.integration-spec.ts`) - Existing ACCESS coverage (current key, legacy HS256 fallback, rotated-out key, revoked key, unknown kid) is preserved. - New `it.each` assertion: `REFRESH`, `WORKSPACE_AGNOSTIC`, and `LOGIN` tokens emitted by the real signUp → signUpInNewWorkspace → getAuthTokensFromLoginToken pipeline are ES256 with a `kid` matching the current signing key — proves end-to-end that the migration didn't regress those flows. ## Open question (separate decision) This PR keeps the legacy HS256 verification fallback **forever**. We may eventually want to sunset it for `API_KEY` once telemetry shows pre-migration tokens are gone, but that's a separate product/security decision and not part of this change. ## Test plan - [ ] CI green - [ ] `npx nx lint:diff-with-main twenty-server` passes - [ ] `npx nx typecheck twenty-server` passes - [ ] `jwt-key-rotation` integration suite passes (new + existing assertions) - [ ] Manually verify: signing in issues an ES256 ACCESS / REFRESH token, generating an API key issues an ES256 token with `kid`, signed file URL JWT is ES256 with `kid` - [ ] Pre-existing HS256 tokens still authenticate (covered by integration test, but worth a manual check with a token from `main`) |
||
|
Paul Rastoin
|
a159a68e2c |
[twenty-server] no floating promises lint rule (#20499)
## Introduction
That's an audit + RFC
## Fire-and-forget (`void`) -- Intentional, correct
These are telemetry, metrics, and audit logging in hot paths or
non-critical contexts. `void` is the right choice.
| File | What was voided |
|---|---|
| `sign-in-up.service.ts` | `metricsService.incrementCounter` (sign-up
metric) + `auditService.insertWorkspaceEvent` (workspace created) |
| `use-graphql-error-handler.hook.ts` | 5x
`metricsService.incrementCounter` (GraphQL operation metrics) |
| `bullmq.driver.ts` | 2x `metricsService.incrementCounter` (job
completed/failed metrics) |
| `call-webhook.job.ts` | 2x `auditService.insertWorkspaceEvent` + 1x
`metricsService.incrementCounter` |
| `custom-domain-manager.service.ts` | `analytics.insertWorkspaceEvent`
(domain activation event) |
| `logic-function-executor.service.ts` |
`auditService.insertWorkspaceEvent` (function execution) |
| `workflow-runner.workspace-service.ts` |
`metricsService.incrementCounter` (throttle metric) |
| `cleaner.workspace-service.ts` | `metricsService.incrementCounter`
(deleted workspace metric) |
| `stream-agent-chat.job.ts` | Detached IIFE for streaming chunks
(intentional concurrent pipeline) |
| `workspace-auth-context.middleware.ts` |
`withWorkspaceAuthContext(...)` (AsyncLocalStorage, returns void anyway)
|
## Top-level script entry points (`void bootstrap()`)
These are module-level calls where the promise has no consumer. `void`
makes the lint rule happy and documents the intent.
| File | What changed |
|---|---|
| `main.ts` | `void bootstrap()` |
| `command.ts` | `void bootstrap()` |
| `queue-worker.ts` | `void bootstrap()` |
| `truncate-db.ts` | `void dropSchemasSequentially()` |
| `codegen/index.ts` | `void generateTests(forceArg)` |
## Now properly awaited -- Real bug fixes
These were floating promises that could silently fail, lose data, or
cause race conditions.
| File | What was fixed |
|---|---|
| `billing-sync-plans-data.command.ts` | `meters.map(async ...)` wrapped
in `Promise.all` -- was returning before upserts finished |
| `cache-storage.service.ts` | `setAdd` and `setPop` had `.then()`
chains that weren't returned/awaited |
| `create-audit-log-from-internal-event.ts` | 4x
`auditService.createObjectEvent` now awaited inside a job |
| `cleaner.workspace-service.ts` | 2x `emailService.send(...)` now
awaited -- emails could silently fail |
| `agent-async-executor.service.ts` | `calculateAndBillUsage` +
`billNativeWebSearchUsage` in `finally` block now awaited |
| `repair-tool-call.util.ts` | `calculateAndBillUsage` now awaited |
| `agent-title-generation.service.ts` | `calculateAndBillUsage` now
awaited |
| `chat-execution.service.ts` | `billNativeWebSearchUsage` now awaited |
| `ai-generate-text.controller.ts` | `calculateAndBillUsage` in
`finally` block now awaited |
| `agent-turn.resolver.ts` | `messageQueueService.add(...)` now awaited
|
| `command.ts` | `app.close()` now awaited (was exiting before graceful
shutdown) |
| `i18n.service.ts` | `loadTranslations()` in `onModuleInit` now awaited
|
| `workspace-query-hook.explorer.ts` | `explore()` in `onModuleInit` now
awaited |
| `message-queue.explorer.ts` | `handleProcessorGroupCollection` in
`onModuleInit` now awaited |
| `ai-billing.service.spec.ts` | Test now properly `await`s the async
call |
| `messaging-messages-import.service.spec.ts` | `expect(...)` now
properly `await`ed for async assertion |
| `archive.finalize()` (3 files) | Voided -- promise resolution already
handled by `pipeline()` / `on('end')` |
## Impersonation & security audit trail -- Upgraded from `void` to
`await`
These were previously fire-and-forget but are
security/compliance-critical events that must be reliably persisted.
| File | What was fixed |
|---|---|
| `impersonation.service.ts` | 4x `auditService.insertWorkspaceEvent`
now awaited (impersonation attempt, token generation
attempt/success/failure) |
| `auth.resolver.ts` | 5x `auditService.insertWorkspaceEvent` now
awaited (impersonation token exchange attempt/success/failure at server
and workspace levels) |
| `auth.service.ts` | 2x `analytics.insertWorkspaceEvent` now awaited
(impersonation attempted/issued) |
## Billing audit -- Upgraded from `void` to `await`
Payment events should be reliably persisted for financial/compliance
reporting.
| File | What was fixed |
|---|---|
| `billing-webhook-invoice.service.ts` |
`auditService.insertWorkspaceEvent(PAYMENT_RECEIVED_EVENT)` now awaited
inside Stripe webhook handler |
## Fire-and-forget with proper error handling -- Upgraded from bare
`void`
These remain non-blocking but now catch and log errors instead of
risking unhandled rejections.
| File | What was fixed |
|---|---|
| `logic-function-executor.service.ts` |
`applicationLogsService.writeLogs` now uses `.catch()` instead of bare
`void` -- user-facing logs should surface errors |
## Systemic infrastructure fixes
| File | What was fixed |
|---|---|
| `metrics.service.ts` | `incrementCounter`: Redis cache write
(`metricsCacheService.updateCounter`) now uses `.catch()` internally
instead of raw `await` -- prevents unhandled rejections across all `void
metricsService.incrementCounter(...)` call sites when Redis is unhealthy
|
| `audit.service.ts` | `preventIfDisabled`: made properly `async` with
`await` and consistent `Promise<{ success: boolean }>` return type.
Removed broken `catch` that returned an `AuditException` as a value
(wrong constructor args, unreachable dead code). Removed unused
`AuditException` import |
## Fixed in this session (beyond original PR)
| File | What changed |
|---|---|
| `telemetry.listener.ts` | Removed misleading `Promise.all` + `void`
combo; replaced with simple `for...of` + `void` |
| `message-queue.explorer.ts` | Changed from `void` to `await` so
startup crashes on registration failure |
|
||
|
Charles Bochet
|
9e515afb13 |
feat(server): asymmetric JWT signing with kid + key rotation table (#20467)
## Context Today every JWT issued by Twenty (access, refresh, login, file, etc.) is HMAC-signed with a per-token-type secret derived from the global `APP_SECRET`. Rotating that secret invalidates **every** active token at once and there is no way to scope a leak to a subset of tokens. This PR is the first slice of a broader effort to **decouple stateful encryption (`APP_SECRET`-derived secrets) from stateless encryption (JWTs)**. It introduces an asymmetric (private/public key) signing path for `ACCESS` and `REFRESH` tokens and a signing-key registry to enable **safe rotation**: leaked keys can be revoked by flipping `revokedAt`/`isCurrent` on the matching row without invalidating tokens issued by other keys. > Out of scope (intentionally): swapping stateful encryption for `APP_SECRET`, asymmetric signing for token types other than `ACCESS`/`REFRESH`, an admin-panel rotation UI, and an enterprise re-encryption command. Those will land in follow-up PRs. ## What changes - **New `core.signingKey` table** (instance command `2.5.0` / `1778550000000`) storing both the public key (PEM, in clear) and the private key (PEM, encrypted with `APP_SECRET` via `SecretEncryptionService`). One row is marked `isCurrent = true` (enforced by a partial unique index). The row's UUID `id` is used directly as the JWT `kid`. - When a key is rotated out, its `privateKey` is nulled (we never keep historical private keys) but the `publicKey` row stays so previously issued tokens can still be verified. - **`JwtKeyManagerService`** lazily loads-or-generates the current signing key on first use: - If a row with `isCurrent = true` exists → decrypts and uses it. - Otherwise → generates a fresh EC P-256 keypair, encrypts the private key, inserts the row (UUID id = kid). Handles concurrent insert races via the unique constraint. - **`JwtWrapperService.signAsync()`** signs `ACCESS`/`REFRESH` payloads with `ES256` and a `kid` header. Falls back to `HS256` if no signing key is available (boot-time DB error, transient failure). - **Dual-path verification** in both `JwtWrapperService.verifyJwtToken` and the Passport `JwtAuthStrategy.secretOrKeyProvider`: - JWT with a `kid` header → resolve the public key PEM by id and verify with `ES256`, - otherwise → fall back to the existing `APP_SECRET`-derived `HS256` path (unchanged). - **`AccessTokenService` / `RefreshTokenService`** now sign through `signAsync` (single public surface; the routing detail stays internal to the wrapper). - **Public key cache**: a new `SigningKeyEntityCacheProviderService` plugs into `CoreEntityCacheService` (`signingKeyPublicKey` namespace) and serves PEMs by id, with the standard local-memo + Redis layering. - **PEM strings end-to-end**: `jsonwebtoken` accepts PEM strings directly for both sign and verify, so the manager never converts to a Node `KeyObject` and the cache hands the PEM straight to `jwt.verify`. ## Why ES256 (and not EdDSA / RS256) - `@nestjs/jwt` is backed by `jsonwebtoken`, which does **not** support EdDSA today. - ES256 keys are tiny (~120 bytes vs 1.6 kB for RS256), signatures are short (~64 bytes), and signing/verification is fast — important since JWT verification runs on every authenticated request. - ES256 is widely supported and standardized (RFC 7518), with mature ecosystem support. ## Why store the private key in DB (not env) - No new secret to provision: existing instances already have `APP_SECRET`, which we reuse to encrypt the private key at rest. - Self-healing: a fresh instance auto-generates its first signing key on first boot. Nothing to copy/paste. - Rotation is a SQL operation against `core.signingKey`, not a redeploy + env mutation. ## Backward compatibility - All previously-issued tokens (no `kid`) keep verifying through the legacy HS256 path with their existing `APP_SECRET`-derived secret. No forced re-login. - Token types not in scope (`WORKSPACE_AGNOSTIC`, `API_KEY`, `FILE`, `LOGIN`, `EMAIL_VERIFICATION`, etc.) keep their current HS256 behavior unchanged — they still go through the synchronous `JwtWrapperService.sign(payload, options)` with a caller-supplied secret. - `signWithAppSecret` is kept intentionally as the HS256 fallback path; it will be deprecated in a follow-up PR. - If the DB lookup/generation fails for any reason, the wrapper logs the error and falls back to HS256 — no startup crash, no silent regression. ## Rotation story 1. Bootstrap: first signing call lazily inserts a row in `core.signingKey` with `isCurrent = true`, `privateKey = encrypt(pem_A)`. New tokens carry `kid_A`. 2. Rotate: `UPDATE core."signingKey" SET "isCurrent" = false, "privateKey" = NULL WHERE id = '<kid_A>';` then insert a new row with `isCurrent = true`. New tokens carry `kid_B`. Tokens still in flight with `kid_A` keep verifying because the public-key row for `kid_A` is still there. 3. Revoke: `UPDATE core."signingKey" SET "revokedAt" = now() WHERE id = '<kid_A>';`. All tokens with `kid_A` now fail verification cleanly with `UNAUTHENTICATED` (no 500). 4. Tokens with no `kid` (legacy) are unaffected throughout. ## Test plan - [x] Unit: `JwtWrapperService` dual-path verification (HS256 no-kid vs ES256 with-kid), unknown-kid → `UNAUTHENTICATED`, `signAsync` happy path + `null` when no key, `signAsync` rejection for non-rotatable types. - [x] Unit: `JwtAuthStrategy` `secretOrKeyProvider` dual-path resolution and algorithm validation. - [x] All existing JWT/auth/application unit tests adjusted to the renamed public method. - [x] Integration (`jwt-key-rotation.integration-spec.ts`): - **Happy path**: signed-up user's `ACCESS` token has `alg=ES256` + correct UUID `kid`, the `isCurrent=true` row exists in `core.signingKey`, `getCurrentUser` resolves. - **Legacy fallback**: hand-crafted no-kid HS256 token verifies via the legacy `APP_SECRET`-derived path. - **Previous-key rotation**: token signed by a hardcoded *previous* key whose row is pre-inserted with `privateKey = NULL` (rotated-out) still verifies — proves the leaked-key revocation flow works in both directions. - **Unknown kid**: token signed with an orphan UUID `kid` is cleanly rejected (no 500). - [x] `npx nx typecheck twenty-server` - [x] `npx nx test twenty-server` - [x] `npx nx run twenty-server:lint` |
||
|
Charles Bochet
|
10876138d2 |
refactor: stop reading joinColumnName from relation field settings (#20304)
## Summary `joinColumnName` on relation field settings is always derivable from the field name (and the target object name for morph relations). This PR stops reading it from settings anywhere in production code; the stored value is no longer used. The settings field is **not** removed from data yet — a follow-up can drop it once we are confident nothing depends on the stored value. ## Helpers The helpers are split by layer because frontend and backend hold morph relations differently: the frontend has a base name plus a `morphRelations[]` array, the backend has one row per target with the name already morph-resolved. | Helper | Layer | When to use | |---|---|---| | `computeRelationGqlFieldJoinColumnName` | Shared / frontend (`gqlField`) | Non-morph relation on the frontend. | | `computeMorphRelationGqlFieldName` | Shared / frontend (`gqlField`) | Need the per-target morph gqlField name (e.g. `targetCompany`). | | `computeMorphRelationGqlFieldJoinColumnName` | Shared / frontend (`gqlField`) | Per-target morph join column on the frontend. Prefer over the non-morph helper for any morph field — it forces the per-target inputs. | | `computeMorphOrRelationFieldJoinColumnName` | Backend (`FlatFieldMetadata.name`) | Any backend read or write — the flat name is already morph-resolved, so one helper covers both cases. | | `computeMorphRelationFlatFieldName` | Backend (`FlatFieldMetadata.name`) | **Mutation paths only** (create / update / object rename). Reads consume the stored `field.name` and never call this. | ## Test plan - [x] Typecheck and lint (front, server, shared) - [x] Existing unit tests pass - [ ] CI green |