9f3ebaaf22061eb306f613a104d65b9da6002fd4
473 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Weiko
|
cc76b7bc50 |
Fix fields widget new field visibility (#21111)
Fixes https://github.com/twentyhq/twenty/issues/21043 ## Context Newly created fields were never added to FIELDS widgets, regardless of the "Set fields created in the future as visible" toggle. The widget's newFieldDefaultVisibility was null on widgets that never explicitly set it (it was never populated at creation), so the backend skipped them and no view field was created. ## Implementation Keep newFieldDefaultVisibility nullable with false (not visible) as the behavior when not provided. The FE now reflects that properly and shows "un-toggled" when it's null (iso with BE behavior). The fix also ensures the value is explicitly set to true wherever it should be: - Set newFieldDefaultVisibility: true at every FIELDS widget creation path (backend default record-page layout, frontend createDefaultFieldsWidget + useTemporaryFieldsConfiguration); - Added a 2-9 workspace upgrade command that backfills true onto existing standard FIELDS widgets where the value is null. |
||
|
Charles Bochet
|
58907b733c |
feat(logic-function): add LIVE / PREBUILT execution modes (#20873)
## Summary
### Why
1. Sending the code to the lambda (~1Mb usually) is heavy on network and
results to a constant traffic of ~30Mb/s on AWS which results into TB of
network data every month
2. eval(1MB of code) is not that fast, it's heavy on memory and CPU on
lambda side
### High level
Adds two execution modes for logic functions, gated behind the new
`IS_LOGIC_FUNCTION_PREBUILT_MODE_ENABLED` workspace feature flag (off
everywhere by default):
- **LIVE** (current behavior, preserved bit-for-bit): the compiled
bundle is read from object storage and shipped in every Lambda invoke
payload. Used for fast iteration in the workflow editor / Settings test
runs.
- **PREBUILT** (new): the bundle is installed onto the per-function
Lambda alongside the unified executor, and invocations carry only `{
params, env, handlerName }` — saving JSON payload egress and warm-start
`import()` cost on every call.
### Key design choices
- **Unified Lambda handler** (`constants/executor/index.mjs`) dispatches
at runtime: `event.code` present ? LIVE (write to `/tmp`, dynamic
import) : `import('./prebuilt-logic-function.mjs')`. Both code paths
always coexist on the deployment package, so the same Lambda can serve
either mode without redeploying.
- **Install runs inside the `validateBuildAndRun` migration pipeline**,
not at execute time. `Create/UpdateLogicFunctionActionHandlerService`
calls `driver.installPrebuiltBundle` when `executionMode` flips
LIVE?PREBUILT or `checksum` changes while PREBUILT, gated on
`isBuildUpToDate=true` and a fresh checksum.
- **Strict execute, no reconciliation**:
`LogicFunctionExecutorService.execute` resolves `effectiveExecutionMode`
(caller override > feature flag > entity column). For PREBUILT it asks
the driver `getInstalledBundleChecksum` (Lambda `twenty:bundle-checksum`
tag for AWS, sidecar file locally) and throws
`LOGIC_FUNCTION_PREBUILT_BUNDLE_NOT_INSTALLED` on mismatch.
- **Feature flag gates every side effect**: with the flag off the
executor forces LIVE, the action-handler install hooks bail before AWS,
and workflow activation does not flip the mode. Rollback is just turning
the flag off.
### Lifecycle
- New workflow CODE step ? `LIVE`, no install.
- Workflow activated ? build + activation flips `executionMode=PREBUILT`
? action-handler installs the bundle + sets the Lambda tag.
- Draft from active version ? duplicated logic function reset to `LIVE`.
- App install ? manifest converter sets `PREBUILT`, create-action
handler installs.
- Test runs (`executeOneFromSource`, workflow editor) pass
`executionMode=LIVE` explicitly.
### Observability
`[lambda-timing]` log lines now include `effectiveExecutionMode` and
`payloadBytes`; the action handler logs `install_duration_ms` for each
install.
## Test plan
- [x] `npx nx typecheck twenty-server` ? passes
- [x] `npx oxlint --type-aware` on all changed files ? 0 warnings, 0
errors
- [x] `npx nx test twenty-server` ? 588 suites / 5009 tests pass (no
regressions vs main)
- [x] New unit suite `flat-logic-function-validator.service.spec.ts` ?
9/9
- [x] Existing
`workflow-version-step-operations.workspace-service.spec.ts` ? 8/8
(verified the new token-based DI avoids a circular-import regression)
- [x] Snapshot for
`ALL_UNIVERSAL_FLAT_ENTITY_PROPERTIES_TO_COMPARE_AND_STRINGIFY` updated
to include `executionMode`
- [x] Integration suite `logic-function-execution.integration-spec.ts`
extended to assert `executionMode=LIVE` on newly-created functions and
continues to exercise the LIVE happy path
- [ ] Manual staging rollout: flip
`IS_LOGIC_FUNCTION_PREBUILT_MODE_ENABLED` per workspace, observe
`[lambda-timing]` `payloadBytes` drop + `install_duration_ms`, then ramp
in prod.
|
||
|
nitin
|
13f09d8946 |
[Dashboards] Remove gauge chart types and code (#20410)
Follow-up cleanup to #20172. |
||
|
martmull
|
961745e9ba |
Fix latency spike on application lookup (#21042)
Fixes https://discord.com/channels/1130383047699738754/1509645089062781058 Caching application entities to improve authentication latency |
||
|
Paul Rastoin
|
ebfaca5b3d |
EncryptedString PlaintextString branded string types (#21001)
## Summary closes https://github.com/twentyhq/core-team-issues/issues/2464 Introduces compile-time branded types to distinguish encrypted ciphertext from plaintext strings, preventing mix-ups like the one fixed in #20819 — but at the type level rather in addition to the one existing at runtime. ### Branded string primitives - Created `EncryptedString` and `PlaintextString` as hard nominal brands using `z.string().brand(...)`, making them non-assignable to each other or to raw `string` - Created `isEncryptedString` type predicate to narrow `string` to `EncryptedString` based on the `enc:v2:` envelope prefix - Retyped `SecretEncryptionService`: `encryptVersioned` accepts `PlaintextString`, `decryptVersioned` returns `PlaintextString` ### Entity typing - Typed encrypted columns across entities: `SigningKeyEntity.privateKey`, `TwoFactorAuthenticationMethodEntity.secret`, `ApplicationRegistrationVariableEntity.encryptedValue`, `ApplicationVariableEntity.value` - Parameterized JSONB types for connected account connection parameters (`ImapSmtpCaldavParams<Pwd>`) with reusable aliases `EncryptedImapSmtpCaldavParams` / `DecryptedImapSmtpCaldavParams` - Typed DTOs (`CreateApplicationRegistrationVariableInput`, `UpdateApplicationRegistrationVariablePayload`, `UpdateApplicationVariableEntityInput`) with `PlaintextString` ### ApplicationVariable always-encrypt uniformization - Retyped `ApplicationVariableEntity.value` to `EncryptedString | ''` — all values are now encrypted regardless of `isSecret` - Updated `ApplicationVariableEntityService` to always encrypt on write and always decrypt on read - Simplified `UpdateApplicationVariableActionHandlerService` by removing conditional encrypt/decrypt-on-isSecret-toggle logic - Added slow instance command (`2.9.0`) to backfill-encrypt existing `isSecret=false` plaintext rows and tighten the `CHECK` constraint ### ConfigStorageService refactor - Split `convertAndSecureValue` (which used `any`) into two well-typed methods: `convertAndDecrypt` and `convertAndEncrypt` - Introduced `isSensitiveStringValue` type predicate to narrow values before encryption/decryption ### What's next - Typeorm entity derivation to strictly type sitemap configuration as code + handler logic for encryption rotation - https://github.com/twentyhq/core-team-issues/issues/2465 |
||
|
neo773
|
c5606212f2 |
Ses outbound followup (#20610)
This pull request unifies outbound with inbound under the new feature and the new email groups feature. These are workspace level shared inboxes that are shared between all workspace members. outbound sending with SES works, we only listen for tenant status events, rest is managed by AWS PR refactors old code and webhook to be split for outbound and inbound for proper separation | Area | Change | |---|---| | AWS SES driver | Split into `AwsSesRegisterDomainService` (tenant + identity + DKIM + MAIL FROM + configuration-set + EventBridge dest + contact list) and `AwsSesSendEmailService` (SendEmail). | | Reputation webhook | New `/webhooks/messaging/ses/outbound` route. SES → EventBridge (`Sending Status Enabled/Disabled` on default bus) → SNS → router → `SesOutboundSendingStateHandlerService` updates `emailing_domain.tenantStatus`. | | Inbound webhook | Refactored into `SesInboundWebhookRouterService` + `SesInboundMailHandlerService`. Shared `SnsSignatureVerifierService` + `SnsSubscriptionConfirmerService` across both routes. | | Global uniqueness | New migration + instance command: `emailing_domain.domain` is now globally unique (one tenant per domain across workspaces). | | Tenant status | New `emailing_domain.tenantStatus` column (`ACTIVE` / `PAUSED`) + `EmailingDomainTenantStatusService`. | | Send-email mutation | New `sendEmailViaDomain` GraphQL mutation + DTOs. | | Cleanup | `EmailingDomainWorkspaceCleanupJob` wired into `WorkspaceService.deleteWorkspace` — tears down SES tenant association + identity on workspace delete. | | Settings UI | Rewritten around reusable `SettingsTableListSection`. "Email Group" → "Email Handle" rename. New cells for status/source/forwarding. Outbound domains surfaced on workspace settings page. | ### Env vars (new) All in `config-variables.ts`, group `AWS_SES_SETTINGS`, all optional: - `AWS_SES_REGION` — `@IsAWSRegion`, consumed by `AwsSesClientProvider` + driver factory - `AWS_SES_ACCOUNT_ID` — used for ARN construction in driver factory - `SES_SNS_TOPIC_ARN_ALLOWLIST` — **shared** by inbound + outbound webhook routers, comma-separated list of accepted SNS topic ARNs (verified via `sns-payload-validator`) ### Migrations - `1778862608620-add-emailing-domain-tenant-status` (fast) — adds `tenantStatus` column. - `1778865501791-unique-emailing-domain-globally` (slow, idempotent) — enforces global uniqueness on `domain`. - Instance commands bumped to `2.5`. ### Infra dependency Two coupled twenty-infra PRs: - `ses-inbound-email` — receipt-rule + inbound SNS topic + S3 bucket policy + KMS grant + `email_group_*` outputs. - `ses-outbound-tf` — EventBridge rule + outbound SNS topic + SES IAM policy + outbound `webhook_url` subscription. **Based on `ses-inbound-email`.** Merge order: inbound first, then outbound. Outbound PR's chart edit owns the comma-joined `SES_SNS_TOPIC_ARN_ALLOWLIST` value (both ARNs). Features lives under `/settings/general` <img width="1496" height="845" alt="SCR-20260519-ofhi-2" src="https://github.com/user-attachments/assets/a025485a-09f7-4131-91cd-0067690ff18d" /> --------- Co-authored-by: Félix Malfait <felix.malfait@gmail.com> Co-authored-by: claude[bot] <41898282+claude[bot]@users.noreply.github.com> Co-authored-by: Félix Malfait <FelixMalfait@users.noreply.github.com> |
||
|
Etienne
|
4a82cddad6 |
remove ai-model-preferences var env and config (#20859)
Split the single AI_MODEL_PREFERENCES JSON config into 4 array configs and migrates existing workspace data. |
||
|
neo773
|
bb5c2bd00c |
Fix/restore channel association scalar field metadata (#20920)
[#20836](https://github.com/twentyhq/twenty/pull/20836) dropped the channel objects but even though calendarChannelId/messageChannelId/messageFolderId already existed in compute standard flat field, there was never an upgrade command to readd them on the surviving association objects so existing workspaces lost the field metadata (columns survived) and import workers throw ```Error: Unknown error importing calendar events for calendar channel <REDACTED> in workspace <REDACTED>: Query runner already released. Cannot run queries anymore.``` This PR adds that command --------- Co-authored-by: prastoin <paul@twenty.com> |
||
|
Félix Malfait
|
d602f35cbd |
feat(data-model): custom-indexes management UI and mutations (#20846)
## Summary
Brings indexes management into the per-object Settings tab as a section
under Search (no feature flag, advanced mode only). Admins can create /
delete non-unique indexes with the UI; apps can declare indexes in code
with `defineIndex`. Composite-typed fields are now indexable by picking
a specific sub-column (e.g. `Address > City`).
A few related polish items also land here (invite-user dropdown lands on
the Invite tab; standard warning callout above the new-index form).
## What ships
### UI — custom indexes on per-object Settings
- New section directly under Search, wrapped in
`AdvancedSettingsWrapper`.
- Filter dropdown on the search bar toggles system-index visibility
(shown by default since advanced mode).
- **+ Add Index** button (disabled with tooltip once the per-object cap
is reached) navigates to a dedicated `SettingsObjectNewIndex` page
(matches the field-creation pattern, not a modal):
- Field picker mirrors the webhook event-form layout (rows of dropdowns,
implicit trailing empty row).
- Composite fields surface their sub-properties (`Address > City`,
`Currency > Amount`, …).
- BTREE / GIN type selector.
- Standard warning Callout: "Use indexes sparingly — each one speeds
reads but slows writes."
- Trash icon on `isCustom: true` rows → confirmation modal →
`deleteOneIndex`.
### Server — `createOneIndex` / `deleteOneIndex` mutations
- Gated by `SettingsPermissionGuard(DATA_MODEL)`.
- `IndexMetadataService` wraps the existing migration runner via
`WorkspaceMigrationValidateBuildAndRunService` so the metadata row and
the SQL index land atomically.
- Validation: rejects empty fields, duplicate `(fieldMetadataId,
subFieldName)` pairs, fields not on the object, requires `subFieldName`
for composite parents, forbids `subFieldName` on scalar/relation,
enforces `MAX_CUSTOM_INDEXES_PER_OBJECT = 10`.
- Delete refuses on `isCustom: false` rows so system indexes can't be
removed via this API.
- Dedicated GraphQL exception handler maps each typed error to the right
transport error class.
### Composite sub-field indexing
- Adds `subFieldName: string | null` column to
`IndexFieldMetadataEntity` (fast instance command).
- The flat-entity flow (`UniversalFlatIndexFieldMetadata`,
`FlatIndexFieldMetadata`, `from-universal-flat-index-to-flat-index`,
runner column resolution) all carry `subFieldName` through.
- For composite parents, the runner uses
`computeCompositeColumnName({...}, property)` for the picked sub-column;
for non-composite parents, behavior is unchanged.
- The `'::'` separator encodes `(fieldMetadataId, subFieldName)` for
dedup on the wire; the frontend uses the same separator inside the
Select component's string value.
### Apps can declare indexes in code (`defineIndex`)
- New `IndexManifest` + `IndexFieldManifest` types in
`twenty-shared/application` wired into the `Manifest` type.
- `defineIndex` SDK helper + `IndexConfig`. CLI manifest builder +
extractor recognize `defineIndex` / `ManifestEntityKey.Indexes`.
- Server: `from-index-manifest-to-universal-flat-index` converter
resolves field IDs, validates composite/scalar `subFieldName` rules, and
delegates to `generateFlatIndexMetadataWithNameOrThrow` for the
deterministic name.
- Orchestrator wires the loop after the field-resolution pass;
per-object cap enforced inline against the manifest.
- Cascade on uninstall is automatic — when an app disappears its indexes
drop with it (universal-flat-entity diff handles it).
- Rich-app fixture ships a real `defineIndex` on `PostCard.status`,
exercising the full manifest → install path in CI.
### Closed for now (open later if needed)
- Apps cannot declare `isUnique` indexes — unique constraints stay with
the field-creation flow.
- Apps cannot use a partial-`indexWhereClause` — the UI surface keeps
the framework's hardcoded allowlist.
- UI cannot create unique or partial indexes either; same reasons.
### Cleanups along the way
- Reused the existing `getCompositeSubFieldLabel` +
`COMPOSITE_FIELD_SUB_FIELD_LABELS` (deleted the duplicates I'd created
early in the PR).
- Moved `MAX_CUSTOM_INDEXES_PER_OBJECT` to `twenty-shared/constants`
(single source for FE + BE).
- Replaced inline `isDefined(x) && x !== ''` with `isNonEmptyString`
(from `@sniptt/guards`).
- Hoisted the per-object fields Map + inlined the cap counter into the
indexes orchestrator loop (drops the install scan from O(indexes ×
totalFields) to O(totalFields + indexes)).
- Per design-feedback: page-based create flow (not a modal), filter
dropdown on the SearchInput (not a separate toggle), webhook-style
picker, field icons.
### Unrelated polish that lands here
- "Invite user" link in the multi-workspace dropdown now lands on the
Invite tab directly (`#invite`) instead of the first tab of the members
page.
## Test plan
- [ ] `npx nx typecheck twenty-server / twenty-front / twenty-sdk /
twenty-shared` — passes
- [ ] `npx nx lint:diff-with-main twenty-server / twenty-front` — clean
- [ ] `npx jest index-metadata.service.spec` — green
- [ ] `npx jest from-index-manifest-to-universal-flat-index` — green
(new converter spec, 8 cases)
- [ ] `npx vitest run
src/sdk/define/indexes/__tests__/define-index.spec.ts` (twenty-sdk) —
green (6 cases)
- [ ] `npx vitest run --config vitest.integration.config.ts -t
"rich-app"` — green (rich-app app-dev integration exercises the new
manifest path with the PostCard.status index)
- [ ] Advanced mode → Settings → any object → Settings tab → Indexes
section is visible under Search
- [ ] Create a single-field BTREE index, confirm SQL index exists
(verify via `pg_indexes`)
- [ ] Create a composite-field index (`Address > City`) and confirm the
column is `addressAddressCity`
- [ ] Create an index spanning two columns; column order matches the
picker order
- [ ] Attempt to create an 11th custom index → button is disabled with
tooltip
- [ ] Delete a custom index → confirmation modal → row disappears, PG
index dropped
- [ ] System indexes have no trash icon and are hidden by default
|
||
|
Marie
|
85d649e831 |
[Fix] Backfill missing command menu items conditional availability expression (#20852)
## Description Following [report in discord](https://discord.com/channels/1130383047699738754/1498690477044793386/1506602927412744242) Some command menu items were showing to all users because they had no conditional availability expression, whereas users did not actually have access to the page or feature behind. For instance: "Go to Admin panel", "Go to AI settings", "Send email" etc. <img width="833" height="1245" alt="image" src="https://github.com/user-attachments/assets/8d2a9404-9b81-4d58-9522-558e9924c457" /> ## Fix - Add conditional availability expressions - Backfill expressions for existing workspaces as they are stored in db (commandMenuItems table) --------- Co-authored-by: Charles Bochet <charles@twenty.com> |
||
|
Paul Rastoin
|
76e144e85a |
Deprecate messageChannel messageFolder calendarChannel standard objects (#20836)
# Introduction Removing old standard objects `messageChannel` and `messageFolder` and `calendarChannel` --------- Co-authored-by: Cursor <cursoragent@cursor.com> |
||
|
Félix Malfait
|
084fa8eaba |
fix(server): auto-index target<X>Id join columns on polymorphic standard objects (#20820)
Closes #20726 ## The bug `timelineActivity` (and the three other polymorphic standard objects — `attachment`, `noteTarget`, `taskTarget`) store relations as N nullable `target<X>Id` columns, one per related object. Each one is a join key queried as `WHERE target<X>Id IN (...) AND deletedAt IS NULL`. For **built-in** related objects (Person, Company, Opportunity, …), each `target<X>Id` column gets a BTREE index, declared statically in `compute-{timelineActivity,attachment,noteTarget,taskTarget}-standard-flat-index-metadata.util.ts`. For **custom** related objects, the same `target<CustomObject>Id` column was added — **without an index**. On a `timelineActivity` table at issue-reporter scale (~21.9M rows, 7.1 GB), this turned record loads into 20–40s sequential scans and produced `QueryFailedError: Query read timeout` for end users. ## Diagnosis The morph/relation field generator (`generateMorphOrRelationFlatFieldMetadataPair`) already creates a BTREE index for the field that owns the join column and returns it alongside the field metadata pair. The two user-driven entry points (`fromRelationCreateFieldInput…`, `fromMorphRelationCreateFieldInput…`) correctly destructure and propagate that index. But the **custom-object creation path** — `buildDefaultRelationFlatFieldMetadatasForCustomObject`, called when a user creates a new custom object — destructured only `{ flatFieldMetadatas }` and threw away `indexMetadatas`. So every `target<CustomObject>Id` column added to the four polymorphic standard objects has been shipping unindexed since custom morph relations went in. ## The fix Three commits. ### 1. `fix(server): index target<CustomObject>Id columns on standard polymorphic objects` 13 lines across 2 files. - `build-default-relation-flat-field-metadatas-for-custom-object.util.ts` — also destructure `indexMetadatas` from the pair generator and accumulate them into the returned record (new field `standardTargetFlatIndexMetadatas`). - `from-create-object-input-to-flat-object-metadata-and-flat-field-metadatas-to-create.util.ts` — append the accumulated indexes to `flatIndexMetadataToCreate`. The migration pipeline at `object-metadata.service.ts:559–562` already passes `flatIndexMetadataToCreate` to the migration runner, so no further wiring is needed. From now on, creating a custom object also creates the four BTREE indexes — one per polymorphic standard object's new `target<CustomObject>Id` column — atomically with the rest of the migration. ### 2. `feat(server): backfill workspace command for relation join column indexes` For existing workspaces whose custom objects were created before the forward-fix. `upgrade:2-8:backfill-relation-join-column-indexes` is a `@RegisteredWorkspaceCommand('2.8.0', 1798100000000)` matching the pattern from `2-7-workspace-command-…-drop-connected-account-standard-object.command.ts`. Per workspace: 1. Load `flatObjectMetadataMaps`, `flatFieldMetadataMaps`, `flatIndexMaps` from the workspace cache. 2. Resolve the four polymorphic standard object IDs by `nameSingular` against `DEFAULT_RELATIONS_OBJECTS_STANDARD_IDS`. 3. Collect every field ID that's already covered by any existing index. 4. Filter `flatFieldMetadataMaps` to MORPH_RELATION fields on those four objects whose `settings.relationType === MANY_TO_ONE` (i.e. owns a join column) and whose ID isn't in the indexed set. 5. Generate a BTREE `UniversalFlatIndexMetadata` for each via `generateIndexForFlatFieldMetadata` (same helper the forward-fix uses). 6. Create the indexes in the workspace schema with **CONCURRENTLY** (see commit 3). 7. Submit the metadata through `WorkspaceMigrationValidateBuildAndRunService` so it lands in `indexMetadata` and the cache — same pipeline as a normal metadata change. The pipeline's own `CREATE INDEX IF NOT EXISTS` no-ops because the index already exists. Properties: - **Idempotent.** Re-running is a no-op once indexes exist. - **Scoped.** Only the four polymorphic standard objects, only their MANY_TO_ONE morph relation fields, only those with no covering index. - **Same code path as the forward-fix.** The backfill produces exactly the indexes the forward-fix would have created at custom-object creation time. - **`--dry-run` supported** via the base `ActiveOrSuspendedWorkspaceCommandRunner`. ### 3. `feat(server): create index CONCURRENTLY in relation join column backfill` Adds an opt-in `concurrently` flag to `WorkspaceSchemaIndexManagerService.createIndex` (threaded through `createIndexInWorkspaceSchema`). When `true`, emits `CREATE INDEX CONCURRENTLY IF NOT EXISTS …`. Defaults to `false` — every existing caller keeps the current transactional `CREATE INDEX` behavior. The backfill command opts in. It creates a QueryRunner **without** `startTransaction()`, issues the CONCURRENTLY indexes one-by-one (each waits for the previous to finish), then submits the metadata through the normal migration pipeline whose own `CREATE INDEX IF NOT EXISTS` is now a no-op. Why not flip the default for the helper: - `CREATE INDEX CONCURRENTLY` cannot run inside a transaction — Postgres errors out. The migration pipeline calls `createIndex` from inside a transactional schema migration. - CONCURRENTLY doesn't roll back with the transaction. If the surrounding migration fails, the index remains and you end up with metadata/schema drift. - Failed CONCURRENTLY builds leave an INVALID index behind that needs manual `DROP`. - UNIQUE indexes have different failure semantics under CONCURRENTLY (deferred, not immediate). So CONCURRENTLY is opt-in, used only where it's the right tool (post-hoc backfills on populated tables). ## Decisions / tradeoffs - **Single-column BTREE vs partial `WHERE deletedAt IS NULL` vs composite.** Twenty's queries always include `deletedAt IS NULL`. A partial index would be slightly better than a plain BTREE (smaller, no wasted seeks on soft-deleted rows). This PR ships single-column to match the existing built-in target index pattern, which already covers >95% of the available speedup (the 20s→4ms drop the reporter saw comes from having any index — composite/partial is a second-order effect). Switching all relation indexes to partial is a separate, broader change. - **CONCURRENTLY operator caveat.** If a CONCURRENTLY build is interrupted (kill, connection drop, OOM), Postgres leaves the index as INVALID. We deliberately don't probe `pg_index` for invalid leftovers on every create — catalog-table queries can be slow at multi-tenant scale and the failure mode is rare. Recovery is manual: `DROP INDEX <name>` and re-run the backfill. - **Forward-fix is not gated** behind a feature flag. The change is metadata-pipeline-internal; before, custom-object creation silently produced a degraded state. After, it produces the correct state. No new public API, no behavioural change for end users besides the indexes existing. ## Risk - Forward-fix: changes only the metadata produced during custom-object creation. New objects get four extra `FlatIndexMetadata` rows and four extra `CREATE INDEX` statements during their creation migration. Tables are empty at that point so the index builds in microseconds. - Helper change: API-compatible, default behavior unchanged. The new `concurrently` parameter is optional. - Backfill: read-only state probe → CONCURRENTLY index creation (no write blocking) → metadata insert via the normal migration pipeline. Idempotent. Reverting is `DROP INDEX`. ## Test plan - [ ] Verify forward-fix: create a custom object, confirm four new BTREE indexes appear on `timelineActivity`, `attachment`, `noteTarget`, `taskTarget` for the new `target<CustomObject>Id` columns, and that `flatIndexMaps` has matching entries. - [ ] Verify backfill on a workspace that had custom objects created before the fix: run `--dry-run` first, confirm the expected indexes are listed; then run for real, confirm the indexes appear in pg (and as `indisvalid = true` in `pg_index`) and in `flatIndexMaps`. Re-run; confirm no-op. - [ ] Verify backfill on a clean workspace: should log "no missing indexes" and exit. - [ ] Verify CONCURRENTLY behavior under load: run backfill against a workspace with active writes on `timelineActivity`; confirm inserts/updates keep working during index build (no `ShareLock` waits in `pg_stat_activity`). - [ ] On the affected reporter-scale workspace, confirm `EXPLAIN ANALYZE` switches from sequential scan to index scan and timeline activity timeouts go away. |
||
|
neo773
|
323e66433e |
lint: migrate prettier to oxfmt (#20783)
Most changes are `implements` being unwrapped this is not a oxfmt regression Prettier in 3.7 (we're on 3.1) changed this behaviour prettier blog [post](https://prettier.io/blog/2025/11/27/3.7.0#change-18094) This unifies our linting tooling --------- Co-authored-by: github-actions <github-actions@twenty.com> Co-authored-by: Charles Bochet <charles@twenty.com> |
||
|
Paul Rastoin
|
abf5902ab5 | Connected account deprecation system build (#20810) | ||
|
Weiko
|
d13cc7c349 |
Drop legacy rolePermissionFlag.flag column + fallback logic (#20730)
## Summary - **New fast migration** `2-7-instance-command-fast-1779600000000-finalize-role-permission-flag-cutover.ts`: - `DROP CONSTRAINT IDX_ROLE_PERMISSION_FLAG_FLAG_ROLE_ID_UNIQUE` - `ALTER COLUMN permissionFlagId SET NOT NULL` - `DROP COLUMN flag` - `down()` repopulates `flag` from the catalog via `permissionFlagId` and restores the old unique. - **Entity**: `RolePermissionFlagEntity` hides the `flag` column by using the new decorator + drops old `@Unique` decorator; `permissionFlagId` and the `permissionFlag` relation become non-nullable. - **Deletes** the synthesizer `synthesize-flat-permission-flag-from-flag.util.ts` and every fallback branch that used it (`from-role-permission-flag-entity-to-flat-role-permission-flag.util.ts`, `from-flat-role-permission-flag-to-role-permission-flag-dto.util.ts`, `permissions.service.ts`, `workspace-roles-permissions-cache.service.ts`, `fromRoleEntityToRoleDto.util.ts`, `flat-role-permission-flag-validator.service.ts`, `role-permission-flag.service.ts:getEffectiveUniversalIdentifier`). - **Write path**: ~~drops `flag` from `CreateRolePermissionFlagInput`, the create util, and the application-manifest converter.~~ - **Metadata configs**: ~~removes `flag` from `all-entity-properties-configuration-by-metadata-name.constant.ts` (rolePermissionFlag block)~~ and flips `permissionFlag.isNullable` to `false` in `all-many-to-one-metadata-relations.constant.ts`. ### Why the `flag` field stays declared in the entity The decorator (`@WasRemovedInUpgrade`) is the right tool for the lifecycle marker, but it's a **reflect-metadata** runtime decorator — TypeScript can't see it at compile time. So while the adapter ([`UpgradeAwareEntityMetadataAdapter`](packages/twenty-server/src/engine/twenty-orm/upgrade-aware/upgrade-aware-entity-metadata.adapter.ts)) now correctly flips `isSelect`/`isInsert`/`isUpdate` to `false` once the drop migration's cursor is crossed, the *static* TypeScript types derived from `RolePermissionFlagEntity` (`UniversalFlatRolePermissionFlag`, `FlatRolePermissionFlag`, `MetadataEntityPropertyConfiguration<'rolePermissionFlag'>`, etc.) still see `flag` as a required scalar property — because the entity declares it. That means every producer of one of those derived types must include `flag`: - `from-create-role-permission-flag-input-to-flat-role-permission-flag-to-create.util.ts` plumbs it through. - `from-permission-flag-to-universal-flat-role-permission-flag.util.ts` (the application-manifest converter) sets `flag: permissionFlag.flag`. - `all-entity-properties-configuration-by-metadata-name.constant.ts` has a `flag` entry under `rolePermissionFlag`. - `CreateRolePermissionFlagInput` keeps the `flag` field. - `RolePermissionFlagService.upsertPermissionFlags` passes `flag: permissionFlag.key as PermissionFlagType` to the create util. Explored phantom-brand approach (`RemovedInUpgrade<T>` wrapper on the field type, key-filter mapped type applied inside `ScalarFlatEntity` / `UniversalFlatEntityFrom`) but previous commands could have `flag === undefined` (downcast from the brand since we can't compare with UpgradeMigrationName like we do with a decorator). That's a **silent-read** failure mode: compiles fine, comparisons against `flag` silently always-false, no error surfaces. Probably worth too much risk for what's a small amount of plumbing? The eventual full deletion of `flag` (entity field included) is a future cleanup once we drop cross-upgrade support for versions ≤ 2.6 Note: Not sure if this PR (and even the decorator) is really needed in the end, seems we need to keep a lot of code in place to handle legacy. Maybe a simple noop [At]Deprecated is enough @charlesBochet (and a migration to set the column nullable if that was not the case before + remove associated constraints) |
||
|
Paul Rastoin
|
9b9c97a049 |
Deprecate and backfill delete ConnectedAccount twenty standard object (#20752)
# Introduction Following connected account permissions refactor and encryption Removing the old workspace schema twenty standard application connectedAccount objects and related standard fields and index - a lot of deadcode - instance command backfill cleaning the connected account object from workspaces |
||
|
Félix Malfait
|
f3aadbbb66 |
chore(server): drop leftover favorite and favoriteFolder workspace objects (#20744)
## Summary - Adds a 2.7.0 workspace upgrade command `upgrade:2-7:drop-favorite-objects` that removes the legacy `favorite` and `favoriteFolder` object metadata (and their workspace tables) from every active or suspended workspace. - The records were migrated to `navigationMenuItem` in the 1.17/1.18 upgrades and the entity code was deleted in #19536, but the per-workspace metadata rows were never cleaned up — so they still surface in the "Existing objects" settings list and expose stale CRUD tools to the AI/MCP layer (e.g. the model can hallucinate `create_favorite_folder` against a real-looking schema). ## Implementation notes - Modeled on [`upgrade:2-3:drop-message-direction-field`](https://github.com/twentyhq/twenty/blob/main/packages/twenty-server/src/database/commands/upgrade-version-command/2-3/2-3-workspace-command-1777400000000-drop-message-direction-field.command.ts), but at object granularity. - Uses `ObjectMetadataService.deleteOneObject({ isSystemBuild: true })` so all cascading is handled by the existing pipeline: field metadata, indexes, relation fields on other workspace entities, command menu items, and the workspace data tables. Views and orphaned `navigationMenuItem` rows pointing at favorite views are removed by the existing `onDelete: 'CASCADE'` FKs. - Deletion order: `favorite` first (holds a relation to `favoriteFolder`), then `favoriteFolder`. - Both objects are flagged `isSystem: true`, hence `isSystemBuild: true` on the call. - Idempotent: workspaces where the object is already absent are logged and skipped. - Honors `--dry-run`. - Universal identifiers are hard-coded because the matching `STANDARD_OBJECTS` entries were deleted in #19536. ## Test plan - [ ] Run on a workspace that still has `favorite` / `favoriteFolder` in `core.objectMetadata` (verify in prod-like DB beforehand) and confirm both objects, their fields, indexes, relation fields on linked objects, views, and the workspace data tables are gone after running. - [ ] Re-run on the same workspace — confirm it logs "already absent" and exits clean (idempotency). - [ ] Run on a workspace where the objects don't exist (e.g. fresh local) — confirm clean no-op. - [ ] Run with \`--dry-run\` first — confirm log output and no DB mutations. - [ ] Confirm the "Existing objects" settings page no longer lists Favorites / Favorite Folders after the migration. ## Safety check before rollout Before running in prod, verify no workspace has live (non-soft-deleted) favorite data that didn't make it to \`navigationMenuItem\`: \`\`\`sql -- Per workspace SELECT count(*) FROM workspace_xxx.favorite WHERE "deletedAt" IS NULL; \`\`\` Should be ~0 in workspaces that ran the 1.17 / 1.18 migrations. --------- Co-authored-by: prastoin <paul@twenty.com> |
||
|
nitin
|
83b10ad698 |
fix(server): sync command menu item availability expressions on existing workspaces (#20719)
Two fixes via one workspace command: 1. Gates 5 standard command menu items behind `pageType == "INDEX_PAGE"` -- `importRecords`, `exportView`, `seeDeletedRecords`, `createNewView`, `hideDeletedRecords`. They currently appear (and crash or do nothing) on RECORD_PAGE. 2. Fixes Edit Layout missing from older workspaces -- root cause is `conditionalAvailabilityExpression` drift between source-of-truth constants and the workspace DB (e.g. #20556 removed a feature flag from the expression without syncing existing workspaces). The 2-6 workspace command iterates all `STANDARD_COMMAND_MENU_ITEMS` and reconciles any `conditionalAvailabilityExpression` that differs from the constant. Idempotent -- already-correct rows are skipped. Deferred: `deleteRecords` doesn't refetch the current record after deletion on RECORD_PAGE (mutation fires but UI shows stale state until refresh) -- different fix shape (frontend handler), separate PR. |
||
|
Paul Rastoin
|
57f13c9b92 |
[CONNECTED_ACCOUNT_BREAKING_CHANGE] Encrypt ConnectedAccount connectionParameters (#20673)
# Introduction Prevent any cross user `connectedAccount` `connectionParamaters` leak Also encrypt in db all `connectionParameters` password Never return any password through `DTO` anymore The settings now allow update mutation without providing the password in edition mode Verified all `connectionParameters.password` interaction ## Integration tests - Added more coverage for both failing and successful paths - Introduced a new env var that allow bypass the provider connection test ## Legacy connected Account decryption support Stop allowing non encrypted decryption on `accessToken` and `refreshToken`, only allow legacy decryption on refactored `connectionParameters` ## Upsert ownership Completely got rid of the connected workspace schema context which is legacy Also now a user can only upsert a connected account for him only.. ## New UI <img width="1770" height="1852" alt="image" src="https://github.com/user-attachments/assets/55c1dc89-42ff-4084-95e2-cc5f9e23753b" /> If in edition the password is by default disabled It needs to be selected as being edited to be enabled ## Next - Refactor tool permissions flag not to include connected accounts - Remove the legacy connected standard object - Refactor and improve connected account resolver auth |
||
|
Charles Bochet
|
77514ad14a |
fix(server): backport relationTargetFieldMetadataId column-add to 2.4 and 2.5 fast instance (#20721)
## Summary Cross-version upgrade from a **v2.3 or v2.4 baseline** to v2.6.x currently fails at the 2.5 workspace command `NormalizeCompositeFieldDefaults`: ``` [QueryFailedError] column ViewFilterEntity.relationTargetFieldMetadataId does not exist at WorkspaceFlatViewFilterMapCacheService.computeForCache ``` Reproduced locally via Docker cross-version upgrade (v2.6.1 against `twentycrm/twenty:v2.3` and `:v2.4` images on a freshly-seeded DB). ### Root cause The column-add is already declared in two places: - `2-3/.../1747234300000-add-relation-target-field-metadata-id-to-view-filter` (backport from #20664) - `2-6/.../1798000005000-add-relation-target-field-metadata-id-to-view-filter` But the runner's `resolveStartCursor` (`upgrade-sequence-runner.service.ts`) advances forward from `lastAttemptedCommandName` and never re-runs commands inserted *behind* the cursor: - **fresh install through 1.23 → 2.6.x**: cursor < 2.3 → 2.3 backport runs → column added before 2.5 workspace ✓ - **v2.3 baseline → 2.6.x**: cursor past 2.3 → 2.3 backport skipped → 2.5 workspace `NormalizeCompositeFieldDefaults` crashes ✗ - **v2.4 baseline → 2.6.x**: cursor past 2.4 → 2.3 backport skipped → same crash ✗ - **v2.5 baseline → 2.6.x**: cursor past 2.5 → 2.5 workspace already applied (ran against v2.5 source's older entity without the column) → 2.6 fast adds the column ✓ The 2.6 fast `1798000005000` runs *after* the 2.5 workspace command, too late to help v2.3 / v2.4 baselines. ### Fix Mirror the existing 2.3 Early backport at two more versions: - `2-4/.../1747234400000-add-relation-target-field-metadata-id-to-view-filter` — covers v2.3 baseline (runs in 2.4 fast, before any 2.4/2.5 workspace command) - `2-5/.../1747234500000-add-relation-target-field-metadata-id-to-view-filter` — covers v2.4 baseline (runs in 2.5 fast, before `NormalizeCompositeFieldDefaults`) Both use `ADD COLUMN IF NOT EXISTS` (idempotent) and `DROP COLUMN IF EXISTS` for the down. No FK / index — those still live in the 2.6 file, which runs as a no-op for the column on already-fixed DBs. Pre-2.6 codebases can't use `@WasIntroducedInUpgrade` (#20686 only lands in 2.6), so this "ladder of backports" remains the operative pattern. ## Audit context Locally walked `v1.23 / v2.0 / v2.1 / v2.2 / v2.3 / v2.4 / v2.5 → v2.6.1`: | Baseline | Result | |---|---| | v1.23 | PASS | | v2.0 | PASS | | v2.1 | PASS | | v2.2 | PASS | | **v2.3** | **FAIL** (this PR) | | **v2.4** | **FAIL** (this PR) | | v2.5 | PASS | |
||
|
Charles Bochet
|
bad1f20012 |
fix(server): handle legacy PK name in 2.6 rename-permission-flag upgrade (#20697)
## Summary The 2.6 `RenamePermissionFlagToRolePermissionFlag` upgrade command failed on staging and dev with: ``` [QueryFailedError] constraint "PK_a02789db60620a1e9f90147b50f" for table "rolePermissionFlag" does not exist in RenamePermissionFlagToRolePermissionFlag1778235340020 (2.6.0) (instance fast) ``` ### Root cause TypeORM names PKs as `PK_<sha1(tableName_sortedColumnNames)[:27]>`. So: - `permissionFlag_id` → `PK_a02789db60620a1e9f90147b50f` - `settingPermission_id` → `PK_8c144a021030d7e3326835a04c8` - `rolePermissionFlag_id` → `PK_76591adc8035c2e7b0cd6115136` On databases initially migrated before the v1.5.5 migration squash (#15183), the table was renamed `settingPermission` → `permissionFlag` via the pre-squash migration `1753149175945-renameSettingPermissionToPermissionFlag.ts`. That migration renamed the table, the column, the unique index, and the role FK, but **never renamed the PK constraint** — and Postgres does not auto-rename constraints on `ALTER TABLE ... RENAME TO`. Those instances therefore still carry the legacy PK name `PK_8c144a021030d7e3326835a04c8`. Fresh installs (squashed `setupMetadataTables` migration) instead have the expected `PK_a02789db60620a1e9f90147b50f`. The 2.6 upgrade only handled the fresh-install name, so it broke for any DB that went through the historical rename chain. ### Fix Replace the brittle `RENAME CONSTRAINT` with `DROP CONSTRAINT IF EXISTS` for both historical PK names, followed by `ADD CONSTRAINT ... PRIMARY KEY ("id")` with the canonical new name. The migration now converges to the same PK name regardless of the DB's history. The same pattern is applied symmetrically in `down()`. ### Why this is safe - The whole instance command runs in a transaction (`InstanceCommandRunnerService.runFastInstanceCommand`). - The first statement (`ALTER TABLE ... RENAME TO`) takes `ACCESS EXCLUSIVE` on the table, so the drop/add window for the PK is invisible to any concurrent writer — they queue on the lock until commit. - No FK references `rolePermissionFlag.id` at this point in the sequence (migration 22 introduces an FK pointing at the new `permissionFlag` catalog created in migration 21, not at the renamed grant table), so dropping the PK does not cascade or block. - `NOT NULL` and the `uuid_generate_v4()` default on `id` are column-level and remain in place when the PK is dropped. ## Test plan - [ ] Run 2.6 upgrade against a fresh-install database (PK = `PK_a02789db60620a1e9f90147b50f`) — should succeed. - [ ] Run 2.6 upgrade against a pre-squash database (PK = `PK_8c144a021030d7e3326835a04c8`, reproducible on current staging/dev) — should now succeed. - [ ] Verify post-migration: `rolePermissionFlag` exists, PK is named `PK_76591adc8035c2e7b0cd6115136`, all FKs and indexes named as expected. - [ ] Run `down()` and verify table returns to `permissionFlag` with PK `PK_a02789db60620a1e9f90147b50f`. - [ ] Subsequent migrations (`1778235340021` permission-flag catalog, `1778235340022` link, `1778235340023` backfill) still apply cleanly. |
||
|
Charles Bochet
|
d03480472c |
perf(server): index messageChannel/calendarChannel for per-workspace sync crons (#20678)
## Summary The messaging/calendar import crons each iterate every active workspace and execute one `find` per workspace against `core."messageChannel"` / `core."calendarChannel"` with the shape: ``` WHERE "workspaceId" = $1 AND "isSyncEnabled" = true AND "syncStage" = $2 [AND "type" <> $3] ``` There is currently no index supporting that shape, so the planner does a seq scan on each table for every iteration. On prod-eu (RDS Performance Insights, `rds-prod-eu-one`), these two queries are the top two by load — together ~12 AAS, ~12 calls/sec — and have been the primary contributor to the sustained 100% CPU since active workspace count grew. This PR adds composite indexes on `(workspaceId, isSyncEnabled, syncStage)` for both tables as an instance migration in 2.6.0. |
||
|
nitin
|
db0547f503 |
[1/3] Rename permissionFlag to rolePermissionFlag + add permissionFlag catalog/backfill (#20481)
Split of #20377. ## Summary This PR separates available permission flags from per-role permission flag grants. Previously, `core.permissionFlag` stored the role assignment directly: `roleId + flag`. This PR renames that legacy grant table to `core.rolePermissionFlag`, then recreates `core.permissionFlag` as the catalog of available permission flags. ## What changed - Rename the existing `core.permissionFlag` grant table to `core.rolePermissionFlag`. - Add the new syncable `core.permissionFlag` catalog entity with key, label, description, icon, permission type, relevance flags, and custom/standard metadata. - Add stable `SystemPermissionFlag` universal identifiers for the built-in `PermissionFlagType` values. - Seed the standard permission flags for every workspace under the Twenty standard application. - Backfill existing role grants: - create missing catalog rows for existing grant keys, - add `rolePermissionFlag.permissionFlagId`, - migrate grants from the old string `flag` column to the new catalog FK, - replace the old `(flag, roleId)` uniqueness with `(permissionFlagId, roleId)`. - Rewire role permission flag caches, permission checks, role DTO mapping, and `upsertPermissionFlags` to resolve through the catalog. - Keep the existing public role permission API shape: product/app surfaces still talk about `permissionFlags` and return `{ id, roleId, flag }`. - Update metadata flat-entity machinery, migration builders, validators, action handlers, snapshots, generated schemas, docs, and app fixtures for the new `permissionFlag` / `rolePermissionFlag` split. ## Behavior after this PR - Existing permission flag grants keep working. - Existing GraphQL role permission flows keep the same public naming. - Standard permission flags are represented as catalog rows. - Permission checks now compare grants through catalog universal identifiers instead of the legacy `flag` column. - Workspace deletion cleanup now verifies both `permissionFlag` and `rolePermissionFlag`. ## What is not in this PR - Public GraphQL CRUD for custom permission flags. - App manifest support for declaring new custom permission flags. - Frontend UI for creating or assigning custom permission flags beyond the existing role permission flow. --------- Co-authored-by: Weiko <corentin@twenty.com> |
||
|
Charles Bochet
|
6b3064e2ba |
fix(server): add relationTargetFieldMetadataId column early in upgrade sequence (#20664)
## Summary Cross-version upgrade fails at the 2.3 `DropMessageDirectionFieldCommand` stage: ``` [QueryFailedError] column ViewFilterEntity.relationTargetFieldMetadataId does not exist at WorkspaceFlatViewFilterMapCacheService.computeForCache ``` (see https://github.com/twentyhq/twenty-infra/actions/runs/25929264129/job/76219964380) Same shape as #20584 (subFieldName), one column over. ### Root cause 1. The 2.3 `DropMessageDirectionFieldCommand` builds a workspace migration that deletes a `fieldMetadata` (the `direction` field). 2. `WorkspaceMigrationRunnerService.run` walks the metadata cascade graph and pulls `viewFilter` into the dependency set because `viewFilter` is the inverse one-to-many of `fieldMetadata`. 3. That maps to cache keys → `flatViewFilterMaps` gets requested → `WorkspaceFlatViewFilterMapCacheService.computeForCache` runs. 4. `computeForCache` does `viewFilterRepository.find({ where: { workspaceId }, withDeleted: true })` with no `select`, so TypeORM emits a SELECT that includes `relationTargetFieldMetadataId` — column only added by the 2.6 fast instance command `1798000005000`, not yet run at the 2.3 stage. 💥 ### Why v2.5.0 / v2.5.1 passed They didn't include #20527 (one-hop relation filters, May 14), which added `relationTargetFieldMetadataId` to `ViewFilterEntity` and the 2.6 instance command. The CI base image (v1.22) seeded the DB, then the v2.5.0/v2.5.1 container ran upgrade commands against an entity that didn't yet know about this column. |
||
|
Félix Malfait
|
c938fbf4d6 |
feat(twenty-front): relation traversal in filter dropdown (stacked) (#20533)
**Stacked on #20527** https://github.com/user-attachments/assets/48995655-401a-4c35-8094-e88da8408bdd ## Summary Surfaces the one-hop relation traversal added in #20527 through the existing **composite sub-field dropdown pattern**. Clicking a MANY_TO_ONE relation field in the "+ Filter" picker now opens the same second-level dropdown that composite fields (FULL_NAME, ADDRESS, CURRENCY, etc.) already use — populated with the target object's filterable fields. Picking one (e.g. `Company → Name`) builds a filter that serializes to the nested GraphQL filter the backend now accepts: `{ company: { name: { ilike: "%X%" } } }`. No new components. The whole feature reuses `AdvancedFilterSubFieldSelectMenu` + the existing `subFieldNameUsedInDropdownComponentState` + the existing `MenuItem hasSubMenu` indicator. Only the conditions that gate the sub-menu (and the sub-menu's content for relations) were broadened. ## What landed | File | Change | |---|---| | `ObjectFilterDropdownFilterSelectMenuItem` | Sub-menu chevron now shows on MANY_TO_ONE relations (`isManyToOneRelationField` util). | | `AdvancedFilterFieldSelectMenu` | Relation clicks open the sub-menu alongside composite clicks. | | `AdvancedFilterSubFieldSelectMenu` | New branch: when the sub-menu type is `'RELATION'`, render the target object's filterable fields via `useFilterableFieldMetadataItems(targetObjectMetadataId)`. Composite logic untouched. | | `objectFilterDropdownSubMenuFieldType` state | Widened to accept a `'RELATION'` sentinel. Role-permissions sub-field menu narrows it back out (it doesn't traverse relations). | | `useSelectFieldUsedInAdvancedFilterDropdown` | New optional `targetFieldMetadataItem` arg. When present, the stored RecordFilter's `type` is the target field's type so the operand picker and value input render the target's operands (`'TEXT'` operators when filtering `company.name`, etc.). | | `turnRecordFilterIntoGqlOperationFilter` (shared) | When the filter targets a `RELATION` field with a `subFieldName`, synthesize a field-metadata for the target, recurse to build the inner filter, then wrap it under the relation field's name → `{ relationName: { targetFieldName: { ...operator } } }`. | `RecordFilter.subFieldName` stays narrowly typed as `CompositeFieldSubFieldName` so the wide downstream consumers (`shouldShowFilterTextInput`, composite handlers in the serializer, etc.) don't change. The relation target field's name is stored through a narrowly-scoped cast at the dropdown's storage point — the serializer checks `filter.type === 'RELATION'` before interpreting it as a target field name, so the cast can't be mis-read by composite-only code paths. ## Test plan - [ ] Open a table view on People, click "+ Filter", click "Company" → sub-menu opens with Company's filterable fields - [ ] Pick "Name" → operand picker shows TEXT operators (Contains, Equals, …) - [ ] Type "Airbnb" → filter applies, table shows people whose company name contains "Airbnb" - [ ] Verify network tab: the GraphQL filter variable is `{ company: { name: { ilike: "%Airbnb%" } } }` - [ ] Same flow with a composite target field (e.g. `Company → annualRecurringRevenue → amountMicros`) — should work end-to-end (backend supports composite-within-relation; #20527 has an integration test covering this) - [ ] Composite fields (FULL_NAME, ADDRESS) still open their normal sub-menu and filter correctly — no regression - [ ] Role-permissions field-select sub-field menu is unaffected (it bails out early on the RELATION sentinel) ## Out of scope - ONE_TO_MANY traversal (no backend support yet) - Aggregates (`people.count > 5`) - Persisting relation-traversal filters into a saved view (ViewFilter has no `relationPath` column yet; that's a separate slice) - REST API DSL changes - AI Tools 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> |
||
|
Charles Bochet
|
eca92ca559 |
fix(server): rebuild unique phone indexes drops legacy non-empty partial WHERE clause (#20606)
## Summary `RebuildUniquePhoneIndexesCommand` reuses each index's existing `indexWhereClause` when recreating the physical index. For workspaces whose unique phone indexes have a legacy clause like `"primaryPhoneNumber" != ''` (created before PR #18024 hardened the validator allowlist), the recreate path fails at `validateAndReturnIndexWhereClause` because the clause isn't in `ALLOWED_INDEX_WHERE_CLAUSES`. Two workspaces are hitting this on the 2.5 upgrade: - `3a797122-…` — `"companyPhonePrimaryPhoneNumber" != ''` - `ea74716f-…` — `"phonesPrimaryPhoneNumber" != ''` ## Fix Detect the legacy `"<col>" != ''` shape via a strict regex. When it's there, before the existing drop+create, do three things inside the workspace transaction: 1. **Normalize the data** that the legacy partial clause was masking — `UPDATE "<schema>"."<table>" SET "<col>" = NULL WHERE "<col>" = ''` for every column the index covers. Without this the next step would fail because the new plain-unique index would see duplicate `''` values across the rows the old partial clause was excluding. 2. **Null out `core."indexMetadata".indexWhereClause`** so the metadata row matches what the UI would have created (`indexWhereClause: null`) and doesn't carry the validator-rejected clause forward to any future re-emit. Uses the same workspace `queryRunner` (Postgres lets one connection write across schemas). 3. **Recreate** with an overridden flat index where `indexWhereClause: null`. `createIndexInWorkspaceSchema` → `indexManager.createIndex` → `validateAndReturnIndexWhereClause` short-circuits on null, no allowlist check. End state matches the shape a fresh "toggle unique in Settings UI" creates: plain unique index, no `WHERE`, NULL semantics doing the "exclude empty phones" work via PG's default NULL-distinct behaviour. For indexes whose clause is already allowlisted (`"deletedAt" IS NULL`) or null, behaviour is unchanged — just the column-list widening this command already does. |
||
|
Charles Bochet
|
ca1571676c |
fix(server): treat plaintext-under-isSecret rows as plaintext in app variable encryption migration (#20590)
## Summary Prod 2.5 upgrade failed on the slow instance command `EncryptApplicationVariableSlowInstanceCommand`: ``` [Nest] LOG [InstanceCommandRunnerService] 2.5.0_EncryptApplicationVariableSlowInstanceCommand_1798000005000 starting data migration... [Nest] WARN [SecretEncryptionService] Decrypted a legacy unprefixed AES-CTR ciphertext... [Nest] ERROR [InstanceCommandRunnerService] data migration failed TypeError: Invalid initialization vector ``` ### Root cause The migration assumes every row matching `isSecret = true AND value <> '' AND value NOT LIKE 'enc:v2:%'` is legacy AES-CTR ciphertext. In prod we found multiple `isSecret = true` rows whose `value` is plaintext (e.g. `SLACK_HOOK_URL = 'https://hooks.slack.com/services/...'`) — most likely the result of `isSecret` being flipped to true on a row that already held a plaintext value, or a write path that bypassed `ApplicationVariableEntityService.update`. Those values can't decode into the 16-byte IV that AES-CTR needs, so `Buffer.from(value, 'base64')` truncates at the first non-base64 char (`:`), the buffer is < 16 bytes, and `createDecipheriv` throws. ### Fix Follow the same policy as `EncryptConnectedAccountTokensSlowInstanceCommand`: anything that isn't already in the `enc:v2:` envelope is plaintext. Concretely: 1. Try `decryptVersioned` — legacy CTR rows decrypt fine. 2. If it throws (mis-classified plaintext), log a warning naming the row id and fall back to treating `row.value` as plaintext. 3. Encrypt the resulting plaintext into the `enc:v2:` envelope and update the row. In-loop `isSecret` guard is kept (alongside the SQL filter) so non-secret rows are never touched even if the SQL filter is ever loosened. ### Integration test coverage Added one new case alongside the existing ones in `…encrypt-application-variable.integration-spec.ts`: - `treats plaintext-under-isSecret=true as plaintext and re-encrypts as v2` — seeds a row with `isSecret = true` and a URL value (`:` and `/` are not base64, so this is the exact failure shape from prod), runs the migration, and asserts the value is now `enc:v2:...` and decrypts back to the original URL. Existing cases unchanged: legacy CTR happy path, non-secret rows untouched, idempotent across re-runs, `up()` adds the CHECK constraint, `down()` removes it. ### Why this is a 2-5 edit `TWENTY_CURRENT_VERSION` is now 2.6.0, so editing a 2-5 file trips the `server-previous-version-upgrade-mutation-guard` — `ci:allow-previous-version-upgrade-mutation` label is on the PR. `up()` and `down()` are unchanged; only `runDataMigration` is modified. ## Test plan - [ ] Re-deploy 2.5 to prod and confirm `EncryptApplicationVariableSlowInstanceCommand` completes - [ ] Inspect warning log to count rows that went through the plaintext fallback - [ ] Verify resulting secret rows all satisfy `value = '' OR value LIKE 'enc:v2:%'` and the CHECK constraint is in place |
||
|
Charles Bochet
|
09daccc3f9 |
fix(server): add subFieldName column early in upgrade sequence (#20584)
## Summary Cross-version upgrades from pre-2.3 still fail after #20581 / #20583 — different column, structurally similar problem: ``` column ViewSortEntity.subFieldName does not exist at WorkspaceFlatViewSortMapCacheService.computeForCache (...flat-view-sort/services/workspace-flat-view-sort-map-cache.service.js:40) ... triggered indirectly by DropMessageDirectionFieldCommand (2.3 workspace command) ``` (see https://github.com/twentyhq/twenty-infra/actions/runs/25862573418/job/75997337604) ### Why narrowing the `select` doesn't fit here In the previous two PRs the offender was a bare `findOne` on `WorkspaceEntity` — easy to narrow. Here the chain is: 1. The 2.3 `DropMessageDirectionFieldCommand` builds a workspace migration that deletes a `fieldMetadata` (the `direction` field). 2. `WorkspaceMigrationRunnerService.run` walks the metadata cascade graph (`getMetadataRelatedMetadataNames`) and pulls `viewSort` into the dependency set because `viewSort` is the inverse one-to-many of `fieldMetadata` (deleting a field cascades to view sorts that reference it). 3. That maps to cache keys → `flatViewSortMaps` gets requested → `WorkspaceFlatViewSortMapCacheService.computeForCache` runs. 4. `computeForCache` does `viewSortRepository.find({ where: { workspaceId }, withDeleted: true })` with no `select`, so TypeORM emits a SELECT that includes `subFieldName` — the column doesn't exist in DB yet (added by a 2.5 instance command much later in the sequence). 💥 Narrowing the cache provider's select would silently drop `subFieldName` from the cache for runtime use too, until something invalidates it. Brittle, and would re-break the next time anyone adds a `viewSort` column. ### Structural fix Ensure the column exists in DB before any 2.3 workspace command can trigger that cascade. Within a version, the upgrade runner sorts: fast instance → slow instance → workspace, so a new 2.3 fast instance command lands before `DropMessageDirectionFieldCommand`. - **Add** `2-3/2-3-instance-command-fast-1747234200000-add-sub-field-name-to-view-sort.ts` — `ALTER TABLE ... ADD COLUMN IF NOT EXISTS "subFieldName"`. Comment in the file explains the cascade and why this lives in 2.3 instead of 2.5. - **Make idempotent** the existing `2-5/...-add-sub-field-name-to-view-sort.ts` — switched to `ADD COLUMN IF NOT EXISTS` / `DROP COLUMN IF EXISTS` so it's a no-op on cross-upgrade paths while still creating the column on fresh-from-2.5 installs. - Register the new command in `instance-commands.constant.ts`. The 2.5 command body change is semantically preserving (idempotent), and v2.5.0 hasn't shipped to any production DB yet — so this doesn't violate the "never rewrite committed instance commands" rule in spirit. ### Note on the previous two PRs #20581 and #20583 narrowed `select` on `WorkspaceEntity` for `isInternalMessagesImportEnabled`. That's a band-aid that works because there's a small, enumerable set of bare `workspaceRepository.findOne` call sites. It could in principle be replaced with the same pattern as this PR (early 2.x instance command that adds the workspace column). Not doing that here to keep the diff tight, but happy to follow up if preferred. ## Test plan - [ ] Re-run twenty-infra cross-version-upgrade CI and confirm 2.3 workspace commands complete - [ ] Verify the new 2.3 instance command and the modified 2.5 instance command are both idempotent (running upgrade twice should not error) - [ ] Verify a fresh install path still ends with `subFieldName` present on `core.viewSort` |
||
|
Charles Bochet
|
a5982b644c |
fix(server): scope workspace findOne in 1-21 backfill-datasource command (#20581)
## Summary Cross-version upgrades from pre-1-21 instances currently fail with: ``` error: column WorkspaceEntity.isInternalMessagesImportEnabled does not exist ``` (see https://github.com/twentyhq/twenty-infra/actions/runs/25857499266/job/75979993686) ### Root cause The 1-21 workspace command `backfill-datasource-to-workspace` does: ```ts const workspace = await this.workspaceRepository.findOne({ where: { id: workspaceId }, }); ``` No `select`, so TypeORM emits a SELECT for every column declared on `WorkspaceEntity`. PR #20457 added `isInternalMessagesImportEnabled` to the entity, but its DB column is only created by the 2-5 fast instance command `1778525104406-add-is-internal-messages-import-enabled`. On a fresh cross-version upgrade, the runner reaches the 1-21 workspace segment before that 2-5 instance command runs, the bare `findOne` issues SELECT on a column that doesn't exist yet, and the upgrade aborts. ### Fix Narrow the select to just the columns this command actually reads (`id`, `databaseSchema`). The query now ignores entity columns added later in the upgrade sequence. ### Why edit a committed workspace command Per `CLAUDE.md`, committed *instance* command `up`/`down` logic is immutable. Workspace commands are idempotent backfills — adding a `select` narrows the read but doesn't change behavior, so it's safe. ### Audit Verified this is the only unguarded `workspaceRepository.find*` across the entire upgrade subtree: - `WorkspaceIteratorService.iterate` uses `select: ['databaseSchema']` - `WorkspaceVersionService.getActiveOrSuspendedWorkspaceIds` uses `select: ['id']` - `UpgradeStatusService.loadActiveOrSuspendedWorkspaces` uses `select: ['id', 'displayName']` ## Test plan - [ ] Re-run the failing cross-version upgrade job and confirm it gets past 1-21 - [ ] Verify the 1-21 backfill still correctly skips workspaces with a non-empty `databaseSchema` and backfills those without |
||
|
Charles Bochet
|
a941f6fe01 |
feat(server): migrate TOTP secret encryption to SecretEncryptionService (#20577)
## Summary
Removes the last `APP_SECRET`-derived at-rest encryption site by
migrating `core.twoFactorAuthenticationMethod.secret` from
`SimpleSecretEncryptionUtil` (AES-256-CBC with key derived from
`sha256(APP_SECRET + userId + workspaceId + 'otp-secret' +
'KEY_ENCRYPTION_KEY')`) to the versioned `enc:v2:` envelope
(ENCRYPTION_KEY → HKDF-SHA256 bound to `workspaceId` → AES-256-GCM).
- New `decrypt-legacy-aes-cbc.util.ts` faithfully reproduces the
pre-migration CBC derivation byte-for-byte;
`SecretEncryptionService.decryptVersioned` dispatches to it when callers
pass `legacyAesCbcPurpose`, with a dedicated one-shot WARN log family.
- `TwoFactorAuthenticationService` now uses `encryptVersioned` /
`decryptVersioned` (passing the legacy purpose so existing rows still
decrypt). `SimpleSecretEncryptionUtil` and its spec are deleted;
`TwoFactorAuthenticationModule` imports `SecretEncryptionModule` in
their place.
- `TwoFactorAuthenticationMethodEntity` gets a `@Check` decorator
(`CHK_twoFactorAuthenticationMethod_secret_encrypted`) restricting
`secret` to the `enc:v2:` envelope; the matching 2.5 slow instance
command (`1798000009000-encrypt-totp-secrets`) cursor-paginates `JOIN`ed
`userWorkspace` rows to recover the legacy `userId`, re-encrypts to
`enc:v2`, and applies the CHECK constraint in `up()`.
### Deviation note
The plan suggested wiring a workspace-only legacy derivation directly
into `decryptVersioned`. In practice the production rows are
user-and-workspace-scoped (the legacy purpose is
`\${userId}\${workspaceId}otp-secret`), so a workspace-only derivation
could not recover them. The PR keeps the public `decryptVersioned` API
intact and adds an optional `legacyAesCbcPurpose` so callers that can
reconstruct the legacy context (the 2FA service and the slow command)
opt in.
### Final state of remaining `APP_SECRET` usages
- HS256 JWT verify (read-only, self-retiring once asymmetric migration
completes).
- Express-session cookie signing.
- Approved-access-domain HMAC (signing root, not at-rest).
- Zero-friction fallback in `resolveEncryptionKeysOrThrow`
(intentional).
No production at-rest data is encrypted with `APP_SECRET`-derived keys
anymore.
## Test plan
- [x] `npx jest src/engine/core-modules/secret-encryption
src/engine/core-modules/two-factor-authentication` — 170 unit tests
pass, including new unit tests for the legacy CBC util and the new
`SecretEncryptionService` fallback branch.
- [x] `npx jest --config ./jest-integration.config.ts
test/integration/upgrade/suites/2-5-instance-command-slow-1798000009000-encrypt-totp-secrets.integration-spec.ts`
— 4 integration tests cover legacy-CBC seed → slow command → `enc:v2`
round-trip, idempotency, CHECK constraint enforcement on `up()`, and
rollback via `down()`.
- [x] `npx oxlint --type-aware` and `npx prettier --check` clean on all
touched files.
- [ ] CI on this PR (server validation, tests, lint, typecheck).
|
||
|
Charles Bochet
|
0d5617d446 |
chore(server): drop unused postgresCredentials feature (#20573)
## Summary Drops the `postgresCredentials` legacy feature: a never-finished "postgres proxy" that would have let users query their workspace data over a standard Postgres connection. Nothing — frontend, e2e, Zapier, docs, other server code — calls these mutations/query. ## History - **Introduced** June 2024 (#5767, Thomas Trompette) as "first step for creating credentials for database proxy", alongside the Postgres FDW / remote-server work and the custom `twenty-postgres-spilo` image. Planned follow-ups (provisioning a DB on the proxy, mapping users, exposing it as a remote server) never landed. - **Abandoned** January 2026 (#17001, Weiko) when the sibling "remote integration" feature was removed as a BREAKING CHANGE — "not maintained for more than a year and never officially launched". The spilo image was then replaced with vanilla `postgres:16` (#19182, March 2026), retiring the FDW infrastructure entirely. - This PR finishes the cleanup: removes the orphaned module, the `allPostgresCredentials` relation, `JwtTokenTypeEnum.POSTGRES_PROXY` + payload, the reserved metadata keywords, and adds a 2.5.0 fast instance command that drops `core.postgresCredentials` (reversible `down`). Regenerated frontend GraphQL types + SDK metadata client. ## Test plan - [x] `tsgo --noEmit` clean on twenty-server + twenty-front; lint + prettier clean on touched files. - [x] `database:migrate:generate` reports no pending schema diff; server boots and serves the new schema. |
||
|
Charles Bochet
|
7fa136f305 |
feat(twenty-server): migrate remaining at-rest encryption sites to versioned envelope (#20550)
## Summary Second PR in the encryption key rotation series. The previous PR (#20528) introduced `ENCRYPTION_KEY` + the versioned `enc:v2:<keyId>:<base64>` envelope inside `SecretEncryptionService` and migrated `ConnectedAccountTokenEncryptionService` as the first consumer. This PR routes every remaining at-rest encryption site through the versioned envelope so that `ENCRYPTION_KEY` (and the future `FALLBACK_ENCRYPTION_KEY`) actually covers them. The legacy unprefixed CTR ciphertext remains readable as a fallback during the rollout window — every migrated read site uses `decryptVersioned`, which transparently delegates to the legacy CTR decrypt when it sees an unprefixed payload. ### Service migrations - **`ApplicationVariableEntityService` (#8)** — workspace-scoped. HKDF info is bound to each row's `workspaceId`. A new `decryptAndMaskVersioned` helper lands on `SecretEncryptionService` for the resolver display path. - **`ApplicationRegistrationVariableService` (#7)** + consumers — **instance-scoped**. Registration variables are server-level config readable by every workspace that installs the application, so HKDF info is `instance`. Updated consumers: - `LogicFunctionExecutorService.buildServerVariableEnvMap` - `ConnectionProviderService.getClientCredentials` - **`LogicFunctionExecutorService.buildEnvVar` (#9)** — workspace-scoped. Each variable's `workspaceId` is threaded into `decryptVersioned`, so per-workspace HKDF contexts are honoured at execution time. - **`UpdateApplicationVariableActionHandlerService`** (workspace-migration runner) — threads `workspaceId` through the secret/non-secret toggle. - **`JwtKeyManagerService` (#3)** — instance-scoped. Signing keys are shared across the JWKS. - **`ConfigStorageService` (#6)** — instance-scoped sensitive STRING config variables. ### Slow instance commands (2.5.0) Each migrated site has a paired backfill that re-encrypts existing rows into the v2 envelope before the column is constrained: | timestamp | command | scope | CHECK constraint | |---|---|---|---| | `1798000005000` | encrypt-application-variable | workspaceId | `"isSecret" = false OR value = '' OR value LIKE 'enc:v2:%'` | | `1798000006000` | encrypt-application-registration-variable | instance | `"encryptedValue" = '' OR value LIKE 'enc:v2:%'` | | `1798000007000` | encrypt-signing-key-private-keys | instance | `"privateKey" IS NULL OR value LIKE 'enc:v2:%'` | | `1798000008000` | encrypt-sensitive-config-storage | instance | _none_ — heterogeneous jsonb column | All backfills are idempotent (the SELECT filter skips rows already in v2 form) and run before their respective `up()` adds the CHECK constraint. Every `down()` deliberately stops at dropping the CHECK constraint — they intentionally do not re-introduce plaintext on rollback. ### Tests - Unit specs for each new slow command cover the v2 upgrade path, the idempotency invariant, and the instance vs workspace HKDF scope. - New `JwtKeyManagerService` spec asserts `decryptVersioned`/`encryptVersioned` are called without `workspaceId` (instance scope). - Updated existing specs for `ApplicationVariableEntityService`, `ConfigStorageService`, and `buildEnvVar` to assert the versioned API and the workspace HKDF context plumbing. - New `SecretEncryptionService.decryptAndMaskVersioned` cases in the service spec. - Updated the `applicationRegistrationVariable` integration spec to assert the column now stores `enc:v2:<keyId>:<base64>` instead of raw legacy CTR. ### Out of scope (future PRs) - `PostgresCredentialsService` — bespoke `jwtWrapperService.generateAppSecret`–derived key + `encryptText`/`decryptText` from `auth.util.ts`; deserves its own migration. - `SimpleSecretEncryptionUtil` (TOTP) — entirely different `aes-256-cbc` `iv:enc` format; deserves its own migration. ## Test plan - [x] `npx nx typecheck twenty-server` - [x] `npx nx lint:diff-with-main twenty-server` (oxlint + prettier) - [x] Local jest run for `secret-encryption | connected-account-token | application-variable | application-registration-variable | build-env-var | jwt-key-manager | config-storage | encrypt-application-variable | encrypt-application-registration-variable | encrypt-signing-key | encrypt-sensitive-config-storage` — 17 suites, 106 tests pass. - [x] Local jest run for `upgrade | instance-command` — 12 suites, 86 tests pass. - [ ] CI green - [ ] Manual review of CHECK constraint shapes by a server reviewer (each one matches `enc:v2:%` rather than `enc:v_:%` since none of the migrated columns can legitimately hold `enc:v1:` ciphertext). |
||
|
Charles Bochet
|
04eb913551 |
chore(page-layout): remove IS_RECORD_PAGE_LAYOUT_* feature flags (#20556)
## Summary
- Both \`IS_RECORD_PAGE_LAYOUT_EDITING_ENABLED\` and
\`IS_RECORD_PAGE_LAYOUT_GLOBAL_EDITION_ENABLED\` are force-enabled on
every existing workspace by the 1.23.0 upgrade command
\`BackfillRecordPageLayoutsCommand\` and seeded enabled for new
workspaces via \`DEFAULT_FEATURE_FLAGS\` +
\`seed-feature-flags.util.ts\`. They are no longer load-bearing.
- Unwrap all \`if (flag) { … }\` conditionals to their enabled branch on
both server and front.
- Delete legacy fallback files that only the disabled branch reached:
\`PageLayoutRelationWidgetsSyncEffect\`,
\`usePageLayoutWithRelationWidgets\`,
\`reInjectDynamicRelationWidgetsFromDraft\`,
\`injectRelationWidgetsIntoLayout\`, \`isDynamicRelationWidget\` (and
their tests).
- Strip the two \`enableFeatureFlags\` calls from the 1.23 upgrade
command — the page-layout backfill data logic itself is kept intact
since old workspaces upgrading from < 1.23 still need it.
- No DB cleanup migration: stale \`featureFlag\` rows are left in place,
matching the precedent set by #20531 and #20460.
Net diff: 37 files, +106 / -1727.
## Test plan
- [x] \`npx nx typecheck twenty-shared twenty-server twenty-front\` —
all pass
- [x] \`npx nx lint:diff-with-main twenty-server twenty-front\` — all
pass
- [x] \`cd packages/twenty-front && npx jest page-layout\` — 1240 tests,
all pass
- [x] \`cd packages/twenty-server && npx jest
workspace-entity-manager.spec\` — pass
- [ ] Manual smoke: open a record page, verify tabs render and \"Edit
Layout\" command-menu action is available
- [ ] Manual smoke: Settings → Data model → object → Layout tab is
visible (and hidden for remote / Dashboard objects)
- [ ] Manual smoke: edit a tab title, save, reload — confirm persistence
|
||
|
Charles Bochet
|
2a9fef2341 |
feat(upgrade): emit structured logfmt logs for upgrade flow (#20539)
## Summary
Adds a small helper that lets every log line in the upgrade flow stay
human-readable while emitting a structured tail that Loki / the
upgrade-status Grafana dashboard can filter on.
Output shape per `logger.log()` call:
```
<humanMessage as-is, may span multiple lines>
[upgrade] event=<event> key=value … ← always single line
```
Same call produces **one** structured Loki event regardless of how
chatty the human-readable part gets — the dashboard's `|= "[upgrade]"`
filter only matches the trailing line.
## Helper API
```ts
formatUpgradeLog({
humanMessage: string, // free-form, multi-line OK, for engineers scrolling raw pod logs
event: string, // required anchor for Loki filtering / dashboards
logFields?: Record<string, // short structured key=value tail
string | number | boolean | null | undefined
>,
});
```
- `humanMessage` is preserved as-is. A thrown `new Error('line one\nline
two')` surfacing through `humanMessage` stays human-readable across
multiple lines.
- `logFields` values are logfmt-escaped: whitespace / `=` / `"` get
quoted, embedded `\` / `"` / `\n` / `\r` / `\t` are escaped, `null` /
`undefined` emit literally (`key=null`, `key=undefined`) instead of
being silently dropped — caught via `isDefined` from
`twenty-shared/utils`.
- `event` itself runs through the same escape so an event name with
whitespace or `=` can't break parsing.
## Example output
```
Initialized upgrade sequence: 8 step(s)
[upgrade] event=sequence.initialized stepCount=8 dryRun=false
Upgrading workspace abc-123 1/10
[upgrade] event=workspace.start workspaceId=abc-123 index=1 total=10 dryRun=false
Upgrade for workspace abc-123 completed.
[upgrade] event=workspace.success workspaceId=abc-123 executedByVersion=1.4.0 dryRun=false
Upgrade summary: 42 workspace(s) succeeded, 1 workspace(s) failed
[upgrade] event=summary totalSuccesses=42 totalFailures=1 dryRun=false
Upgrade failed: Workspace migration runner failed:
- Option id is required
- Option id is invalid
[upgrade] event=aborted totalSuccesses=41 totalFailures=2 dryRun=false
```
Loki query for the dashboard: `{namespace="twenty"} |= "[upgrade]" |
logfmt event, workspaceId, command, executedByVersion`
## Scope
Only the **upgrade-specific** call sites carry the tag:
- `upgrade.command.ts` — `sequence.initialized`, `sequence.step`
(verbose), `summary`, `aborted`
- `upgrade-sequence-runner.service.ts` — `sequence.stopped`,
`sequence.aborted`
- `workspace-command-runner.service.ts` — `workspace.start`,
`workspace.success`, `cache.invalidate.failed`
`instance-command-runner.service.ts` is intentionally **not** tagged —
`runFastInstanceCommand` / `runSlowInstanceCommand` are also invoked
from `RunInstanceCommandsCommand` (DB init / `run-instance-commands`),
so an `[upgrade]` tag there would mislead at init time. Those lines stay
plain-text; stacks still flow on their own via NestJS
`logger.error(message, error.stack)`.
`chalk` is dropped from `upgrade.command.ts` — ANSI escapes break log
parsers and chalk is a no-op without a TTY anyway.
## Tests
9 inline-snapshot tests in `format-upgrade-log.util.spec.ts` surface the
actual output of every interesting shape (summary call site, multi-line
humanMessage, quoted / escaped / control-character logField values,
null/undefined fields, event name escaping). Snapshots double as
documentation of what a real upgrade log line looks like.
## Test plan
- [x] Unit tests green (`jest format-upgrade-log`)
- [x] oxlint + prettier clean
- [x] tsgo typecheck clean on the upgrade module
- [x] CI green
- [ ] Smoke test on staging: run `upgrade` command, confirm `[upgrade]`
structured lines surface in Loki and `| logfmt` extracts fields
|
||
|
Félix Malfait
|
e16977f97b |
[breaking: deploy server before front] feat(view-sort): pick sort sub-field inline on the chip (#20445)
## Summary Lets users choose which sub-field of a composite column to sort by — directly from the sort chip — by clicking the sub-field label and picking from a dropdown. Persists per view via a new nullable \`subFieldName\` column on \`ViewSort\`. Replaces #20438, which proposed a field-settings (admin) configuration for the same problem. The chip-level approach is more discoverable (the option lives where the user is looking) and per-view, so different views on the same object can sort by different sub-fields. ### What changes for users - **FullName columns**: previously sorted by \`firstName\` and \`lastName\` together as a stable dual-key sort. Now the user can pick which sub-field is primary (the other is the tie-breaker). Default remains \`firstName\` primary, \`lastName\` tie-breaker. - **Address columns**: previously not sortable at all (not in \`SORTABLE_FIELD_METADATA_TYPES\`). Now sortable, with a chip dropdown listing each enabled sub-field. Default is \`addressCity\` if enabled, else the first enabled sub-field. Disabling a sub-field at the field-metadata level (existing setting) removes it from the dropdown. - **Other composite types** (Currency, Phones, Emails, Links, Actor) and scalar fields keep their existing single-key sort behavior. ### UX ``` ┌─────────────────────────┐ ┌─────────────────────────┐ │ ↑ Name · Last name ✕ │ │ ↑ Address · City ✕ │ └────────┬────────────────┘ └────────┬────────────────┘ ▼ (click sub-field) ▼ ┌────────────┐ ┌────────────┐ │ First name │ │ Address 1 │ │ Last name ✓│ │ Address 2 │ └────────────┘ │ City ✓│ │ State │ │ Postcode │ │ Country │ └────────────┘ ``` The chip body still toggles direction on click — the \`Dropdown\`'s internal wrapper calls \`stopPropagation\` so the sub-field click doesn't bubble to the chip's onClick. ## What changed **Backend:** - \`ViewSortEntity\` — new nullable \`subFieldName: varchar\` column - \`ViewSortDTO\`, \`CreateViewSortInput\`, \`UpdateViewSortInputUpdates\` — new \`@Field(() => String, { nullable: true })\` - \`FLAT_VIEW_SORT_EDITABLE_PROPERTIES\` — \`'subFieldName'\` added so the property flows through the update merge path - \`ALL_ENTITY_PROPERTIES_CONFIGURATION_BY_METADATA_NAME.viewSort\` — new \`subFieldName\` entry with \`toCompare: true\` so cache diffs notice it - \`fromCreateViewSortInputToFlatViewSortToCreate\` — threads \`subFieldName\` through - Instance command migration (\`add-sub-field-name-to-view-sort\`) — single \`ALTER TABLE core.viewSort ADD subFieldName varchar\` / \`DROP\` **Frontend:** - \`RecordSort\` and \`ViewSort\` types — \`subFieldName?: string | null\` - \`VIEW_SORT_FRAGMENT\` — adds \`subFieldName\` so the field round-trips - \`mapRecordSortToViewSort\` + \`areViewSortsEqual\` — carry the new field through, include it in the diff so the usual \`useSaveRecordSortsToViewSorts\` create/update flow fires when it changes - \`useSaveRecordSortsToViewSorts\` — passes \`subFieldName\` in both \`CreateViewSortInput\` and \`UpdateViewSortInputUpdates\` - \`getOrderByForFieldMetadataType(field, direction, subFieldName?)\` — new optional third arg. \`turnSortsIntoOrderBy\` threads \`sort.subFieldName\` into it. - \`Address\` added to \`SORTABLE_FIELD_METADATA_TYPES\` - New helpers: \`getEnabledAddressSubFields\` (filters by the field's \`subFields\` setting, falls back to the 6 default visible address sub-fields), \`getDefaultSortSubFieldForAddress\`, \`getDefaultSortSubFieldForFullName\` - New shared types/constants: \`AllowedFullNameSubField\`, \`ALLOWED_FULL_NAME_SUBFIELDS\`, \`DEFAULT_VISIBLE_ADDRESS_SUBFIELDS\` - \`SortOrFilterChip\` — new \`labelSubField?: ReactNode\` slot; renders as \` · {sub-field}\` with subdued weight after the main label - \`EditableSortChip\` — builds options from field metadata (\`ALLOWED_FULL_NAME_SUBFIELDS\` for FullName, \`getEnabledAddressSubFields\` for Address), uses i18n-wrapped labels, persists picks via \`upsertRecordSort\` ## Test plan - [x] \`npx nx typecheck\` passes for twenty-shared, twenty-front, twenty-server - [x] \`oxlint --type-aware\` on all 19 frontend + 9 server changed files: 0 errors - [x] \`prettier --check\`: clean - [x] 16 unit tests pass — \`getOrderByForFieldMetadataType\` covers the new \`subFieldName\` override branch for FULL_NAME and ADDRESS; \`getDefaultSortSubFieldForAddress\` covers the city/first-enabled fallback path; \`getDefaultSortSubFieldForFullName\` exercises its constant - [ ] Manual: sort a People view by Full Name → click the chip's sub-field label → switch between First name and Last name → reload page → choice is preserved - [ ] Manual: sort a Company view by Address → confirm dropdown lists only enabled sub-fields → disable Address \`addressCity\` in field settings → confirm dropdown options update and runtime falls back to the first enabled sub-field 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> Co-authored-by: claude[bot] <41898282+claude[bot]@users.noreply.github.com> |
||
|
Etienne
|
fcd2d586ee |
chore(billing) - remove feature flag (#20531)
- remove feature flag - remove old enforce cap usage logic |
||
|
Charles Bochet
|
e0b4c9918b |
feat(twenty-server): introduce ENCRYPTION_KEY env var with versioned envelope (#20528)
## Summary
- Adds `ENCRYPTION_KEY` (primary) and `FALLBACK_ENCRYPTION_KEY`
(decrypt-only fallback for rotation) env vars to twenty-server, with
backward-compatible fallback to `APP_SECRET` when `ENCRYPTION_KEY` is
unset.
- Introduces a versioned ciphertext envelope `enc:v2:<keyId>:<base64>`
using AES-256-GCM with HKDF-SHA256 derived per-context keys. The 8-hex
`keyId` fingerprint lets every row identify which physical key encrypted
it, so rotation routes directly to primary or fallback without trial
decryption; GCM's auth tag gives true integrity (legacy CTR has none).
- Migrates `ConnectedAccountTokenEncryptionService` to the new envelope
and plumbs `workspaceId` through every caller, so per-workspace HKDF
context binds each row to its tenant.
The remaining encryption sites (`jwt-key-manager`, `config-storage`,
`postgres-credentials`, `application-variable`, TOTP) stay on the legacy
unprefixed CTR path and will be migrated in follow-up PRs. The
operator-facing rotation runbook is out of scope here.
### Format details
`enc:v{N}:{keyId}:{base64}` — `N=2` is the only version produced by new
writes (`v1` exists for backward-compatible decryption of existing
connected-account rows). `keyId =
sha256(rawKey).slice(0,4).toString('hex')`. The CHECK constraint on
`core.connectedAccount.{accessToken,refreshToken}` is relaxed from `LIKE
'enc:v1:%'` to `LIKE 'enc:v_:%'` so both versions pass.
### Key resolution
| `ENCRYPTION_KEY` | `FALLBACK_ENCRYPTION_KEY` | `APP_SECRET` | Encrypt
with | Decrypt try order |
|---|---|---|---|---|
| set | set | (any) | `ENCRYPTION_KEY` | match `keyId` → primary →
fallback |
| set | unset | (any) | `ENCRYPTION_KEY` | match `keyId` → primary |
| unset | set | set | `APP_SECRET` | match `keyId` → `APP_SECRET` →
fallback |
| unset | unset | set | `APP_SECRET` | match `keyId` → `APP_SECRET` |
| unset | unset | unset | startup error | n/a |
## Test plan
- [x] `npx nx typecheck twenty-server` — clean
- [x] `npx jest
'secret-encryption|connected-account-token-encryption|connected-account-refresh-tokens|encrypt-connected-account-tokens|connection-provider-oauth-flow'`
— 87 tests pass
- [x] New `secret-encryption.service.versioned.spec.ts` covers: key
resolution table (no-key error, APP_SECRET fallback, ENCRYPTION_KEY
precedence), v2 round-trip with/without workspaceId, GCM tamper
rejection, workspaceId-mismatch rejection, keyId-based primary→fallback
routing, missing-key error names the fingerprint, v1 legacy decryption,
no-prefix legacy decryption, malformed envelope rejection.
- [x] Updated `connected-account-token-encryption.service.spec.ts`
covers workspaceId binding and HKDF context isolation.
- [x] Updated slow instance command spec verifies workspaceId is
threaded through encryption and the relaxed `enc:v_:%` LIKE pattern
matches both v1 and v2.
- [ ] Manual E2E: connect a Gmail account on a freshly deployed instance
with `APP_SECRET` only → confirm `core.connectedAccount.accessToken` is
`enc:v2:<keyId>:<base64>`.
- [ ] Manual E2E: rotate — set `ENCRYPTION_KEY=<new>` and
`FALLBACK_ENCRYPTION_KEY=<old APP_SECRET>`, restart, confirm
pre-rotation rows still decrypt and new rows carry the new `keyId`.
- [ ] Manual E2E: missing key — set `ENCRYPTION_KEY=<new>` without the
fallback, confirm decrypt error names the old `keyId` so the operator
can identify the missing key.
|
||
|
MkDev11
|
bbc55193f5 |
Fix phone unique constraints (#20261)
## Summary Closes #20195 Fix phone field unique constraints so phone numbers are considered unique by both `primaryPhoneNumber` and `primaryPhoneCallingCode`. - Include `primaryPhoneCallingCode` in the shared phone composite unique constraint metadata - Align the frontend settings composite field config with the backend metadata - Return all included unique composite subfields when building create-many conflict fields - Match composite unique conflict fields as a group during create-many upserts ## Root Cause Phone composite metadata only marked `primaryPhoneNumber` as part of the unique constraint. That made different international phone numbers with the same national number conflict, for example `+1 123456789` and `+32 123456789`. ## Test Plan - `yarn workspace twenty-shared build` - `jest --runTestsByPath <index action handler and create-many utility specs>` - `prettier --check <touched files>` - `oxlint --type-aware <touched files>` - `nx run twenty-shared:typecheck` - `nx run twenty-server:typecheck` - `nx run twenty-front:typecheck` --------- Co-authored-by: mkdev11 <MkDev11@users.noreply.github.com> Co-authored-by: Etienne <45695613+etiennejouan@users.noreply.github.com> Co-authored-by: prastoin <paul@twenty.com> |
||
|
Charles Bochet
|
9e515afb13 |
feat(server): asymmetric JWT signing with kid + key rotation table (#20467)
## Context Today every JWT issued by Twenty (access, refresh, login, file, etc.) is HMAC-signed with a per-token-type secret derived from the global `APP_SECRET`. Rotating that secret invalidates **every** active token at once and there is no way to scope a leak to a subset of tokens. This PR is the first slice of a broader effort to **decouple stateful encryption (`APP_SECRET`-derived secrets) from stateless encryption (JWTs)**. It introduces an asymmetric (private/public key) signing path for `ACCESS` and `REFRESH` tokens and a signing-key registry to enable **safe rotation**: leaked keys can be revoked by flipping `revokedAt`/`isCurrent` on the matching row without invalidating tokens issued by other keys. > Out of scope (intentionally): swapping stateful encryption for `APP_SECRET`, asymmetric signing for token types other than `ACCESS`/`REFRESH`, an admin-panel rotation UI, and an enterprise re-encryption command. Those will land in follow-up PRs. ## What changes - **New `core.signingKey` table** (instance command `2.5.0` / `1778550000000`) storing both the public key (PEM, in clear) and the private key (PEM, encrypted with `APP_SECRET` via `SecretEncryptionService`). One row is marked `isCurrent = true` (enforced by a partial unique index). The row's UUID `id` is used directly as the JWT `kid`. - When a key is rotated out, its `privateKey` is nulled (we never keep historical private keys) but the `publicKey` row stays so previously issued tokens can still be verified. - **`JwtKeyManagerService`** lazily loads-or-generates the current signing key on first use: - If a row with `isCurrent = true` exists → decrypts and uses it. - Otherwise → generates a fresh EC P-256 keypair, encrypts the private key, inserts the row (UUID id = kid). Handles concurrent insert races via the unique constraint. - **`JwtWrapperService.signAsync()`** signs `ACCESS`/`REFRESH` payloads with `ES256` and a `kid` header. Falls back to `HS256` if no signing key is available (boot-time DB error, transient failure). - **Dual-path verification** in both `JwtWrapperService.verifyJwtToken` and the Passport `JwtAuthStrategy.secretOrKeyProvider`: - JWT with a `kid` header → resolve the public key PEM by id and verify with `ES256`, - otherwise → fall back to the existing `APP_SECRET`-derived `HS256` path (unchanged). - **`AccessTokenService` / `RefreshTokenService`** now sign through `signAsync` (single public surface; the routing detail stays internal to the wrapper). - **Public key cache**: a new `SigningKeyEntityCacheProviderService` plugs into `CoreEntityCacheService` (`signingKeyPublicKey` namespace) and serves PEMs by id, with the standard local-memo + Redis layering. - **PEM strings end-to-end**: `jsonwebtoken` accepts PEM strings directly for both sign and verify, so the manager never converts to a Node `KeyObject` and the cache hands the PEM straight to `jwt.verify`. ## Why ES256 (and not EdDSA / RS256) - `@nestjs/jwt` is backed by `jsonwebtoken`, which does **not** support EdDSA today. - ES256 keys are tiny (~120 bytes vs 1.6 kB for RS256), signatures are short (~64 bytes), and signing/verification is fast — important since JWT verification runs on every authenticated request. - ES256 is widely supported and standardized (RFC 7518), with mature ecosystem support. ## Why store the private key in DB (not env) - No new secret to provision: existing instances already have `APP_SECRET`, which we reuse to encrypt the private key at rest. - Self-healing: a fresh instance auto-generates its first signing key on first boot. Nothing to copy/paste. - Rotation is a SQL operation against `core.signingKey`, not a redeploy + env mutation. ## Backward compatibility - All previously-issued tokens (no `kid`) keep verifying through the legacy HS256 path with their existing `APP_SECRET`-derived secret. No forced re-login. - Token types not in scope (`WORKSPACE_AGNOSTIC`, `API_KEY`, `FILE`, `LOGIN`, `EMAIL_VERIFICATION`, etc.) keep their current HS256 behavior unchanged — they still go through the synchronous `JwtWrapperService.sign(payload, options)` with a caller-supplied secret. - `signWithAppSecret` is kept intentionally as the HS256 fallback path; it will be deprecated in a follow-up PR. - If the DB lookup/generation fails for any reason, the wrapper logs the error and falls back to HS256 — no startup crash, no silent regression. ## Rotation story 1. Bootstrap: first signing call lazily inserts a row in `core.signingKey` with `isCurrent = true`, `privateKey = encrypt(pem_A)`. New tokens carry `kid_A`. 2. Rotate: `UPDATE core."signingKey" SET "isCurrent" = false, "privateKey" = NULL WHERE id = '<kid_A>';` then insert a new row with `isCurrent = true`. New tokens carry `kid_B`. Tokens still in flight with `kid_A` keep verifying because the public-key row for `kid_A` is still there. 3. Revoke: `UPDATE core."signingKey" SET "revokedAt" = now() WHERE id = '<kid_A>';`. All tokens with `kid_A` now fail verification cleanly with `UNAUTHENTICATED` (no 500). 4. Tokens with no `kid` (legacy) are unaffected throughout. ## Test plan - [x] Unit: `JwtWrapperService` dual-path verification (HS256 no-kid vs ES256 with-kid), unknown-kid → `UNAUTHENTICATED`, `signAsync` happy path + `null` when no key, `signAsync` rejection for non-rotatable types. - [x] Unit: `JwtAuthStrategy` `secretOrKeyProvider` dual-path resolution and algorithm validation. - [x] All existing JWT/auth/application unit tests adjusted to the renamed public method. - [x] Integration (`jwt-key-rotation.integration-spec.ts`): - **Happy path**: signed-up user's `ACCESS` token has `alg=ES256` + correct UUID `kid`, the `isCurrent=true` row exists in `core.signingKey`, `getCurrentUser` resolves. - **Legacy fallback**: hand-crafted no-kid HS256 token verifies via the legacy `APP_SECRET`-derived path. - **Previous-key rotation**: token signed by a hardcoded *previous* key whose row is pre-inserted with `privateKey = NULL` (rotated-out) still verifies — proves the leaked-key revocation flow works in both directions. - **Unknown kid**: token signed with an orphan UUID `kid` is cleanly rejected (no 500). - [x] `npx nx typecheck twenty-server` - [x] `npx nx test twenty-server` - [x] `npx nx run twenty-server:lint` |
||
|
Charles Bochet
|
5003fcbbf2 |
chore: remove dead feature flags (#20460)
## Summary Two related cleanups, following the same pattern as #19916 and #19074. ### Dead feature flags Drops four feature flags whose only references are the enum entry and the generated GraphQL/SDK files: - `IS_COMMAND_MENU_ITEM_ENABLED` — never read anywhere. - `IS_DATASOURCE_MIGRATED` — already commented `@deprecated`. Zero non-generated consumers. - `IS_RICH_TEXT_V1_MIGRATED` — the 1-19 migration that gated it was removed in #19074; the flag became dead at that point. - `IS_CONNECTED_ACCOUNT_MIGRATED` — only read by the 1-21 `migrate-messaging-infrastructure-to-metadata` command as an early-return guard, but the flag was never written anywhere in the codebase, so the guard never fired (and that workspace command is now removed entirely — see below). Generated GraphQL/SDK schemas and the `workspace-entity-manager` test mock are trimmed to match. ### 1-21 workspace commands Same pattern as #19074 (which removed workspace commands ≤ 1.18). Twenty is now on 2-5; the 1-21 workspace commands have long since run on every active workspace and are dead code. Removes: - All 14 workspace commands under `upgrade-version-command/1-21/` (compose-email menu item, key-value-pair index, datasource backfill, message-thread backfill, dedup engine commands, select-all fixes, AI response format migration, edit-layout label, drop messaging FKs, folder parent-id migration, messaging-infra-to-metadata, navigation refactor, message-thread label fix, search-menu-item label). - The `1-21-upgrade-version-command.module.ts` registration and the `V1_21_UpgradeVersionCommandModule` import from `WorkspaceCommandProviderModule`. **Kept** (intentionally): the 3 `1-21-instance-command-fast-*` files. Unlike workspace commands (which mutate data), instance commands carry **schema deltas** still required by current entity definitions (`AddViewFieldGroupIdIndex`, `MigrateMessagingCalendarToCore`, `AddEmailThreadWidgetType`). They remain registered in `INSTANCE_COMMANDS` and `'1.21.0'` stays in `TWENTY_PREVIOUS_VERSIONS`. They will fold away naturally on a future version bump when a `CoreMigrationCheck`-style snapshot picks them up. ## Test plan - [x] `npx nx typecheck twenty-shared` - [x] `npx nx typecheck twenty-server` - [x] `npx nx typecheck twenty-front` - [x] `npx prettier --check` on the changed files - [x] `npx oxlint` on the changed server files - [x] `npx jest feature-flag` (4 suites, 20 tests pass) - [x] `npx jest workspace-entity-manager` (1 suite, 5 tests pass) |
||
|
neo773
|
b03f044d0f |
feat(messaging): add workspace toggle to sync internal emails (#20457)
Co-authored-by: cubic-dev-ai[bot] <191113872+cubic-dev-ai[bot]@users.noreply.github.com> |
||
|
Paul Rastoin
|
6c18bacb93 |
Encrypt connected account accessToken and refreshToken (#20441)
# Introduction Encrypt the `connectedAccount` `accessToken` and `refreshToken` using `APP_SECRET` in order to mitigate potential data leak or `core` table compromise ## Decrypt Temporary allow already plain text stored token to be retrieve without decryption until the slow instance has been passed Will uncomment the invariant check in a patch when the instance slow has fully be run ## Standards - Token are encrypted as quickly as possible - A token cannot be written in database non encrypted by mistake using a custom constraint ( `enc:` prefix ) ## What's next We should standardize not managing secret as is in the the services and layer, they should be encrypted on the flight the earliest and should never be logged Will create a dedicated pattern afterwards for `applicationVariables` secrets too --------- Co-authored-by: Charles Bochet <charlesBochet@users.noreply.github.com> |
||
|
Félix Malfait
|
34b927ff23 |
feat(public-domain): bind public domains to apps + reorganize settings (#20360)
## Summary - **Public domains can now be bound to a specific app.** When a request hits an app-bound public domain, route resolution restricts logic-function matching to that app's HTTP-routed functions only — isolating each app's routes to its own domain instead of letting routes from other apps in the workspace match nondeterministically. - **Settings sidebar reorganized.** Removed the standalone Domains page. Workspace Domain → General. Approved Domains + Invitations → Members "Access" tab. Emailing Domains + Public Domains → Apps "Developer" tab. Roles → Members "Roles" tab. ## Why The use case: someone building a partner portal app or a lead-collection app declares private objects (leads, partners…) plus a few public HTTP routes. Each app needs its own domain (`partners.acme.com`, `leads.acme.com`) without those domains exposing every other app's routes in the same workspace. Today's PublicDomainEntity is workspace-scoped only, so all HTTP-routed logic functions in a workspace compete for any public domain — first match wins nondeterministically. ## Backend - Added nullable `applicationId` FK to `PublicDomainEntity` (cascade-deleted with the app); indexed for the route-trigger lookup. - New fast instance command `2-4-instance-command-fast-1798000003000-add-application-id-to-public-domain` adds the column, index, and FK constraint. - `createPublicDomain(domain, applicationId)` accepts an optional app binding; new `updatePublicDomain(domain, applicationId)` mutation rebinds/unbinds an existing domain. Both validate the application belongs to the workspace. - `WorkspaceDomainsService.resolveWorkspaceAndPublicDomain(origin)` returns both the workspace and the matched public domain in one query — replacing the old back-to-back lookups in the route-trigger hot path. `getWorkspaceByOriginOrDefaultWorkspace` is preserved as a thin wrapper. - `RouteTriggerService` filters `logicFunction` by `applicationId` when the matched public domain is app-scoped; falls back to workspace-wide when unbound. - Three sequential validation queries in `createPublicDomain` now run in parallel via `Promise.all`. ## Frontend | Old location | New location | |---|---| | Settings sidebar → Domains (standalone page) | Removed | | Domains page → Workspace Domain | General page | | Domains page → Approved Domains | Members → Access tab | | Domains page → Emailing Domains | Apps → Developer tab | | Domains page → Public Domains | Apps → Developer tab | | Settings sidebar → Roles (standalone) | Members → Roles tab | | `pages/settings/roles/` | `pages/settings/members/roles/` | - The Public Domain detail page has an Application picker that uses `Select`'s native `emptyOption` + `null` value pattern (matches `SettingsDataModelObjectIdentifiersForm`). - Members page tabs use the existing `TabListFromUrlOptionalEffect` mechanism (rendered automatically by `TabList`) for hash-based tab activation. - `/settings/members/roles` redirects to `/settings/members#roles` so role sub-pages' `navigate(SettingsPath.Roles)` lands on the Members page with the Roles tab pre-selected. - All affected breadcrumbs updated to nest under their new parents. - `SettingsPath.Roles` and friends now nest under `members/`; `Subdomain` and `CustomDomain` under `general/`; `PublicDomain` and `EmailingDomain` under `applications/`. ## Test plan - [x] `nx typecheck twenty-front` passes - [x] `nx typecheck twenty-server` passes - [x] `oxlint --type-aware` clean on all touched files - [x] `prettier --check` clean on all touched files - [x] Migration applied locally; `publicDomain.applicationId` (uuid, nullable) confirmed in DB - [x] GraphQL schema exposes `PublicDomain.applicationId`, `createPublicDomain.applicationId`, `updatePublicDomain` mutation - [x] **End-to-end route resolution scenarios verified locally:** - Domain bound to App A, function in App A → route matches ✅ - Domain bound to App B, function in App A → route does NOT match (HTTP 404 `TRIGGER_NOT_FOUND`) ✅ - Domain unbound (`applicationId = NULL`) → route matches workspace-wide ✅ - Unknown path on bound domain → returns 404 cleanly ✅ - [x] UI sanity (browser-tested at `apple.localhost:3001`): - General page shows Workspace Domain card - Members page shows Team / Access / Roles tabs - Access tab combines Invite by link + by email + Approved Domains - Roles tab embeds the role list - `/settings/members/roles` direct URL → redirects + Roles tab pre-selected - Apps Developer tab shows Emailing Domains + Public Domains sections - Public Domain detail page has Application picker dropdown listing workspace apps - Sidebar nav: "Domains" and "Roles" no longer present (now folded into General/Members) ## Notes for reviewers - Creating a public domain via the UI still requires Cloudflare credentials in the dev `.env` (`CLOUDFLARE_API_KEY`, `CLOUDFLARE_PUBLIC_DOMAIN_ZONE_ID`, `PUBLIC_DOMAIN_URL`). The DNS step is unchanged from main. - The `applicationId` column is nullable, so existing public-domain rows continue to work workspace-wide — no data backfill required. - `SettingsRolesContainer` was deleted (no longer referenced after `SettingsRoles` index page was removed). 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
Félix Malfait
|
4da8878697 |
feat: add email forwarding message channel (#19535)
## Summary - Add email forwarding as a new message channel type, allowing users to forward emails from addresses like `support@mycompany.com` into Twenty - Inbound emails arrive via S3 (SES → S3 bucket), are polled by a cron job, parsed, routed to the correct workspace/channel, and persisted as messages - Dedicated settings page at `/settings/accounts/new-email-forwarding` where users provide their source email handle and receive a unique forwarding address - Forwarding channels bypass the IMAP/mailbox sync state machine — they skip cron-driven sync, relaunch, and message-list-fetch lifecycle stages - Forwarding address section shown at the top of the Emails settings page so users can find/copy their addresses after initial setup - Tab names for forwarding channels display the user-provided handle (e.g. `support@mycompany.com`) instead of the internal routing address - Shared utilities extracted from IMAP driver: `extractThreadId`, `extractParticipants`, `extractAddresses` to avoid code duplication - Uses the existing S3 bucket (STORAGE_S3_*) with `inbound-email/` prefix — no separate bucket needed - Feature gated behind `isEmailForwardingEnabled` client config (requires `INBOUND_EMAIL_DOMAIN` + S3 storage) ## New backend modules - `InboundEmailS3ClientProvider` — lazy-initialized S3 client using existing storage config - `InboundEmailStorageService` — S3 operations (get, move to processed/unmatched/failed) - `InboundEmailParserService` — RFC 822 parsing via `postal-mime`, builds `MessageWithParticipants` - `InboundEmailImportService` — orchestrates download → parse → route → persist → archive - `MessagingInboundEmailPollCronJob` — polls S3 `incoming/` prefix, enqueues import jobs - `CreateEmailForwardingChannelInput` DTO — accepts user-provided `handle` ## New frontend components - `SettingsAccountsNewEmailForwardingChannel` — dedicated page with handle input form + forwarding address result - `SettingsAccountsEmailForwardingSection` — forwarding address list on the Emails settings page - `useConnectedAccountHandleMap` — shared hook for account ID → handle lookup - `useCreateEmailForwardingChannel` — mutation hook accepting handle parameter ## Test plan - [x] 17 unit tests for inbound email import service (all outcomes: imported, unmatched, loop_dropped, unconfigured, parse_failed, persist_failed) - [x] 16 tests for `computeSyncStatus` including EMAIL_FORWARDING cases - [x] 11 tests for `extractEnvelopeRecipient` utility - [x] TypeScript typechecks pass for both twenty-server and twenty-front - [x] Lint passes for both packages - [ ] Manual: create forwarding channel, verify forwarding address generated - [ ] Manual: send email to forwarding address, verify it appears in Twenty https://claude.ai/code/session_01KpyF6p4cUEnuaT4h8DP5Pm --------- Co-authored-by: Claude <noreply@anthropic.com> Co-authored-by: Copilot Autofix powered by AI <223894421+github-code-quality[bot]@users.noreply.github.com> Co-authored-by: claude[bot] <41898282+claude[bot]@users.noreply.github.com> Co-authored-by: neo773 <62795688+neo773@users.noreply.github.com> Co-authored-by: neo773 <neo773@protonmail.com> |
||
|
Charles Bochet
|
546ab0a036 |
fix: handle widgets with missing universalConfiguration in 2.3 delete-gauge-widgets command (#20393)
## Summary
The 2.3 `upgrade:2-3:delete-gauge-widgets` workspace command crashed in
production for ~10 workspaces (out of 5000) with:
```
[Error] Cannot read properties of undefined (reading 'configurationType')
at .../2-3-workspace-command-1798000000000-delete-gauge-widgets.command.js:35:164
at Array.filter (<anonymous>)
```
### Root cause
Those workspaces have legacy `pageLayoutWidget` rows whose
`configuration` JSONB does not contain a recognized `configurationType`.
This is consistent with the 1.15 backfill
(`MigratePageLayoutWidgetConfigurationCommand`) only migrating widgets
with the deprecated `graphType` and the `IFRAME` /
`STANDALONE_RICH_TEXT` types — any other widget type that was already
missing `configurationType` (or has a value not in the current enum) was
left as-is.
When the cache is recomputed,
`fromPageLayoutWidgetConfigurationToUniversalConfiguration` switches on
`configuration.configurationType`. With no matching case, the function
falls through and returns `undefined`, so the cached
`widget.universalConfiguration` ends up `undefined`. The gauge filter
then dereferences `.configurationType` and throws.
We can't reproduce the affected data locally, but the symptom uniquely
points at this fall-through path — every other code path either throws
earlier (e.g. when `configuration` itself is null) or yields a defined
`universalConfiguration`.
### Fix
In
`2-3-workspace-command-1798000000000-delete-gauge-widgets.command.ts`:
- Skip widgets whose `universalConfiguration` is `undefined` — by
definition they aren't gauge widgets, so they don't belong in the
deletion set.
- Log them as a warning (id and count) so we still have visibility on
the corrupt rows for follow-up cleanup.
- Use optional chaining when comparing the configuration type so the
filter is robust to the same shape going forward.
The fix is minimal and additive: workspaces without corrupt widgets
behave exactly as before, and the upgrade can now succeed on the
affected workspaces.
## Test plan
- [ ] CI lint + typecheck green
- [ ] Run the upgrade on a healthy workspace locally — gauge widgets are
still deleted, no warnings logged
- [ ] On production, verify the 2.3 upgrade no longer fails on the
affected ~10 workspaces and that the warning logs surface the offending
widget ids for follow-up
## Follow-ups (out of scope of this PR)
- Investigate the corrupt widgets surfaced by the new warning log and
decide whether to backfill / delete them in a dedicated upgrade command
- Consider hardening
`fromPageLayoutWidgetConfigurationToUniversalConfiguration` so the
switch fall-through fails loudly (or returns a sentinel) instead of
silently yielding `undefined`
|
||
|
Etienne
|
9fc5be1c4c |
Billing - Migrate from Stripe metering (#20298)
**Overall strategy** **1. Introduce “Billing V2” behind a workspace flag** Gate the new model with FeatureFlagKey.IS_BILLING_V2_ENABLED so existing workspaces stay on the old behavior until they’re migrated or explicitly on V2. **2. Replace workflow metered SKUs with a resource-credit product** Conceptually, billable “workflow execution” usage is not the primary subscription line item anymore. Add a RESOURCE_CREDIT product (and keep WORKFLOW_NODE_EXECUTION as deprecated for the transition). Usage and limits are expressed through credit buckets (e.g. price metadata like credit_amount), so one product can represent pooled credits instead of a narrow workflow-only meter. **3. Migrate subscriptions in two layers** Schema/catalog: persist extra price metadata (instance upgrade) so the server knows credit amounts and can match Stripe prices to the new model. Per workspace: the registered workspace command upgrade:2-2:migrate-to-billing-v2 finds subscriptions that still have WORKFLOW_NODE_EXECUTION, swaps those items to the right RESOURCE_CREDIT prices (using existing Stripe schedule + BillingSubscriptionUpdateService stack), then treats the workspace as V2 (flag). Workspaces without that legacy item or without a subscription are skipped. **4. Unify subscription lifecycle + usage on the server** **5. Refresh the product surface in Settings** Test : - [x] Subscribe v1 + Update subscribe + Migrate - [x] Subscribe v2 + Update subscribe |
||
|
martmull
|
2158266fcc |
Fix migration (#20321)
as title |
||
|
martmull
|
617f571400 |
20215 convert application variable to a syncable entity (#20269)
## Summary - Converts applicationVariable from a bespoke sync path to a proper SyncableEntity, unifying it with the workspace migration pipeline used by all other manifest-managed entities (agent, skill, frontComponent, webhook, etc.) - Removes the upsertManyApplicationVariableEntities method and its direct-DB-mutation approach in favor of the standard validate → build → run action handler pipeline - Adds universalIdentifier, deletedAt columns and makes applicationId NOT NULL via an instance command migration ## Motivation Before this change, applicationVariable was the only manifest-managed entity that bypassed ApplicationManifestMigrationService.syncMetadataFromManifest(). It used a bespoke service method called directly from syncApplication(), creating two mental models, two validation styles, and two cache invalidation patterns. Now there's one unified pipeline for all manifest entities. ## What changed ### Entity refactor: - ApplicationVariableEntity now extends SyncableEntity (gains universalIdentifier, non-nullable applicationId with CASCADE, soft-delete via deletedAt) ### New flat entity layer (flat-application-variable/): - Type, maps type, editable properties constant, entity-to-flat converter, cache service, module ### New migration pipeline wiring: - Manifest converter (fromApplicationVariableManifestToUniversalFlatApplicationVariable) - Validator service (FlatApplicationVariableValidatorService) - Builder service (WorkspaceMigrationApplicationVariableActionsBuilderService) - Create/Update/Delete action handlers with secret encryption hooks - Registered in orchestrator, builder module, runner module, and all type registries ### Removed bespoke path: - Deleted upsertManyApplicationVariableEntities from ApplicationVariableEntityService - Removed its call from ApplicationSyncService.syncApplication() - Kept update() (operator-set value at runtime) and getDisplayValue() (runtime display) ### Database migration: - Instance command to add columns, backfill universalIdentifier, enforce NOT NULL constraints, and update indexes ## Test plan - npx nx typecheck twenty-server passes (0 errors) - Unit tests pass (application-variable.service.spec.ts, build-env-var.spec.ts) - Install an app with applicationVariables in its manifest → variables appear with correct universalIdentifier - Update app manifest (add/remove/modify a variable) → migration pipeline handles diff correctly - Operator-set value via update endpoint persists correctly with encryption - Uninstall app → variables cascade-deleted - app dev --once on example app syncs without errors |
||
|
nitin
|
bbd9720ab3 |
[Dashboards] [Warning] Remove gauge chart support and delete existing widgets (#20172)
## Summary Removes gauge chart from the chart-type picker and deletes existing gauge widgets via a workspace migration. The gauge was rendering a hardcoded `0.7 / "Progress"` stub regardless of configuration -- never wired to real data. The contract stays in place. We keep `WidgetConfigurationType.GAUGE_CHART`, the DTO, the GraphQL union member, and the gauge folder -- so stored gauge JSON still resolves through the schema. The render path falls through to `default: return null`, so any un-migrated gauge widget renders as an empty cell, not a crash. This PR just removes existing gauge widgets if there are any (via `upgrade:2-3:delete-gauge-widgets`). The deliberate cleanup -- deleting the type definitions, the gauge folder, the DTO -- comes in a follow-up PR after the migration has run. --------- Co-authored-by: Charles Bochet <charles@twenty.com> |
||
|
neo773
|
d040756fcf |
remove direction from messages (#20026)
This was a leftover column removed in https://github.com/twentyhq/twenty/pull/6743 but was accidentally added again when we migrated to `buildMessageStandardFlatFieldMetadatas` from workspace decorator /closes #20011 |