3ee93b5ec925890467e107c6ebdb5351daa95006
447 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Weiko
|
3ee93b5ec9 |
feat(server): add isSystemSideEffect & merge createOneObject/createOneField side-effect migrations (#21673)
## Context When an object is created via the metadata API, `createOneObject` creates its side-effect entities (INDEX view + viewFields, indexes, navigation menu item, "go to" command menu item, record-page fields view, page layout/tabs/widgets) across **three separate `validateBuildAndRunWorkspaceMigration` calls**, purely because the protection behavior (mutations → overrides, delete → deactivate, reset → reactivate) was keyed on *"owned by the standard app"*, forcing the side effects into batches with different application owners. This misrepresents ownership and breaks atomicity. This PR separates two orthogonal concepts: - **Ownership** (`applicationId`), the true owner: the caller's application (the workspace custom app today, 3rd-party apps later). - **Protection** (`isSystemSideEffect`), the row was generated by the system, so user mutations route to overrides, deletion becomes deactivation, and reset restores defaults. Once side effects are re-owned to the caller, the old `applicationId === standardApp` check can no longer tell an original side-effect row from a user-added one so a dedicated `isSystemSideEffect` flag carries the protection instead. This is **PR 1 of 2** (forward-only). It makes newly created objects and fields correct; existing workspaces are handled by a follow-up backfill (see *Out of scope*). ## What this PR does - **`isSystemSideEffect` column** on the 8 affected entities (`view`, `viewField`, `indexMetadata`, `commandMenuItem`, `pageLayout`, `pageLayoutTab`, `pageLayoutWidget`, `fieldMetadata`), with `@WasIntroducedInUpgrade` + an entry in the flat-entity property configuration (`toCompare: true`, read-only). - **Single atomic migration in `createOneObject`**: the three `validateBuildAndRunWorkspaceMigration` calls are merged into one, owned by the caller (`resolvedOwnerFlatApplication`) and the record-page view/fields, page layout, and navigation command item are re-owned to the caller and flagged `isSystemSideEffect: true`. `buildNavigationFlatCommandMenuItem` is parameterized with `applicationUniversalIdentifier` (no longer hardcoded to the standard app). - **Field-creation side effects** (`createManyFields`/`createOneField` already run as a single caller-owned migration, so no re-ownership/merge was needed): the auto-created viewField is flagged `isSystemSideEffect: true`, and a new field now also propagates to the object's **INDEX/table view** (added there as a **hidden** column, `isVisible: false`) in addition to the record-page FIELDS widget. The INDEX view is targeted directly by `key = INDEX` (it is not a page-layout widget), de-duplicated per `(viewId, fieldMetadataUniversalIdentifier)` to respect the per-view unique index. The unique-field index is likewise flagged the inverse relation field stays unflagged (`isSystem: false`). - **Protection predicate** extended: `isCallerOverridingEntity` and the removal/reset split strategies now treat `isSystemSideEffect` rows as protected even when caller-owned (route to overrides / deactivate / reset) and the page-layout-reset guards allow resetting flagged entities. - **Standard compute maps** set the flag consistently so a re-sync produces no diff (standard-object side effects stay `false`; per-object nav command items and custom-object base fields are `true`). - **Read-only GraphQL exposure** of `isSystemSideEffect` on the view / view-field / page-layout / tab / widget / command-menu-item DTOs (not exposed on create/update inputs). => Todo: needs to take this new flag into account. This is fine for now because isSystem remains on object/field. - **Fast instance command** (`2-14`) adding the 8 columns (`NOT NULL DEFAULT false`). ## Scope decisions - **`pageLayout` is not an `OverridableEntity`**, its own row has nothing user-overridable (all customization lives on tabs/widgets). It's dual-purpose (`RECORD_PAGE` side-effect vs. user `DASHBOARD`), so it gets `isSystemSideEffect` for protection only, no `overrides` jsonb. - **`navigationMenuItem` is out of scope.**: Those are side effects only for the metadata API and not marked as "system" (they can be deleted/updated etc...) - **`viewFieldGroup` is not a side effect**, it's only created via the explicit view-field-group API, never by object/field creation, so it gets no flag. ## Out of scope (follow-ups) **PR 2** — slow per-workspace backfill (re-own + flag existing side effects, recreate missing ones) and deterministic v5 identifiers for base fields / pageLayout / tab. <!-- This is an auto-generated description by cubic. --> <a href="https://cubic.dev/pr/twentyhq/twenty/pull/21673?utm_source=github" target="_blank" rel="noopener noreferrer" data-no-image-dialog="true"><picture><source media="(prefers-color-scheme: dark)" srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source media="(prefers-color-scheme: light)" srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img alt="Review in cubic" src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a> <!-- End of auto-generated description by cubic. --> |
||
|
neo773
|
d8d5991977 |
fix(messaging): honor IMAP/SMTP encryption setting instead of inferring it from the port (#21562)
This pull request makes the IMAP and SMTP encryption setting actually honor what the user selects. As per spec there's 3 modes: SSL/TLS (implicit TLS from the start), STARTTLS (it will attempt TLS but if the server doesn't support it, it gracefully falls back to plaintext), NONE (plaintext) Current implementation had a boolean flag for this, this replaces it with the 3 modes Upgrade command to migrate all existing accounts, to not risk breaking anyone's existing account in production we map each account to the mode that matches its current behavior, so nothing changes on the wire /closes #21300 <!-- This is an auto-generated description by cubic. --> <a href="https://cubic.dev/pr/twentyhq/twenty/pull/21562?utm_source=github" target="_blank" rel="noopener noreferrer" data-no-image-dialog="true"><picture><source media="(prefers-color-scheme: dark)" srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source media="(prefers-color-scheme: light)" srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img alt="Review in cubic" src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a> <!-- End of auto-generated description by cubic. --> --------- Co-authored-by: Charles Bochet <charles@twenty.com> |
||
|
Charles Bochet
|
b327999b04 |
fix(server): run 2-14 standard relation label/icon heal as a system build (#21667)
## Problem The `2-14:fix-standard-relation-field-labels-icons` workspace upgrade command aborts the upgrade, failing for every workspace that has drift to heal with: ``` FIELD_MUTATION_NOT_ALLOWED: System fields only allow updating: universalSettings, isActive. Forbidden properties: icon ``` The default relation fields it heals (note/task/attachment/timeline) on standard objects are **system-owned**. The flat-field-metadata validator forbids mutating any property other than `universalSettings`/`isActive` on a system field **unless the migration runs as a system build** (`buildOptions.isSystemBuild`). The command omitted `isSystemBuild`, so it defaulted to `false` and the heal was rejected. ## Fix Pass `isSystemBuild: true` when building the heal migration — consistent with every other standard-metadata upgrade command (2-3, 2-5, 2-7, 2-8, 2-9, 2-10, 2-13, other 2-14 commands). ## Notes - Follow-up to #21658, which added the error-surfacing diagnostics that revealed this root cause but did not include this fix. - `label` and `icon` are both in `FLAT_FIELD_METADATA_RELATION_PROPERTIES_TO_COMPARE`, so the relation-field validator (not gated on `isSystemBuild`) already permits them — the system-build flag was the only blocker. - Dry-run returns before the build step, so this only manifests on real runs. <!-- This is an auto-generated description by cubic. --> <a href="https://cubic.dev/pr/twentyhq/twenty/pull/21667?utm_source=github" target="_blank" rel="noopener noreferrer" data-no-image-dialog="true"><picture><source media="(prefers-color-scheme: dark)" srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source media="(prefers-color-scheme: light)" srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img alt="Review in cubic" src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a> <!-- End of auto-generated description by cubic. --> |
||
|
Charles Bochet
|
ee6c9db33a |
fix(server): surface validation errors in 2-14 fix-standard-relation-field-labels-icons upgrade command (#21658)
## Problem
The \`2-14:fix-standard-relation-field-labels-icons\` workspace upgrade
command threw a generic error on migration build failure:
\`\`\`ts
if (result.status === 'fail') {
throw new Error(\`Migration failed for workspace \${workspaceId} while
healing standard relation field labels/icons\`);
}
\`\`\`
This discarded \`result.report\` entirely — the structured per-field
validation failures (\`code\`, \`message\`, \`value\`, offending field)
— making real-world upgrade failures impossible to diagnose from logs.
On a recent staging/app-main upgrade, 13 workspaces failed here with no
actionable detail.
## Change
Flatten \`result.report\` into both the logged error and the thrown
message, so failures now print the actual validation errors per field,
e.g.:
\`\`\`
[fieldMetadata] <universalIdentifier> -> SOME_VALIDATION_CODE: <real
reason>
\`\`\`
No behavior change beyond logging/error content — the command still
aborts on failure as before.
## Notes
- Dry-run still returns before the build step, so this only surfaces on
real runs (unchanged).
<!-- This is an auto-generated description by cubic. -->
<a
href="https://cubic.dev/pr/twentyhq/twenty/pull/21658?utm_source=github"
target="_blank" rel="noopener noreferrer"
data-no-image-dialog="true"><picture><source
media="(prefers-color-scheme: dark)"
srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source
media="(prefers-color-scheme: light)"
srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img
alt="Review in cubic"
src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a>
<!-- End of auto-generated description by cubic. -->
|
||
|
nitin
|
8a866dba54 |
Add call recording schema and meeting bot scaffold (#21584)
## Summary - add 2.13 upgrade commands for call recording request status and dropping CalendarEvent recordingPreference - remove the recording preference from the core CalendarEvent standard object - add a scaffold-generated twenty-meeting-bot app with logo and the CalendarEvent meetingBotPreference field ## Tests - yarn install - yarn lint - yarn twenty dev:typecheck - git diff --check <!-- This is an auto-generated description by cubic. --> <a href="https://cubic.dev/pr/twentyhq/twenty/pull/21584?utm_source=github" target="_blank" rel="noopener noreferrer" data-no-image-dialog="true"><picture><source media="(prefers-color-scheme: dark)" srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source media="(prefers-color-scheme: light)" srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img alt="Review in cubic" src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a> <!-- End of auto-generated description by cubic. --> |
||
|
martmull
|
c848ac34fd |
Fix default view widget visibility (#21590)
Tim logged on the left, Phil on the right, Tim created view widget ## Before <img width="1512" height="938" alt="image" src="https://github.com/user-attachments/assets/7f993f41-a244-42db-b56a-b17c15fb3409" /> ## After Fix Tim created a second view widget, Phil can see it <img width="1512" height="982" alt="image" src="https://github.com/user-attachments/assets/3908a70d-538b-4adf-95df-7373a2f6e269" /> ## After slow migration Phil can see first widget <img width="1512" height="982" alt="image" src="https://github.com/user-attachments/assets/6b30429e-1acf-4119-ba32-26db3155975e" /> <!-- This is an auto-generated description by cubic. --> <a href="https://cubic.dev/pr/twentyhq/twenty/pull/21590?utm_source=github" target="_blank" rel="noopener noreferrer" data-no-image-dialog="true"><picture><source media="(prefers-color-scheme: dark)" srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source media="(prefers-color-scheme: light)" srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img alt="Review in cubic" src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a> <!-- End of auto-generated description by cubic. --> --------- Co-authored-by: Weiko <corentin@twenty.com> |
||
|
Manish Kumar
|
25b0e4d81c |
fix: #19173 correct labels and icons for custom object default relations (#19224)
**### Problem**
When creating a custom Data Model object, the auto-generated Note and
Task relations had incorrect labels ("Note Targets", "Task Targets") and
a wrong hardcoded icon (IconBuildingSkyscraper).
Expected behavior is to use user-friendly labels ("Notes", "Tasks") and
proper icons, consistent with standard objects like Company and Person.
**Root causes:**
* `icon` in `createFieldInput` was hardcoded to
`'IconBuildingSkyscraper'`
* `label` was derived from `targetFlatObjectMetadata.labelPlural`, which
returns system labels (e.g., "Note Targets") instead of display labels
---
**Fix**
* Added `sourceFieldOverridesByRelationObjectNameSingular` map to define
correct labels and icons for all default relation types
* Ensures consistency with standard objects
Mappings:
* noteTarget: "Note Targets" → "Notes", IconBuildingSkyscraper →
IconNotes
* taskTarget: "Task Targets" → "Tasks", IconBuildingSkyscraper →
IconCheckbox
* attachment: "Attachments" → "Attachments", IconBuildingSkyscraper →
IconFileImport
* timelineActivity: "Timeline Activities" → "Timeline Activities",
IconBuildingSkyscraper → IconTimelineEvent
* favorite: "Favorites" → "Favorites", IconBuildingSkyscraper →
IconHeart
* Added type safety using:
`satisfies Record<(typeof
DEFAULT_RELATIONS_OBJECTS_STANDARD_IDS)[number], ...>`
This ensures new default relations must be explicitly defined
* Renamed variable:
`icon` → `targetFieldIcon`
for better clarity (it is only used for the target field)
---
**Limitations**
* Applies only to newly created custom objects
* Existing objects will keep incorrect labels/icons
* Requires a separate data migration to fix existing data
---
**Testing**
1. Go to Settings → Data Model
2. Create a new custom object
3. Verify:
* Labels show "Notes" and "Tasks" (not "Note Targets"/"Task Targets")
* Icons match those used in standard objects (e.g., Company, Person)
---
## Update (reworked while merging main)
The original approach was reworked:
- The label/icon mapping now lives in a shared
`STANDARD_RELATION_FIELD_PROPERTIES_BY_RELATION_OBJECT` constant
(`msg`-based, so labels stay translatable), used as the single source of
truth. Dropped the unused `favorite` entry.
- Standard objects now reference that same constant explicitly at each
call site (uniformization) instead of duplicating the values. Objects
that intentionally differ keep their explicit overrides: note/task →
`Relations`, person/workspaceMember → `Events`, workflow attachments →
`IconFileUpload`.
- Fixed an unrelated typo found along the way: Company's
`timelineActivities` icon was `IconIconTimelineEvent`.
- For the history (supersedes the "Limitations" above): added a `2.9.0`
workspace upgrade command
`upgrade:2-9:fix-standard-relation-field-labels-icons` that re-syncs
**standard** objects' default relation labels/icons against the source
of truth. It deliberately leaves **custom** objects untouched — their
relation fields are user-editable and must not be overwritten by an
upgrade.
## Testing / Verification
Verified locally end-to-end:
**New custom objects**
- Created a custom object via the Data Model UI and via the metadata API
— its note/task/attachment/timeline relations now show `Notes` / `Tasks`
/ `Attachments` / `Timeline Activities` with the correct icons instead
of `Note Targets` + `IconBuildingSkyscraper`.
**Standard uniformization (value-preserving)**
- Re-seeded a workspace on this branch and inspected all 25
default-relation field definitions across the 10 standard objects: every
canonical value is unchanged, every intentional variant (Relations /
Events / IconFileUpload) is preserved, and the only diff vs `main` is
the Company `IconIconTimelineEvent` → `IconTimelineEvent` fix.
**Upgrade command (existing workspaces)**
- Simulated a real upgrade: seeded a workspace on `main` (Company icon
typo present), created a custom object via the metadata API (it came out
with the old buggy labels, as expected on `main`), then switched to this
branch and ran the command.
- Confirmed via both the metadata API and direct DB inspection:
Company's standard `timelineActivities` icon healed to
`IconTimelineEvent`, while the custom object's relations were left
untouched.
- Idempotent: re-running reports "already up to date".
**CI**: typecheck, lint, server unit tests, and all server
integration-test shards green.
---------
Co-authored-by: Manish Kumar <manishkumar@Mac.lan>
Co-authored-by: Charles Bochet <charles@twenty.com>
|
||
|
Félix Malfait
|
0baa333809 |
feat(lint): forbid data mutations in fast instance command up() (#21547)
## Why Fast instance commands run in the ArgoCD **PreSync** hook, before the new pods roll. A bulk `UPDATE`/`INSERT`/`DELETE` held in the **same transaction** as an `ADD COLUMN`/`ALTER` keeps an `ACCESS EXCLUSIVE` lock on the table for the whole write, blocking every read of it. That is what froze prod during the 2.13 `isUIReadOnly → isUIEditable` rename — a bulk `UPDATE "fieldMetadata"` inside the same `up()` transaction as the `ADD COLUMN`s → read timeouts → failed PreSync → aborted sync. @charlesBochet already caught this exact pattern by hand on #21527 ("data migration => make a slow instance command :)"). This turns that manual review into something CI enforces. ## What New oxlint rule **`twenty/no-data-mutation-in-fast-instance-command`**: - Flags statement-leading `UPDATE`/`INSERT`/`DELETE`/`MERGE` passed to `.query(...)` **inside `up()`** of a `*-instance-command-fast-*` file. - Allows: schema DDL (`ALTER`/`CREATE`/`DROP`); `ON DELETE CASCADE` / a column named `updatedAt` (not statement-leading, so never matched); rollback DML in `down()`; and data migrations in **slow** commands' `runDataMigration()`. - The error message points the author straight at the slow-command pattern. Enabled as `error` in `twenty-server`. ## Grandfathering Scoping to `up()` means **only one** existing file violates the rule: the already-shipped 2.13 rename command. It's recorded complete in cloud and must not be rewritten, so it's grandfathered with a documented file-level `oxlint-disable` (the comment makes clear it's an exception, not a precedent). The four other fast commands that contain DML keep theirs in `down()` and are correctly unaffected. ## Tests - 9 RuleTester cases — valid: DDL, FK cascade, `updatedAt`, `down()` DML, slow-command DML, non-upgrade files; invalid: `UPDATE`/`INSERT`/`DELETE` in `up()`. - Verified end-to-end with oxlint: a throwaway violating file → 1 error; all 141 upgrade-command files → 0 errors; full oxlint-rules suite 225/225; typecheck clean. Part of the v2.13 deploy post-mortem follow-ups. https://claude.ai/code/session_013Az1etaGyxWRRVhgjhPWeB --- _Generated by [Claude Code](https://claude.ai/code/session_013Az1etaGyxWRRVhgjhPWeB)_ <!-- This is an auto-generated description by cubic. --> <a href="https://cubic.dev/pr/twentyhq/twenty/pull/21547?utm_source=github" target="_blank" rel="noopener noreferrer" data-no-image-dialog="true"><picture><source media="(prefers-color-scheme: dark)" srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source media="(prefers-color-scheme: light)" srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img alt="Review in cubic" src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a> <!-- End of auto-generated description by cubic. --> --------- Co-authored-by: Claude <noreply@anthropic.com> |
||
|
Félix Malfait
|
9901fa93d9 |
fix(server): write 2.13 UI capability flags directly, bypassing validation (#21543)
## Context The `v1.22 → v2.13.x` cross-version upgrade test (twenty-infra) still fails *after* #21537. This is the **same root cause from a deeper layer**, and the fix here ends the class. ## What's actually happening The 2.13 `SyncStandardUiCapabilityFlags` workspace command heals drifted `isUIEditable`/`isUICreatable` on standard metadata by running a **bulk update through `validateBuildAndRunWorkspaceMigration`** — the validation pipeline meant for *user-initiated* metadata edits. That pipeline has multiple "you may not mutate property X on entity Y" guards, and `isSystemBuild: true` only bypasses **some** of them: | guard | gated on `isSystemBuild`? | |---|---| | system-**field** allow-list (`flat-field-metadata-validator.ts:83`) | ✅ bypassed | | system-**object** guard (`flat-object-metadata-validator.ts:63`) | ✅ bypassed | | **relation-field** allow-list (`flat-field-metadata-validator.ts:143`) | ❌ not gated | So the healing command fails on exactly the workspaces that have real drift (the genuinely cross-version-upgraded ones). #21537 patched the relation allow-list by adding `isUIEditable` to it — one guard — and the build then failed on the next. From this run's logs: `Upgrade summary: 42 workspace(s) succeeded, 2 workspace(s) failed` (the 2 drifted workspaces; the build returns `status=fail`, the per-workspace error detail isn't surfaced in logs). **Root cause:** a trusted system flag-backfill should not run through the user-mutation validation layer at all. ## Fix (direct metadata write) `isUIEditable`/`isUICreatable` are UI-affordance columns on `core.fieldMetadata`/`core.objectMetadata` — changing them needs **no workspace-schema migration**. The command now writes them **directly** to those tables (mirroring the 2.13 slow backfill's raw `UPDATE core."objectMetadata"`) and invalidates the flat-metadata cache, bypassing the validation pipeline entirely. Drift detection is unchanged. This removes the whole class of guard rejections instead of patching guards one at a time. ## Verification - `nx typecheck twenty-server` ✅, `oxlint --type-aware` ✅ (the file is intentionally oxfmt-ignored via `**/upgrade-version-command/**`). - ⚠️ I could **not** run a live cross-version repro from the dev container (no Docker/Postgres available here). The fix categorically can't hit the previous failure (the validation pipeline is gone), but the definitive runtime gate is the twenty-infra `cross-version-upgrade` job against a new image. Quick local repro to confirm: on a reset dev DB, flip `isUIEditable` on a standard relation field (e.g. an `activityTargets` `target*` field) so it drifts, run `yarn command:prod upgrade:2-13:sync-standard-ui-capability-flags -w <workspaceId>`, and confirm it completes (pre-fix it threw on the relation field). --- _Generated by [Claude Code](https://claude.ai/code/session_013Az1etaGyxWRRVhgjhPWeB)_ <!-- This is an auto-generated description by cubic. --> <a href="https://cubic.dev/pr/twentyhq/twenty/pull/21543?utm_source=github" target="_blank" rel="noopener noreferrer" data-no-image-dialog="true"><picture><source media="(prefers-color-scheme: dark)" srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source media="(prefers-color-scheme: light)" srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img alt="Review in cubic" src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a> <!-- End of auto-generated description by cubic. --> --------- Co-authored-by: Claude <noreply@anthropic.com> |
||
|
Félix Malfait
|
c05f01f2a3 |
fix(server): repair 2.13 isUIReadOnly→isUIEditable rename fallout (#21504) (#21537)
## Context Follow-up to #21504 ("Rename isUIReadOnly to isUIEditable, add isUICreatable…"), which surfaced two issues: 1. **`column FieldMetadataEntity.isUIReadOnly does not exist`** on twenty-main.com. 2. **`cross-version-upgrade` CI failure** — `SyncStandardUiCapabilityFlags` aborts the v1.22 → 2.13 upgrade. ## Fix 1 — don't drop `isUIReadOnly` in the 2.13 rename command The 2.13 fast instance command physically dropped `isUIReadOnly` from `core."fieldMetadata"` and `core."objectMetadata"`. But migrations run in an ArgoCD **PreSync** hook **before** the new pods roll out (`charts/prod-eu/apps/twenty-server` migration Job is `hook: PreSync`, sync-wave `2`; the api/worker Deployments are sync-wave `10`). So the **previous** release's pods keep serving and still `SELECT isUIReadOnly`, throwing `column ... does not exist` from the moment the column is dropped until the rollout finishes. This keeps the column (already hidden from the app via `@WasRemovedInUpgrade` on both entities) and **defers the physical drop** to a later release. Since 2.13 hasn't shipped to self-hosters yet, the committed command is amended in place. Both tables handled; `isUICreatable` (new, additive column) is unaffected. The eventual physical drop + GraphQL-compat removals are tracked in twentyhq/core-team-issues#2542. ## Fix 2 — allow `isUIEditable` updates on relation field metadata `SyncStandardUiCapabilityFlags` re-syncs `isUIEditable` on standard fields, including morph/relation fields (the activityTargets `target*` relations). The flat-field-metadata validator only permits a fixed property allow-list on relation fields, which omitted `isUIEditable`, so the command failed with `FIELD_MUTATION_NOT_ALLOWED` and aborted the upgrade (leaving workspaces in a FAILED state). `isUIEditable` is a per-field UI-affordance flag that applies to relation fields too, so it's added to the relation-field updatable properties (a constant used **only** by that validator — no diff-engine side effects). ## Coherence notes - Object-level is covered: the drop is deferred on **both** tables, and `ObjectMetadataEntity` has the identical decorators. - `isUICreatable` needs no change: object-only and additive (no drop → no rolling-deploy hazard), and never reaches the field relation allow-list. - The object-metadata validator has no relation allow-list, so there's no object-level analog to change. ## Verification - `nx typecheck twenty-server` ✅ (the `satisfies` guard holds — `isUIEditable` is a `toCompare` property of `fieldMetadata`) - `oxlint --type-aware` + `oxfmt --check` ✅ on changed files - Fix 2's path is exercised end-to-end by the `cross-version-upgrade` CI that originally caught it. --------- Co-authored-by: Claude <noreply@anthropic.com> |
||
|
neo773
|
5d892bdfd0 |
[WIP] Feat/marketing emails (#21173)
Marketing/campaign emails on top of the emailing-domain (SES) feature:
send a broadcast to a hand-picked list, with per-customer-domain
unsubscribe links and opt-out-only **unsubscribe topics**.
## Model
Standard objects (workspace schema, flat-metadata):
- `messageCampaign` — a campaign send (subject, body template, from
address, status, list, optional unsubscribe topic).
- `messageList` + `messageListMember` — the hand-picked audience (person
↔ list join). A campaign's recipients are its list's members; everyone
is sendable unless suppressed.
Core entities (`core` schema, workspace-scoped — readable by the public
unsubscribe flow without a workspace context):
- `unsubscribeTopic` — an opt-out-only category (name, description,
visibility). There is no opt-in subscription state.
- `messageSuppression` — the single consent store: a row with
`unsubscribeTopicId` NULL is a global block; a row with an
`unsubscribeTopicId` and reason `UNSUBSCRIBE` is a per-topic opt-out.
Two partial unique indexes dedupe global vs per-topic rows (Postgres
treats NULLs as distinct).
- `emailingDomain` — the workspace's SES sending domain,
auto-provisioned when an email channel is added (and cleaned up when its
last channel is removed), with verification status + DNS records.
Campaign messages reuse the existing `message` / `messageThread` /
`messageParticipant` model — one outbound `message` per recipient with a
`deliveryStatus` state machine.
## Sending
- `sendMessageCampaign` resolves the audience **under the caller's
permissions**, creates the campaign, and enqueues a single fan-out job
(the request never materializes per-recipient rows or jobs).
- The fan-out job materializes one QUEUED message per recipient
(deterministic ids → idempotent re-runs, reconciles crash-orphaned rows)
and fans out per-recipient send jobs carrying **only ids**.
- Each send job renders per-recipient `{{variable}}` merge fields and
sends via `EmailingDomainSenderService`, which applies suppression
(global + per-topic) and the unsubscribe footer/headers. Suppressed
recipients are recorded `SKIPPED`.
- The campaign finalizes `SENT`, or `SENT_WITH_ERRORS` if any recipient
terminally failed.
- `previewMessageCampaignAudience` returns a pre-send breakdown (total /
without-email / duplicate / globally-unsubscribed / topic-unsubscribed /
sendable), shown as a hint under the composer pickers.
## Unsubscribe
- Encrypted (AES-256-GCM) token carrying workspaceId, address, optional
`unsubscribeTopicId`, `issuedAt`, and a `preview` flag.
- One-click POST (RFC 8058) + `mailto:` — topic-scoped when the token
carries a topic, global otherwise.
- Preferences page: a checkbox per visible topic (checked = still
receiving); submitting creates per-topic opt-outs for unchecked topics
and lifts re-checked ones (UNSUBSCRIBE only — never
`BOUNCE`/`COMPLAINT`, never a global block).
- A **Preview** action in settings opens the live page via a
preview-claim token; opt-out POSTs are no-ops for preview tokens, so
previewing never mutates state.
- SES webhooks: inbound unsubscribe + outbound bounce/complaint →
suppression (race-safe against at-least-once delivery, with reason
escalation that never downgrades).
- Per-customer unsubscribe hostname (Cloudflare DNS); sends are gated on
it being active, except in LOG/demo mode.
## Architecture
Campaign orchestration, suppression, the sender, the unsubscribe
controller, and the SES webhook handlers live in `src/modules/emailing`
+ `src/modules/messaging-webhooks` (the workspace-feature layer).
`core-modules/emailing-domain` keeps the SES driver, domain
provisioning, the `unsubscribeTopic` / `messageSuppression` core
entities, and the unsubscribe token/hostname plumbing. Domain creation
is validated (`CreateEmailingDomainInput` — domain-format regex,
lowercased) before any value reaches SES or the unsubscribe hostname.
## Frontend
- Campaign composer side panel (from / list / unsubscribe topic /
subject / body) with a live audience-preview hint.
- Email settings: email channels each showing their auto-provisioned
sending domain in a single section (status + DNS records + a "Check
verification" action), plus an **Unsubscribe Topics** section to
create/manage topics and preview the recipient page. A demo-mode banner
is shown when the LOG driver is active.
---------
Co-authored-by: Félix Malfait <felix@twenty.com>
Co-authored-by: Félix Malfait <felix.malfait@gmail.com>
|
||
|
Félix Malfait
|
76f69efb43 |
Keep synced messages and events when removing a workspace member (#21443)
## Context Removing a workspace member deletes their connected accounts, which cascades into deleting every message and calendar event those accounts synced. For a CRM, losing the email history of departed teammates is a big deal. ## What this does Connected accounts are now kept and reassigned instead of deleted when a member is removed: - Ownership moves to the acting user (whoever removed the member). When members remove themselves (leave workspace, account deletion), it falls back to the oldest admin. - OAuth tokens are revoked, credentials wiped, message/calendar channels get `isSyncEnabled = false`, and the account is stamped with a new `archivedAt` column (fast instance command included). - Synced messages, threads and calendar events stay in the workspace. Channel visibility settings keep applying as before, since channels and associations survive. - The reassigned account appears in the new owner's Settings → Accounts, where it can still be deleted (with its data) like any other account. The transfer happens synchronously during removal, while the member's userWorkspace row still exists. This also removes `DeleteWorkspaceMemberConnectedAccountsCleanupJob` and its listener: the async job had to reconstruct the account-owner link from rows the removal flow had just deleted, which was race-prone (see 2181fb541e). Archived accounts are excluded from the workflow send-email default account resolution, and both removal confirmation modals now mention what happens to synced data. --------- Co-authored-by: claude[bot] <41898282+claude[bot]@users.noreply.github.com> |
||
|
Weiko
|
f63f053444 |
CommandMenuItem overridable entity (#21486)
## Context Second PR of the overridable-entities track (after #21436 for views): command menu items become overridable so that edits on non-owned items are stored as overrides instead of mutating the row, and deletion/deactivation becomes reversible. ## What this does - `CommandMenuItemEntity` now extends `OverridableEntity<CommandMenuItemOverrides>` (adds `isActive` + `overrides`). All editable properties are overridable for now (to discuss). - **Update**: mutations on a command item not owned by the caller (standard items) are written into `overrides`; reads merge them in the DTO. The command palette edit mode (pin, reorder, shortLabel) now preserves standard values, "Reset label to default" gains true post-save semantics. - **Delete**: protected items are deactivated (`isActive = false`) instead of deleted; custom items still hard-delete. - **Object deactivate/enable toggle**: now flips `isActive` on the command item (merged into the main migration call) instead of delete/recreate; a create-if-missing fallback covers legacy deactivated objects. - **Front**: inactive command items are filtered out of the palette selector. <!-- This is an auto-generated description by cubic. --> <a href="https://cubic.dev/pr/twentyhq/twenty/pull/21486?utm_source=github" target="_blank" rel="noopener noreferrer" data-no-image-dialog="true"><picture><source media="(prefers-color-scheme: dark)" srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source media="(prefers-color-scheme: light)" srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img alt="Review in cubic" src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a> <!-- End of auto-generated description by cubic. --> --------- Co-authored-by: Charles Bochet <charles@twenty.com> |
||
|
Félix Malfait
|
036d9a2bcf |
feat: gate record creation on isUICreatable only, decoupled from isSystem (#21527)
## Context The generic "create a record" UI affordance previously required `!isSystem`, conflating two distinct concerns: **"hidden from Data Model"** and **"not user-creatable"**. This blocked legitimately creatable system objects (e.g. marketing message lists kept `isSystem: true` only to stay out of the Data Model). This PR makes creatability depend on `isUICreatable` alone, so visibility (`isSystem`) and creatability (`isUICreatable`) become independent. ## Changes - **Front gate** (`canCreateRecordsForObjectMetadataItem.ts`): drop the `!isSystem` clause — creatability is now `isUICreatable && !readOnly`. Updated comment + unit test. - **Command menu** (`standard-command-menu-item.constant.ts`): drop the matching `not objectMetadataItem.isSystem` clause from the `createNewRecord` availability expression so the "Create new X" command mirrors the front gate. Existing workspaces get this via the already-present `SyncCreateRecordCommandAvailabilityExpressionCommand`, which re-syncs from the live definition. - **Standard object audit** (`create-standard-flat-object-metadata.util.ts`): the 15 sync/system-created standard objects that relied on `!isSystem` to stay non-creatable now set `isUICreatable: false` (attachment, blocklist, calendar*/message*/note/task targets, message, messageThread, messageParticipant, timelineActivity, callRecording, workflowAutomatedTrigger, …). `workflowRun`/`workflowVersion`/`workspaceMember` were already `false`. Non-system objects (company, person, note, opportunity, dashboard, task, workflow) are untouched. - **Backfill**: new fast instance command (`2-13-…-1781277480000-backfill-non-ui-creatable-standard-system-objects.ts`) runs `UPDATE core.objectMetadata SET isUICreatable = false` for those standard system objects (symmetric `down`), registered in `instance-commands.constant.ts`. `isSystem` and Data-Model visibility logic are unchanged. ## Verification - Front gate unit test (7 pass), standard-application suite incl. callRecording (10 pass) - `oxlint` clean on all changed files - `twenty-server` and `twenty-front` typecheck green 🤖 https://claude.ai/code/session_01TF4kjD56hHjP31wkHPxPv3 --- _Generated by [Claude Code](https://claude.ai/code/session_01TF4kjD56hHjP31wkHPxPv3)_ <!-- This is an auto-generated description by cubic. --> <a href="https://cubic.dev/pr/twentyhq/twenty/pull/21527?utm_source=github" target="_blank" rel="noopener noreferrer" data-no-image-dialog="true"><picture><source media="(prefers-color-scheme: dark)" srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source media="(prefers-color-scheme: light)" srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img alt="Review in cubic" src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a> <!-- End of auto-generated description by cubic. --> --------- Co-authored-by: Claude <noreply@anthropic.com> |
||
|
Félix Malfait
|
1efa3567ef |
Rename isUIReadOnly to isUIEditable, add isUICreatable, expose both to app developers (#21504)
<!-- CURSOR_AGENT_PR_BODY_BEGIN -->
# UI capability flags: `isUIEditable` + `isUICreatable`
## Per-verb capability model
This PR replaces the negative `isUIReadOnly` metadata flag with
positive, per-verb capability flags (à la Salesforce
`createable`/`updateable`):
- **`isUIEditable: boolean`, default `true`** — rename of `isUIReadOnly`
with inverted polarity, on **both** `objectMetadata` and
`fieldMetadata`. It is one concept ("can the user edit this through the
generic UI?") at two altitudes, so it carries one name at both levels.
- **`isUICreatable: boolean`, default `true`** — new, **object-level
only** (fields have no create verb). When `false`, no generic UI
affordance to create a record of this object appears anywhere (table "+"
buttons, board column add, calendar add, relation-section "Add new",
record picker "Add new", command-menu create action and its keyboard
shortcut).
Both flags are **UI-affordance flags only**: the server does not block
create/edit mutations based on them, so the system, API, and workflows
continue to mutate these records freely. They are orthogonal statements
about the object's nature with no implication rule in the data model.
Because today's inline creation UX creates a blank record the user must
then edit, the frontend create predicate currently requires both
`isUICreatable` and effective editability.
There is no CREATE permission in `ObjectPermissions`; the frontend keeps
gating creation on `canUpdateObjectRecords` as a proxy, ANDed with the
new flags.
## Unified create predicate
All generic creation entry points now flow through one predicate,
`canCreateRecordsForObjectMetadataItem` (`isUICreatable` && not
`isSystem` && not effectively read-only, where effective read-only
covers `isUIEditable`, `isRemote`, and the `canUpdateObjectRecords`
proxy via `isObjectMetadataReadOnly`). This deletes the previously
hardcoded suppression lists:
- `isRecordTableCreateDisabled.ts` and its hardcoded
`WorkflowRun`/`WorkflowVersion` list — deleted; those objects (plus
`workspaceMember`) now declare `isUICreatable: false` in the standard
application instead.
- The hardcoded `workspaceMember` guard inside
`useAddNewRecordAndOpenSidePanel.ts` — deleted.
- The `CREATE_NEW_RECORD` command menu item's availability expression
now checks `objectMetadataItem.isUICreatable`, `isUIEditable`,
`isSystem`, and `isRemote`; a workspace upgrade command re-syncs the
expression in existing workspaces.
Component-local conditions (soft-delete filter active, layout
customization mode) stay in their components.
## GraphQL compatibility and removal plan
The schema delta versus main is **purely additive plus deprecations —
zero breaking changes**:
- `isUIReadOnly` remains on both the ObjectMetadata and FieldMetadata
GraphQL output types for **one release** as a deprecated field computed
as `!isUIEditable` (`deprecationReason: 'Use isUIEditable'`). The Twenty
frontend no longer queries it.
- `isUIReadOnly` also remains on the **input side** for one release
(`CreateFieldInput`, `UpdateFieldInput`, `FieldFilter`, `ObjectFilter`),
keeping the schema shape identical to main for those members. On create
it acts as a legacy alias mapped to `!isUIReadOnly` (`isUIEditable` wins
when both are provided); on update it is ignored, exactly as on main (it
was never an editable property). Filtering on the deprecated member
keeps working until the column is dropped at upgrade time; after that it
is a deprecated no-op surface kept only for schema compatibility.
**Removal plan for next release: drop `isUIReadOnly` from the output
DTOs (and resolvers' `@ResolveField`s), from the input/filter types,
from the create-input mapping, and the `@WasRemovedInUpgrade`-retained
entity columns and decorators.**
## ⚠️ Webhook / database-event payload shape change
The `database-event-payload` type in `twenty-shared` got a clean rename
(no alias): metadata snapshots in webhook and database-event payloads
now carry `isUIEditable` (and `isUICreatable` at object level) **instead
of** `isUIReadOnly`, with inverted polarity. Consumers of these payloads
that read `isUIReadOnly` must switch to `isUIEditable`.
## New manifest properties (app-developer DX)
Application developers can now set these flags in their app manifests
(purely additive — existing manifests and older `twenty-sdk` versions
are unaffected, defaults apply when omitted):
- `objects[].isUICreatable?: boolean` (default `true`)
- `objects[].isUIEditable?: boolean` (default `true`)
- `fields[].isUIEditable?: boolean` (default `true`)
The manifest converters previously hardcoded `isUIReadOnly: false`; they
now read the manifest values with `?? true` defaults. The types are
re-exported through `twenty-sdk` from `twenty-shared`.
## Migration & backfill
- One fast instance command: adds `isUIEditable` (NOT NULL default
`true`) on `core."objectMetadata"` and `core."fieldMetadata"`, backfills
`isUIEditable = false` exactly where `isUIReadOnly = true`, drops
`isUIReadOnly`, and adds `isUICreatable` (default `true`) on
`objectMetadata`. The `down` is the exact inverse. Uses `ADD/DROP COLUMN
IF (NOT) EXISTS`, matching the 2-12 drop-`isCustom` precedent. Verified
up and down in separate transactions against a dev database with exact
backfill counts.
- **Cross-version upgrade safety (multi-version self-hosted jumps):**
the upgrade sequence interleaves per version (instance → workspace
commands), so pre-2.13 workspace commands run **before** the 2.13 rename
when an old instance jumps several versions. Following the `isCustom`
precedent: `isUIEditable`/`isUICreatable` are marked
`@WasIntroducedInUpgrade` and `isUIReadOnly` stays on both entities as
`@WasRemovedInUpgrade`, so the upgrade-aware entity metadata adapter
hides the not-yet-existing columns (and keeps the legacy column live) at
pre-2.13 cursors. **No committed upgrade command outside the 2-13
directory is modified**: the old 1-21/2-8/2-9 commands keep their
original `isUIReadOnly: true` inputs, which still compile (entity
property retained, deprecated create-input alias mapped) and still
produce the correct legacy column writes pre-rename.
- A 2-13 workspace command (`sync-standard-ui-capability-flags`)
re-syncs `isUICreatable` **and** `isUIEditable` on standard objects and
`isUIEditable` on standard fields from the standard-application
definitions. This backfills `isUICreatable: false` on
`workflowRun`/`workflowVersion`/`workspaceMember` and heals fields
created mid-cross-upgrade by pre-2.13 commands (whose hidden
`isUIEditable` value cannot reach the insert). Both 2-13 sync commands
pass `isSystemBuild: true` — the flat metadata validator otherwise
rejects direct updates to system objects (verified against a
deliberately drifted dev database; the run is idempotent).
- A second 2-13 workspace command re-syncs the create-record command
availability expression.
## Testing
- Unit tests for `canCreateRecordsForObjectMetadataItem`
(flag/permission/system combinations) and for the manifest converters
(flags set / omitted → defaults).
- Full `upgrade --dry-run` boots the sequence (107 steps) and validates
the upgrade-aware decorator references; both 2-13 sync commands verified
end to end against real drift and re-run idempotently.
- Schema verified by live introspection after the input-alias restore:
all four input/filter members match main, output deprecations intact;
frontend metadata types and `twenty-client-sdk` schema regenerated from
the running server.
- Read-only-related and touched jest suites pass on both packages;
typecheck and lint pass on `twenty-server` and `twenty-front`.
<!-- CURSOR_AGENT_PR_BODY_END -->
<div><a
href="https://cursor.com/agents/bc-0f3e04cb-b04a-40be-8330-5609c4538e8a"><picture><source
media="(prefers-color-scheme: dark)"
srcset="https://cursor.com/assets/images/open-in-web-dark.png"><source
media="(prefers-color-scheme: light)"
srcset="https://cursor.com/assets/images/open-in-web-light.png"><img
alt="Open in Web" width="114" height="28"
src="https://cursor.com/assets/images/open-in-web-dark.png"></picture></a> <a
href="https://cursor.com/background-agent?bcId=bc-0f3e04cb-b04a-40be-8330-5609c4538e8a"><picture><source
media="(prefers-color-scheme: dark)"
srcset="https://cursor.com/assets/images/open-in-cursor-dark.png"><source
media="(prefers-color-scheme: light)"
srcset="https://cursor.com/assets/images/open-in-cursor-light.png"><img
alt="Open in Cursor" width="131" height="28"
src="https://cursor.com/assets/images/open-in-cursor-dark.png"></picture></a> </div>
<!-- This is an auto-generated description by cubic. -->
<a
href="https://cubic.dev/pr/twentyhq/twenty/pull/21504?utm_source=github"
target="_blank" rel="noopener noreferrer"
data-no-image-dialog="true"><picture><source
media="(prefers-color-scheme: dark)"
srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source
media="(prefers-color-scheme: light)"
srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img
alt="Review in cubic"
src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a>
<!-- End of auto-generated description by cubic. -->
---------
Co-authored-by: Cursor Agent <cursoragent@cursor.com>
|
||
|
Charles Bochet
|
577b22df46 |
fix(upgrade): invalidate upgrade-status cache on command end (#21497)
## Problem The "Twenty / Upgrade Status" Grafana dashboard shows stale workspace counts (e.g. `N behind / 0 up-to-date` while the instance reads `UP_TO_DATE`) that disagree with `command:prod upgrade:status`. The CLI is correct; the dashboard lags, sometimes for the full hour. ## Root cause The dashboard is fed by the `twenty_upgrade_workspaces_*` gauges, which read their workspace counts from a Redis snapshot (`UpgradeStatusCacheService`). That snapshot is only invalidated **per-command, inside the runners' `finally` blocks**. Two gaps: 1. An instance command that is already applied returns **before** its invalidation runs (`isAlreadyCompleted` early-return in `InstanceCommandRunnerService`). So a plain **redeploy** — which changes the deployed upgrade sequence, and thus the "behind" answer, without executing any command — never refreshes the snapshot. This is most visible on an instance-only release. 2. The snapshot then stays frozen until its 60-minute TTL, while the CLI reads live and disagrees. "Behind" is derived from the deployed sequence, not just the ledger, so the correct answer changes on events (deploys) that run no command — which is exactly why per-command invalidation isn't enough on its own. ## Fix Invalidate the upgrade-status cache **once, unconditionally, at the end of both upgrade entrypoints** — `run-instance-commands` (the deploy/migrate step) and `upgrade` — in a `finally`. Every run, including a no-op redeploy where all commands are already applied, now clears the snapshot, so the next gauge scrape recomputes against the current sequence. Best-effort (failures are logged, never block the command). The existing per-command invalidation is kept for mid-run progress. This keeps the read path untouched. ## Reproduction + verification (live, local) Served twenty-server (`NODE_PORT=4000`, `METER_DRIVER=prometheus`) against the seeded DB, whose latest version `2.12.0` is instance-only. 1. Froze the gauge at `behind 4 / up_to_date 0` while the DB was brought up-to-date (snapshot not invalidated) — reproduced the dashboard/CLI divergence. 2. Ran the **patched** `run-instance-commands --force`. Every step logged `already executed, skipping` — and the `finally` still deleted the Redis snapshot. 3. On the next recompute the gauge self-healed to `instance_health 1, behind 0, up_to_date 4`, matching the live CLI. With the old code the snapshot stayed frozen at `behind 4` until the TTL. |
||
|
Weiko
|
cfb9772179 |
feat(server): convert view to overridable entity (#21436)
## Context Every entity created as a side effect of object creation must support the overridable pattern (`isActive` + `overrides` + override routing) before we can re-own side effects to their true application. Starting with View. viewField, viewFieldGroup, pageLayoutTab and pageLayoutWidget already extend `OverridableEntity`. This PR brings `view` to the same pattern. ## What this does - `ViewEntity` now extends `OverridableEntity<ViewOverrides>` (adds `isActive` boolean + `overrides` jsonb). All editable view properties are overridable; the 3 fieldMetadata foreign keys are converted to/from universal identifiers like viewField's `viewFieldGroupId`. - **Update**: mutations on a view not owned by the caller (e.g. standard views like "All Companies") are written into `overrides` instead of mutating the row. Reads merge overrides in the DTO. - **Delete/destroy**: views not owned by the caller are deactivated (`isActive = false`) instead of deleted. ~~- **INDEX invariant**: `key = INDEX` views can only be created via object-creation side effect. The API now rejects creating, deleting or destroying INDEX views (object-deletion cascade is unaffected). This was not really needed for this migration but was flagged during implementation.~~ - **Front**: views with `isActive = false` are filtered out of the views selector. - Fast instance command adds the two columns (`2-12-instance-command-fast-...-view-overridable-entity.ts`). ## Notes - Custom (caller-owned) views behave exactly as before: direct updates, soft delete. - View-group side effects (kanban groups) are computed on the override-merged view so overridden `mainGroupByFieldMetadataId` works. |
||
|
Weiko
|
9c66975520 |
isCustom deprecation for Objects and Fields (#21228)
## Context
`isCustom` was a legacy denormalized boolean on `ObjectMetadataEntity`
and `FieldMetadataEntity`.
Now that every metadata row carries `applicationId` (via
`SyncableEntity`), "is this custom" is fully derivable, and the stored
boolean was a redundant second source of truth that could drift.
The real meaning of `isCustom` is **"the owning application is not the
twenty-standard application"** — i.e. `!belongsToTwentyStandardApp`.
Note this is *not* "belongs to the workspace custom app" as I initially
thought: third-party-application
objects/fields are custom too.
The standard application has a globally stable `universalIdentifier`, so
the value derives with no per-workspace lookup.
## Changed
## `isCustom` checks — before → after
`isCustom` is no longer a stored column. The table below lists every
site that branched on it and how it resolves now. The unifying rule:
`isCustom ≡
!isTwentyStandardApplicationUniversalIdentifier(applicationUniversalIdentifier)`.
### Server — behavioural checks
| Location | Purpose | Before | Now |
|---|---|---|---|
| `utils/compute-object-target-table.util.ts` | Physical table name `_`
prefix | `computeTableName(nameSingular, objectMetadata.isCustom)` |
derives from `applicationUniversalIdentifier` (single source for all
table-name callers) |
| `twenty-orm/factories/entity-schema.factory.ts` +
`…/entity-schema-metadata.type.ts` | ORM table name (hot path) |
`object.isCustom` | `object.applicationId !== standardApplicationId`
(computed in `buildEntitySchemaMetadataMaps`) |
|
`twenty-orm/repository/workspace-{delete,soft-delete,update}-query-builder.ts`
| Table name for mutations | `computeTableName(nameSingular,
objectMetadata.isCustom)` | `computeObjectTargetTable(objectMetadata)` |
| `index-metadata/utils/generate-deterministic-index-name-v2.ts` | Index
name hash (must stay bit-identical) | `flatObjectMetadata.isCustom` |
derives from `applicationUniversalIdentifier` |
| `object-metadata/object-record-count.service.ts` | Table name for
record count | `computeTableName(nameSingular, isCustom)` |
`computeObjectTargetTable(flatObjectMetadata)` |
|
`workspace-manager/dev-seeder/data/services/dev-seeder-data.service.ts`
| Match seed config by table name | `computeTableName(item.nameSingular,
item.isCustom)` | `computeObjectTargetTable(item)` |
| `commands/workspace-export/workspace-export.service.ts` +
`…/utils/generate-workspace-schema-ddl.util.ts` | Export table name (raw
entity) | `objectMetadata.isCustom` |
`!isTwentyStandard…(objectMetadata.application?.universalIdentifier)` |
|
`flat-field-metadata/services/flat-field-metadata-type-validator.service.ts`
| Block users creating reserved field types |
`args.flatEntityToValidate.isCustom` |
`!args.flatEntityToValidate.isSystem` |
| `api/common/.../common-create-many-query-runner.service.ts` | Don't
let client overwrite system `createdBy` |
`createdByFieldMetadata.isCustom === false` |
`createdByFieldMetadata.isSystem === true` |
|
`field-metadata/utils/resolve-field-metadata-standard-override.util.ts`
| Skip i18n/overrides for custom fields | `if (fieldMetadata.isCustom)
return raw` | **removed** — falls through on
`isDefined(standardOverrides)` |
|
`object-metadata/utils/resolve-object-metadata-standard-override.util.ts`
| Skip i18n/overrides for custom objects | `if (objectMetadata.isCustom)
return raw` | **removed** — same fall-through |
|
`command-menu-item/utils/build-navigation-interpolation-context.util.ts`
| Override context for nav labels | passed `isCustom` into resolver |
dropped (resolver no longer needs it) |
| `api/common/.../data-arg-processor.service.ts` | `isCustom` for
record-position table name | `flatObjectMetadata.isCustom` | derives
from `applicationUniversalIdentifier` |
| `metadata-modules/minimal-metadata/minimal-metadata.service.ts` |
Minimal DTO + override context | `flatObjectMetadata.isCustom` | derives
from `applicationUniversalIdentifier` |
|
`commands/upgrade-version-command/1-23/…backfill-record-page-layouts.command.ts`
| Filter to custom objects | `objectMetadata.isCustom` |
`!isTwentyStandard…(applicationUniversalIdentifier)` |
### Server — DTO / API population
| Location | Before | Now |
|---|---|---|
|
`flat-object-metadata/utils/from-flat-object-metadata-to-object-metadata-dto.util.ts`
| passthrough `isCustom` | derives from `applicationUniversalIdentifier`
|
|
`flat-field-metadata/utils/from-flat-field-metadata-to-field-metadata-dto.util.ts`
| passthrough `isCustom` | derives from `applicationUniversalIdentifier`
|
|
`object-metadata/utils/from-object-metadata-entity-to-object-metadata-dto.util.ts`
(REST) | `entity.isCustom` | `entity.applicationId !==
standardApplicationId` |
|
`field-metadata/utils/from-field-metadata-entity-to-field-metadata-dto.util.ts`
(REST) | `entity.isCustom` | `entity.applicationId !==
standardApplicationId` |
| `dataloaders/dataloader.service.ts` | passed
`flatFieldMetadata.isCustom` into override resolver | dropped (resolver
no longer needs it) |
> REST controllers (`object-metadata.controller.ts`,
`field-metadata.controller.ts`) resolve `standardApplicationId` once per
request from the cached `flatApplicationMaps`.
### Frontend
| Location | Purpose | Before | Now |
|---|---|---|---|
| `settings/.../SettingsObjectFieldDisabledActionDropdown.tsx` | Whether
an inactive field is deletable | `isDeletable = isCustomField` |
`isDeletable = isCustomField && !isSystemField` |
### Unchanged (out of scope)
`isCustom` on `IndexMetadata` / `View` / `Skill` / `Agent` and their
guards still read the persisted column.
Breaking change is on the isCustom filter on field and object APIs, this
is never used in the FE and unlikely used by external consumers
|
||
|
Félix Malfait
|
dfb3da1f8d |
feat(emailing-domain): add LOG driver for local development (#21286)
## What Adds a `LOG` driver to the emailing-domain feature, selected via a new `EMAILING_DOMAIN_DRIVER` config variable (defaults to `AWS_SES`, so production behavior is unchanged). The LOG driver: - resolves domains to `VERIFIED` instantly (no DNS / SES setup) - logs each `sendEmail` and returns a synthetic `messageId` instead of calling SES It also dev-seeds a pre-verified domain per workspace (`<workspaceId>.dev.twenty.local`) so the feature works out of the box. ## Why The emailing-domain feature currently ships only the AWS SES driver, so the verify → send flow can't be exercised locally (or in CI) without real AWS credentials. This unblocks local development and review of anything built on emailing domains. ## Usage ``` EMAILING_DOMAIN_DRIVER=LOG ``` The seeded `*.dev.twenty.local` domain is already verified; sends are logged (`[log-driver] sendEmail ...`). --------- Co-authored-by: claude[bot] <41898282+claude[bot]@users.noreply.github.com> |
||
|
nitin
|
f72898063a | upgrade command patch - warn instead of throw when no calendarEvent object found in the workspace (#21258) | ||
|
nitin
|
e485b679ea |
[Call Recording] Add standard object (#21158)
Adds **Call Recording** as a first-class standard object (Twenty's
flat-metadata
standard-object system), with a hidden junction to calendar events and a
backfill
command for existing workspaces. Everything is gated behind the
`IS_CALL_RECORDING_ENABLED` feature flag.
### What's included
- **`CallRecording`**: audio/video files, transcript, status, recording
policy,
timing, external bot/recording ids. Label identifier is
`meetingOccurrenceKey`.
- **`CallRecordingCalendarEventAssociation`**: hidden junction linking a
recording
to a calendar event (dedupes one bot to many subscribers of the same
meeting).
- Full metadata graph via the flat-metadata builders: fields, indexes,
views,
view fields/groups, record page layout, and navigation items.
- **Metadata-only reverse relation** on `CalendarEvent`: present in
standard
metadata, omitted from the TS entity class to avoid expanding recursive
nested-insert types.
- **Upgrade command (2.9.0)** backfilling active/suspended workspaces:
- Creates the full graph; idempotent (skips when it already exists).
- Moves a colliding custom `callRecording` object aside to
`callRecordingOld`
(numeric suffix if that name is also taken).
- Navigation items (commands) are flag-gated by `universalIdentifier`,
so a custom object
reusing the name is never gated.
### QA
Run locally against existing workspaces (with and without a name
collision) and a
freshly created workspace:
- [x] Backfill, collision: custom `callRecording` renamed to
`callRecordingOld`;
standard graph created.
- [x] Backfill, no collision: standard graph created; unrelated custom
object untouched.
- [x] Idempotent: re-run is a no-op, with no duplicate metadata and
counts unchanged.
- [x] New workspace via `init()` produces an identical graph to the
backfill
(`universalIdentifier` set-diff = 0).
- [x] Label identifier (`meetingOccurrenceKey`) holds position 0 in
non-widget views.
- [x] Nav items gated behind the feature flag; collision-renamed
object's nav
expression re-pointed to its new name.
- [x] Unit tests cover collision name resolution and nav-gating logic.
|
||
|
Raphaël Bosi
|
41d5d80a65 |
Migrate Company and Person standard fields in preparation for the enrichment app (#21171)
# Migrate Company and Person standard fields in preparation for the
enrichment app
## Why
Our standard `Person`/`Company` objects accumulated fields that aren't
generic to every
business, while missing a more universal revenue field that essentially
every CRM ships.
This PR makes the **Standard application** hold a tighter, more
universal set of fields,
and sets the stage for a follow-up PR that introduces a **People Data
Labs enrichment app**
to populate them.
## What changes
### Standard fields
**Demoted (Standard → Workspace Custom application)** — not generic
enough to ship as standard:
| Object | Field | Type |
| ------- | ------------------------------ | -------- |
| Company | annualRecurringRevenue (ARR) | CURRENCY |
| Company | employees | NUMBER |
| Company | idealCustomerProfile (ICP) | BOOLEAN |
| Company | xLink (X/Twitter) | LINKS |
| Person | xLink (X/Twitter) | LINKS |
| Person | city | TEXT |
**Added (new generic Standard field)** — present in
Salesforce/HubSpot/Zoho, PDL-populatable:
| Object | Field | Type |
| ------- | ------------- |
-------------------------------------------------------- |
| Company | annualRevenue | CURRENCY (generic total revenue; replaces
the niche ARR) |
### Behavior by workspace
* **New workspaces:** demoted fields are gone; `annualRevenue` is
**active**.
* **Existing workspaces:** demoted fields are **preserved as active
custom fields, data intact**;
`annualRevenue` is created **inactive (opt-in)** with its column ready,
so a later activation
is a metadata-only toggle.
### Upgrade commands (v2.9)
Three idempotent, per-workspace commands, run in timestamp order:
1. **`upgrade:2-9:move-demoted-standard-fields-to-custom-application`**
(1799000040000) —
re-owns the 6 demoted fields to the workspace custom application
(`isCustom = true`,
new `applicationId` + fresh `universalIdentifier`), keeping their data
and active state.
2. **`upgrade:2-9:rename-conflicting-custom-fields`** (1799000045000) —
if a workspace already
has a *custom* field named `annualRevenue`, renames it to
`annualRevenueCustom`
(data preserved via column rename) so the standard field can be added.
Skips non-custom matches.
3. **`upgrade:2-9:add-inactive-generic-standard-fields`**
(1799000050000) — creates
`Company.annualRevenue` on existing workspaces as inactive, guarded to
skip workspaces
missing the target object or where the name is still taken.
**Failure model:** the workspace iterator isolates failures per
workspace (one workspace failing
never affects others); within a workspace the runner records per-command
status and resumes on the
next run, and every command is idempotent, so partial runs self-heal.
### Supporting changes
* **Field-option color palette:** widened the `TagColor` union
(`twenty-shared` `FieldMetadataOptions`
+ the field-metadata `options.input` DTO) from 10 colors to the full
theme palette, benefiting any
future SELECT/MULTI_SELECT field.
* **Dev seeder:**
* The default "Annual Recurring Revenue" dashboard widget now points at
the generic
`annualRevenue` field (renamed to "Annual Revenue").
* Removed the "Companies by Size (Stacked by City)" widget (relied on
the demoted `employees`).
* `employees` is dropped from company data seeds and re-added as a
**custom** field seed, so dev
workspaces still get an `employees` column matching the demoted
behavior.
### Cleanup
Front-end record types (`Company.ts`/`Person.ts`), the
`getDisplayNameFromParticipant` test mock,
metadata integration specs, the Zapier `crud_record` test, and the
regenerated
`get-standard-object-metadata-related-entity-ids` snapshot.
## ⚠️ Breaking change (intentional)
Removes standard fields `Company.annualRecurringRevenue`,
`Company.employees`,
`Company.idealCustomerProfile`, `Company.xLink`, `Person.xLink`, and
`Person.city` from the core
GraphQL schema (replaced by `Company.annualRevenue`).
This is why the breaking-changes check reports a large number of
removals — `graphql-inspector`
flags any removed object field plus its derived
aggregate/order-by/filter/update types.
**Mitigation:** the
`upgrade:2-9:move-demoted-standard-fields-to-custom-application` command
re-owns these fields as custom fields per workspace, preserving their
name and data, so existing
tenants keep working. New workspaces won't have them.
|
||
|
nitin
|
475cd9e7fe | fix(upgrade): resequence workflow step logs command (#21213) | ||
|
Marie
|
4b15b949f3 |
Provide additional logsobservability to workflow runs (per node) (#21142)
Surfaces per-step "Logs" tabs in the workflow run side panel so users can see what each step actually did (model + tokens + tool calls for AI, console output for serverless functions, request/response for HTTP, recipients/body for Email). <img width="546" height="501" alt="ai_agent_without_websearch" src="https://github.com/user-attachments/assets/c6ca3518-9489-4484-a570-3d0569ff3b03" /> ## Storage - New `stepLogs` JSONB column on the `workflowRun` workspace entity, typed as `Record<string, WorkflowRunStepLog>` (keyed by step id). - Schema lives in `twenty-shared`: `workflowRunStepLogSchema` with a discriminated `details.type` union for `AI_AGENT | CODE | HTTP_REQUEST | EMAIL` — frontends and backends consume the same Zod-inferred type. - Field is added to existing workspaces via a workspace upgrade command (`2-9 add-workflow-run-step-logs-field`); the standard-object metadata declares it for new workspaces. - Writes happen atomically per step in `WorkflowRunStepLogWorkspaceService.setStepLog` using `jsonb_set`. That lets concurrent steps in the same run write their own keys without contending with the existing lock around `workflowRun.state`. - Per-step payload is hard-capped at 256 KB; anything larger is dropped with a `logger.warn`, so a pathological tool call can never bloat a row. See below for more information. ## How logs are produced **Aalmost everything was already being collected; this PR mostly persists and renders it.** - **AI agent** — `AgentAsyncExecutorService` already tracked token usage, model id, native web-search count, and the AI SDK's `steps[]`. We map those into the log via `mapAiStepsToToolCallLogs` (`searchVector` stripped from record outputs, per-call input/output capped at 32/64 KB, max 200 tool calls per step). The only new measurement is a wall-clock `durationMs` taken around `executeAgent`, and we now fold native web-search cost into the displayed `totalCostInDollars` (it was already billed, just not shown). - **Code / serverless function** — reuses the `console.log` output the function runner already returns (`logsByLevel`); `build-code-step-log.util` only repackages it. - **HTTP request** — built from the action's existing input/output via `build-http-request-step-log.util`. No new signals collected. - **Email (send / draft)** — added `sanitizedHtmlBody` + `plainTextBody` to the existing tool outputs (a small additive change), then `build-email-step-log.util` consumes them. No additional AI inference or external calls are made for logging — the cost is a small CPU overhead per step plus the JSONB write. ## Security The log surface intentionally shows whatever the workflow touched, which made redaction and sanitization the main design concern. - **HTTP — secrets in headers**: existing `SENSITIVE_HEADER_NAMES` set (Authorization, Cookie, …) replaced with `[redacted]` in both request and response. - **HTTP — secrets in URLs**: `SENSITIVE_URL_PARAM_NAMES` (e.g. `api_key`, `token`, `access_token`) replaced in the query string via `URL`-based parsing. - **HTTP — secrets in bodies**: `SENSITIVE_BODY_KEY_REGEX` deep-walks JSON request/response bodies (object input or stringified JSON) and redacts matching keys. Applied to the `error` field too, since transport-layer errors sometimes embed structured payloads. - **Email — XSS risk in body preview**: tool outputs now expose a server-side `sanitizedHtmlBody`; the log builder prefers it over the raw user-authored `input.body`, with `plainTextBody` as a second fallback. The original raw body is only used if sanitization didn't happen (e.g. tool failed before composing). - **AI — internal/noisy data**: `searchVector` (Postgres tsvector strings) is stripped from record outputs returned by Twenty tools to avoid leaking internal full-text-search payloads. - **DB bloat / runaway agents**: 256 KB per-step cap + 32 KB / 64 KB per-tool-call input/output cap + 200 tool calls per step. <img width="547" height="307" alt="logic_function" src="https://github.com/user-attachments/assets/dd4a3d16-67f2-434b-95b3-bdcaf9ed053d" /> ## More details on Log size & truncation Logs are stored in `workflowRun.stepLogs` (JSONB), keyed by `stepId`. ### Per-step cap Each step's log is hard-capped at **256 KB** (`MAX_STEP_LOG_BYTES` in `WorkflowRunStepLogWorkspaceService.setStepLog`). For ~99% of workflows this is roomy — typical real-world sizes: - Code / serverless function: 1–20 KB - HTTP request: 5–70 KB - Email: 5–30 KB - AI agent (a handful of tool calls): 5–50 KB ### Two layers of bounding 1. **Per-field truncation** in each builder (before writing): - **Code**: ≤ 500 entries, ≤ 4 KB per message, ≤ 8 KB stack trace - **HTTP**: ≤ 32 KB per body (request + response), UTF-8 byte-aware - **Email**: ≤ 8 KB body preview, UTF-8 byte-aware - **AI agent**: ≤ 32 KB tool input, ≤ 64 KB tool output, ≤ 200 tool calls/step 2. **Global per-step safety net** at write time: if the assembled `stepLog` still exceeds 256 KB, the write is **dropped entirely** with a `logger.warn`. The workflow itself keeps running unaffected. ### What this means in practice - **Safe**: workflow execution, step results, downstream steps — never blocked by log size. - **Safe**: iterators (each iteration overwrites the previous log for that `stepId`, so they can't accumulate). - **Safe**: step retries (same `stepId` is overwritten, not appended). - **Possible**: an AI agent step with many large tool outputs (e.g., 50+ heavy `web_search` calls) can exceed 256 KB → the **entire** step's log is dropped, side panel shows "No logs were recorded for this step". The user has no explicit signal that the log was dropped due to size (only server-side warn). - **Possible** (theoretical): a workflow with hundreds of distinct steps could push the row toward Postgres's internal ~256 MB jsonb limit. Beyond that, individual `jsonb_set` writes would error and be swallowed by the action's try/catch — workflow still completes. ### Possible future hardening (not in this PR) - Replace "drop entire log" with a stub that preserves the summary card (cost, duration, status) and marks `truncated.reason = 'size_cap'`. - Surface size-drops in the UI (similar to the existing `<StyledTruncatedNotice>`). - Emit a metric so dropped logs are observable in dashboards. |
||
|
Weiko
|
cc76b7bc50 |
Fix fields widget new field visibility (#21111)
Fixes https://github.com/twentyhq/twenty/issues/21043 ## Context Newly created fields were never added to FIELDS widgets, regardless of the "Set fields created in the future as visible" toggle. The widget's newFieldDefaultVisibility was null on widgets that never explicitly set it (it was never populated at creation), so the backend skipped them and no view field was created. ## Implementation Keep newFieldDefaultVisibility nullable with false (not visible) as the behavior when not provided. The FE now reflects that properly and shows "un-toggled" when it's null (iso with BE behavior). The fix also ensures the value is explicitly set to true wherever it should be: - Set newFieldDefaultVisibility: true at every FIELDS widget creation path (backend default record-page layout, frontend createDefaultFieldsWidget + useTemporaryFieldsConfiguration); - Added a 2-9 workspace upgrade command that backfills true onto existing standard FIELDS widgets where the value is null. |
||
|
Charles Bochet
|
58907b733c |
feat(logic-function): add LIVE / PREBUILT execution modes (#20873)
## Summary
### Why
1. Sending the code to the lambda (~1Mb usually) is heavy on network and
results to a constant traffic of ~30Mb/s on AWS which results into TB of
network data every month
2. eval(1MB of code) is not that fast, it's heavy on memory and CPU on
lambda side
### High level
Adds two execution modes for logic functions, gated behind the new
`IS_LOGIC_FUNCTION_PREBUILT_MODE_ENABLED` workspace feature flag (off
everywhere by default):
- **LIVE** (current behavior, preserved bit-for-bit): the compiled
bundle is read from object storage and shipped in every Lambda invoke
payload. Used for fast iteration in the workflow editor / Settings test
runs.
- **PREBUILT** (new): the bundle is installed onto the per-function
Lambda alongside the unified executor, and invocations carry only `{
params, env, handlerName }` — saving JSON payload egress and warm-start
`import()` cost on every call.
### Key design choices
- **Unified Lambda handler** (`constants/executor/index.mjs`) dispatches
at runtime: `event.code` present ? LIVE (write to `/tmp`, dynamic
import) : `import('./prebuilt-logic-function.mjs')`. Both code paths
always coexist on the deployment package, so the same Lambda can serve
either mode without redeploying.
- **Install runs inside the `validateBuildAndRun` migration pipeline**,
not at execute time. `Create/UpdateLogicFunctionActionHandlerService`
calls `driver.installPrebuiltBundle` when `executionMode` flips
LIVE?PREBUILT or `checksum` changes while PREBUILT, gated on
`isBuildUpToDate=true` and a fresh checksum.
- **Strict execute, no reconciliation**:
`LogicFunctionExecutorService.execute` resolves `effectiveExecutionMode`
(caller override > feature flag > entity column). For PREBUILT it asks
the driver `getInstalledBundleChecksum` (Lambda `twenty:bundle-checksum`
tag for AWS, sidecar file locally) and throws
`LOGIC_FUNCTION_PREBUILT_BUNDLE_NOT_INSTALLED` on mismatch.
- **Feature flag gates every side effect**: with the flag off the
executor forces LIVE, the action-handler install hooks bail before AWS,
and workflow activation does not flip the mode. Rollback is just turning
the flag off.
### Lifecycle
- New workflow CODE step ? `LIVE`, no install.
- Workflow activated ? build + activation flips `executionMode=PREBUILT`
? action-handler installs the bundle + sets the Lambda tag.
- Draft from active version ? duplicated logic function reset to `LIVE`.
- App install ? manifest converter sets `PREBUILT`, create-action
handler installs.
- Test runs (`executeOneFromSource`, workflow editor) pass
`executionMode=LIVE` explicitly.
### Observability
`[lambda-timing]` log lines now include `effectiveExecutionMode` and
`payloadBytes`; the action handler logs `install_duration_ms` for each
install.
## Test plan
- [x] `npx nx typecheck twenty-server` ? passes
- [x] `npx oxlint --type-aware` on all changed files ? 0 warnings, 0
errors
- [x] `npx nx test twenty-server` ? 588 suites / 5009 tests pass (no
regressions vs main)
- [x] New unit suite `flat-logic-function-validator.service.spec.ts` ?
9/9
- [x] Existing
`workflow-version-step-operations.workspace-service.spec.ts` ? 8/8
(verified the new token-based DI avoids a circular-import regression)
- [x] Snapshot for
`ALL_UNIVERSAL_FLAT_ENTITY_PROPERTIES_TO_COMPARE_AND_STRINGIFY` updated
to include `executionMode`
- [x] Integration suite `logic-function-execution.integration-spec.ts`
extended to assert `executionMode=LIVE` on newly-created functions and
continues to exercise the LIVE happy path
- [ ] Manual staging rollout: flip
`IS_LOGIC_FUNCTION_PREBUILT_MODE_ENABLED` per workspace, observe
`[lambda-timing]` `payloadBytes` drop + `install_duration_ms`, then ramp
in prod.
|
||
|
nitin
|
13f09d8946 |
[Dashboards] Remove gauge chart types and code (#20410)
Follow-up cleanup to #20172. |
||
|
martmull
|
961745e9ba |
Fix latency spike on application lookup (#21042)
Fixes https://discord.com/channels/1130383047699738754/1509645089062781058 Caching application entities to improve authentication latency |
||
|
Paul Rastoin
|
ebfaca5b3d |
EncryptedString PlaintextString branded string types (#21001)
## Summary closes https://github.com/twentyhq/core-team-issues/issues/2464 Introduces compile-time branded types to distinguish encrypted ciphertext from plaintext strings, preventing mix-ups like the one fixed in #20819 — but at the type level rather in addition to the one existing at runtime. ### Branded string primitives - Created `EncryptedString` and `PlaintextString` as hard nominal brands using `z.string().brand(...)`, making them non-assignable to each other or to raw `string` - Created `isEncryptedString` type predicate to narrow `string` to `EncryptedString` based on the `enc:v2:` envelope prefix - Retyped `SecretEncryptionService`: `encryptVersioned` accepts `PlaintextString`, `decryptVersioned` returns `PlaintextString` ### Entity typing - Typed encrypted columns across entities: `SigningKeyEntity.privateKey`, `TwoFactorAuthenticationMethodEntity.secret`, `ApplicationRegistrationVariableEntity.encryptedValue`, `ApplicationVariableEntity.value` - Parameterized JSONB types for connected account connection parameters (`ImapSmtpCaldavParams<Pwd>`) with reusable aliases `EncryptedImapSmtpCaldavParams` / `DecryptedImapSmtpCaldavParams` - Typed DTOs (`CreateApplicationRegistrationVariableInput`, `UpdateApplicationRegistrationVariablePayload`, `UpdateApplicationVariableEntityInput`) with `PlaintextString` ### ApplicationVariable always-encrypt uniformization - Retyped `ApplicationVariableEntity.value` to `EncryptedString | ''` — all values are now encrypted regardless of `isSecret` - Updated `ApplicationVariableEntityService` to always encrypt on write and always decrypt on read - Simplified `UpdateApplicationVariableActionHandlerService` by removing conditional encrypt/decrypt-on-isSecret-toggle logic - Added slow instance command (`2.9.0`) to backfill-encrypt existing `isSecret=false` plaintext rows and tighten the `CHECK` constraint ### ConfigStorageService refactor - Split `convertAndSecureValue` (which used `any`) into two well-typed methods: `convertAndDecrypt` and `convertAndEncrypt` - Introduced `isSensitiveStringValue` type predicate to narrow values before encryption/decryption ### What's next - Typeorm entity derivation to strictly type sitemap configuration as code + handler logic for encryption rotation - https://github.com/twentyhq/core-team-issues/issues/2465 |
||
|
neo773
|
c5606212f2 |
Ses outbound followup (#20610)
This pull request unifies outbound with inbound under the new feature and the new email groups feature. These are workspace level shared inboxes that are shared between all workspace members. outbound sending with SES works, we only listen for tenant status events, rest is managed by AWS PR refactors old code and webhook to be split for outbound and inbound for proper separation | Area | Change | |---|---| | AWS SES driver | Split into `AwsSesRegisterDomainService` (tenant + identity + DKIM + MAIL FROM + configuration-set + EventBridge dest + contact list) and `AwsSesSendEmailService` (SendEmail). | | Reputation webhook | New `/webhooks/messaging/ses/outbound` route. SES → EventBridge (`Sending Status Enabled/Disabled` on default bus) → SNS → router → `SesOutboundSendingStateHandlerService` updates `emailing_domain.tenantStatus`. | | Inbound webhook | Refactored into `SesInboundWebhookRouterService` + `SesInboundMailHandlerService`. Shared `SnsSignatureVerifierService` + `SnsSubscriptionConfirmerService` across both routes. | | Global uniqueness | New migration + instance command: `emailing_domain.domain` is now globally unique (one tenant per domain across workspaces). | | Tenant status | New `emailing_domain.tenantStatus` column (`ACTIVE` / `PAUSED`) + `EmailingDomainTenantStatusService`. | | Send-email mutation | New `sendEmailViaDomain` GraphQL mutation + DTOs. | | Cleanup | `EmailingDomainWorkspaceCleanupJob` wired into `WorkspaceService.deleteWorkspace` — tears down SES tenant association + identity on workspace delete. | | Settings UI | Rewritten around reusable `SettingsTableListSection`. "Email Group" → "Email Handle" rename. New cells for status/source/forwarding. Outbound domains surfaced on workspace settings page. | ### Env vars (new) All in `config-variables.ts`, group `AWS_SES_SETTINGS`, all optional: - `AWS_SES_REGION` — `@IsAWSRegion`, consumed by `AwsSesClientProvider` + driver factory - `AWS_SES_ACCOUNT_ID` — used for ARN construction in driver factory - `SES_SNS_TOPIC_ARN_ALLOWLIST` — **shared** by inbound + outbound webhook routers, comma-separated list of accepted SNS topic ARNs (verified via `sns-payload-validator`) ### Migrations - `1778862608620-add-emailing-domain-tenant-status` (fast) — adds `tenantStatus` column. - `1778865501791-unique-emailing-domain-globally` (slow, idempotent) — enforces global uniqueness on `domain`. - Instance commands bumped to `2.5`. ### Infra dependency Two coupled twenty-infra PRs: - `ses-inbound-email` — receipt-rule + inbound SNS topic + S3 bucket policy + KMS grant + `email_group_*` outputs. - `ses-outbound-tf` — EventBridge rule + outbound SNS topic + SES IAM policy + outbound `webhook_url` subscription. **Based on `ses-inbound-email`.** Merge order: inbound first, then outbound. Outbound PR's chart edit owns the comma-joined `SES_SNS_TOPIC_ARN_ALLOWLIST` value (both ARNs). Features lives under `/settings/general` <img width="1496" height="845" alt="SCR-20260519-ofhi-2" src="https://github.com/user-attachments/assets/a025485a-09f7-4131-91cd-0067690ff18d" /> --------- Co-authored-by: Félix Malfait <felix.malfait@gmail.com> Co-authored-by: claude[bot] <41898282+claude[bot]@users.noreply.github.com> Co-authored-by: Félix Malfait <FelixMalfait@users.noreply.github.com> |
||
|
Etienne
|
4a82cddad6 |
remove ai-model-preferences var env and config (#20859)
Split the single AI_MODEL_PREFERENCES JSON config into 4 array configs and migrates existing workspace data. |
||
|
neo773
|
bb5c2bd00c |
Fix/restore channel association scalar field metadata (#20920)
[#20836](https://github.com/twentyhq/twenty/pull/20836) dropped the channel objects but even though calendarChannelId/messageChannelId/messageFolderId already existed in compute standard flat field, there was never an upgrade command to readd them on the surviving association objects so existing workspaces lost the field metadata (columns survived) and import workers throw ```Error: Unknown error importing calendar events for calendar channel <REDACTED> in workspace <REDACTED>: Query runner already released. Cannot run queries anymore.``` This PR adds that command --------- Co-authored-by: prastoin <paul@twenty.com> |
||
|
Félix Malfait
|
d602f35cbd |
feat(data-model): custom-indexes management UI and mutations (#20846)
## Summary
Brings indexes management into the per-object Settings tab as a section
under Search (no feature flag, advanced mode only). Admins can create /
delete non-unique indexes with the UI; apps can declare indexes in code
with `defineIndex`. Composite-typed fields are now indexable by picking
a specific sub-column (e.g. `Address > City`).
A few related polish items also land here (invite-user dropdown lands on
the Invite tab; standard warning callout above the new-index form).
## What ships
### UI — custom indexes on per-object Settings
- New section directly under Search, wrapped in
`AdvancedSettingsWrapper`.
- Filter dropdown on the search bar toggles system-index visibility
(shown by default since advanced mode).
- **+ Add Index** button (disabled with tooltip once the per-object cap
is reached) navigates to a dedicated `SettingsObjectNewIndex` page
(matches the field-creation pattern, not a modal):
- Field picker mirrors the webhook event-form layout (rows of dropdowns,
implicit trailing empty row).
- Composite fields surface their sub-properties (`Address > City`,
`Currency > Amount`, …).
- BTREE / GIN type selector.
- Standard warning Callout: "Use indexes sparingly — each one speeds
reads but slows writes."
- Trash icon on `isCustom: true` rows → confirmation modal →
`deleteOneIndex`.
### Server — `createOneIndex` / `deleteOneIndex` mutations
- Gated by `SettingsPermissionGuard(DATA_MODEL)`.
- `IndexMetadataService` wraps the existing migration runner via
`WorkspaceMigrationValidateBuildAndRunService` so the metadata row and
the SQL index land atomically.
- Validation: rejects empty fields, duplicate `(fieldMetadataId,
subFieldName)` pairs, fields not on the object, requires `subFieldName`
for composite parents, forbids `subFieldName` on scalar/relation,
enforces `MAX_CUSTOM_INDEXES_PER_OBJECT = 10`.
- Delete refuses on `isCustom: false` rows so system indexes can't be
removed via this API.
- Dedicated GraphQL exception handler maps each typed error to the right
transport error class.
### Composite sub-field indexing
- Adds `subFieldName: string | null` column to
`IndexFieldMetadataEntity` (fast instance command).
- The flat-entity flow (`UniversalFlatIndexFieldMetadata`,
`FlatIndexFieldMetadata`, `from-universal-flat-index-to-flat-index`,
runner column resolution) all carry `subFieldName` through.
- For composite parents, the runner uses
`computeCompositeColumnName({...}, property)` for the picked sub-column;
for non-composite parents, behavior is unchanged.
- The `'::'` separator encodes `(fieldMetadataId, subFieldName)` for
dedup on the wire; the frontend uses the same separator inside the
Select component's string value.
### Apps can declare indexes in code (`defineIndex`)
- New `IndexManifest` + `IndexFieldManifest` types in
`twenty-shared/application` wired into the `Manifest` type.
- `defineIndex` SDK helper + `IndexConfig`. CLI manifest builder +
extractor recognize `defineIndex` / `ManifestEntityKey.Indexes`.
- Server: `from-index-manifest-to-universal-flat-index` converter
resolves field IDs, validates composite/scalar `subFieldName` rules, and
delegates to `generateFlatIndexMetadataWithNameOrThrow` for the
deterministic name.
- Orchestrator wires the loop after the field-resolution pass;
per-object cap enforced inline against the manifest.
- Cascade on uninstall is automatic — when an app disappears its indexes
drop with it (universal-flat-entity diff handles it).
- Rich-app fixture ships a real `defineIndex` on `PostCard.status`,
exercising the full manifest → install path in CI.
### Closed for now (open later if needed)
- Apps cannot declare `isUnique` indexes — unique constraints stay with
the field-creation flow.
- Apps cannot use a partial-`indexWhereClause` — the UI surface keeps
the framework's hardcoded allowlist.
- UI cannot create unique or partial indexes either; same reasons.
### Cleanups along the way
- Reused the existing `getCompositeSubFieldLabel` +
`COMPOSITE_FIELD_SUB_FIELD_LABELS` (deleted the duplicates I'd created
early in the PR).
- Moved `MAX_CUSTOM_INDEXES_PER_OBJECT` to `twenty-shared/constants`
(single source for FE + BE).
- Replaced inline `isDefined(x) && x !== ''` with `isNonEmptyString`
(from `@sniptt/guards`).
- Hoisted the per-object fields Map + inlined the cap counter into the
indexes orchestrator loop (drops the install scan from O(indexes ×
totalFields) to O(totalFields + indexes)).
- Per design-feedback: page-based create flow (not a modal), filter
dropdown on the SearchInput (not a separate toggle), webhook-style
picker, field icons.
### Unrelated polish that lands here
- "Invite user" link in the multi-workspace dropdown now lands on the
Invite tab directly (`#invite`) instead of the first tab of the members
page.
## Test plan
- [ ] `npx nx typecheck twenty-server / twenty-front / twenty-sdk /
twenty-shared` — passes
- [ ] `npx nx lint:diff-with-main twenty-server / twenty-front` — clean
- [ ] `npx jest index-metadata.service.spec` — green
- [ ] `npx jest from-index-manifest-to-universal-flat-index` — green
(new converter spec, 8 cases)
- [ ] `npx vitest run
src/sdk/define/indexes/__tests__/define-index.spec.ts` (twenty-sdk) —
green (6 cases)
- [ ] `npx vitest run --config vitest.integration.config.ts -t
"rich-app"` — green (rich-app app-dev integration exercises the new
manifest path with the PostCard.status index)
- [ ] Advanced mode → Settings → any object → Settings tab → Indexes
section is visible under Search
- [ ] Create a single-field BTREE index, confirm SQL index exists
(verify via `pg_indexes`)
- [ ] Create a composite-field index (`Address > City`) and confirm the
column is `addressAddressCity`
- [ ] Create an index spanning two columns; column order matches the
picker order
- [ ] Attempt to create an 11th custom index → button is disabled with
tooltip
- [ ] Delete a custom index → confirmation modal → row disappears, PG
index dropped
- [ ] System indexes have no trash icon and are hidden by default
|
||
|
Marie
|
85d649e831 |
[Fix] Backfill missing command menu items conditional availability expression (#20852)
## Description Following [report in discord](https://discord.com/channels/1130383047699738754/1498690477044793386/1506602927412744242) Some command menu items were showing to all users because they had no conditional availability expression, whereas users did not actually have access to the page or feature behind. For instance: "Go to Admin panel", "Go to AI settings", "Send email" etc. <img width="833" height="1245" alt="image" src="https://github.com/user-attachments/assets/8d2a9404-9b81-4d58-9522-558e9924c457" /> ## Fix - Add conditional availability expressions - Backfill expressions for existing workspaces as they are stored in db (commandMenuItems table) --------- Co-authored-by: Charles Bochet <charles@twenty.com> |
||
|
Paul Rastoin
|
76e144e85a |
Deprecate messageChannel messageFolder calendarChannel standard objects (#20836)
# Introduction Removing old standard objects `messageChannel` and `messageFolder` and `calendarChannel` --------- Co-authored-by: Cursor <cursoragent@cursor.com> |
||
|
Félix Malfait
|
084fa8eaba |
fix(server): auto-index target<X>Id join columns on polymorphic standard objects (#20820)
Closes #20726 ## The bug `timelineActivity` (and the three other polymorphic standard objects — `attachment`, `noteTarget`, `taskTarget`) store relations as N nullable `target<X>Id` columns, one per related object. Each one is a join key queried as `WHERE target<X>Id IN (...) AND deletedAt IS NULL`. For **built-in** related objects (Person, Company, Opportunity, …), each `target<X>Id` column gets a BTREE index, declared statically in `compute-{timelineActivity,attachment,noteTarget,taskTarget}-standard-flat-index-metadata.util.ts`. For **custom** related objects, the same `target<CustomObject>Id` column was added — **without an index**. On a `timelineActivity` table at issue-reporter scale (~21.9M rows, 7.1 GB), this turned record loads into 20–40s sequential scans and produced `QueryFailedError: Query read timeout` for end users. ## Diagnosis The morph/relation field generator (`generateMorphOrRelationFlatFieldMetadataPair`) already creates a BTREE index for the field that owns the join column and returns it alongside the field metadata pair. The two user-driven entry points (`fromRelationCreateFieldInput…`, `fromMorphRelationCreateFieldInput…`) correctly destructure and propagate that index. But the **custom-object creation path** — `buildDefaultRelationFlatFieldMetadatasForCustomObject`, called when a user creates a new custom object — destructured only `{ flatFieldMetadatas }` and threw away `indexMetadatas`. So every `target<CustomObject>Id` column added to the four polymorphic standard objects has been shipping unindexed since custom morph relations went in. ## The fix Three commits. ### 1. `fix(server): index target<CustomObject>Id columns on standard polymorphic objects` 13 lines across 2 files. - `build-default-relation-flat-field-metadatas-for-custom-object.util.ts` — also destructure `indexMetadatas` from the pair generator and accumulate them into the returned record (new field `standardTargetFlatIndexMetadatas`). - `from-create-object-input-to-flat-object-metadata-and-flat-field-metadatas-to-create.util.ts` — append the accumulated indexes to `flatIndexMetadataToCreate`. The migration pipeline at `object-metadata.service.ts:559–562` already passes `flatIndexMetadataToCreate` to the migration runner, so no further wiring is needed. From now on, creating a custom object also creates the four BTREE indexes — one per polymorphic standard object's new `target<CustomObject>Id` column — atomically with the rest of the migration. ### 2. `feat(server): backfill workspace command for relation join column indexes` For existing workspaces whose custom objects were created before the forward-fix. `upgrade:2-8:backfill-relation-join-column-indexes` is a `@RegisteredWorkspaceCommand('2.8.0', 1798100000000)` matching the pattern from `2-7-workspace-command-…-drop-connected-account-standard-object.command.ts`. Per workspace: 1. Load `flatObjectMetadataMaps`, `flatFieldMetadataMaps`, `flatIndexMaps` from the workspace cache. 2. Resolve the four polymorphic standard object IDs by `nameSingular` against `DEFAULT_RELATIONS_OBJECTS_STANDARD_IDS`. 3. Collect every field ID that's already covered by any existing index. 4. Filter `flatFieldMetadataMaps` to MORPH_RELATION fields on those four objects whose `settings.relationType === MANY_TO_ONE` (i.e. owns a join column) and whose ID isn't in the indexed set. 5. Generate a BTREE `UniversalFlatIndexMetadata` for each via `generateIndexForFlatFieldMetadata` (same helper the forward-fix uses). 6. Create the indexes in the workspace schema with **CONCURRENTLY** (see commit 3). 7. Submit the metadata through `WorkspaceMigrationValidateBuildAndRunService` so it lands in `indexMetadata` and the cache — same pipeline as a normal metadata change. The pipeline's own `CREATE INDEX IF NOT EXISTS` no-ops because the index already exists. Properties: - **Idempotent.** Re-running is a no-op once indexes exist. - **Scoped.** Only the four polymorphic standard objects, only their MANY_TO_ONE morph relation fields, only those with no covering index. - **Same code path as the forward-fix.** The backfill produces exactly the indexes the forward-fix would have created at custom-object creation time. - **`--dry-run` supported** via the base `ActiveOrSuspendedWorkspaceCommandRunner`. ### 3. `feat(server): create index CONCURRENTLY in relation join column backfill` Adds an opt-in `concurrently` flag to `WorkspaceSchemaIndexManagerService.createIndex` (threaded through `createIndexInWorkspaceSchema`). When `true`, emits `CREATE INDEX CONCURRENTLY IF NOT EXISTS …`. Defaults to `false` — every existing caller keeps the current transactional `CREATE INDEX` behavior. The backfill command opts in. It creates a QueryRunner **without** `startTransaction()`, issues the CONCURRENTLY indexes one-by-one (each waits for the previous to finish), then submits the metadata through the normal migration pipeline whose own `CREATE INDEX IF NOT EXISTS` is now a no-op. Why not flip the default for the helper: - `CREATE INDEX CONCURRENTLY` cannot run inside a transaction — Postgres errors out. The migration pipeline calls `createIndex` from inside a transactional schema migration. - CONCURRENTLY doesn't roll back with the transaction. If the surrounding migration fails, the index remains and you end up with metadata/schema drift. - Failed CONCURRENTLY builds leave an INVALID index behind that needs manual `DROP`. - UNIQUE indexes have different failure semantics under CONCURRENTLY (deferred, not immediate). So CONCURRENTLY is opt-in, used only where it's the right tool (post-hoc backfills on populated tables). ## Decisions / tradeoffs - **Single-column BTREE vs partial `WHERE deletedAt IS NULL` vs composite.** Twenty's queries always include `deletedAt IS NULL`. A partial index would be slightly better than a plain BTREE (smaller, no wasted seeks on soft-deleted rows). This PR ships single-column to match the existing built-in target index pattern, which already covers >95% of the available speedup (the 20s→4ms drop the reporter saw comes from having any index — composite/partial is a second-order effect). Switching all relation indexes to partial is a separate, broader change. - **CONCURRENTLY operator caveat.** If a CONCURRENTLY build is interrupted (kill, connection drop, OOM), Postgres leaves the index as INVALID. We deliberately don't probe `pg_index` for invalid leftovers on every create — catalog-table queries can be slow at multi-tenant scale and the failure mode is rare. Recovery is manual: `DROP INDEX <name>` and re-run the backfill. - **Forward-fix is not gated** behind a feature flag. The change is metadata-pipeline-internal; before, custom-object creation silently produced a degraded state. After, it produces the correct state. No new public API, no behavioural change for end users besides the indexes existing. ## Risk - Forward-fix: changes only the metadata produced during custom-object creation. New objects get four extra `FlatIndexMetadata` rows and four extra `CREATE INDEX` statements during their creation migration. Tables are empty at that point so the index builds in microseconds. - Helper change: API-compatible, default behavior unchanged. The new `concurrently` parameter is optional. - Backfill: read-only state probe → CONCURRENTLY index creation (no write blocking) → metadata insert via the normal migration pipeline. Idempotent. Reverting is `DROP INDEX`. ## Test plan - [ ] Verify forward-fix: create a custom object, confirm four new BTREE indexes appear on `timelineActivity`, `attachment`, `noteTarget`, `taskTarget` for the new `target<CustomObject>Id` columns, and that `flatIndexMaps` has matching entries. - [ ] Verify backfill on a workspace that had custom objects created before the fix: run `--dry-run` first, confirm the expected indexes are listed; then run for real, confirm the indexes appear in pg (and as `indisvalid = true` in `pg_index`) and in `flatIndexMaps`. Re-run; confirm no-op. - [ ] Verify backfill on a clean workspace: should log "no missing indexes" and exit. - [ ] Verify CONCURRENTLY behavior under load: run backfill against a workspace with active writes on `timelineActivity`; confirm inserts/updates keep working during index build (no `ShareLock` waits in `pg_stat_activity`). - [ ] On the affected reporter-scale workspace, confirm `EXPLAIN ANALYZE` switches from sequential scan to index scan and timeline activity timeouts go away. |
||
|
neo773
|
323e66433e |
lint: migrate prettier to oxfmt (#20783)
Most changes are `implements` being unwrapped this is not a oxfmt regression Prettier in 3.7 (we're on 3.1) changed this behaviour prettier blog [post](https://prettier.io/blog/2025/11/27/3.7.0#change-18094) This unifies our linting tooling --------- Co-authored-by: github-actions <github-actions@twenty.com> Co-authored-by: Charles Bochet <charles@twenty.com> |
||
|
Paul Rastoin
|
abf5902ab5 | Connected account deprecation system build (#20810) | ||
|
Weiko
|
d13cc7c349 |
Drop legacy rolePermissionFlag.flag column + fallback logic (#20730)
## Summary - **New fast migration** `2-7-instance-command-fast-1779600000000-finalize-role-permission-flag-cutover.ts`: - `DROP CONSTRAINT IDX_ROLE_PERMISSION_FLAG_FLAG_ROLE_ID_UNIQUE` - `ALTER COLUMN permissionFlagId SET NOT NULL` - `DROP COLUMN flag` - `down()` repopulates `flag` from the catalog via `permissionFlagId` and restores the old unique. - **Entity**: `RolePermissionFlagEntity` hides the `flag` column by using the new decorator + drops old `@Unique` decorator; `permissionFlagId` and the `permissionFlag` relation become non-nullable. - **Deletes** the synthesizer `synthesize-flat-permission-flag-from-flag.util.ts` and every fallback branch that used it (`from-role-permission-flag-entity-to-flat-role-permission-flag.util.ts`, `from-flat-role-permission-flag-to-role-permission-flag-dto.util.ts`, `permissions.service.ts`, `workspace-roles-permissions-cache.service.ts`, `fromRoleEntityToRoleDto.util.ts`, `flat-role-permission-flag-validator.service.ts`, `role-permission-flag.service.ts:getEffectiveUniversalIdentifier`). - **Write path**: ~~drops `flag` from `CreateRolePermissionFlagInput`, the create util, and the application-manifest converter.~~ - **Metadata configs**: ~~removes `flag` from `all-entity-properties-configuration-by-metadata-name.constant.ts` (rolePermissionFlag block)~~ and flips `permissionFlag.isNullable` to `false` in `all-many-to-one-metadata-relations.constant.ts`. ### Why the `flag` field stays declared in the entity The decorator (`@WasRemovedInUpgrade`) is the right tool for the lifecycle marker, but it's a **reflect-metadata** runtime decorator — TypeScript can't see it at compile time. So while the adapter ([`UpgradeAwareEntityMetadataAdapter`](packages/twenty-server/src/engine/twenty-orm/upgrade-aware/upgrade-aware-entity-metadata.adapter.ts)) now correctly flips `isSelect`/`isInsert`/`isUpdate` to `false` once the drop migration's cursor is crossed, the *static* TypeScript types derived from `RolePermissionFlagEntity` (`UniversalFlatRolePermissionFlag`, `FlatRolePermissionFlag`, `MetadataEntityPropertyConfiguration<'rolePermissionFlag'>`, etc.) still see `flag` as a required scalar property — because the entity declares it. That means every producer of one of those derived types must include `flag`: - `from-create-role-permission-flag-input-to-flat-role-permission-flag-to-create.util.ts` plumbs it through. - `from-permission-flag-to-universal-flat-role-permission-flag.util.ts` (the application-manifest converter) sets `flag: permissionFlag.flag`. - `all-entity-properties-configuration-by-metadata-name.constant.ts` has a `flag` entry under `rolePermissionFlag`. - `CreateRolePermissionFlagInput` keeps the `flag` field. - `RolePermissionFlagService.upsertPermissionFlags` passes `flag: permissionFlag.key as PermissionFlagType` to the create util. Explored phantom-brand approach (`RemovedInUpgrade<T>` wrapper on the field type, key-filter mapped type applied inside `ScalarFlatEntity` / `UniversalFlatEntityFrom`) but previous commands could have `flag === undefined` (downcast from the brand since we can't compare with UpgradeMigrationName like we do with a decorator). That's a **silent-read** failure mode: compiles fine, comparisons against `flag` silently always-false, no error surfaces. Probably worth too much risk for what's a small amount of plumbing? The eventual full deletion of `flag` (entity field included) is a future cleanup once we drop cross-upgrade support for versions ≤ 2.6 Note: Not sure if this PR (and even the decorator) is really needed in the end, seems we need to keep a lot of code in place to handle legacy. Maybe a simple noop [At]Deprecated is enough @charlesBochet (and a migration to set the column nullable if that was not the case before + remove associated constraints) |
||
|
Paul Rastoin
|
9b9c97a049 |
Deprecate and backfill delete ConnectedAccount twenty standard object (#20752)
# Introduction Following connected account permissions refactor and encryption Removing the old workspace schema twenty standard application connectedAccount objects and related standard fields and index - a lot of deadcode - instance command backfill cleaning the connected account object from workspaces |
||
|
Félix Malfait
|
f3aadbbb66 |
chore(server): drop leftover favorite and favoriteFolder workspace objects (#20744)
## Summary - Adds a 2.7.0 workspace upgrade command `upgrade:2-7:drop-favorite-objects` that removes the legacy `favorite` and `favoriteFolder` object metadata (and their workspace tables) from every active or suspended workspace. - The records were migrated to `navigationMenuItem` in the 1.17/1.18 upgrades and the entity code was deleted in #19536, but the per-workspace metadata rows were never cleaned up — so they still surface in the "Existing objects" settings list and expose stale CRUD tools to the AI/MCP layer (e.g. the model can hallucinate `create_favorite_folder` against a real-looking schema). ## Implementation notes - Modeled on [`upgrade:2-3:drop-message-direction-field`](https://github.com/twentyhq/twenty/blob/main/packages/twenty-server/src/database/commands/upgrade-version-command/2-3/2-3-workspace-command-1777400000000-drop-message-direction-field.command.ts), but at object granularity. - Uses `ObjectMetadataService.deleteOneObject({ isSystemBuild: true })` so all cascading is handled by the existing pipeline: field metadata, indexes, relation fields on other workspace entities, command menu items, and the workspace data tables. Views and orphaned `navigationMenuItem` rows pointing at favorite views are removed by the existing `onDelete: 'CASCADE'` FKs. - Deletion order: `favorite` first (holds a relation to `favoriteFolder`), then `favoriteFolder`. - Both objects are flagged `isSystem: true`, hence `isSystemBuild: true` on the call. - Idempotent: workspaces where the object is already absent are logged and skipped. - Honors `--dry-run`. - Universal identifiers are hard-coded because the matching `STANDARD_OBJECTS` entries were deleted in #19536. ## Test plan - [ ] Run on a workspace that still has `favorite` / `favoriteFolder` in `core.objectMetadata` (verify in prod-like DB beforehand) and confirm both objects, their fields, indexes, relation fields on linked objects, views, and the workspace data tables are gone after running. - [ ] Re-run on the same workspace — confirm it logs "already absent" and exits clean (idempotency). - [ ] Run on a workspace where the objects don't exist (e.g. fresh local) — confirm clean no-op. - [ ] Run with \`--dry-run\` first — confirm log output and no DB mutations. - [ ] Confirm the "Existing objects" settings page no longer lists Favorites / Favorite Folders after the migration. ## Safety check before rollout Before running in prod, verify no workspace has live (non-soft-deleted) favorite data that didn't make it to \`navigationMenuItem\`: \`\`\`sql -- Per workspace SELECT count(*) FROM workspace_xxx.favorite WHERE "deletedAt" IS NULL; \`\`\` Should be ~0 in workspaces that ran the 1.17 / 1.18 migrations. --------- Co-authored-by: prastoin <paul@twenty.com> |
||
|
nitin
|
83b10ad698 |
fix(server): sync command menu item availability expressions on existing workspaces (#20719)
Two fixes via one workspace command: 1. Gates 5 standard command menu items behind `pageType == "INDEX_PAGE"` -- `importRecords`, `exportView`, `seeDeletedRecords`, `createNewView`, `hideDeletedRecords`. They currently appear (and crash or do nothing) on RECORD_PAGE. 2. Fixes Edit Layout missing from older workspaces -- root cause is `conditionalAvailabilityExpression` drift between source-of-truth constants and the workspace DB (e.g. #20556 removed a feature flag from the expression without syncing existing workspaces). The 2-6 workspace command iterates all `STANDARD_COMMAND_MENU_ITEMS` and reconciles any `conditionalAvailabilityExpression` that differs from the constant. Idempotent -- already-correct rows are skipped. Deferred: `deleteRecords` doesn't refetch the current record after deletion on RECORD_PAGE (mutation fires but UI shows stale state until refresh) -- different fix shape (frontend handler), separate PR. |
||
|
Paul Rastoin
|
57f13c9b92 |
[CONNECTED_ACCOUNT_BREAKING_CHANGE] Encrypt ConnectedAccount connectionParameters (#20673)
# Introduction Prevent any cross user `connectedAccount` `connectionParamaters` leak Also encrypt in db all `connectionParameters` password Never return any password through `DTO` anymore The settings now allow update mutation without providing the password in edition mode Verified all `connectionParameters.password` interaction ## Integration tests - Added more coverage for both failing and successful paths - Introduced a new env var that allow bypass the provider connection test ## Legacy connected Account decryption support Stop allowing non encrypted decryption on `accessToken` and `refreshToken`, only allow legacy decryption on refactored `connectionParameters` ## Upsert ownership Completely got rid of the connected workspace schema context which is legacy Also now a user can only upsert a connected account for him only.. ## New UI <img width="1770" height="1852" alt="image" src="https://github.com/user-attachments/assets/55c1dc89-42ff-4084-95e2-cc5f9e23753b" /> If in edition the password is by default disabled It needs to be selected as being edited to be enabled ## Next - Refactor tool permissions flag not to include connected accounts - Remove the legacy connected standard object - Refactor and improve connected account resolver auth |
||
|
Charles Bochet
|
77514ad14a |
fix(server): backport relationTargetFieldMetadataId column-add to 2.4 and 2.5 fast instance (#20721)
## Summary Cross-version upgrade from a **v2.3 or v2.4 baseline** to v2.6.x currently fails at the 2.5 workspace command `NormalizeCompositeFieldDefaults`: ``` [QueryFailedError] column ViewFilterEntity.relationTargetFieldMetadataId does not exist at WorkspaceFlatViewFilterMapCacheService.computeForCache ``` Reproduced locally via Docker cross-version upgrade (v2.6.1 against `twentycrm/twenty:v2.3` and `:v2.4` images on a freshly-seeded DB). ### Root cause The column-add is already declared in two places: - `2-3/.../1747234300000-add-relation-target-field-metadata-id-to-view-filter` (backport from #20664) - `2-6/.../1798000005000-add-relation-target-field-metadata-id-to-view-filter` But the runner's `resolveStartCursor` (`upgrade-sequence-runner.service.ts`) advances forward from `lastAttemptedCommandName` and never re-runs commands inserted *behind* the cursor: - **fresh install through 1.23 → 2.6.x**: cursor < 2.3 → 2.3 backport runs → column added before 2.5 workspace ✓ - **v2.3 baseline → 2.6.x**: cursor past 2.3 → 2.3 backport skipped → 2.5 workspace `NormalizeCompositeFieldDefaults` crashes ✗ - **v2.4 baseline → 2.6.x**: cursor past 2.4 → 2.3 backport skipped → same crash ✗ - **v2.5 baseline → 2.6.x**: cursor past 2.5 → 2.5 workspace already applied (ran against v2.5 source's older entity without the column) → 2.6 fast adds the column ✓ The 2.6 fast `1798000005000` runs *after* the 2.5 workspace command, too late to help v2.3 / v2.4 baselines. ### Fix Mirror the existing 2.3 Early backport at two more versions: - `2-4/.../1747234400000-add-relation-target-field-metadata-id-to-view-filter` — covers v2.3 baseline (runs in 2.4 fast, before any 2.4/2.5 workspace command) - `2-5/.../1747234500000-add-relation-target-field-metadata-id-to-view-filter` — covers v2.4 baseline (runs in 2.5 fast, before `NormalizeCompositeFieldDefaults`) Both use `ADD COLUMN IF NOT EXISTS` (idempotent) and `DROP COLUMN IF EXISTS` for the down. No FK / index — those still live in the 2.6 file, which runs as a no-op for the column on already-fixed DBs. Pre-2.6 codebases can't use `@WasIntroducedInUpgrade` (#20686 only lands in 2.6), so this "ladder of backports" remains the operative pattern. ## Audit context Locally walked `v1.23 / v2.0 / v2.1 / v2.2 / v2.3 / v2.4 / v2.5 → v2.6.1`: | Baseline | Result | |---|---| | v1.23 | PASS | | v2.0 | PASS | | v2.1 | PASS | | v2.2 | PASS | | **v2.3** | **FAIL** (this PR) | | **v2.4** | **FAIL** (this PR) | | v2.5 | PASS | |
||
|
Charles Bochet
|
bad1f20012 |
fix(server): handle legacy PK name in 2.6 rename-permission-flag upgrade (#20697)
## Summary The 2.6 `RenamePermissionFlagToRolePermissionFlag` upgrade command failed on staging and dev with: ``` [QueryFailedError] constraint "PK_a02789db60620a1e9f90147b50f" for table "rolePermissionFlag" does not exist in RenamePermissionFlagToRolePermissionFlag1778235340020 (2.6.0) (instance fast) ``` ### Root cause TypeORM names PKs as `PK_<sha1(tableName_sortedColumnNames)[:27]>`. So: - `permissionFlag_id` → `PK_a02789db60620a1e9f90147b50f` - `settingPermission_id` → `PK_8c144a021030d7e3326835a04c8` - `rolePermissionFlag_id` → `PK_76591adc8035c2e7b0cd6115136` On databases initially migrated before the v1.5.5 migration squash (#15183), the table was renamed `settingPermission` → `permissionFlag` via the pre-squash migration `1753149175945-renameSettingPermissionToPermissionFlag.ts`. That migration renamed the table, the column, the unique index, and the role FK, but **never renamed the PK constraint** — and Postgres does not auto-rename constraints on `ALTER TABLE ... RENAME TO`. Those instances therefore still carry the legacy PK name `PK_8c144a021030d7e3326835a04c8`. Fresh installs (squashed `setupMetadataTables` migration) instead have the expected `PK_a02789db60620a1e9f90147b50f`. The 2.6 upgrade only handled the fresh-install name, so it broke for any DB that went through the historical rename chain. ### Fix Replace the brittle `RENAME CONSTRAINT` with `DROP CONSTRAINT IF EXISTS` for both historical PK names, followed by `ADD CONSTRAINT ... PRIMARY KEY ("id")` with the canonical new name. The migration now converges to the same PK name regardless of the DB's history. The same pattern is applied symmetrically in `down()`. ### Why this is safe - The whole instance command runs in a transaction (`InstanceCommandRunnerService.runFastInstanceCommand`). - The first statement (`ALTER TABLE ... RENAME TO`) takes `ACCESS EXCLUSIVE` on the table, so the drop/add window for the PK is invisible to any concurrent writer — they queue on the lock until commit. - No FK references `rolePermissionFlag.id` at this point in the sequence (migration 22 introduces an FK pointing at the new `permissionFlag` catalog created in migration 21, not at the renamed grant table), so dropping the PK does not cascade or block. - `NOT NULL` and the `uuid_generate_v4()` default on `id` are column-level and remain in place when the PK is dropped. ## Test plan - [ ] Run 2.6 upgrade against a fresh-install database (PK = `PK_a02789db60620a1e9f90147b50f`) — should succeed. - [ ] Run 2.6 upgrade against a pre-squash database (PK = `PK_8c144a021030d7e3326835a04c8`, reproducible on current staging/dev) — should now succeed. - [ ] Verify post-migration: `rolePermissionFlag` exists, PK is named `PK_76591adc8035c2e7b0cd6115136`, all FKs and indexes named as expected. - [ ] Run `down()` and verify table returns to `permissionFlag` with PK `PK_a02789db60620a1e9f90147b50f`. - [ ] Subsequent migrations (`1778235340021` permission-flag catalog, `1778235340022` link, `1778235340023` backfill) still apply cleanly. |
||
|
Charles Bochet
|
d03480472c |
perf(server): index messageChannel/calendarChannel for per-workspace sync crons (#20678)
## Summary The messaging/calendar import crons each iterate every active workspace and execute one `find` per workspace against `core."messageChannel"` / `core."calendarChannel"` with the shape: ``` WHERE "workspaceId" = $1 AND "isSyncEnabled" = true AND "syncStage" = $2 [AND "type" <> $3] ``` There is currently no index supporting that shape, so the planner does a seq scan on each table for every iteration. On prod-eu (RDS Performance Insights, `rds-prod-eu-one`), these two queries are the top two by load — together ~12 AAS, ~12 calls/sec — and have been the primary contributor to the sustained 100% CPU since active workspace count grew. This PR adds composite indexes on `(workspaceId, isSyncEnabled, syncStage)` for both tables as an instance migration in 2.6.0. |
||
|
nitin
|
db0547f503 |
[1/3] Rename permissionFlag to rolePermissionFlag + add permissionFlag catalog/backfill (#20481)
Split of #20377. ## Summary This PR separates available permission flags from per-role permission flag grants. Previously, `core.permissionFlag` stored the role assignment directly: `roleId + flag`. This PR renames that legacy grant table to `core.rolePermissionFlag`, then recreates `core.permissionFlag` as the catalog of available permission flags. ## What changed - Rename the existing `core.permissionFlag` grant table to `core.rolePermissionFlag`. - Add the new syncable `core.permissionFlag` catalog entity with key, label, description, icon, permission type, relevance flags, and custom/standard metadata. - Add stable `SystemPermissionFlag` universal identifiers for the built-in `PermissionFlagType` values. - Seed the standard permission flags for every workspace under the Twenty standard application. - Backfill existing role grants: - create missing catalog rows for existing grant keys, - add `rolePermissionFlag.permissionFlagId`, - migrate grants from the old string `flag` column to the new catalog FK, - replace the old `(flag, roleId)` uniqueness with `(permissionFlagId, roleId)`. - Rewire role permission flag caches, permission checks, role DTO mapping, and `upsertPermissionFlags` to resolve through the catalog. - Keep the existing public role permission API shape: product/app surfaces still talk about `permissionFlags` and return `{ id, roleId, flag }`. - Update metadata flat-entity machinery, migration builders, validators, action handlers, snapshots, generated schemas, docs, and app fixtures for the new `permissionFlag` / `rolePermissionFlag` split. ## Behavior after this PR - Existing permission flag grants keep working. - Existing GraphQL role permission flows keep the same public naming. - Standard permission flags are represented as catalog rows. - Permission checks now compare grants through catalog universal identifiers instead of the legacy `flag` column. - Workspace deletion cleanup now verifies both `permissionFlag` and `rolePermissionFlag`. ## What is not in this PR - Public GraphQL CRUD for custom permission flags. - App manifest support for declaring new custom permission flags. - Frontend UI for creating or assigning custom permission flags beyond the existing role permission flow. --------- Co-authored-by: Weiko <corentin@twenty.com> |
||
|
Charles Bochet
|
6b3064e2ba |
fix(server): add relationTargetFieldMetadataId column early in upgrade sequence (#20664)
## Summary Cross-version upgrade fails at the 2.3 `DropMessageDirectionFieldCommand` stage: ``` [QueryFailedError] column ViewFilterEntity.relationTargetFieldMetadataId does not exist at WorkspaceFlatViewFilterMapCacheService.computeForCache ``` (see https://github.com/twentyhq/twenty-infra/actions/runs/25929264129/job/76219964380) Same shape as #20584 (subFieldName), one column over. ### Root cause 1. The 2.3 `DropMessageDirectionFieldCommand` builds a workspace migration that deletes a `fieldMetadata` (the `direction` field). 2. `WorkspaceMigrationRunnerService.run` walks the metadata cascade graph and pulls `viewFilter` into the dependency set because `viewFilter` is the inverse one-to-many of `fieldMetadata`. 3. That maps to cache keys → `flatViewFilterMaps` gets requested → `WorkspaceFlatViewFilterMapCacheService.computeForCache` runs. 4. `computeForCache` does `viewFilterRepository.find({ where: { workspaceId }, withDeleted: true })` with no `select`, so TypeORM emits a SELECT that includes `relationTargetFieldMetadataId` — column only added by the 2.6 fast instance command `1798000005000`, not yet run at the 2.3 stage. 💥 ### Why v2.5.0 / v2.5.1 passed They didn't include #20527 (one-hop relation filters, May 14), which added `relationTargetFieldMetadataId` to `ViewFilterEntity` and the 2.6 instance command. The CI base image (v1.22) seeded the DB, then the v2.5.0/v2.5.1 container ran upgrade commands against an entity that didn't yet know about this column. |
||
|
Félix Malfait
|
c938fbf4d6 |
feat(twenty-front): relation traversal in filter dropdown (stacked) (#20533)
**Stacked on #20527** https://github.com/user-attachments/assets/48995655-401a-4c35-8094-e88da8408bdd ## Summary Surfaces the one-hop relation traversal added in #20527 through the existing **composite sub-field dropdown pattern**. Clicking a MANY_TO_ONE relation field in the "+ Filter" picker now opens the same second-level dropdown that composite fields (FULL_NAME, ADDRESS, CURRENCY, etc.) already use — populated with the target object's filterable fields. Picking one (e.g. `Company → Name`) builds a filter that serializes to the nested GraphQL filter the backend now accepts: `{ company: { name: { ilike: "%X%" } } }`. No new components. The whole feature reuses `AdvancedFilterSubFieldSelectMenu` + the existing `subFieldNameUsedInDropdownComponentState` + the existing `MenuItem hasSubMenu` indicator. Only the conditions that gate the sub-menu (and the sub-menu's content for relations) were broadened. ## What landed | File | Change | |---|---| | `ObjectFilterDropdownFilterSelectMenuItem` | Sub-menu chevron now shows on MANY_TO_ONE relations (`isManyToOneRelationField` util). | | `AdvancedFilterFieldSelectMenu` | Relation clicks open the sub-menu alongside composite clicks. | | `AdvancedFilterSubFieldSelectMenu` | New branch: when the sub-menu type is `'RELATION'`, render the target object's filterable fields via `useFilterableFieldMetadataItems(targetObjectMetadataId)`. Composite logic untouched. | | `objectFilterDropdownSubMenuFieldType` state | Widened to accept a `'RELATION'` sentinel. Role-permissions sub-field menu narrows it back out (it doesn't traverse relations). | | `useSelectFieldUsedInAdvancedFilterDropdown` | New optional `targetFieldMetadataItem` arg. When present, the stored RecordFilter's `type` is the target field's type so the operand picker and value input render the target's operands (`'TEXT'` operators when filtering `company.name`, etc.). | | `turnRecordFilterIntoGqlOperationFilter` (shared) | When the filter targets a `RELATION` field with a `subFieldName`, synthesize a field-metadata for the target, recurse to build the inner filter, then wrap it under the relation field's name → `{ relationName: { targetFieldName: { ...operator } } }`. | `RecordFilter.subFieldName` stays narrowly typed as `CompositeFieldSubFieldName` so the wide downstream consumers (`shouldShowFilterTextInput`, composite handlers in the serializer, etc.) don't change. The relation target field's name is stored through a narrowly-scoped cast at the dropdown's storage point — the serializer checks `filter.type === 'RELATION'` before interpreting it as a target field name, so the cast can't be mis-read by composite-only code paths. ## Test plan - [ ] Open a table view on People, click "+ Filter", click "Company" → sub-menu opens with Company's filterable fields - [ ] Pick "Name" → operand picker shows TEXT operators (Contains, Equals, …) - [ ] Type "Airbnb" → filter applies, table shows people whose company name contains "Airbnb" - [ ] Verify network tab: the GraphQL filter variable is `{ company: { name: { ilike: "%Airbnb%" } } }` - [ ] Same flow with a composite target field (e.g. `Company → annualRecurringRevenue → amountMicros`) — should work end-to-end (backend supports composite-within-relation; #20527 has an integration test covering this) - [ ] Composite fields (FULL_NAME, ADDRESS) still open their normal sub-menu and filter correctly — no regression - [ ] Role-permissions field-select sub-field menu is unaffected (it bails out early on the RELATION sentinel) ## Out of scope - ONE_TO_MANY traversal (no backend support yet) - Aggregates (`people.count > 5`) - Persisting relation-traversal filters into a saved view (ViewFilter has no `relationPath` column yet; that's a separate slice) - REST API DSL changes - AI Tools 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> |
||
|
Charles Bochet
|
eca92ca559 |
fix(server): rebuild unique phone indexes drops legacy non-empty partial WHERE clause (#20606)
## Summary `RebuildUniquePhoneIndexesCommand` reuses each index's existing `indexWhereClause` when recreating the physical index. For workspaces whose unique phone indexes have a legacy clause like `"primaryPhoneNumber" != ''` (created before PR #18024 hardened the validator allowlist), the recreate path fails at `validateAndReturnIndexWhereClause` because the clause isn't in `ALLOWED_INDEX_WHERE_CLAUSES`. Two workspaces are hitting this on the 2.5 upgrade: - `3a797122-…` — `"companyPhonePrimaryPhoneNumber" != ''` - `ea74716f-…` — `"phonesPrimaryPhoneNumber" != ''` ## Fix Detect the legacy `"<col>" != ''` shape via a strict regex. When it's there, before the existing drop+create, do three things inside the workspace transaction: 1. **Normalize the data** that the legacy partial clause was masking — `UPDATE "<schema>"."<table>" SET "<col>" = NULL WHERE "<col>" = ''` for every column the index covers. Without this the next step would fail because the new plain-unique index would see duplicate `''` values across the rows the old partial clause was excluding. 2. **Null out `core."indexMetadata".indexWhereClause`** so the metadata row matches what the UI would have created (`indexWhereClause: null`) and doesn't carry the validator-rejected clause forward to any future re-emit. Uses the same workspace `queryRunner` (Postgres lets one connection write across schemas). 3. **Recreate** with an overridden flat index where `indexWhereClause: null`. `createIndexInWorkspaceSchema` → `indexManager.createIndex` → `validateAndReturnIndexWhereClause` short-circuits on null, no allowlist check. End state matches the shape a fresh "toggle unique in Settings UI" creates: plain unique index, no `WHERE`, NULL semantics doing the "exclude empty phones" work via PG's default NULL-distinct behaviour. For indexes whose clause is already allowlisted (`"deletedAt" IS NULL`) or null, behaviour is unchanged — just the column-list widening this command already does. |
