69a7e614ffa1f8a5f8d8ea791dfdd57dbcebe959
658 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Charles Bochet
|
615c3d8dbe |
security: drop end-of-life apollo-server-core (#735, #736) (#21418)
Closes the `apollo-server-core` alerts (**#735**, **#736**) by
**removing the dependency** — no Apollo migration, no resolution.
### Why these were flagged "no patch available"
`apollo-server-core` is **Apollo Server v3, which is end-of-life** (per
its npm deprecation notice). No patched release of this package will
ever exist — the CVE fix lives only in the renamed `@apollo/server` v4
package.
### Why we can just drop it
twenty-server **doesn't use Apollo Server** — its GraphQL runtime is
**GraphQL Yoga** (`YogaDriver`). `apollo-server-core` was imported for
one thing only: the `gql` template tag in **6 integration test files**.
`gql` from `graphql-tag` is identical (apollo-server-core merely
re-exports it), `graphql-tag` is **already a direct dependency**, and
**15 other twenty-server tests already import `gql` from it**.
### Change
- Swapped `import { gql } from 'apollo-server-core'` → `import { gql }
from 'graphql-tag'` in the 6 test files.
- Removed `apollo-server-core` from
`packages/twenty-server/package.json`.
- Result: `apollo-server-core` (and its transitive surface) is gone from
`yarn.lock` entirely.
### Verification
- `yarn install --immutable` ✓
- No `apollo-server-core` references remain in source or lockfile
- Integration tests (which exercise the swapped `gql` imports) run in CI
|
||
|
Weiko
|
9c66975520 |
isCustom deprecation for Objects and Fields (#21228)
## Context
`isCustom` was a legacy denormalized boolean on `ObjectMetadataEntity`
and `FieldMetadataEntity`.
Now that every metadata row carries `applicationId` (via
`SyncableEntity`), "is this custom" is fully derivable, and the stored
boolean was a redundant second source of truth that could drift.
The real meaning of `isCustom` is **"the owning application is not the
twenty-standard application"** — i.e. `!belongsToTwentyStandardApp`.
Note this is *not* "belongs to the workspace custom app" as I initially
thought: third-party-application
objects/fields are custom too.
The standard application has a globally stable `universalIdentifier`, so
the value derives with no per-workspace lookup.
## Changed
## `isCustom` checks — before → after
`isCustom` is no longer a stored column. The table below lists every
site that branched on it and how it resolves now. The unifying rule:
`isCustom ≡
!isTwentyStandardApplicationUniversalIdentifier(applicationUniversalIdentifier)`.
### Server — behavioural checks
| Location | Purpose | Before | Now |
|---|---|---|---|
| `utils/compute-object-target-table.util.ts` | Physical table name `_`
prefix | `computeTableName(nameSingular, objectMetadata.isCustom)` |
derives from `applicationUniversalIdentifier` (single source for all
table-name callers) |
| `twenty-orm/factories/entity-schema.factory.ts` +
`…/entity-schema-metadata.type.ts` | ORM table name (hot path) |
`object.isCustom` | `object.applicationId !== standardApplicationId`
(computed in `buildEntitySchemaMetadataMaps`) |
|
`twenty-orm/repository/workspace-{delete,soft-delete,update}-query-builder.ts`
| Table name for mutations | `computeTableName(nameSingular,
objectMetadata.isCustom)` | `computeObjectTargetTable(objectMetadata)` |
| `index-metadata/utils/generate-deterministic-index-name-v2.ts` | Index
name hash (must stay bit-identical) | `flatObjectMetadata.isCustom` |
derives from `applicationUniversalIdentifier` |
| `object-metadata/object-record-count.service.ts` | Table name for
record count | `computeTableName(nameSingular, isCustom)` |
`computeObjectTargetTable(flatObjectMetadata)` |
|
`workspace-manager/dev-seeder/data/services/dev-seeder-data.service.ts`
| Match seed config by table name | `computeTableName(item.nameSingular,
item.isCustom)` | `computeObjectTargetTable(item)` |
| `commands/workspace-export/workspace-export.service.ts` +
`…/utils/generate-workspace-schema-ddl.util.ts` | Export table name (raw
entity) | `objectMetadata.isCustom` |
`!isTwentyStandard…(objectMetadata.application?.universalIdentifier)` |
|
`flat-field-metadata/services/flat-field-metadata-type-validator.service.ts`
| Block users creating reserved field types |
`args.flatEntityToValidate.isCustom` |
`!args.flatEntityToValidate.isSystem` |
| `api/common/.../common-create-many-query-runner.service.ts` | Don't
let client overwrite system `createdBy` |
`createdByFieldMetadata.isCustom === false` |
`createdByFieldMetadata.isSystem === true` |
|
`field-metadata/utils/resolve-field-metadata-standard-override.util.ts`
| Skip i18n/overrides for custom fields | `if (fieldMetadata.isCustom)
return raw` | **removed** — falls through on
`isDefined(standardOverrides)` |
|
`object-metadata/utils/resolve-object-metadata-standard-override.util.ts`
| Skip i18n/overrides for custom objects | `if (objectMetadata.isCustom)
return raw` | **removed** — same fall-through |
|
`command-menu-item/utils/build-navigation-interpolation-context.util.ts`
| Override context for nav labels | passed `isCustom` into resolver |
dropped (resolver no longer needs it) |
| `api/common/.../data-arg-processor.service.ts` | `isCustom` for
record-position table name | `flatObjectMetadata.isCustom` | derives
from `applicationUniversalIdentifier` |
| `metadata-modules/minimal-metadata/minimal-metadata.service.ts` |
Minimal DTO + override context | `flatObjectMetadata.isCustom` | derives
from `applicationUniversalIdentifier` |
|
`commands/upgrade-version-command/1-23/…backfill-record-page-layouts.command.ts`
| Filter to custom objects | `objectMetadata.isCustom` |
`!isTwentyStandard…(applicationUniversalIdentifier)` |
### Server — DTO / API population
| Location | Before | Now |
|---|---|---|
|
`flat-object-metadata/utils/from-flat-object-metadata-to-object-metadata-dto.util.ts`
| passthrough `isCustom` | derives from `applicationUniversalIdentifier`
|
|
`flat-field-metadata/utils/from-flat-field-metadata-to-field-metadata-dto.util.ts`
| passthrough `isCustom` | derives from `applicationUniversalIdentifier`
|
|
`object-metadata/utils/from-object-metadata-entity-to-object-metadata-dto.util.ts`
(REST) | `entity.isCustom` | `entity.applicationId !==
standardApplicationId` |
|
`field-metadata/utils/from-field-metadata-entity-to-field-metadata-dto.util.ts`
(REST) | `entity.isCustom` | `entity.applicationId !==
standardApplicationId` |
| `dataloaders/dataloader.service.ts` | passed
`flatFieldMetadata.isCustom` into override resolver | dropped (resolver
no longer needs it) |
> REST controllers (`object-metadata.controller.ts`,
`field-metadata.controller.ts`) resolve `standardApplicationId` once per
request from the cached `flatApplicationMaps`.
### Frontend
| Location | Purpose | Before | Now |
|---|---|---|---|
| `settings/.../SettingsObjectFieldDisabledActionDropdown.tsx` | Whether
an inactive field is deletable | `isDeletable = isCustomField` |
`isDeletable = isCustomField && !isSystemField` |
### Unchanged (out of scope)
`isCustom` on `IndexMetadata` / `View` / `Skill` / `Agent` and their
guards still read the persisted column.
Breaking change is on the isCustom filter on field and object APIs, this
is never used in the FE and unlikely used by external consumers
|
||
|
Marie
|
c27c8c88b0 |
Fix various graphs bugs (#21311)
Some bugs fixed in this PR
1. From UI any field could be chosen to group the query by it, while for
instance, RAW_JSON type (eg workflowRun.state) is not supported by
PostgreSQL to group a query by. Fix: removed it from the "group by"
fields options in FE + in BE -->
2. The BE check existed (isFlatFieldMetadataSupportedInGroupBy) but the
signature was malformed: it expected`{ fieldMetadataType,
fieldMetadataName, fieldMetadataIsSystem }` while every caller passes a
flat field metadata object with type/name/isSystem. So the check is
mis-wired — at runtime the destructured props are undefined, making it
always return true (validation bypassed). Fixed this.
3. Group by does not work with Morph relations if their direction is
ONE_TO_MANY. Added that constraint.
4. Group by with morph relations were broken even for MANY_TO_ONE,
because a morph is stored as one field per target
(polymorphicOwnerRocket, polymorphicOwnerSurveyResult…), each with its
own join column, but the frontend collapsed them into a single
polymorphicOwner field — so the backend tried to resolve a non-existent
polymorphicOwnerId. Fix: Frontend: added a target picker so you choose
the specific morph target (then its sub-field), storing the real
per-target field id. Backend: fixed validate-relation-subfield to use
the per-target field's own relationTargetObjectMetadataId instead of the
multi-target resolver that returned null.
5. (improvement) When an error occured in the query, the graph showed
"No data". Updated it to "error". (screenshot 1)
6. When a field used as a filter on a graph is deleted, it is not
deleted as a graph filter (which is ok because it would involve parsing
all the graph's configuration json to find whether a field is
referenced; there is no foreign key), which prevented from further
modifying the graph's filters. Fixed this + add an indicator that the
filter is can/should be removed (see screenshot 2)
7. "Ambiguous column name" PG error occurs when ordering by "creation
date" of a related field, because both objects have createdAt field.
Fixed it by adding table alias as prefix.
8. (improvement) While working on #5 I did not understand why we could
directly do `"objectMetadataNameSingular"."columnName" `while I expected
that for custom objects it would have to be
`_objectMetadataNameSingular`. that's simply because we use an alias
from the beginning. To add clarity, within groupBy code I replaced
`objectMetadataNameSingular` with `objectAlias` everywhere it is indeed
inherited from us using objectAlias.
<img width="685" height="391" alt="Screenshot 2026-06-08 at 12 01 45"
src="https://github.com/user-attachments/assets/f2b15ca5-da39-4114-8188-69f58f3c4cbf"
/>
<img width="598" height="341" alt="Screenshot 2026-06-08 at 11 53 55"
src="https://github.com/user-attachments/assets/66372811-4a37-40d9-b43a-4af51f89b6e6"
/>
|
||
|
Charles Bochet
|
fcaf2b4d9b |
chore(twenty-server): temporary instrumentation for app-install 504 (#21365)
## Why App installs on cloud intermittently fail with a 504, surfacing in Sentry as `Migration action 'update' for 'logicFunction' failed` + `Failed to rollback transaction: Query runner already released`. This is **temporary instrumentation** to pin down where the time goes — to be reverted once the bottleneck is fixed. Everything is greppable via `[install-perf]` and marked `// TODO(install-perf)`. ## What the local repro already told us I instrumented the manifest-sync/migration path and ran a local harness (new skipped spec) installing **1 / 8 / 30 logic functions**, for both create and the checksum-bump **update** (the incident path): | stage (N=30, update) | ms | |---|---| | flat-maps recompute | ~1 | | build migration | ~11 | | transaction (all actions + commit) | ~79 | | post-commit cache invalidate | ~6 | | **full sync** | **~135** | Nothing approached 1s, let alone 10s; no slow queries logged. So the migration/cache code is **not** the algorithmic cause. Given the in-transaction `UPDATE ... WHERE id=?` is intrinsically fast, a >10s in prod almost certainly means it was **blocked on a lock**, and the 10s node-pg `query_timeout` (`core.datasource.ts`) then killed the connection → the observed errors + 504. Local can't reproduce prod lock contention / table sizes, hence this instrumentation. ## What this adds (all `TODO`-marked) - **hrtime per-stage timing** — flat-maps recompute, build vs run, per-action (`>50ms`), transaction summary, post-commit cache invalidation. Uses `process.hrtime` because the integration harness enables fake timers (so `Date.now()` is useless there). - **`maxQueryExecutionTime`** slow-query logging on the core datasource (logs the offending SQL). - **Scoped `SET LOCAL lock_timeout = '8s'`** on the migration transaction (below the 10s `query_timeout`) → a blocked action fails fast with a clear *"canceling statement due to lock timeout"* instead of the opaque connection kill. - **Best-effort `pg_stat_activity` snapshot on failure** (on a fresh pooled connection) to identify the blocking session, plus a **guarded rollback** so a released connection stops masking the real error. - **Skipped local perf harness** (`logic-function-install-performance.integration-spec.ts`) — run manually with `nx test:integration:with-db-reset -- --testPathPattern "logic-function-install-performance"`. ## How we'll use it Deploy, reproduce the failing install, and read the `[install-perf]` logs: the per-action timing names the action, the `lock_timeout` message + `pg_stat_activity` snapshot name the **blocking** query/PID. Then revert this PR and fix the actual contention. Typecheck (`nx typecheck twenty-server`) is clean. |
||
|
martmull
|
77d1e8ced6 |
feat(app-dev): sync error hints, flatEntity labels, dev-mode summary UI, and docs (#21252)
Split out of #21240 — all remaining app-dev improvements. Stacked on #21251 (review/merge that first). - Actionable recovery hints on failed syncs; unified diff renderer; `--dry-run` guard. - Return `flatEntity` on update/delete sync actions and unify the diff label. - Summarize the dev-mode entity list unless `--verbose`. - Docs: syncing & recovery guide + dry-run + open-an-issue prompt. - Live execution mode for synced logic functions; clearer manifest warnings. <img width="1018" height="700" alt="image" src="https://github.com/user-attachments/assets/5e9ce19e-0f1d-4f99-8524-4e118bde932b" /> |
||
|
neo773
|
186d5b8faa | revert #21177 (#21284) | ||
|
Félix Malfait
|
91f2f08995 |
feat(server): unify workspace-event ingestion behind one EventSink pipeline (#21197)
## Why The five event-log streams (`workspaceEvent`, `pageview`, `objectEvent`, `usageEvent`, `applicationLog`) each wrote to ClickHouse through their own fire-and-forget writer (`AuditService`, `UsageEventWriterService`, and the `application-logs` driver), with the per-type knowledge (table names, normalization, access rules) spread across several modules. Three of them reimplemented the same ClickHouse insert, and the read side, the live stream, and the producers lived in different modules under two different names. This consolidates them into one `core-modules/event-logs/` subsystem (emit, write, live, read), with the per-type config in a single registry so adding an event type is roughly one file. The base Logs settings tab and free application logs shipped separately in #21180 (merged). This PR adds the unified backend, the registry, and the viewer's live mode and entitlement gating. ## Pipeline ```mermaid flowchart TB subgraph PROD["Producers"] A["auth, billing, impersonation,<br/>webhook, custom-domain"] U["usage listener"] F["logic-function executor (app logs)"] R["record CRUD (entity events)"] end EM["EventLogEmitterService<br/>createContext().insert* / dispatch()"] EQ(["entityEventsToDbQueue<br/>(existing, shared with timeline)"]) CIE["CreateEventLogFromInternalEvent"] SINK["WorkspaceEventSinkService.ingest()"] C1["ClickHouseEventSink"] C2["ConsoleEventSink"] LIVE["EventLogLiveService.publishWatched()<br/>(presence-gated)"] CH[("ClickHouse, 5 tables, async_insert")] CHAN(["WORKSPACE_EVENTS_CHANNEL"]) RS["EventLogsService (registry-driven read)"] LR["EventLogsLiveResolver"] UI["Settings > Logs"] A --> EM U --> EM F --> EM EM -->|direct| SINK R --> EQ --> CIE -->|ingest| SINK SINK --> C1 --> CH SINK --> C2 SINK --> LIVE -.->|if a viewer is watching| CHAN --> LR --> UI CH --> RS --> UI ``` ## What it does - Producers call `EventLogEmitterService.createContext().insert*()`, which builds a typed `WorkspaceEventEnvelope` and writes it through `WorkspaceEventSinkService` to the configured sinks (ClickHouse, Console) plus a presence-gated live fan-out. Record/CRUD events reach the same sink through the existing `entityEventsToDbQueue`. There is no dedicated queue; ClickHouse `async_insert` batches server-side. Writes are best-effort, as on main today. - `EVENT_LOG_TYPES[table]` is the per-type source of truth: the ClickHouse table, the required entitlement, the free-text filter column, and the row-to-GraphQL mapping. Read row shapes derive from the write rows. - Four modules along their dependency boundaries: `EventLogEmitterModule` (producer API), `EventLogIngestionModule` (sink layer), `EventLogLiveModule` (fan-out), and `EventLogsViewerModule` (the entitlement-gated GraphQL read, which is where billing/enterprise/permissions stay so producers stay light). - Logs viewer: per-table columns, filters (text, date, record), live mode, and an upgrade card that points to Billing on Cloud or the Admin Panel on self-hosted. Application logs are free on every plan; the other four require the `AUDIT_LOGS` entitlement (with a `NO_ENTITLEMENT` fallback to the upgrade card). - Renames `AuditService` to `EventLogEmitterService`, and the generic `Monitoring` event to a typed `Impersonation` event (`level` + `action`). - Removes `UsageEventWriterService`, the `application-logs` driver/module, and `AuditService`'s direct inserts. ## Durability Writes are best-effort, the same as main today (the old writers were fire-and-forget). A dedicated queue was tried mid-PR and removed: `async_insert` already batches server-side, so the queue only added durability, which isn't a requirement right now. The `EventSink` seam keeps a durable transport (e.g. a Redis-Streams buffer) easy to add later without touching producers. ## Out of scope S3 peer sink (seam only), Postgres or any second read path, `ReplicatedMergeTree`, ClickHouse table-schema changes, and the record-data `EVENT_STREAM_CHANNEL` (unchanged, separate concern). ## Testing Unit tests cover the registry definitions and row normalization, the entitlement gating, the envelope builders, and the producers. Integration tests cover the write paths (record create produces an `objectEvent`; the track mutation produces a `workspaceEvent`) and the read/query path across all five tables. Verified with typecheck, lint, a server boot, and GraphQL/SDK codegen. |
||
|
martmull
|
6c65d26ced |
feat(app-dev): add dry-run preview to dev sync (#21251)
Split out of #21240. Stacked on #21250 (review/merge that first). `yarn twenty dev --once --dry-run` computes the migration plan and prints the diff **without applying anything** (no migration, no app-record update, no SDK generation). Also renders the diff on a normal `dev --once` sync. <img width="646" height="179" alt="image" src="https://github.com/user-attachments/assets/59f3ddcd-2a5b-4b8a-b21a-c659abe16af0" /> |
||
|
neo773
|
bfb83e93b2 |
fix(metadata): resolve junction targets order-independently during mgration (#21193)
A junction relation points at a target field on the join object that another action may create later (two junctions into the same join reference each other). The builder validator now also looks up the target in the to be created set, and the runner mints every field id up front so the target resolves regardless of action order, the same way relation pairs are already handled. --------- Co-authored-by: Félix Malfait <felix@twenty.com> |
||
|
martmull
|
128d2d394d |
feat: allow apps to add view fields to existing views (defineViewField) (#21160)
## Summary
Lets a Twenty application add **view fields (columns) to an existing
view it does not own** — including standard views like the People index
view — without redeclaring/owning that view. This mirrors the existing,
working pattern by which an app adds a custom field to a standard object
via `defineField` + `objectUniversalIdentifier`.
The asymmetry being removed was purely in the manifest schema:
`ViewFieldManifest` only existed *nested* inside
`ViewManifest.fields[]`, so adding a view field forced declaring a
`ViewManifest` — which the sync treats as a view the app creates and
owns, and rejects when the UID is a standard view's. Validation,
persistence, the FK aggregator machinery, and uninstall cleanup were
already generic and cross-app-safe, so no engine changes were needed.
### Changes
- **twenty-shared:** new top-level `StandaloneViewFieldManifest`
(`ViewFieldManifest & { viewUniversalIdentifier }`),
`Manifest.viewFields`, and a `SyncableEntity.ViewField` member.
- **twenty-sdk:** `defineViewField` (validates `universalIdentifier` +
`viewUniversalIdentifier` + `fieldMetadataUniversalIdentifier`), CLI
manifest assembly of a top-level `viewFields` list, and `dev:add
viewField` scaffolding.
- **twenty-server:** one top-level loop over `manifest.viewFields` that
reuses the existing `fromViewFieldManifestToUniversalFlatViewField`
converter (already parameterized by `viewUniversalIdentifier`). No
validator/persistence/aggregator changes.
### Notes for maintainers
- Confirm the `Manifest.viewFields` optionality convention — implemented
as a **required** array to mirror `fields`/`views`.
- Two different apps adding a column for the same field to the same view
conflicts on the existing unique `(fieldMetadataId, viewId)` partial
index; the existing `flat-view-field-validator` duplicate check surfaces
this as a structured validation error.
- `dev:add viewField` scaffolding is included (was optional in the
plan).
## Test Plan
- [x] `twenty-shared` typecheck
- [x] `twenty-sdk` 364 unit tests + `buildManifest` assembly test
(rich-app fixture) + typecheck + prettier
- [x] `twenty-server` typecheck + `lint:diff-with-main`
- [x] **Server integration suite**
`successful-manifest-update-view-field.integration-spec.ts` (4/4):
- standalone view field attaches to the standard `allPeople` view
without recreating it (sync succeeds, no
`INVALID_VIEW_DATA`/`ENTITY_ALREADY_EXISTS`)
- uninstall removes the contributed column while the standard view + its
columns remain intact
- duplicate `(view, field)` rejected with `METADATA_VALIDATION_FAILED`
- unknown target view rejected
- [x] Sibling `successful-manifest-update-field.integration-spec.ts`
still green (no harness regression)
🤖 Generated with [Claude Code](https://claude.com/claude-code)
|
||
|
Ratish jain
|
979047d004 |
fix: allow email change verification on self-hosted instances (#20123)
fixes #20117 ## Technical Details Flow after fix: 1. User submits email change request 2. user.service.ts:517-524 calls sendVerificationEmail() with verificationTrigger: EMAIL_UPDATE 3. Guard checks: verificationTrigger === SIGN_UP → false → guard skipped 4. Verification token generated, email rendered and sent via emailService.send() 5. User receives confirmation email at new address 6. User clicks confirmation link → email update completes --- Impact - Minimal change: Only 3 lines modified in a single file - No breaking changes: Sign-up verification behavior unchanged - Security preserved: Email changes always require verification (correct security behavior) - Self-hosted friendly: Instance admins can disable sign-up verification while keeping email change verification active --------- Co-authored-by: Charles Bochet <charles@twenty.com> |
||
|
Paul Rastoin
|
0877ba2ffd |
Fix getApplicationSubAllFlatEntityMaps to prune unrelated appIds universal identifier aggregators (#21234)
# Introduction Atm when computing the `fromAllFlatEntityMaps` we're retrieving all the applicationIds related metadata entities to build a flat entity maps scoped to them ( atm always the applicationId + twenty-standard application id ) only inter app dependency we manage for the moment A flat entity contains universal identifier aggregator to its related entities The issue was that the `getApplicationSubAllFlatEntityMaps` wasn't pruning the aggregator by app Now added a new process phase after the initial one that will check that all the aggregators contains universal identifiers that has been retrieve from the appId + appId standard intersection ## TDD test Created a very human readable ( that's a joke ) test |
||
|
Raphaël Bosi
|
41d5d80a65 |
Migrate Company and Person standard fields in preparation for the enrichment app (#21171)
# Migrate Company and Person standard fields in preparation for the
enrichment app
## Why
Our standard `Person`/`Company` objects accumulated fields that aren't
generic to every
business, while missing a more universal revenue field that essentially
every CRM ships.
This PR makes the **Standard application** hold a tighter, more
universal set of fields,
and sets the stage for a follow-up PR that introduces a **People Data
Labs enrichment app**
to populate them.
## What changes
### Standard fields
**Demoted (Standard → Workspace Custom application)** — not generic
enough to ship as standard:
| Object | Field | Type |
| ------- | ------------------------------ | -------- |
| Company | annualRecurringRevenue (ARR) | CURRENCY |
| Company | employees | NUMBER |
| Company | idealCustomerProfile (ICP) | BOOLEAN |
| Company | xLink (X/Twitter) | LINKS |
| Person | xLink (X/Twitter) | LINKS |
| Person | city | TEXT |
**Added (new generic Standard field)** — present in
Salesforce/HubSpot/Zoho, PDL-populatable:
| Object | Field | Type |
| ------- | ------------- |
-------------------------------------------------------- |
| Company | annualRevenue | CURRENCY (generic total revenue; replaces
the niche ARR) |
### Behavior by workspace
* **New workspaces:** demoted fields are gone; `annualRevenue` is
**active**.
* **Existing workspaces:** demoted fields are **preserved as active
custom fields, data intact**;
`annualRevenue` is created **inactive (opt-in)** with its column ready,
so a later activation
is a metadata-only toggle.
### Upgrade commands (v2.9)
Three idempotent, per-workspace commands, run in timestamp order:
1. **`upgrade:2-9:move-demoted-standard-fields-to-custom-application`**
(1799000040000) —
re-owns the 6 demoted fields to the workspace custom application
(`isCustom = true`,
new `applicationId` + fresh `universalIdentifier`), keeping their data
and active state.
2. **`upgrade:2-9:rename-conflicting-custom-fields`** (1799000045000) —
if a workspace already
has a *custom* field named `annualRevenue`, renames it to
`annualRevenueCustom`
(data preserved via column rename) so the standard field can be added.
Skips non-custom matches.
3. **`upgrade:2-9:add-inactive-generic-standard-fields`**
(1799000050000) — creates
`Company.annualRevenue` on existing workspaces as inactive, guarded to
skip workspaces
missing the target object or where the name is still taken.
**Failure model:** the workspace iterator isolates failures per
workspace (one workspace failing
never affects others); within a workspace the runner records per-command
status and resumes on the
next run, and every command is idempotent, so partial runs self-heal.
### Supporting changes
* **Field-option color palette:** widened the `TagColor` union
(`twenty-shared` `FieldMetadataOptions`
+ the field-metadata `options.input` DTO) from 10 colors to the full
theme palette, benefiting any
future SELECT/MULTI_SELECT field.
* **Dev seeder:**
* The default "Annual Recurring Revenue" dashboard widget now points at
the generic
`annualRevenue` field (renamed to "Annual Revenue").
* Removed the "Companies by Size (Stacked by City)" widget (relied on
the demoted `employees`).
* `employees` is dropped from company data seeds and re-added as a
**custom** field seed, so dev
workspaces still get an `employees` column matching the demoted
behavior.
### Cleanup
Front-end record types (`Company.ts`/`Person.ts`), the
`getDisplayNameFromParticipant` test mock,
metadata integration specs, the Zapier `crud_record` test, and the
regenerated
`get-standard-object-metadata-related-entity-ids` snapshot.
## ⚠️ Breaking change (intentional)
Removes standard fields `Company.annualRecurringRevenue`,
`Company.employees`,
`Company.idealCustomerProfile`, `Company.xLink`, `Person.xLink`, and
`Person.city` from the core
GraphQL schema (replaced by `Company.annualRevenue`).
This is why the breaking-changes check reports a large number of
removals — `graphql-inspector`
flags any removed object field plus its derived
aggregate/order-by/filter/update types.
**Mitigation:** the
`upgrade:2-9:move-demoted-standard-fields-to-custom-application` command
re-owns these fields as custom fields per workspace, preserving their
name and data, so existing
tenants keep working. New workspaces won't have them.
|
||
|
Paul Rastoin
|
3d49642d12 |
[AUDIT] Run knip over twenty-server (#21159)
# Introduction Run [knip](https://knip.dev/) over twenty-server Used config: ```json { "$schema": "https://unpkg.com/knip@5/schema.json", "workspaces": { "packages/twenty-server": { "entry": [ "src/main.ts", "src/command/command.ts", "src/queue-worker/queue-worker.ts", "src/database/scripts/setup-db.ts", "src/database/scripts/truncate-db.ts", "src/database/clickHouse/migrations/run-migrations.ts", "src/database/clickHouse/seeds/run-seeds.ts", "src/instrument.ts", "lingui.config.ts", "test/integration/graphql/codegen/index.ts", "test/integration/utils/setup-test.ts", "test/integration/utils/teardown-test.ts", "scripts/**/*.ts", "**/*.spec.ts", "**/*.integration-spec.ts" ], "project": ["src/**/*.ts", "test/**/*.ts", "scripts/**/*.ts"], "ignore": [ "src/database/typeorm/**/migrations/**", "src/database/typeorm/**/*.entity.ts", "**/*.workspace-entity.ts", "**/logic-function-resource/constants/seed-project/**" ], "ignoreDependencies": ["@types/psl", "@types/aws-lambda"], "ignoreBinaries": ["nest", "lingui", "typeorm"] } } } ``` |
||
|
Etienne
|
15eaabdbc1 |
fix(ai) - optimize crud tools (#21133)
- **Add delete many**, `delete_many_{object}` added alongside the
existing `delete_one_{object}`.
- **Uniformize naming**, crud module, type names, and MCP helper
constants renamed for consistency.
- **Optimize tool schema (learn phase)**
- `find_many(_companies)`: **7 158 → 2 700 tokens**
- `find_one(_company)`: **280 → 126 tokens**
- ....
- Main mechanism: `reused: 'ref'` (line 7 of
`to-tool-json-schema.util.ts`). Zod walks the schema tree, tracks which
Zod schema instances appear more than once, and emits each reused
instance exactly once in `$defs`, replacing all subsequent occurrences
with a `$ref`. Works because filter and value schemas are now extracted
as shared objects.
- **Optimize system prompt (tool catalog)**, DATABASE_CRUD section
restructured to list operation patterns (`find_many_{object}`, …) once +
objects once, instead of the full N×M cross-product of tool names.
- **Optimize execute_tool**, shared record-properties schema (same
`$defs` deduplication applies at call time); introduced `upsert_many`;
added `selectedFields` to `find_*` so the agent only fetches the fields
it needs.
|
||
|
Etienne
|
dd0039ca1c |
feat(mcp) - optimize instruction prompt and hide get_tool_catalog (#21183)
Workspace-aware initialize.instructions - Deleted the static mcp-server-instructions.const.ts - Created build-mcp-server-instructions.util.ts — a comprehensive system prompt with identity, object list, tool grammar, routing decision tree, intent mapping, skills vs tools, safety constraints, and data efficiency guidelines - Created McpInstructionBuilderService — fetches workspace-specific object names + skill names and injects them into the instructions Hide/deprecate get_tool_catalog Benefit : skip first MCP call (tools are included in instruction) |
||
|
Weiko
|
cc76b7bc50 |
Fix fields widget new field visibility (#21111)
Fixes https://github.com/twentyhq/twenty/issues/21043 ## Context Newly created fields were never added to FIELDS widgets, regardless of the "Set fields created in the future as visible" toggle. The widget's newFieldDefaultVisibility was null on widgets that never explicitly set it (it was never populated at creation), so the backend skipped them and no view field was created. ## Implementation Keep newFieldDefaultVisibility nullable with false (not visible) as the behavior when not provided. The FE now reflects that properly and shows "un-toggled" when it's null (iso with BE behavior). The fix also ensures the value is explicitly set to true wherever it should be: - Set newFieldDefaultVisibility: true at every FIELDS widget creation path (backend default record-page layout, frontend createDefaultFieldsWidget + useTemporaryFieldsConfiguration); - Added a 2-9 workspace upgrade command that backfills true onto existing standard FIELDS widgets where the value is null. |
||
|
Paul Rastoin
|
164a5b1e8d |
Refactor email composer (#21177)
# Introduction Gate what connected account can be ingested in case of ai mcp user workspace agnostic funnel to only the workspace shared connected account Added a quick win intregration tests on seeded connected accounts ( that wasn't covered but already protected fix impacts only the mcp ) Refactored the API slightly too ## Notice This mean there's a breaking change in the product behavior Whereas before a non user workspace related mcp interaction would might have fallback on any private user connected account it will now only search for workspace visible listed ones |
||
|
Charles Bochet
|
58907b733c |
feat(logic-function): add LIVE / PREBUILT execution modes (#20873)
## Summary
### Why
1. Sending the code to the lambda (~1Mb usually) is heavy on network and
results to a constant traffic of ~30Mb/s on AWS which results into TB of
network data every month
2. eval(1MB of code) is not that fast, it's heavy on memory and CPU on
lambda side
### High level
Adds two execution modes for logic functions, gated behind the new
`IS_LOGIC_FUNCTION_PREBUILT_MODE_ENABLED` workspace feature flag (off
everywhere by default):
- **LIVE** (current behavior, preserved bit-for-bit): the compiled
bundle is read from object storage and shipped in every Lambda invoke
payload. Used for fast iteration in the workflow editor / Settings test
runs.
- **PREBUILT** (new): the bundle is installed onto the per-function
Lambda alongside the unified executor, and invocations carry only `{
params, env, handlerName }` — saving JSON payload egress and warm-start
`import()` cost on every call.
### Key design choices
- **Unified Lambda handler** (`constants/executor/index.mjs`) dispatches
at runtime: `event.code` present ? LIVE (write to `/tmp`, dynamic
import) : `import('./prebuilt-logic-function.mjs')`. Both code paths
always coexist on the deployment package, so the same Lambda can serve
either mode without redeploying.
- **Install runs inside the `validateBuildAndRun` migration pipeline**,
not at execute time. `Create/UpdateLogicFunctionActionHandlerService`
calls `driver.installPrebuiltBundle` when `executionMode` flips
LIVE?PREBUILT or `checksum` changes while PREBUILT, gated on
`isBuildUpToDate=true` and a fresh checksum.
- **Strict execute, no reconciliation**:
`LogicFunctionExecutorService.execute` resolves `effectiveExecutionMode`
(caller override > feature flag > entity column). For PREBUILT it asks
the driver `getInstalledBundleChecksum` (Lambda `twenty:bundle-checksum`
tag for AWS, sidecar file locally) and throws
`LOGIC_FUNCTION_PREBUILT_BUNDLE_NOT_INSTALLED` on mismatch.
- **Feature flag gates every side effect**: with the flag off the
executor forces LIVE, the action-handler install hooks bail before AWS,
and workflow activation does not flip the mode. Rollback is just turning
the flag off.
### Lifecycle
- New workflow CODE step ? `LIVE`, no install.
- Workflow activated ? build + activation flips `executionMode=PREBUILT`
? action-handler installs the bundle + sets the Lambda tag.
- Draft from active version ? duplicated logic function reset to `LIVE`.
- App install ? manifest converter sets `PREBUILT`, create-action
handler installs.
- Test runs (`executeOneFromSource`, workflow editor) pass
`executionMode=LIVE` explicitly.
### Observability
`[lambda-timing]` log lines now include `effectiveExecutionMode` and
`payloadBytes`; the action handler logs `install_duration_ms` for each
install.
## Test plan
- [x] `npx nx typecheck twenty-server` ? passes
- [x] `npx oxlint --type-aware` on all changed files ? 0 warnings, 0
errors
- [x] `npx nx test twenty-server` ? 588 suites / 5009 tests pass (no
regressions vs main)
- [x] New unit suite `flat-logic-function-validator.service.spec.ts` ?
9/9
- [x] Existing
`workflow-version-step-operations.workspace-service.spec.ts` ? 8/8
(verified the new token-based DI avoids a circular-import regression)
- [x] Snapshot for
`ALL_UNIVERSAL_FLAT_ENTITY_PROPERTIES_TO_COMPARE_AND_STRINGIFY` updated
to include `executionMode`
- [x] Integration suite `logic-function-execution.integration-spec.ts`
extended to assert `executionMode=LIVE` on newly-created functions and
continues to exercise the LIVE happy path
- [ ] Manual staging rollout: flip
`IS_LOGIC_FUNCTION_PREBUILT_MODE_ENABLED` per workspace, observe
`[lambda-timing]` `payloadBytes` drop + `install_duration_ms`, then ramp
in prod.
|
||
|
Paul Rastoin
|
6a908b7876 |
Front component s3 redirect (#21116)
# Introduction Unload the server of the file stream when possible Also fix inconsistent pipeline exception management Needs to highly be QA, not sure how the cors will behave here |
||
|
Félix Malfait
|
75df1f3997 |
chore(settings): address review comments from PR 21072 (#21121)
## Summary Round through bosiraphael's 31 review threads on the merged PR #21072 (discovery hero + ephemeral playground token). The user asked to apply each suggestion only where it adds value, so this PR is split into three buckets. ### Comments (~17 threads) - Tightened security-rationale / CSS-gotcha / API-doc comments to one or two factual lines - Kept (shortened) the comments above `RequireAccessTokenGuard` call sites — without them a future reader could remove the guard and silently reopen the escalation hole - Kept (shortened) the in-memory-only rationale on `playgroundApiKeyState` for the same reason - Kept `flex: 1 + min-height: 0` CSS gotcha on `SubMenuTopBarContainer` — non-obvious and easy to break ### Structure / extraction - Move `WEBHOOK_TABLE_ROW_GRID_TEMPLATE_COLUMNS` to its own constants file (one-export-per-file) - Split `SettingsAgentToolsTab` and `SettingsAgentToolsTable` across queries/, hooks/, types/, utils/: - `graphql/queries/findManyApplicationsForToolTable.ts` - `graphql/queries/findManyMarketplaceAppsForToolTable.ts` - `hooks/useSettingsAgentToolsTable.ts` (data loading + index merging) - `types/SettingsAgentToolItem|Application|MarketplaceApp` - `utils/getToolApplicationId|getToolLink` - Extract `SettingsAiModelsTab` optimistic mutations into `hooks/useSettingsAiModelsActions` (handleModelFieldChange, handleUseRecommendedToggle, handleModelToggle, handleToggleAllVisibleModels) - Extract `SettingsAI.handleCreateTool` into `hooks/useCreateTool` - Drop unnecessary `useMemo` wrappers on `heroTabs` arrays (SettingsObjects, SettingsLayout) - Simplify `MenuItemToggle` handler in SettingsAgentSkillsTab: `onToggleChange={setShowDeactivated}` (no longer wrapping with arrow + read of stale `!showDeactivated`) ### Hero assets - Replace placeholder `customize-illustration` with per-page exports - Rename `layout/customize-illustration-{light,dark}.png` → `layout/cover-{light,dark}.png` - Add `cover-{light,dark}.png` for **applications** and **members** (they were both pointing at the layout placeholder as a TODO) - Overwrite `data-model/cover-*.png`, `playground/cover-*.png`, `ai/ai-tools-cover-*.png` with the new exports ## Test plan - [ ] `npx nx typecheck twenty-front` ✅ - [ ] `npx nx typecheck twenty-server` ✅ - [ ] `npx nx lint twenty-front` ✅ (oxlint + oxfmt, 0 warnings/errors) - [ ] `/settings/layout`, `/settings/data-model`, `/settings/applications`, `/settings/ai`, `/settings/api-webhooks`, `/settings/members` each render the new hero illustration (light + dark) - [ ] AI tab: tool list still loads, search + Custom/Managed/Standard filters still work, "New Tool" still navigates to detail - [ ] AI tab: Models tab — smart/fast model select, "Use best models only" toggle, per-model checkboxes, toggle-all all still optimistic+revert on error - [ ] Skills tab: "Deactivated" toggle still flips show/hide - [ ] Webhooks table still uses the 1fr 28px grid |
||
|
Weiko
|
d86e827563 |
fix: return proper FORBIDDEN GraphQL errors from ApiKeyResolver (#21107)
## Context CI is broken on main, regression introduced in https://github.com/twentyhq/twenty/pull/21072 Guard-rejected ApiKey mutations returned malformed GraphQL responses. RequireAccessTokenGuard (and SettingsPermissionGuard) throw plain AuthException/PermissionException classes, which are not GraphQLErrors. ApiKeyResolver had no @UseFilters, so these exceptions were never translated, they surfaced as request-level errors with no data key (data: undefined) and a non-FORBIDDEN code, instead of data: null + FORBIDDEN. This broke the `createApiKey › should reject a non-ACCESS token even with API key permission` integration test (expect(res.body.data).toBeNull() received undefined). The sibling generateApiKeyToken test passed only because it lives on AuthResolver, which already declares these filters. ## Fix Add the standard exception filters to ApiKeyResolver, matching the idiom used by other guard-protected resolvers ```ts @UseFilters(AuthGraphqlApiExceptionFilter, PermissionsGraphqlApiExceptionFilter) ``` |
||
|
Félix Malfait
|
b338a7a1d2 |
feat(settings): discovery hero rollout + ephemeral playground token (#21072)
## Summary
Two intertwined streams of work:
### UI — discovery hero pattern, settings shell, AI/API redesign
- **Generalize `SettingsDiscoveryHeroCard`** and use it on Layout, Data
Model, Apps, AI, API/Webhooks, Members. Drops 4 per-page wrapper files
(`SettingsObjectCoverImage`, `SettingsLayoutCoverImage`,
`SettingsLayoutCustomizeVideoModal`,
`SettingsDataModelVisualizeVideoModal`). Each page now supplies cover
src, modal id, and tab list.
- **Modal**: swap `<video>` placeholder for the Vimeo iframe pattern
from `twenty-docs`, per-tab `vimeoId`. Drop the parallel border-bottom
on the header (TabList draws its own baseline) and the grey background
behind the video. Note: Vimeo's embed allowlist applies — the iframes
load with the correct URL on `localhost` but the player itself requires
the video owner to allow the dev/staging domains in Vimeo settings.
- **AI page** rebuilt into a Cockpit pattern (Overview / Models / Skills
/ Tools / Usage). New `SettingsAiOverviewTab` with default Smart/Fast
pickers, at-a-glance stats, and an MCP signpost that deep-links to
`/settings/api-webhooks#mcp`. System Prompt link moved under Models.
Advanced tab removed.
- **API & Webhooks** now has 4 tabs (Playground / MCP / API Keys /
Webhooks). Hero card above tabs. Playground tab inverted to "Core API" /
"Metadata API" sections, each containing REST + GraphQL cards — schema
is the meaningful axis, protocol is secondary. Hash deep-link sync
delegated to the shared `TabListFromUrlOptionalEffect`.
- **Settings shell**: unified drawer outer padding (kill `isSettings`
branch), extract `CollapsibleNavigationDrawerSection`, add `iconColor`
on settings nav items, fix Exit Settings button alignment, 880px content
cap.
### Backend — strategy C: ephemeral playground token
The legacy paste-your-API-key flow is replaced by an on-demand
short-lived token scoped to the calling user's permissions. No shared
"Playground" API key to manage or revoke.
- New `JwtTokenTypeEnum.PLAYGROUND`. `PlaygroundTokenJwtPayload =
Omit<AccessTokenJwtPayload, 'type' | impersonation fields>` so any
future ACCESS claim flows through automatically.
- `AccessTokenService.generatePlaygroundToken` signs an access-shaped
JWT with `type: PLAYGROUND` and a configurable short TTL. A shared
private `resolveTokenSubject` helper parallelizes the user / workspace /
userWorkspace lookups for both generators.
- `JwtAuthStrategy.validateAccessToken` widened to accept
`AccessTokenJwtPayload | PlaygroundTokenJwtPayload`; impersonation gated
on `payload.type === ACCESS` so the union narrows without `as unknown
as` casts. The two branches in `validate()` collapse into one.
- New `PLAYGROUND_TOKEN_EXPIRES_IN` config var (default `2h`).
- New `generatePlaygroundToken` mutation (`WorkspaceAuthGuard`, no args,
returns `AuthToken`).
- Frontend `useOpenPlayground` hook centralizes mint → atom write →
navigate, with Apollo `onError` snackbar and a "use cached PLAYGROUND
token if still fresh" short-circuit (decodes via `jwt-decode`, checks
both `type` AND `exp`). Old API_KEY tokens left in localStorage from the
prior paste-form flow are rejected on `type` alone and force a re-mint —
this is what was causing the "This API Key is revoked" symptom on stale
browsers.
### Drive-by cleanups
- `PlaygroundToken` DTO removed (identical shape to `AuthToken` already
in use).
- 5 `customize-sidebar.webm` imports and the dead placeholder pipeline
removed.
## Test plan
### Discovery hero
- [ ] `/settings/layout`, `/settings/data-model`,
`/settings/applications`, `/settings/ai`, `/settings/api-webhooks`,
`/settings/members` each render the discovery hero card with its
illustration + play button + tabbed modal
- [ ] Modal tabs show the correct Vimeo embed URL per tab; aspect ratio
stays at 1440/900; no parallel border-bottom jog at the tab baseline
- [ ] AI Overview tab shows Smart/Fast model pickers + stats grid + MCP
signpost card; the MCP card lands on `/settings/api-webhooks#mcp` with
the MCP tab active
### API playground (ephemeral token)
- [ ] With an empty `playgroundApiKeyState` in localStorage, clicking
REST or GraphQL playground card opens the playground and the cached
token has `type: "PLAYGROUND"` with ~2h exp
- [ ] Clicking the card again within the freshness window does **not**
re-mint (`iat` / fingerprint stable across visits)
- [ ] Planting a fake API_KEY-shaped JWT in localStorage and clicking
the card forces a fresh mint (old token rejected on `type`)
- [ ] `GET /rest/companies?limit=1` with the cached token returns 200 +
real data
- [ ] `POST /graphql { __typename }` returns 200
### Settings shell
- [ ] Settings nav matches main app drawer padding; sections collapse;
Exit Settings button aligns with the workspace links above
- [ ] Active nav items have a right-gap (cleaner active state)
- [ ] Content area capped at 880px
### Verify
- [ ] `npx nx typecheck twenty-front` passes
- [ ] `npx nx typecheck twenty-server` passes
- [ ] `npx nx lint:diff-with-main twenty-front` passes
- [ ] `npx nx lint:diff-with-main twenty-server` passes
|
||
|
nitin
|
13f09d8946 |
[Dashboards] Remove gauge chart types and code (#20410)
Follow-up cleanup to #20172. |
||
|
nitin
|
996cdaf3ff |
refactor(agents): split tool resolution into native and action rails (#20331)
## Summary
Splits AI agent tool resolution into two independent rails:
- **Native tools** — capabilities baked into the model SDK
(Anthropic/OpenAI `web_search`, xAI `web`/`x` provider options). Bound
by `NativeToolBinderService`, controlled by per-agent
`modelConfiguration` toggles. Opaque to Twenty — executed on the model
provider's servers.
- **Action tools** — registry-scoped tools from `ToolRegistryService`
(code interpreter, send email, record CRUD, etc.). Permission-gated via
the agent's role. Executed on Twenty's server.
Both rails merge into a single `ToolSet` at call time. When both
surfaces expose a search tool the model picks at runtime — coexistence
is intentional (relevant once Exa returns as an action, see below).
## Notable changes worth calling out
**Contract change: `AgentAsyncExecutorService.executeAgent` no longer
accepts `rolePermissionConfig`.** Workflow agents now scope exclusively
by the agent's own permission-tab role (`unionOf: [agentRoleId]`). The
previous role-merging path (caller role intersected with agent role) is
removed. No agent role → no registry tools (fail-closed by design).
**`NativeToolBinderService` relocated** from
`core-modules/tool-provider/native/` →
`metadata-modules/ai/ai-models/services/`. The binder needs SDK-package
knowledge, which lives in `ai-models`. Old location created a backwards
module dependency.
**`NATIVE_MODEL_TOOLS_BY_SDK_PACKAGE` is exhaustive over
`AiSdkPackage`** (`Record<>`, not `Partial<Record<>>`). Adding a new SDK
without thinking about native tools now fails the build. SDKs without
native tools (Bedrock, Google, Mistral, Azure, OpenAI-compatible) get
explicit `{}` entries.
**Discriminated union `kind: 'sdk-tool' | 'provider-option'`** lets one
registry describe both function tools (Anthropic/OpenAI) and runtime
sources (xAI). Follows the local `tool-provider` convention from #19321.
## Deferred to follow-ups
- **Exa web search is dropped from this PR** (along with its
`WEB_SEARCH_TOOL` permission flag and the Exa-specific gating). Exa
comes back as an **action/app tool** once apps can define permission
flags through the SDK — ongoing work in #20481.
- **xAI native search currently errors.** xAI deprecated its Live Search
API (the `web`/`x` provider-option sources this rail maps to), so xAI
returns `410` when native search is actually exercised. The code path
itself is clear — it's only hit if you test xAI native tools. Fixed
separately alongside the broader xAI model fixes.
## Conscious non-decisions
- **No "twenty-native" category.** `native` is reserved for
model/provider SDK features; everything Twenty-owned is just a
tool/action.
- **Coexistence over precedence.** No rule forcing an action search tool
to override native search (or vice-versa) — when both exist, it's the
user's choice in workflow agents and the model's choice in chat.
---------
Co-authored-by: Félix Malfait <felix.malfait@gmail.com>
Co-authored-by: Félix Malfait <felix@twenty.com>
|
||
|
Paul Rastoin
|
ebfaca5b3d |
EncryptedString PlaintextString branded string types (#21001)
## Summary closes https://github.com/twentyhq/core-team-issues/issues/2464 Introduces compile-time branded types to distinguish encrypted ciphertext from plaintext strings, preventing mix-ups like the one fixed in #20819 — but at the type level rather in addition to the one existing at runtime. ### Branded string primitives - Created `EncryptedString` and `PlaintextString` as hard nominal brands using `z.string().brand(...)`, making them non-assignable to each other or to raw `string` - Created `isEncryptedString` type predicate to narrow `string` to `EncryptedString` based on the `enc:v2:` envelope prefix - Retyped `SecretEncryptionService`: `encryptVersioned` accepts `PlaintextString`, `decryptVersioned` returns `PlaintextString` ### Entity typing - Typed encrypted columns across entities: `SigningKeyEntity.privateKey`, `TwoFactorAuthenticationMethodEntity.secret`, `ApplicationRegistrationVariableEntity.encryptedValue`, `ApplicationVariableEntity.value` - Parameterized JSONB types for connected account connection parameters (`ImapSmtpCaldavParams<Pwd>`) with reusable aliases `EncryptedImapSmtpCaldavParams` / `DecryptedImapSmtpCaldavParams` - Typed DTOs (`CreateApplicationRegistrationVariableInput`, `UpdateApplicationRegistrationVariablePayload`, `UpdateApplicationVariableEntityInput`) with `PlaintextString` ### ApplicationVariable always-encrypt uniformization - Retyped `ApplicationVariableEntity.value` to `EncryptedString | ''` — all values are now encrypted regardless of `isSecret` - Updated `ApplicationVariableEntityService` to always encrypt on write and always decrypt on read - Simplified `UpdateApplicationVariableActionHandlerService` by removing conditional encrypt/decrypt-on-isSecret-toggle logic - Added slow instance command (`2.9.0`) to backfill-encrypt existing `isSecret=false` plaintext rows and tighten the `CHECK` constraint ### ConfigStorageService refactor - Split `convertAndSecureValue` (which used `any`) into two well-typed methods: `convertAndDecrypt` and `convertAndEncrypt` - Introduced `isSensitiveStringValue` type predicate to narrow values before encryption/decryption ### What's next - Typeorm entity derivation to strictly type sitemap configuration as code + handler logic for encryption rotation - https://github.com/twentyhq/core-team-issues/issues/2465 |
||
|
Félix Malfait
|
531410f64a |
fix(ai): expose MORPH_RELATION join columns in AI/MCP tool schemas (#21012)
## Summary
- Fixes a bug where `noteTarget` (and any other morph-relation join
object) created via AI/MCP would land with `targetCompanyId` /
`targetPersonId` / `targetOpportunityId` left null, even though the tool
reported success.
- Root cause: the Zod schema generators for the AI tools only branched
on `FieldMetadataType.RELATION`. MORPH_RELATION fields fell through to
the default case — for `create_*` they were exposed as `targetCompany:
string` instead of `targetCompanyId: uuid`, and for `group_by_*` they
were silently skipped entirely. Downstream
(`data-arg-processor.service.ts` and the group-by arg processor) already
accept the join-column form for both kinds of relations via
`computeMorphOrRelationFieldJoinColumnName` and
`isMorphOrRelationFlatFieldMetadata`, so the fix is purely in the schema
generators.
## Changes
- `record-properties.zod-schema.ts` — extend the existing RELATION
MANY_TO_ONE / ONE_TO_MANY branches to also match MORPH_RELATION.
- `group-by-tool.zod-schema.ts` — replace the silent MORPH_RELATION skip
with the same treatment as RELATION MANY_TO_ONE (exposes `${name}Id` as
a groupBy option).
- `test/integration/ai/suites/mcp-tool-execution.integration-spec.ts` —
new file. First integration test for tool execution end-to-end. Drives
the real MCP JSON-RPC endpoint with the seeded API key (`learn_tools`
for schema introspection, `execute_tool` for invocation):
- asserts `create_note_target`'s schema exposes `targetCompanyId` /
`targetPersonId` / `targetOpportunityId` as UUIDs and does **not**
expose `targetCompany` / `targetPerson` / `targetOpportunity`.
- creates a company + note + noteTarget via MCP, then queries the
workspace schema to confirm `targetCompanyId` is actually persisted in
the FK column.
- asserts `group_by_note_targets` schema accepts `targetCompanyId` as a
groupBy key.
- sets up 3 noteTargets (2 → company A, 1 → company B), calls
`group_by_note_targets` by `targetCompanyId`, and asserts the counts.
Out of scope: `record-filter.zod-schema.ts` has the same pattern (only
RELATION) — left for a follow-up so this PR stays focused on what was
reported.
## Test plan
- [x] `npx nx typecheck twenty-server`
- [x] `npx oxlint --type-aware` on changed files — clean
- [x] `npx oxfmt --check` on changed files — clean
- [x] Integration tests pass (4/4) after `database:reset`:
- `should expose the morph-relation join columns as \`${name}Id\` UUID
parameters`
- `should persist targetCompanyId when create_note_target is invoked via
MCP`
- `should expose targetCompanyId as a valid groupBy option`
- `should group noteTargets by targetCompanyId via MCP`
|
||
|
neo773
|
c5606212f2 |
Ses outbound followup (#20610)
This pull request unifies outbound with inbound under the new feature and the new email groups feature. These are workspace level shared inboxes that are shared between all workspace members. outbound sending with SES works, we only listen for tenant status events, rest is managed by AWS PR refactors old code and webhook to be split for outbound and inbound for proper separation | Area | Change | |---|---| | AWS SES driver | Split into `AwsSesRegisterDomainService` (tenant + identity + DKIM + MAIL FROM + configuration-set + EventBridge dest + contact list) and `AwsSesSendEmailService` (SendEmail). | | Reputation webhook | New `/webhooks/messaging/ses/outbound` route. SES → EventBridge (`Sending Status Enabled/Disabled` on default bus) → SNS → router → `SesOutboundSendingStateHandlerService` updates `emailing_domain.tenantStatus`. | | Inbound webhook | Refactored into `SesInboundWebhookRouterService` + `SesInboundMailHandlerService`. Shared `SnsSignatureVerifierService` + `SnsSubscriptionConfirmerService` across both routes. | | Global uniqueness | New migration + instance command: `emailing_domain.domain` is now globally unique (one tenant per domain across workspaces). | | Tenant status | New `emailing_domain.tenantStatus` column (`ACTIVE` / `PAUSED`) + `EmailingDomainTenantStatusService`. | | Send-email mutation | New `sendEmailViaDomain` GraphQL mutation + DTOs. | | Cleanup | `EmailingDomainWorkspaceCleanupJob` wired into `WorkspaceService.deleteWorkspace` — tears down SES tenant association + identity on workspace delete. | | Settings UI | Rewritten around reusable `SettingsTableListSection`. "Email Group" → "Email Handle" rename. New cells for status/source/forwarding. Outbound domains surfaced on workspace settings page. | ### Env vars (new) All in `config-variables.ts`, group `AWS_SES_SETTINGS`, all optional: - `AWS_SES_REGION` — `@IsAWSRegion`, consumed by `AwsSesClientProvider` + driver factory - `AWS_SES_ACCOUNT_ID` — used for ARN construction in driver factory - `SES_SNS_TOPIC_ARN_ALLOWLIST` — **shared** by inbound + outbound webhook routers, comma-separated list of accepted SNS topic ARNs (verified via `sns-payload-validator`) ### Migrations - `1778862608620-add-emailing-domain-tenant-status` (fast) — adds `tenantStatus` column. - `1778865501791-unique-emailing-domain-globally` (slow, idempotent) — enforces global uniqueness on `domain`. - Instance commands bumped to `2.5`. ### Infra dependency Two coupled twenty-infra PRs: - `ses-inbound-email` — receipt-rule + inbound SNS topic + S3 bucket policy + KMS grant + `email_group_*` outputs. - `ses-outbound-tf` — EventBridge rule + outbound SNS topic + SES IAM policy + outbound `webhook_url` subscription. **Based on `ses-inbound-email`.** Merge order: inbound first, then outbound. Outbound PR's chart edit owns the comma-joined `SES_SNS_TOPIC_ARN_ALLOWLIST` value (both ARNs). Features lives under `/settings/general` <img width="1496" height="845" alt="SCR-20260519-ofhi-2" src="https://github.com/user-attachments/assets/a025485a-09f7-4131-91cd-0067690ff18d" /> --------- Co-authored-by: Félix Malfait <felix.malfait@gmail.com> Co-authored-by: claude[bot] <41898282+claude[bot]@users.noreply.github.com> Co-authored-by: Félix Malfait <FelixMalfait@users.noreply.github.com> |
||
|
martmull
|
fbae66de8a |
Fix error when token invalid (#20972)
## Before <img width="914" height="519" alt="image" src="https://github.com/user-attachments/assets/8933bf9e-d8db-4670-ad08-69e900083fc1" /> ## After <img width="1105" height="523" alt="image" src="https://github.com/user-attachments/assets/d8638dbb-adee-4b7d-9d29-1a2a0185ab4f" /> |
||
|
nitin
|
863f3f29a2 |
Fix page layout widget tab moves (#20915)
Fixes widget moves between page layout tabs by making pageLayoutTabId part of the flat-entity diff, so the save mutation no longer silently drops the new tab assignment. https://discord.com/channels/1130383047699738754/1508737039128985680 |
||
|
Félix Malfait
|
8cb88cabee |
fix(role): rebind API keys + agents before deleting their role (#20935)
## Customer-reported bug
A customer hit this when using the AI chat:
```json
{
"message": "API key 760d4822-da40-4b3f-9031-40563d7ed6c9 has no role assigned",
"extensions": {
"code": "INTERNAL_SERVER_ERROR",
"userFriendlyMessage": "This API key has no role assigned."
}
}
```
Their integration authenticates via API key. Somewhere along the way,
the role bound to that API key was deleted, leaving the API key
authenticated but role-less. Any request that hits a permission check
(`getRoleIdForApiKeyId`) blows up.
## Root cause
In `RoleService.deleteManyRoles`, the pre-deletion cleanup
(`assignDefaultRoleToMembersWithRoleToDelete`) only rebinds **user
workspaces** to the workspace default role. API keys and agents pointing
at the role are ignored. Because `RoleTargetEntity.role` declares
`onDelete: 'CASCADE'`, the FK then drops the role_target rows for those
API keys / agents — but the API keys themselves stay in `api_key`, now
orphaned in `apiKeyRoleMap`.
A previous read-side workaround
([2767ddac44](https://github.com/twentyhq/twenty/commit/2767ddac44) —
make the `role` ResolveField nullable) handled the API-key-details page,
but did not address the write paths (`getRoleIdForApiKeyId`).
## Fix
- Rename `assignDefaultRoleToMembersWithRoleToDelete` →
`rebindTargetsOfRoleToDeleteToDefaultRole` and extend it to rebind API
keys (via `ApiKeyRoleService.assignRoleToApiKey`) and agents (via
`AiAgentRoleService.assignRoleToAgent`) in the same step, before the
role is deleted.
- If the workspace default role doesn't satisfy `canBeAssignedToApiKeys`
/ `canBeAssignedToAgents`, the inner `assignRoleTo*` validation throws.
We catch that and rethrow as a `PermissionsException` with a
role-deletion-context message and two new codes —
`ROLE_CANNOT_BE_ASSIGNED_TO_API_KEYS` /
`ROLE_CANNOT_BE_ASSIGNED_TO_AGENTS` — so the admin sees a clear
"reassign these first" prompt rather than a confusing inner error.
## Scope / non-goals
- **Already-orphaned API keys are not auto-healed.** The customer still
needs to reassign a role to their existing orphan API key via the UI
(Settings > API Keys > [the key] > role). A separate cleanup command for
existing orphans is a follow-up.
- I did not investigate *why* the customer's session was authenticated
via API key in the AI chat — that may be their integration setup. Worth
confirming with them separately.
## Test plan
- [ ] Workspace with default role `Admin` (which has
`canBeAssignedToApiKeys: true`): create an API key with a custom role,
delete the custom role → API key is rebound to Admin, requests keep
working.
- [ ] Workspace with default role `Member` (default, has
`canBeAssignedToApiKeys: false`): create an API key with a custom role,
delete the custom role → role deletion fails with the new
`ROLE_CANNOT_BE_ASSIGNED_TO_API_KEYS` error explaining the admin must
reassign first. API key + custom role are both unchanged.
- [ ] Same two scenarios for agents (`canBeAssignedToAgents`).
- [ ] Existing user-workspace rebind behavior is unchanged.
- [ ] Role deletion with no dependent API keys / agents still works.
|
||
|
Félix Malfait
|
ac89d2ff56 |
feat: raise FILES field max number of values from 10 to 60 (#20950)
## Summary Raises the artificial hardcoded ceiling on `maxNumberOfValues` for custom FILES fields from `10` to `60` so users can attach more files per record. - Bumped `FILES_FIELD_MAX_NUMBER_OF_VALUES` constant in `twenty-shared` from `10` to `60` - Updated validator unit test (inline snapshots + "exceeds max" case) - Updated create/update files-field metadata integration tests and Jest snapshots The frontend Zod schema only enforces a `min`, so no frontend changes are required — the backend constant is the single source of truth for the upper bound. Refs #20942 🤖 Generated with [Claude Code](https://claude.ai/code) Co-authored-by: claude[bot] <41898282+claude[bot]@users.noreply.github.com> |
||
|
Etienne
|
5eb79e7797 |
fix(billing) - fix orphaned stripe subs 2/2 (#20916)
Fixes https://sonarly.com/issue/40688 Should have been included in https://github.com/twentyhq/twenty/commit/0edd8d400c646cd6a40ff0fea5342a0f61645d5e |
||
|
Paul Rastoin
|
b8de469f37 |
Refactor and centralize file mimeType integrity check and sanitization (#20889)
# Introduction closes https://github.com/twentyhq/private-issues/issues/484 This PR refactors the writeFile API to never expect to be passed a mimetype, its extract is done programmatically low level so any callers will pass through Same for the file sanitization ## IANA override Disclaimer for consistency we existing behavior we wanted to always have `application/typescript` - should we rather consider fallbacking to octect-steam instead ? - Any pulbic assets that has .ts will now also fallback to `application/typescript` instead of the official IANA ## Integration Added coverage |
||
|
Weiko
|
08682fb3f5 |
Various fixes - some AI findings (#20921)
App permissions tab: - The fallback uuidv4() for a marketplace field was generated twice, so id and universalIdentifier could diverge; it's now computed once and reused as it seemed to be the intention (even though I don't really think it's a good idea) - Renamed buildobjectMetadataItemsFromMarketplaceApp → buildObjectMetadataItemsFromMarketplaceApp to follow camelCase. Morph relation validation: - Fixed the user-facing message "At least one relation is require" → "...is required" - Typos in the related test descriptions (Morh → Morph, samefield → same field) and their snapshots. Docs - The UUID field-type row in views.mdx only listed IS; updated to the full set supported by FILTER_OPERANDS_MAP (IS, IS_NOT, IS_EMPTY, IS_NOT_EMPTY). |
||
|
Paul Rastoin
|
076c05cbd0 |
File service uniformize not found behavior and stream management (#20891)
# Introduction closes https://github.com/twentyhq/private-issues/issues/485 |
||
|
Weiko
|
90f711361c |
Add definePermissionFlag for app-defined permission flags (#20887)
## Context
Adds the SDK plumbing for apps to declare custom permission flags and
the server-side manifest pipeline to persist them.
```typescript
import { definePermissionFlag } from 'twenty-sdk/define';
export const MANAGE_INVOICES_PERMISSION_FLAG_UNIVERSAL_IDENTIFIER = '…';
export default definePermissionFlag({
universalIdentifier: MANAGE_INVOICES_PERMISSION_FLAG_UNIVERSAL_IDENTIFIER,
key: 'MANAGE_INVOICES',
label: 'Manage Invoices',
description: 'Create, edit, and delete invoices',
icon: 'IconReceipt',
});
```
```typescript
import { defineApplicationRole, SystemPermissionFlag } from 'twenty-sdk/define';
import { MANAGE_INVOICES_PERMISSION_FLAG_UNIVERSAL_IDENTIFIER } from './permission-flags/manage-invoices';
export default defineApplicationRole({
universalIdentifier: DEFAULT_ROLE_UNIVERSAL_IDENTIFIER,
label: `${APP_DISPLAY_NAME} default function role`,
// ...
permissionFlagUniversalIdentifiers: [
SystemPermissionFlag.UPLOAD_FILE,
MANAGE_INVOICES_PERMISSION_FLAG_UNIVERSAL_IDENTIFIER,
],
});
```
The flag can then be referenced by UUID in a role's
permissionFlagUniversalIdentifiers. On sync, the catalog row lands in
core.permissionFlag and the link in core.rolePermissionFlag.
## Not in this PR
- Runtime permission checks.
PermissionsService.getUserWorkspacePermissions still builds its result
from Object.values(PermissionFlagType), so custom flags are stored but
not yet enforced, code asking "does this role have MANAGE_INVOICES?"
won't get a meaningful answer. Widening PermissionsService and
UserWorkspacePermissions.permissionFlags to support arbitrary flag keys
is the next PR.
- PermissionFlag from apps can only define "tool" permissions and not
"settings" as a permissionType, this parameter is not mutable. This is
because "settings" are for settings page (until we might decide to
separate both type of permissions into 2 different entities) and apps
can't declare settings page or interact with them so this parameter
would be unnecessary.
|
||
|
Paul Rastoin
|
69d89f8cfc |
Early return in public assets (#20881)
# Introduction Related https://github.com/twentyhq/twenty/issues/20879 More abstracted response error and cleaner integrity check before performing any in database search Nothing critical patched here Also added integration coverage to the related endpoint Fixed the stream on error throw that would have been bubbling up into node process ## Next Once this has been approved will re-apply to all the existing prone file.getBy* methods and controllers endpoints |
||
|
Paul Rastoin
|
b8b115f4e3 |
FileStorageService Dedicated file and folder code flow + integrity check (#20831)
# Introduction
Next handling mimetype integrity check and checksum integrity check for
s3 storage type
Always expecting a trailing end slash when deleting a folder etc
## Application
Uninstalling an application now deletes all its related files
## File storage service
Making a distincton between folder path and file path
## Validation Pipeline
Every file operation in `FileStorageService.buildOnStoragePath` runs
through `validateResourcePath`, which chains three validators in order:
**1. `validateSafeRelativePath`** -- rejects path traversal attacks
| Input | Result | Error |
|---|---|---|
| `../../../etc/passwd` | Rejected | `Resource path must not contain
path traversal (..)` |
| `/etc/passwd` | Rejected | `Resource path must be relative, not
absolute` |
| `file\0.txt` | Rejected | `Resource path contains null bytes` |
| `..\\..\\etc\\passwd` | Rejected | `Resource path must not contain
backslashes` |
| _(empty)_ | Rejected | `Resource path must not be empty` |
**2. `validateFilenameIntegrity`** -- enforces safe characters, length
limits, extension required
| Input | Result | Error |
|---|---|---|
| `my folder/file.mjs` | Rejected | `A path segment contains invalid
characters...` |
| `Makefile` | Rejected | `Filename must have an extension` |
| `aaa...(256 chars).mjs` | Rejected | `A path segment exceeds the
maximum length of 255 characters` |
| `a/b/.../file.mjs` (1025+ chars) | Rejected | `Resource path exceeds
maximum length of 1024 characters` |
| `src/handlers/index.mjs` | Accepted | -- |
| `my-app/my_file.tsx` | Accepted | -- |
| `v1.0/module.config.mjs` | Accepted | -- |
Allowed characters per segment: `a-z`, `A-Z`, `0-9`, `.`, `-`, `_`
**3. `validateResourceExtension`** -- checks extension against the
`FileFolder` allowlist
| Input | FileFolder | Result | Error |
|---|---|---|---|
| `handler.js` | `BuiltLogicFunction` | Rejected | `Invalid file
extension. Allowed extensions: .mjs` |
| `card.tsx` | `BuiltFrontComponent` | Rejected | `Invalid file
extension. Allowed extensions: .mjs` |
| `script.js` | `PublicAsset` | Rejected | `Invalid file extension.
Allowed extensions: .png, .jpg, ...` |
| `index.mjs` | `BuiltLogicFunction` | Accepted | -- |
| `app.tsx` | `Source` | Accepted | -- |
| `photo.png` | `CorePicture` | Accepted | -- (unconfigured folder,
passes through) |
## Consumers
- **`FileStorageService`** -- calls `validateResourcePath`, throws
`FileStorageException` on failure (last-resort defense)
- **Resolver (`uploadApplicationFile`)** -- calls
`validateResourcePath`, throws `ApplicationException` on failure
(user-facing)
- **Flat validators** -- call `validateResourcePath`, push the error to
`validationResult.errors` (non-throwing, collects all errors)
All error messages are translated via Lingui `t` and returned in a
discriminated union `{ isValid: true } | { isValid: false, error: string
}`, letting each consumer decide how to handle failures.
|
||
|
Weiko
|
3bda05ea57 |
[Breaking change] Prepare non-system permission flags (#20847)
# Summary Replaces the enum-keyed `permissionFlags: PermissionFlag[]` on roles with `permissionFlagUniversalIdentifiers: string[]` This unlocks mixing system flags (`SystemPermissionFlag.*`) with app-defined flags in a role config. This is a breaking change. Existing app source must switch to the new field. # Breaking changes - `RoleManifest.permissionFlags` removed. Use `RoleManifest.permissionFlagUniversalIdentifiers: string[]`. - `RoleConfig.permissionFlags` removed (was `PermissionFlagType[]`). Use `RoleConfig.permissionFlagUniversalIdentifiers: string[]`. - `PermissionFlagManifest` type removed from `twenty-shared/application`. - `PermissionFlag` re-export removed from `twenty-sdk/define`. `SystemPermissionFlag` is re-exported in its place. - Retargeting a permission flag between roles is now classified as delete + create instead of update ### Not in this PR - definePermissionFlag SDK function and top-level Manifest.permissionFlags catalog (apps defining their own custom flags). Until those land, permissionFlagUniversalIdentifiers only accepts SystemPermissionFlag.* UUIDs; arbitrary UUIDs fail validation. |
||
|
Paul Rastoin
|
76e144e85a |
Deprecate messageChannel messageFolder calendarChannel standard objects (#20836)
# Introduction Removing old standard objects `messageChannel` and `messageFolder` and `calendarChannel` --------- Co-authored-by: Cursor <cursoragent@cursor.com> |
||
|
Félix Malfait
|
084fa8eaba |
fix(server): auto-index target<X>Id join columns on polymorphic standard objects (#20820)
Closes #20726 ## The bug `timelineActivity` (and the three other polymorphic standard objects — `attachment`, `noteTarget`, `taskTarget`) store relations as N nullable `target<X>Id` columns, one per related object. Each one is a join key queried as `WHERE target<X>Id IN (...) AND deletedAt IS NULL`. For **built-in** related objects (Person, Company, Opportunity, …), each `target<X>Id` column gets a BTREE index, declared statically in `compute-{timelineActivity,attachment,noteTarget,taskTarget}-standard-flat-index-metadata.util.ts`. For **custom** related objects, the same `target<CustomObject>Id` column was added — **without an index**. On a `timelineActivity` table at issue-reporter scale (~21.9M rows, 7.1 GB), this turned record loads into 20–40s sequential scans and produced `QueryFailedError: Query read timeout` for end users. ## Diagnosis The morph/relation field generator (`generateMorphOrRelationFlatFieldMetadataPair`) already creates a BTREE index for the field that owns the join column and returns it alongside the field metadata pair. The two user-driven entry points (`fromRelationCreateFieldInput…`, `fromMorphRelationCreateFieldInput…`) correctly destructure and propagate that index. But the **custom-object creation path** — `buildDefaultRelationFlatFieldMetadatasForCustomObject`, called when a user creates a new custom object — destructured only `{ flatFieldMetadatas }` and threw away `indexMetadatas`. So every `target<CustomObject>Id` column added to the four polymorphic standard objects has been shipping unindexed since custom morph relations went in. ## The fix Three commits. ### 1. `fix(server): index target<CustomObject>Id columns on standard polymorphic objects` 13 lines across 2 files. - `build-default-relation-flat-field-metadatas-for-custom-object.util.ts` — also destructure `indexMetadatas` from the pair generator and accumulate them into the returned record (new field `standardTargetFlatIndexMetadatas`). - `from-create-object-input-to-flat-object-metadata-and-flat-field-metadatas-to-create.util.ts` — append the accumulated indexes to `flatIndexMetadataToCreate`. The migration pipeline at `object-metadata.service.ts:559–562` already passes `flatIndexMetadataToCreate` to the migration runner, so no further wiring is needed. From now on, creating a custom object also creates the four BTREE indexes — one per polymorphic standard object's new `target<CustomObject>Id` column — atomically with the rest of the migration. ### 2. `feat(server): backfill workspace command for relation join column indexes` For existing workspaces whose custom objects were created before the forward-fix. `upgrade:2-8:backfill-relation-join-column-indexes` is a `@RegisteredWorkspaceCommand('2.8.0', 1798100000000)` matching the pattern from `2-7-workspace-command-…-drop-connected-account-standard-object.command.ts`. Per workspace: 1. Load `flatObjectMetadataMaps`, `flatFieldMetadataMaps`, `flatIndexMaps` from the workspace cache. 2. Resolve the four polymorphic standard object IDs by `nameSingular` against `DEFAULT_RELATIONS_OBJECTS_STANDARD_IDS`. 3. Collect every field ID that's already covered by any existing index. 4. Filter `flatFieldMetadataMaps` to MORPH_RELATION fields on those four objects whose `settings.relationType === MANY_TO_ONE` (i.e. owns a join column) and whose ID isn't in the indexed set. 5. Generate a BTREE `UniversalFlatIndexMetadata` for each via `generateIndexForFlatFieldMetadata` (same helper the forward-fix uses). 6. Create the indexes in the workspace schema with **CONCURRENTLY** (see commit 3). 7. Submit the metadata through `WorkspaceMigrationValidateBuildAndRunService` so it lands in `indexMetadata` and the cache — same pipeline as a normal metadata change. The pipeline's own `CREATE INDEX IF NOT EXISTS` no-ops because the index already exists. Properties: - **Idempotent.** Re-running is a no-op once indexes exist. - **Scoped.** Only the four polymorphic standard objects, only their MANY_TO_ONE morph relation fields, only those with no covering index. - **Same code path as the forward-fix.** The backfill produces exactly the indexes the forward-fix would have created at custom-object creation time. - **`--dry-run` supported** via the base `ActiveOrSuspendedWorkspaceCommandRunner`. ### 3. `feat(server): create index CONCURRENTLY in relation join column backfill` Adds an opt-in `concurrently` flag to `WorkspaceSchemaIndexManagerService.createIndex` (threaded through `createIndexInWorkspaceSchema`). When `true`, emits `CREATE INDEX CONCURRENTLY IF NOT EXISTS …`. Defaults to `false` — every existing caller keeps the current transactional `CREATE INDEX` behavior. The backfill command opts in. It creates a QueryRunner **without** `startTransaction()`, issues the CONCURRENTLY indexes one-by-one (each waits for the previous to finish), then submits the metadata through the normal migration pipeline whose own `CREATE INDEX IF NOT EXISTS` is now a no-op. Why not flip the default for the helper: - `CREATE INDEX CONCURRENTLY` cannot run inside a transaction — Postgres errors out. The migration pipeline calls `createIndex` from inside a transactional schema migration. - CONCURRENTLY doesn't roll back with the transaction. If the surrounding migration fails, the index remains and you end up with metadata/schema drift. - Failed CONCURRENTLY builds leave an INVALID index behind that needs manual `DROP`. - UNIQUE indexes have different failure semantics under CONCURRENTLY (deferred, not immediate). So CONCURRENTLY is opt-in, used only where it's the right tool (post-hoc backfills on populated tables). ## Decisions / tradeoffs - **Single-column BTREE vs partial `WHERE deletedAt IS NULL` vs composite.** Twenty's queries always include `deletedAt IS NULL`. A partial index would be slightly better than a plain BTREE (smaller, no wasted seeks on soft-deleted rows). This PR ships single-column to match the existing built-in target index pattern, which already covers >95% of the available speedup (the 20s→4ms drop the reporter saw comes from having any index — composite/partial is a second-order effect). Switching all relation indexes to partial is a separate, broader change. - **CONCURRENTLY operator caveat.** If a CONCURRENTLY build is interrupted (kill, connection drop, OOM), Postgres leaves the index as INVALID. We deliberately don't probe `pg_index` for invalid leftovers on every create — catalog-table queries can be slow at multi-tenant scale and the failure mode is rare. Recovery is manual: `DROP INDEX <name>` and re-run the backfill. - **Forward-fix is not gated** behind a feature flag. The change is metadata-pipeline-internal; before, custom-object creation silently produced a degraded state. After, it produces the correct state. No new public API, no behavioural change for end users besides the indexes existing. ## Risk - Forward-fix: changes only the metadata produced during custom-object creation. New objects get four extra `FlatIndexMetadata` rows and four extra `CREATE INDEX` statements during their creation migration. Tables are empty at that point so the index builds in microseconds. - Helper change: API-compatible, default behavior unchanged. The new `concurrently` parameter is optional. - Backfill: read-only state probe → CONCURRENTLY index creation (no write blocking) → metadata insert via the normal migration pipeline. Idempotent. Reverting is `DROP INDEX`. ## Test plan - [ ] Verify forward-fix: create a custom object, confirm four new BTREE indexes appear on `timelineActivity`, `attachment`, `noteTarget`, `taskTarget` for the new `target<CustomObject>Id` columns, and that `flatIndexMaps` has matching entries. - [ ] Verify backfill on a workspace that had custom objects created before the fix: run `--dry-run` first, confirm the expected indexes are listed; then run for real, confirm the indexes appear in pg (and as `indisvalid = true` in `pg_index`) and in `flatIndexMaps`. Re-run; confirm no-op. - [ ] Verify backfill on a clean workspace: should log "no missing indexes" and exit. - [ ] Verify CONCURRENTLY behavior under load: run backfill against a workspace with active writes on `timelineActivity`; confirm inserts/updates keep working during index build (no `ShareLock` waits in `pg_stat_activity`). - [ ] On the affected reporter-scale workspace, confirm `EXPLAIN ANALYZE` switches from sequential scan to index scan and timeline activity timeouts go away. |
||
|
nitin
|
068d365731 |
feat(sdk): error on incompatible view filter operand at sync time (#20763)
view filters with mismatched operand + field type now error at sync -- was silently failing before |
||
|
neo773
|
323e66433e |
lint: migrate prettier to oxfmt (#20783)
Most changes are `implements` being unwrapped this is not a oxfmt regression Prettier in 3.7 (we're on 3.1) changed this behaviour prettier blog [post](https://prettier.io/blog/2025/11/27/3.7.0#change-18094) This unifies our linting tooling --------- Co-authored-by: github-actions <github-actions@twenty.com> Co-authored-by: Charles Bochet <charles@twenty.com> |
||
|
martmull
|
9a7f50fcb7 |
Fix breaking change in install app command (#20825)
add backward compatibility for twenty-sdk install command |
||
|
Charles Bochet
|
edf2bfbf76 |
feat(server): rotate connectedAccount.connectionParameters via secret-encryption:rotate (#20807)
## Summary Adds a new `connected-account-connection-parameters` site to the `secret-encryption:rotate` CLI introduced in #20613, so the nested password envelopes inside `connectedAccount.connectionParameters` (IMAP / SMTP / CALDAV — encrypted at-rest in #20673) are re-encrypted under the current `ENCRYPTION_KEY` alongside every other at-rest secret site. Without this, rotating `ENCRYPTION_KEY` on a 2.7+ instance would silently leave IMAP / SMTP / CalDav passwords on the old key id. ### Why a new handler A dedicated handler is required (rather than reusing `ColumnRotationSiteHandler`) because the envelope lives at `connectionParameters->'<PROTOCOL>'->>'password'`, not in the whole column, and up to three independent envelopes may need rotating per row. The handler: - Uses the same cursor-based, idempotent, online pattern as the existing handlers, with a SQL predicate that skips rows where every non-null protocol password is already on the current key id. - Threads \`workspaceId\` into HKDF, matching how \`EncryptConnectionParametersSlowInstanceCommand\` backfilled. - Rebuilds only the protocols whose passwords are not yet current, so a partial mid-row failure cannot cause unnecessary re-encryption on resume. - Guards the UPDATE with jsonb-level deep equality (\`IS NOT DISTINCT FROM CAST(:json AS jsonb)\`) so optimistic concurrency is unaffected by Postgres's internal jsonb key ordering vs. JSON.stringify ordering. - Refuses to rotate plaintext passwords (counted as \`errors\`) — operators must finish the 2.7 slow instance command (\`EncryptConnectionParametersSlowInstanceCommand\`) before running rotation. ### Sites covered (now) | Site | Location | Scope | | --- | --- | --- | | \`connected-account-access-token\` | \`connectedAccount.accessToken\` | workspace | | \`connected-account-refresh-token\` | \`connectedAccount.refreshToken\` | workspace | | **\`connected-account-connection-parameters\`** (new) | \`connectedAccount.connectionParameters.{IMAP,SMTP,CALDAV}.password\` | workspace | | \`application-variable\` | \`applicationVariable.value\` (isSecret) | workspace | | \`application-registration-variable\` | \`applicationRegistrationVariable.encryptedValue\` | instance | | \`signing-key-private-key\` | \`signingKey.privateKey\` | instance | | \`totp-secret\` | \`twoFactorAuthenticationMethod.secret\` | workspace | | \`sensitive-config-storage\` | \`keyValuePair.value\` (sensitive STRING configs) | instance | |
||
|
Paul Rastoin
|
570c57563a |
Upload application file resolver exception management and integration coverage (#20803)
# Introduction Earlier and better exception handling of the upload application file resolver + coverage |
||
|
martmull
|
2963fa9324 |
Navigate to installed page after app install (#20797)
as title |
||
|
Paul Rastoin
|
a3c92311e3 |
Application file storage service (#20793)
# Introduction Fix unsafe resource path join with expected prefix at file storage directly Add early paths transversal detections in metadata validators |
||
|
Félix Malfait
|
923c3beead |
fix(auth): clarify error when joining a non-active workspace (#20769)
## Summary When an existing user accepts an invite into a workspace whose `activationStatus` is not `ACTIVE` (e.g. `SUSPENDED`, `INACTIVE`, `PENDING_CREATION`), the throw in `throwIfWorkspaceIsNotReadyForSignInUp` returns: > User is not part of the workspace The message describes the symptom (they aren't a member yet) instead of the cause (the workspace can't accept new members), which makes invitees assume their invite is broken when the real issue is the target workspace's state. The sibling branch a few lines above — for brand-new users hitting the same non-ACTIVE workspace — already returns `"Workspace is not ready to welcome new members"`. This PR reuses the same message in the existing-user branch so both paths give a consistent, accurate explanation. Single file, two string literals. ## Test plan - [ ] Sign in via Google with an existing Twenty account, accepting an invite to a `SUSPENDED` workspace → confirm the new message is shown instead of "User is not part of the workspace". - [ ] Confirm the happy path (sign-in to an `ACTIVE` workspace via invite) is unchanged — early-return on `ACTIVE` is untouched. --------- Co-authored-by: Charles Bochet <charles@twenty.com> |