2aa118cb0d0f95c63c7bb0b1f8e4f904ff6303e1
1205 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Raphaël Bosi
|
2aa118cb0d |
Add a roles standard skill for the AI chat agent (#23636)
Role management tools shipped in #23613, but no skill was added, so the agent could call them with no guidance. This adds a `roles` standard skill covering the traps that are easy to get wrong: `upsert_object_permissions` replaces the role's entire override list (so omitted objects silently revert to global permissions), granting write without read is rejected, system-managed roles like Admin cannot be modified, and the lockout guard rejects mutations that would strip the acting admin's own access. It also tells the agent to call `list_roles` first, since every workspace already ships with Admin and Member, and reuses the confirmation-gate pattern from `dashboard-building` before any create/update/delete. New workspaces get the skill from the standard application. There is no generic sync for existing ones, so this also adds a `2-27` backfill command that diffs computed standard skills against existing ones and creates the missing ones. <!-- This is an auto-generated description by cubic. --> <a href="https://cubic.dev/pr/twentyhq/twenty/pull/23636?utm_source=github" target="_blank" rel="noopener noreferrer" data-no-image-dialog="true"><picture><source media="(prefers-color-scheme: dark)" srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source media="(prefers-color-scheme: light)" srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img alt="Review in cubic" src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a> <!-- End of auto-generated description by cubic. --> |
||
|
Weiko
|
ad271ee639 |
Add connected account handle/provider index (#23580)
## Context
Google messaging webhook notifications resolve connected accounts with
an equality lookup on both `handle` and `provider`:
```ts
connectedAccountRepository.find({
where: {
handle: decodedData.emailAddress,
provider: ConnectedAccountProvider.GOOGLE,
},
});
```
This lookup runs for incoming Gmail notifications, but
`connectedAccount` currently has no index matching either predicate. As
the table grows, PostgreSQL has to inspect unrelated connected-account
rows for each notification. Under sustained webhook traffic, that adds
avoidable database work and keeps database connections occupied longer.
## What changed
- Add a composite B-tree index on `connectedAccount(handle, provider)`.
- Register the index in the TypeORM entity metadata.
- Add an idempotent 2.26 fast instance command to create the index for
existing installations and remove it on rollback.
The webhook handler and query behavior remain unchanged.
## Why this index
- Both query predicates are equality conditions, so the composite index
supports a targeted lookup.
- `handle` is first because it is the more selective value and also
makes the index useful for handle-prefixed lookups.
- The index is intentionally non-unique. The same provider handle may
legitimately belong to connected accounts in different workspaces, and
this change must not introduce a new data constraint.
- Connected accounts are read by webhooks much more frequently than
their handle or provider changes, so index maintenance overhead should
remain small.
## Expected impact
Webhook account resolution should use an index lookup instead of
scanning the connected-account table. This reduces cumulative PostgreSQL
work and connection occupancy on the Gmail notification path.
This is a targeted database optimization. It should reduce pressure
generated by this high-frequency query, but it is not expected to
resolve every source of API tail latency by itself.
## Safety and rollout
- The instance command uses `CREATE INDEX IF NOT EXISTS` and `DROP INDEX
IF EXISTS`.
- No uniqueness or application behavior changes are introduced.
- Existing rows require no data backfill.
- The index adds bounded storage and write-maintenance overhead.
## Validation
- `yarn nx typecheck twenty-server`
- Type-aware Oxlint on the changed files
- Oxfmt on the changed files
- `git diff --check`
<!-- This is an auto-generated description by cubic. -->
<a
href="https://cubic.dev/pr/twentyhq/twenty/pull/23580?utm_source=github"
target="_blank" rel="noopener noreferrer"
data-no-image-dialog="true"><picture><source
media="(prefers-color-scheme: dark)"
srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source
media="(prefers-color-scheme: light)"
srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img
alt="Review in cubic"
src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a>
<!-- End of auto-generated description by cubic. -->
|
||
|
Weiko
|
e4e1d24731 |
Prevent overlapping workspace cleanup executions (#23522)
## Context The suspended-workspace cleanup is a long-running scheduled job. Under database or cache pressure, BullMQ can consider an execution stalled and start a replacement on another worker while the original execution is still running. Both executions can then enumerate the same suspended workspaces and run destructive cleanup concurrently. This amplifies the initial slowdown: 1. Multiple cleanup transactions target the same workspace data. 2. Transactions wait on each other's locks. 3. Database connections remain occupied while waiting. 4. Other workers and API requests have fewer connections available. There is a second source of unnecessary lock duration in workspace deletion. The deletion transaction currently starts before field metadata is read from the workspace cache. If that lookup is slow, the transaction stays open during an unrelated cache wait. ## What changed ### Prevent overlapping scheduled cleanups - Acquire a non-blocking PostgreSQL advisory lock before listing suspended workspaces. - Skip the execution when another worker already holds the lock. - Keep the lock on one dedicated PostgreSQL session for the full callback. - Release the lock in all normal and error paths. - Discard the database connection if lock acquisition or release has an ambiguous failure, preventing a session that may still own the lock from returning to the pool. - Encapsulate this lifecycle in `PostgresAdvisoryLockService`, exported by `TypeORMModule`, so other coarse-grained jobs can reuse it without handling acquisition and release themselves. ### Shorten the workspace deletion transaction - Read field metadata and build deletion chunks before starting the transaction. - Pass the precomputed chunks into the transactional deletion loop. - Keep the existing deletion order and SQL behavior unchanged. ## Why a PostgreSQL advisory lock The lock needs to coordinate workers running in different pods. A PostgreSQL session advisory lock provides the required behavior: - It is shared across all workers using the same database. - Acquisition is non-blocking, a duplicate execution can exit immediately. - It has no TTL or renewal heartbeat that could expire during the same event-loop stall that caused BullMQ to recover the job. - PostgreSQL automatically releases it when the owning session or process disappears. This is deliberately scoped to `CleanSuspendedWorkspacesJob`. It prevents overlapping scheduled executions, but it is not an exactly-once mechanism or a global mutex around every workspace-deletion entry point. ## Expected impact - Prevent one slow cleanup execution from becoming several concurrent cleanup executions. - Reduce database lock contention and connection-pool pressure during cleanup. - Avoid holding deletion transaction locks while waiting for workspace-cache data. - Reduce cleanup-related API latency bursts without changing normal cleanup semantics. The advisory lock holds one core database connection for the duration of the scheduled cleanup. This is intentional and bounded to the single lock owner. ## Validation - Focused advisory-lock tests cover successful execution, contention, callback failure, and unsafe connection disposal when unlock fails. - Cleanup-job tests cover both the lock-owner and skipped-execution paths. - Workspace-service coverage verifies that field metadata is loaded before the deletion transaction starts. - `yarn nx typecheck twenty-server` - Oxlint, Prettier, and Oxfmt checks on the changed files |
||
|
Paul Rastoin
|
133b3375b6 |
[Upgrade] Stop command gracefully on SIGINT/SIGTERM (#23481)
Ctrl+C on `upgrade` used to kill the process wherever it happened to be, potentially in the middle of a workspace command. It now stops at the next iteration boundary instead. ## Behavior - **First SIGINT/SIGTERM** — the runner finishes what it started, then stops instead of starting new work. Exits with `130` (SIGINT) or `143` (SIGTERM), following the 128+signal convention, so orchestrators can tell an interruption apart from a failure. - **Second signal** — immediate exit, leaving the command in progress unfinished. - **SIGKILL** — untrappable, same outcome as a second signal. Nothing is rolled back on stop: the run resumes from the last command recorded in `upgradeMigration`. ## Opt-in per command Registering a `SIGINT` listener removes Node's default kill-on-signal behavior, so a command that installs a handler without honoring the flag would ignore the first Ctrl+C entirely. Handlers are therefore opt-in via `CommandShutdownService.listenToShutdownSignals()`, called by the two commands that stop at a boundary: - `UpgradeCommand` - `WorkspaceCommandRunner`, the base for standalone workspace commands Everything else keeps today's behavior and dies on the first signal, `run-instance-commands` included: instance commands are transactional and cursor-guarded, so a hard kill rolls back and a rerun skips what completed. `install-application`, `rebuild-application-default-deps` and `install-pre-installed-apps` iterate over workspaces without going through `WorkspaceCommandRunner`, so they are not armed either; they are one call away if we want them. The server and worker processes share these services and never arm anything, so their shutdown semantics are unchanged. ## Where the flag is checked `CommandShutdownService` exposes a single boolean, `isShutdownRequested()`, read only by the iteration runners: - `UpgradeSequenceRunnerService.runInner` — before each sequence step - `WorkspaceIteratorService.iterate` — before each workspace There is deliberately no `AbortSignal`: in-flight work is never cancelled, it is allowed to finish. Individual commands know nothing about shutdown, so a workspace that has started runs its whole pending segment before the run stops. Each workspace ends up either fully done with the segment or untouched, never scattered at some cursor inside it. That keeps resume state coarse and the change out of the command layer, at the cost of a longer stop latency, which the second Ctrl+C covers. `WorkspaceIteratorReport` gained an `interrupted` flag. The sequence runner needs it: stopping partway through the workspace list and then advancing the cursor would run an instance step against workspaces that are not aligned yet, so it returns instead. ## Deployment note Under Kubernetes, `terminationGracePeriodSeconds` must exceed the time for one workspace to finish its segment, otherwise the SIGTERM path degrades into a SIGKILL. Documented in `docs/UPGRADE_COMMANDS.md`. ## Testing - New unit test for `CommandShutdownService` (7 cases); 293 tests pass across `database/commands` and `core-modules/upgrade` - `tsgo -p tsconfig.json` clean - oxlint and oxfmt clean on all touched files |
||
|
nitin
|
742f76e318 |
[BREAKING-CHANGE] Add NOT_RECORDED call recording status (#23478)
Adds NOT_RECORDED to the CallRecording status select, for meetings where nothing was captured (bot never admitted, meeting not started, nobody joined). First part of twentyhq/core-team-issues#2706, split out so existing workspaces are upgraded before the call-recorder app starts writing the new status. - NOT_RECORDED enum value + standard select option - 2-26 workspace upgrade command adding the option to existing workspaces (idempotent, same option id as the standard definition) App-side classification from Recall sub codes comes in a follow-up PR. <!-- This is an auto-generated description by cubic. --> <a href="https://cubic.dev/pr/twentyhq/twenty/pull/23478?utm_source=github" target="_blank" rel="noopener noreferrer" data-no-image-dialog="true"><picture><source media="(prefers-color-scheme: dark)" srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source media="(prefers-color-scheme: light)" srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img alt="Review in cubic" src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a> <!-- End of auto-generated description by cubic. --> |
||
|
Paul Rastoin
|
4ec65ed08d |
System view tooling explicit params key naming (#23506)
# Introduction View field system always result from a field existence, the application universal identifier should be the related field one Same but for views and object <!-- This is an auto-generated description by cubic. --> <a href="https://cubic.dev/pr/twentyhq/twenty/pull/23506?utm_source=github" target="_blank" rel="noopener noreferrer" data-no-image-dialog="true"><picture><source media="(prefers-color-scheme: dark)" srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source media="(prefers-color-scheme: light)" srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img alt="Review in cubic" src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a> <!-- End of auto-generated description by cubic. --> |
||
|
Félix Malfait
|
33970dd45d |
Align campaign view column positions with the standard layout (#23496)
Follow-up to #23493 (merged). Now based on `main`. ## Problem `AddMessageCampaignNameFieldCommand` places the new `name` column below the lowest existing position when it is the label identifier (`min - 1`). That satisfies `isViewFieldInLowestPosition` once, but does not hold: `viewField.position` is compared by the standard-application sync (`position: { toCompare: true }`), so the stored position is pulled back to the standard one and re-fails `validateLabelIdentifierFieldMetadataIdFlatViewField`, throwing `WorkspaceMigrationBuilderException` on every sync. That sync runs during `WorkspaceManagerService.init`, which is what `ActivateWorkspace` calls. cubic flagged this on the original PR: [discussion_r3653452416](https://github.com/twentyhq/twenty/pull/23188#discussion_r3653452416). ## Confirmed against prod Queried the workspace failing in Sentry ([TWENTY-SERVER-JJ2](https://twenty-v7.sentry.io/issues/7639768673/)): the `allMessageCampaigns` view has no `name` column at all, `subject` at position `0` (still the label identifier), and every column at the old layout. The activation sync tries to create `name` at standard position `0` while repointing the label identifier in the same batch, ties with `subject` at `0`, and throws. Exactly the mechanism this PR fixes. ## Change A `2-25` workspace command (`upgrade:2-25:align-message-campaign-view-field-positions`, timestamp `1785332560000`) that aligns the `allMessageCampaigns` columns to the standard layout, so the sync has nothing left to change: - Standard columns take their standard positions (`subject 0→1`, `status 1→2`, …), freeing slot `0` for `name`. - Columns the standard application does not know about keep their relative order and move above the standard ones. - Updates run in two migration passes: everything else first, then the lowest-target column alone. The migration builder validates updates one at a time against optimistic maps it mutates as it goes, so a single batch is order-dependent (caught by Greptile below); two passes keep every intermediate state valid. Placed in `2-25/` rather than `2-26/` so it can ship in a 2.25.x patch — the failures are on `v2.25.0` instances. This is why `server-previous-version-upgrade-mutation-guard` is red: it needs the `ci:allow-previous-version-upgrade-mutation` label (same as #23493). The position arithmetic and the pass-splitting are pure utils with 12 tests. ## Traced against the confirmed prod state Pass 1 moves `status`…`createdAt` up while `subject` (label identifier) stays lowest at `0`; pass 2 moves `subject` to `1`, still strictly lowest. The subsequent activation sync then creates `name` at `0` below `subject` at `1` and repoints the label identifier. Converges. Run order matters only relative to the sibling command from #23493: removal (`…550000`) sorts before alignment (`…560000`), which is correct. |
||
|
Félix Malfait
|
e79424ddb0 |
Stop building the Campaigns navigation menu item (#23493)
Campaigns show up in the sidebar of every workspace, including brand new ones, while the feature is still behind `IS_EMAIL_GROUP_ENABLED`. ## Why it leaked `navigationMenuItem` has no `conditionalAvailabilityExpression` column. Only two entities do: - `page-layout-widget.entity.ts` - `command-menu-item.entity.ts` So there was no flag to attach. #23188 gated every surface that supports gating — the campaign command menu items carry `featureFlags.IS_EMAIL_GROUP_ENABLED` (`standard-command-menu-item.constant.ts:720` and `:735`) and the page layout widgets are gated the same way — but `allMessageCampaigns` was added to `FLAT_NAVIGATION_MENU_ITEM_NAMES`, which builds unconditionally. `messageCampaign` is `isSystem: true`, so the navigation item was the only thing exposing the feature. ## Change - Drop `allMessageCampaigns` from `FLAT_NAVIGATION_MENU_ITEM_NAMES`. Its definition stays in `STANDARD_NAVIGATION_MENU_ITEMS` so the identifier remains reserved and re-enabling is a one-line change. - Add workspace command `1785324390000` to delete the rows from workspaces already provisioned with the item. It collects every matching row rather than looking the identifier up once, because each user workspace gets its own. ## Trade-off Campaigns become unreachable from the sidebar even with the flag on, which restores the pre-#23188 state. The durable fix is adding `conditionalAvailabilityExpression` to `navigationMenuItem` so the item can be gated like the command menu items — a schema change, deliberately not done here. ## Verification Reproduced on a fresh `database:reset` against `main`: two `20202020-b00b-4b0b-8b0b-c0aba11c000b` rows (type `OBJECT`, position 7) for the single seeded workspace, structurally identical to the other nav items. Typecheck, `oxfmt --check src/` and `oxlint --type-aware src/` clean on this branch. The post-fix database check did not complete — local Postgres died mid-reset — so the removal is verified by construction and by the build-list change, not yet by a second reset. --- _Generated by [Claude Code](https://claude.ai/code/session_01NZKeMSUBG3VQPuoy8i5Awe)_ <!-- This is an auto-generated description by cubic. --> <a href="https://cubic.dev/pr/twentyhq/twenty/pull/23493?utm_source=github" target="_blank" rel="noopener noreferrer" data-no-image-dialog="true"><picture><source media="(prefers-color-scheme: dark)" srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source media="(prefers-color-scheme: light)" srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img alt="Review in cubic" src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a> <!-- End of auto-generated description by cubic. --> |
||
|
Paul Rastoin
|
0c545bcdeb |
[BREAKING-CHANGE] Centralize system View viewField side effect (#23081)
# Introduction Closes https://github.com/twentyhq/core-team-issues/issues/2669 Part of the `isSystemSideEffect` engine-ownership effort. Until now, a custom object's default **INDEX** table view (`All {objectLabelPlural}`) and its view fields were built imperatively in `ObjectMetadataService` with random `v4()` identifiers, while `twenty-standard` authored its own copies with hardcoded literals. The two never converged, an object rename could drift the view, and nothing marked these rows as engine-owned. This PR makes the metadata side-effect engine the **single owner** of the INDEX view and its view fields, on name-free deterministic identifiers, for custom and standard objects alike. ## Core design - **Name-free deterministic identity.** The INDEX view identifier derives from `object identifier + ViewKey.INDEX` (`getSystemViewUniversalIdentifier`); each view-field identifier derives from `view identifier + field identifier` (`getViewFieldUniversalIdentifier`). An object rename (with a pinned object identifier) keeps the same view, losslessly. - **`isSystemSideEffect: true` is provenance.** Every INDEX view / view field the engine emits is flagged system-owned, so manifest deletion inference never drops it. The flag follows the view: a view field inherits its parent view's flag. - **The engine is the sole owner of the INDEX view.** It always emits it; a caller providing one with the same derived identifier is a genuine conflict surfaced by the engine's reserved-identifier collision, not silently deferred. ## Changes ### Shared (`twenty-shared`) - `getIndexViewUniversalIdentifier` → `getSystemViewUniversalIdentifier`, now taking a `viewKey` (generalizes to any singleton engine-owned view). - Standard field identifiers extracted into a new `STANDARD_OBJECT_FIELDS` constant, so both an object's `fields` and its INDEX view read the same field identifiers. - `buildStandardObjectIndexView` derives the standard INDEX view + view-field identifiers from `STANDARD_OBJECT_FIELDS`, replacing the hardcoded literals in `standard-object.constant.ts`. ### Metadata side-effect engine (custom objects) - **`objectSystemFieldsAndIndexViewOnCreate`** (replaces `objectSystemFieldsOnCreate`): on object creation, provisions the 7 reserved system fields **and** the INDEX view with one view field per displayable system field, all `isSystemSideEffect: true`. - **`fieldIndexViewFieldOnCreate`** (new): on field creation, provisions the field's INDEX view field. Object created in the same batch → visible, positioned before the system view fields; pre-existing object → hidden, appended (preserving the historical `createOneField` behavior). Both branches resolve the INDEX view by its derived identifier (single map access, never a scan). - **`fieldSystemViewFieldsOnDelete`** (new): on field deletion, cascade-deletes every engine-owned view field displaying it. - **`objectSystemSideEffectsOnDelete`** (extended): now also cascade-deletes the object's engine-owned views and their view fields (in addition to system fields, indexes, searchFieldMetadata). Every lookup walks a foreign-key aggregator down from the deleted object, so the work is proportional to what the object owns, never to workspace size. - Object-create and field-create positions are derived from the same caller-input field list, so the INDEX view layout is contiguous with no handler-ordering dependency. - `view` / `viewField` added to the side-effect companion metadata names for `fieldMetadata` and `objectMetadata`. ### Reserved-identifier invariant A caller can never define an entity whose identifier collides with one a system side effect produces: caller inputs are forced `isSystemSideEffect: false` at every entry point (API and app-manifest transpilers), and the engine raises `RESERVED_SYSTEM_UNIVERSAL_IDENTIFIER`, aborting the operation, when a system emission lands on a caller-claimed identifier. Covered by a new engine-level test. ### Caller-side provisioning removed The imperative INDEX view + view-field provisioning is removed from `ObjectMetadataService.createOneObject`. The record-page `FIELDS_WIDGET` view is intentionally left caller-side and deferred to the follow-up (see below). ### `twenty-standard` convergence Standard INDEX views and their view fields converge on the same derived-identifier + `isSystemSideEffect: true` scheme as the engine. `twenty-standard` syncs through the from/to migration path (which never runs the side-effect engine), so it authors this INDEX surface itself, matching what the engine produces for custom objects. ## Rollout Two `2.26.0` workspace commands, running after the `2.25` messageCampaign commands: - `upgrade:2-26:reconcile-index-view-universal-identifier` re-owns the INDEX views of the **twenty-standard and workspace-custom applications** and all their view fields to the derived identifiers with `isSystemSideEffect: true`, in a single per-workspace transaction. Each view field identifier is keyed on the application of the **displayed field** (an app or user column on a standard INDEX view converges too). Soft-deleted views and view fields are skipped: one can coexist with an active successor on the same derivation inputs and both would derive the same identifier. Children reference the view by primary key, so the re-own is lossless. - `upgrade:2-26:demote-and-backfill-application-index-view` handles **manifest-installed applications**, which never had their INDEX view auto-provisioned: every caller-authored INDEX view of another application is demoted to `key: null` (a plain additional view under its manifest identifier), then every application object gets the engine-owned INDEX view and its full view-field layout backfilled through the migration pipeline's legacy path (no side-effect expansion), views committed before view fields across applications since a view field belongs to the application owning its field. Idempotent and retry-safe: engine-owned INDEX views are neither demoted nor re-backfilled, and view creation and view-field creation are gated independently, so a retry after a partial failure still backfills the missing view fields of an already-committed view. Both support `--dry-run` and invalidate the full flat-maps closure (parents aggregate the re-owned identifiers, children resolve them as universal foreign keys, and page-layout widget universal configurations resolve view PKs at cache-build time). The `2.25` `upgrade:2-25:add-message-campaign-name-field` command is adapted to resolve the campaign INDEX view by its INDEX key on the object instead of by universal identifier: it now runs before the reconcile, on workspaces still holding legacy identifiers. ## ⚠️ Breaking change This PR **mutates 187 previously hardcoded universal identifiers** — the standard objects' INDEX views and their view fields (the literals removed from `standard-object.constant.ts`), now derived. - **Handled by the `2.26` commands above** for all existing workspaces. - **The INDEX key is now engine-reserved.** The flat view validator rejects caller-created INDEX views (API and manifest inputs are forced `isSystemSideEffect: false`) and enforces a single non-deleted INDEX view per object; `view.key` is no longer a comparable/updatable property, so no writer can promote or demote a view after creation. `ViewManifest.key` is deprecated and ignored (manifest views are always additional views, so old apps keep syncing and demoted views are not promoted back); the REST/GraphQL create path now rejects `key: INDEX`. In-repo example apps (`hello-world`, `document-generator`) no longer declare it. - **12 declared-but-never-seeded standard INDEX view field identifiers deleted** (the former `preservedViewFields` on `timelineActivity`, `workflowRun` and `workspaceMember`): after the reconcile, no workspace row references them. - **`computeFlatViewFieldsToCreate` now derives view field identifiers** instead of drawing `v4()` ones, which also changes what the committed `1-23` record-page backfill produces going forward (deliberate, documented in-code). - **Record-page views and view fields are not affected** (identifiers unchanged). - **In-repo apps: `twenty-last-contact` updated.** It was the only app declaring explicit INDEX view fields (10 columns across `allPeople` / `allCompanies` / `allOpportunities`) through manifest `viewFields`. Those target identifiers are now engine-owned and derived, so the manifest inputs no longer resolve and install failed with `View not found`. The app now declares only its fields; the engine's `fieldIndexViewFieldOnCreate` provisions the matching INDEX view field automatically. No other app under `packages/twenty-apps` references any of the 187 mutated identifiers, and apps that target standard views point at record-page views (e.g. `real-estate` → `opportunityRecordPageFields`) or their own objects (`twenty-partners`), all unchanged. ### Loss of granularity for app maintainers The engine now owns the INDEX view field of every field a caller adds to an object, so app maintainers lose direct control over those columns. Previously an app could target the engine-owned INDEX view with an explicit manifest `viewField` and set its `position` and `isVisible`. Now `fieldIndexViewFieldOnCreate` appends a **hidden** view field in caller-input order on field creation, so: - Columns an app previously showed at a **dedicated position** and **visible** (e.g. `twenty-last-contact`'s last-contact columns) become **hidden** and **appended in input order** after install. - There is currently **no manifest way to override** the engine-provisioned INDEX view field's position, visibility, or size. This is a deliberate regression accepted for the sake of single-ownership, and app maintainers should expect their INDEX columns to move/hide after upgrading. A follow-up override API will let maintainers reclaim per-field control over the engine-provisioned INDEX view field. ## Testing - Unit specs for each handler: object create (system fields + INDEX view/view fields, override, position offset), field create (same-batch vs existing-object, non-displayable noop, no-INDEX-view noop), field delete, object delete (fields/indexes/searchFieldMetadata/views/view fields cascade, reverse-relation view field on another object). - Engine-level test for the reserved-identifier collision. - `twenty-standard` guard test that its INDEX views/view fields stay on the derived scheme and stay system-owned. - Integration test: full engine provisioning of the INDEX view/view fields on object creation, same view id preserved across an object rename, and cascade delete on object deletion. ## Follow-up The full record-page stack (record-page view, its view fields, view field groups, page layout / tab / widget) is still built imperatively and moves into the engine in https://github.com/twentyhq/core-team-issues/issues/2721. |
||
|
martmull
|
f5da810e59 |
feat(server): add application:install command (#23430)
Adds `application:install` to install an application on workspaces that do not have it yet, and moves both application commands onto `WorkspaceIteratorService`. ### Behavior - Iterates provisioned workspaces through `WorkspaceIteratorService` (workspace id resolution, workspace context, per-workspace success/fail report), or the ones passed with `-w`. - Workspaces where the application is already installed are skipped, with a log line pointing at `application:upgrade`. This command never upgrades. - Installed workspaces are detected by `universalIdentifier`, the same identity `ApplicationInstallService` uses to tell a fresh install from a version upgrade, checked per workspace on the `(universalIdentifier, workspaceId)` unique index. - `--workspace-count-limit` caps both the iterator's own selection and an explicitly targeted `-w` list. - Fails fast for `LOCAL` and `OAUTH_ONLY` registrations, which have no code artifacts to install. - Per-workspace failures are collected in the iterator report and never abort the run; the command ends with an installed / skipped / failed summary. ### Options | Flag | Description | | --- | --- | | `-u, --application-registration-universal-identifier` | Application registration universal identifier (required) | | `-w, --workspace-id` | Target a specific workspace, repeatable | | `--workspace-count-limit` | Cap the number of workspaces to iterate over (max 50) | | `-d, --dry-run` | Print the workspaces that would be installed without installing | | `-y, --yes` | Skip the confirmation prompt | ### Example ``` yarn command:prod application:install -u UNIVERSAL_IDENTIFIER --dry-run ``` ### Changes to application:upgrade - `ApplicationUpgradeService.upgradeApplications` iterates through `WorkspaceIteratorService` and returns its report, replacing the hand-rolled parallel batching. - `--batch-size` dropped from the command, and `batchSize` dropped from the service and from `UpgradeApplicationsJobData`, since `iterate()` is sequential. - `parseBoundedPositiveInteger` moved to `src/database/commands/utils/` and is shared by both commands. ### Files - `application-install/commands/install-application.command.ts` (new) - `src/database/commands/utils/parse-bounded-positive-integer.util.ts` (new) - `application-upgrade/application-upgrade.service.ts`, `application-upgrade/commands/upgrade-application.command.ts`, `jobs/upgrade-applications.job*`: iterator instead of batching - `application-install.module.ts` / `application-upgrade.module.ts`: register the command, wire `WorkspaceIteratorModule` - `database-command.module.ts`: import `ApplicationInstallModule` so the command is discovered by the CLI ### Testing - `npx jest src/engine/core-modules/application` (32 suites, 181 tests passing) - `npx nx typecheck twenty-server` - `oxlint --type-aware` and `oxfmt` on the changed files |
||
|
Paul Rastoin
|
1fb1232a17 |
Message campaign backfill method sort view field (#23433)
<!-- This is an auto-generated description by cubic. --> <a href="https://cubic.dev/pr/twentyhq/twenty/pull/23433?utm_source=github" target="_blank" rel="noopener noreferrer" data-no-image-dialog="true"><picture><source media="(prefers-color-scheme: dark)" srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source media="(prefers-color-scheme: light)" srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img alt="Review in cubic" src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a> <!-- End of auto-generated description by cubic. --> |
||
|
Raphaël Bosi
|
f15fabb5d9 |
Enrich workspace company via People Data Labs during onboarding (#23199)
https://github.com/user-attachments/assets/fb9001c4-195d-4735-898b-07ccbab01677 During onboarding, the workspace creator's work-email domain is enriched through People Data Labs and stored client-side. The stacked workspace-setup PR folds it into the invisible prompt that kicks off the setup chat, so the assistant knows the company from its first reply. - New `enrichWorkspaceCompany` mutation: throttled, creator-only, work domains only. Off by default: requires the `IS_WORKSPACE_COMPANY_ENRICHMENT_ENABLED` instance config variable (default false), a `PEOPLE_DATA_LABS_API_KEY`, and the `IS_ONBOARDING_AI_CHAT_ENABLED` workspace feature flag (the enrichment only feeds the AI-chat workspace setup). Every attempt past the throttle is recorded per workspace in a `keyValuePair`. - The frontend fetches once during onboarding and stores a matched result in localStorage. This PR does not deliver it to the model: the hidden-message plumbing it adds (`isHidden` on `agentMessage`, excluded from the chat UI, thread ranking and the admin transcript, included in the model conversation) is what the stacked workspace-setup PR uses to send the context and the setup prompt as one invisible first message. - The PDL wire protocol (base URL, wire types, envelope parsing, error extraction) is kept as a small self-contained copy inside the server `company-enrichment` module. The standalone people-data-labs app keeps its own copy; the two are intentionally not shared, since the app and the core-engine usage are expected to evolve independently. - `WorkspaceCompanyEnrichment` lives in `twenty-shared/workspace` so server and front share one shape. ## Flow ```mermaid flowchart LR effect[Onboarding effect] -- enrichWorkspaceCompany --> checks{creator + work domain?} checks -- no --> unavailable[unavailable] checks -- yes --> throttle{throttle 10/h/workspace} throttle -- limited --> transient[transientError] throttle -- ok --> pdl[PDL GET /company/enrich] pdl --> log[(keyValuePair attempt log)] pdl --> matched[matched] matched --> storage[(localStorage)] storage -- consumed by the stacked workspace-setup PR --> kickoff[hidden kickoff prompt] ``` 1. **Onboarding effect** — mounted app-wide, fires once per session while onboarding is in progress (before workspace activation), guarded by a sessionStorage attempt flag and the cached value. 2. **enrichWorkspaceCompany** — metadata-schema mutation returning a typed `WorkspaceCompanyEnrichmentResult` (`outcome` enum `matched`/`unavailable`/`transientError` + `enrichment` JSON). 3. **Creator + work domain checks** — only the workspace's earliest user, only non-consumer email domains, only when the config flag, API key and `IS_ONBOARDING_AI_CHAT_ENABLED` workspace flag are all on; anything else returns `unavailable` without consuming throttle quota. 4. **Throttle** — token bucket, 10 requests/hour per workspace, the sole cost bound on PDL calls; when limited the mutation returns `transientError` instead of surfacing an error. 5. **PDL call** — `GET /v5/company/enrich` with `website` + `min_likelihood` per the PDL spec; body-level statuses win over HTTP ones, 408/429/5xx map to `transientError`, other failures to `unavailable`. Every attempt past the throttle is recorded (`domain`, the pre-collapse PDL `outcome`, `httpStatus`/`message` when present, `attemptedAt`) in a workspace-scoped `keyValuePair`. 6. **matched** — the PDL payload is mapped to `WorkspaceCompanyEnrichment` through the same sanitizer as client input (all fields length-capped and control-character-stripped; summary 600 chars, 8 tags max) and returned. 7. **localStorage** — the frontend stores only a matched enrichment and never refetches it, making it the only cache; cleared on sign-out. Non-matched outcomes are not persisted; a sessionStorage flag caps retries at one attempt per browser session. 8. **Delivery** — out of scope here. The stacked workspace-setup PR reads the stored enrichment and combines it with the data-model proposal prompt into a single hidden `USER` message when the setup chat starts; it is never injected into the system prompt. Reviewer notes: sending the creator's email domain to a third party at signup is not yet disclosed in onboarding copy. <!-- This is an auto-generated description by cubic. --> <a href="https://cubic.dev/pr/twentyhq/twenty/pull/23199?utm_source=github" target="_blank" rel="noopener noreferrer" data-no-image-dialog="true"><picture><source media="(prefers-color-scheme: dark)" srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source media="(prefers-color-scheme: light)" srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img alt="Review in cubic" src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a> <!-- End of auto-generated description by cubic. --> |
||
|
Paul Rastoin
|
ceb699c43f |
Message campaign backfill search field metadata (#23428)
<!-- This is an auto-generated description by cubic. --> <a href="https://cubic.dev/pr/twentyhq/twenty/pull/23428?utm_source=github" target="_blank" rel="noopener noreferrer" data-no-image-dialog="true"><picture><source media="(prefers-color-scheme: dark)" srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source media="(prefers-color-scheme: light)" srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img alt="Review in cubic" src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a> <!-- End of auto-generated description by cubic. --> |
||
|
Paul Rastoin
|
7b46c3ed31 |
use legacy validate build and run for standard metadatas (#23419)
<!-- This is an auto-generated description by cubic. --> <a href="https://cubic.dev/pr/twentyhq/twenty/pull/23419?utm_source=github" target="_blank" rel="noopener noreferrer" data-no-image-dialog="true"><picture><source media="(prefers-color-scheme: dark)" srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source media="(prefers-color-scheme: light)" srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img alt="Review in cubic" src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a> <!-- End of auto-generated description by cubic. --> |
||
|
neo773
|
1e58c3073c |
Feat/email composer improvements (#23188)
- Move composer to dedicated page - Add test email option - Auto saved as draft can be revisited from `objects/messageCampaigns` later - Campaign stats component https://github.com/user-attachments/assets/9e523116-e79b-496d-9c9d-3887e0c9213f <!-- This is an auto-generated description by cubic. --> <a href="https://cubic.dev/pr/twentyhq/twenty/pull/23188?utm_source=github" target="_blank" rel="noopener noreferrer" data-no-image-dialog="true"><picture><source media="(prefers-color-scheme: dark)" srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source media="(prefers-color-scheme: light)" srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img alt="Review in cubic" src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a> <!-- End of auto-generated description by cubic. --> --------- Co-authored-by: Félix Malfait <felix.malfait@gmail.com> |
||
|
Weiko
|
19903f89e5 |
Add indexes for channel webhook subscription external IDs (#23386)
## Context Incoming Microsoft messaging, Microsoft calendar, and Google calendar webhook notifications resolve their channel through `webhookSubscriptionExternalId`. The column was added without an index, so PostgreSQL has to scan the corresponding channel table for every notification. Under sustained webhook traffic, these repeated scans add unnecessary database work and keep core database connections occupied longer. ## What changed - Add a partial B-tree index on `messageChannel.webhookSubscriptionExternalId` - Add the equivalent index on `calendarChannel.webhookSubscriptionExternalId` - Register both indexes in the TypeORM entity metadata - Add an idempotent 2.25 fast instance upgrade command to create and remove them The webhook handlers and their queries remain unchanged. ## Why this design - The indexes contain only non-null subscription IDs, channels without an active subscription do not add index entries - A single-column index supports both the equality lookup used by Google and the `IN` lookup used by Microsoft - The indexes are intentionally non-unique, this preserves existing behavior and avoids making the upgrade fail if historical duplicate values exist - Subscription IDs are read much more often than they are updated, so index maintenance overhead should be negligible ## Expected impact Webhook channel resolution should require a targeted index lookup instead of a table scan. This reduces database work, shortens connection occupancy, and improves latency on webhook notification paths. This is a targeted database optimization. It complements the database pool changes, but is not expected to resolve every source of API tail latency by itself. ## Validation - Server typecheck passes - Oxlint and formatting checks pass - Upgrade command uses idempotent `CREATE INDEX IF NOT EXISTS` and `DROP INDEX IF EXISTS` statements <!-- This is an auto-generated description by cubic. --> <a href="https://cubic.dev/pr/twentyhq/twenty/pull/23386?utm_source=github" target="_blank" rel="noopener noreferrer" data-no-image-dialog="true"><picture><source media="(prefers-color-scheme: dark)" srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source media="(prefers-color-scheme: light)" srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img alt="Review in cubic" src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a> <!-- End of auto-generated description by cubic. --> |
||
|
Raphaël Bosi
|
56245a35af |
Stop leaking the refresh token in the social SSO redirect URL (#23061)
The Google/Microsoft callback for a sign-in with no target workspace
redirected to `/sign-in-up?tokenPair={...}`, putting a 60-day refresh
token in a query string. Those persist in browser history, `Referer`
headers and access logs.
It now carries a single-use, 5-minute opaque token in the URL fragment,
which the frontend exchanges over POST. Browsers never send the fragment
on the wire, so the token stays out of access logs, proxies and
`Referer` headers entirely. Redemption claims the row with a `DELETE`
guarded on `revokedAt`/`deletedAt` being null, so concurrent requests
cannot each mint a refresh token and a revoked token cannot redeem.
Enterprise SSO (OIDC/SAML) already used a POST exchange and is
unchanged.
```mermaid
sequenceDiagram
participant Browser
participant Server
participant DB
Note over Browser,Server: before, the redirect carried access + 60-day refresh in ?tokenPair
Browser->>Server: GET /auth/google/redirect
Server->>DB: store sha256(token), expires in 5 min
Server-->>Browser: 302 /sign-in-up#ssoExchangeToken=opaque
Note over Browser: fragment never sent back to any server
Browser->>Server: POST getAuthTokensFromSSOExchangeToken
Server->>DB: guarded DELETE, single-use claim
Server-->>Browser: access + refresh token, in the response body
```
Since the token is single-use, the refresh token is minted at redemption
instead of at callback, so an abandoned redirect leaves an inert expired
hash rather than a live credential.
Redemption lives in its own `SignInUpSSOExchangeTokenEffect` +
`useRedeemSSOExchangeToken`, mirroring the existing
`VerifyLoginTokenEffect` + `useVerifyLogin` pair, so
`SignInUpGlobalScopeFormEffect` only loses the vulnerable branch. Like
`useVerifyLogin`, the hook clears any stale token pair before
exchanging. The effect reads `window.location.hash` live and strips it
synchronously, which doubles as the StrictMode double-invocation latch.
Remaining exposure is the browser itself (history until the synchronous
strip, client-side scripts), same as any fragment-based OAuth response.
`loginToken` on the workspace-targeted branch still travels as
`/verify?loginToken=` and is replayable for 15 minutes; moving it to the
fragment too is a separate change.
A fast instance command adds a unique partial index on `("type",
"value")` for live SSO exchange tokens, so redemption is an index lookup
instead of a full scan of the shared token table and at most one row can
ever match.
|
||
|
Weiko
|
a96dc335ab |
fix: apply configured pool size to core database (#23322)
## Context `PG_POOL_MAX_CONNECTIONS` is the server setting for the maximum number of PostgreSQL clients in a connection pool. The workspace primary and replica data sources already apply this setting, but the core TypeORM data source did not. Without an explicit `poolSize`, `node-postgres` uses its default limit of 10. As a result, deployments configured with a larger pool still kept the core pool at 10 connections per server process. During bursts of core database work, requests could therefore wait for a local pool connection even when PostgreSQL itself still had available capacity. That acquisition queue adds latency before the query starts, so database-level utilization alone does not reveal the bottleneck. ## What changes The core data source now applies: ```ts poolSize: Number(process.env.PG_POOL_MAX_CONNECTIONS ?? 10) ``` This makes the core data source consistent with the workspace data sources and with the documented meaning of `PG_POOL_MAX_CONNECTIONS`. ## Expected impact Deployments that configure a value above 10 can use that capacity for core database operations instead of queueing behind the driver's default limit. This targets short acquisition spikes affecting operations backed by the core database. The pool remains lazy, so this changes the maximum number of connections available to each process, it does not eagerly open every configured connection. ## Safety - Deployments without `PG_POOL_MAX_CONNECTIONS` keep the previous limit of 10. - Query behavior, transaction behavior, and timeouts are unchanged. - Workspace pool configuration is unchanged. - Operators remain responsible for choosing a value compatible with their total PostgreSQL connection budget and maximum server replica count. ## Scope This removes an unintended local connection-pool bottleneck. It does not address the source of synchronized database bursts, which should be handled separately by reducing unnecessary work. ## Testing Added a regression test that loads the core data source with `PG_POOL_MAX_CONNECTIONS=40` and verifies that TypeORM receives `poolSize: 40`. <!-- This is an auto-generated description by cubic. --> <a href="https://cubic.dev/pr/twentyhq/twenty/pull/23322?utm_source=github" target="_blank" rel="noopener noreferrer" data-no-image-dialog="true"><picture><source media="(prefers-color-scheme: dark)" srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source media="(prefers-color-scheme: light)" srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img alt="Review in cubic" src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a> <!-- End of auto-generated description by cubic. --> |
||
|
neo773
|
3fb29db28a |
Feat/email settings v2 (#23180)
Settings pages changes - Add `displayName` - Unsubscribers Page <img width="1496" height="844" alt="Screenshot 2026-07-22 at 8 52 15 PM" src="https://github.com/user-attachments/assets/69bc1993-4547-4a64-83a6-b47fef1a4e40" /> <!-- This is an auto-generated description by cubic. --> <a href="https://cubic.dev/pr/twentyhq/twenty/pull/23180?utm_source=github" target="_blank" rel="noopener noreferrer" data-no-image-dialog="true"><picture><source media="(prefers-color-scheme: dark)" srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source media="(prefers-color-scheme: light)" srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img alt="Review in cubic" src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a> <!-- End of auto-generated description by cubic. --> --------- Co-authored-by: Félix Malfait <felix.malfait@gmail.com> |
||
|
Etienne
|
3a1067ec6d |
fix(server): index page-layout FKs to fix workspace cleanup timeout (#23289)
The cleanSuspendedWorkspacesJob cron timed out every run (Sentry monitor
"a timeout check-in was detected"): hard-deleting soft-deleted
workspaces hung on `DELETE FROM core.pageLayout`, hit the 10s query
timeout, rolled back, so those workspaces were never destroyed and got
retried hourly.
Root cause: the FKs in the pageLayout -> pageLayoutTab ->
pageLayoutWidget tree had no usable index on the referencing column. The
existing indexes lead with workspaceId and are partial ("deletedAt" IS
NULL), so ON DELETE CASCADE / SET NULL fell back to full sequential
scans of the shared core tables per deleted row; on layout-heavy
workspaces this exceeded 10s.
- Add non-partial FK-column indexes on pageLayoutTab(pageLayoutId) and
pageLayoutWidget(pageLayoutTabId)
<!-- This is an auto-generated description by cubic. -->
<a
href="https://cubic.dev/pr/twentyhq/twenty/pull/23289?utm_source=github"
target="_blank" rel="noopener noreferrer"
data-no-image-dialog="true"><picture><source
media="(prefers-color-scheme: dark)"
srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source
media="(prefers-color-scheme: light)"
srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img
alt="Review in cubic"
src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a>
<!-- End of auto-generated description by cubic. -->
|
||
|
Paul Rastoin
|
bc7922da7d |
fix: move add-agent-foreign-key-to-role-target instance command to 2.25 (#23285)
## What Moves the `add-agent-foreign-key-to-role-target` fast instance command from `2.24.0` to `2.25.0`. Introduced in #23206, the command was registered under version `2.24.0`. Since `TWENTY_CURRENT_VERSION` is now `2.25.0`, `2.24.0` is an already-released version, so its instance commands do not re-run on upgrade and the foreign-key migration would never execute. This is the same issue #23271 fixed for the message-list-members backfill workspace command. ## Changes - Moved the command file from `upgrade-version-command/2-24/` to `2-25/` (renamed the file prefix). - Updated the decorator from `@RegisteredInstanceCommand('2.24.0', ...)` to `('2.25.0', ...)`. - Updated the import in `instance-commands.constant.ts` to the new relative path and reordered both the import and the array entry to sit after the 2-24 commands. The timestamp (`1784820332810`) and command logic (`up`/`down`) are unchanged. --- _Generated by [Claude Code](https://claude.ai/code/session_01KpCb9BK1eoM1WWXJU6g95e)_ <!-- This is an auto-generated description by cubic. --> <a href="https://cubic.dev/pr/twentyhq/twenty/pull/23285?utm_source=github" target="_blank" rel="noopener noreferrer" data-no-image-dialog="true"><picture><source media="(prefers-color-scheme: dark)" srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source media="(prefers-color-scheme: light)" srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img alt="Review in cubic" src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a> <!-- End of auto-generated description by cubic. --> |
||
|
martmull
|
25b0b2601f |
Replace admin app rollout buttons with upgrade-application CLI command (#23212)
## What Removes the two rollout buttons from the admin application detail page and replaces the upgrade flow with a CLI command that can be run directly from a server or worker pod. Also restructures application stop into its own module with a kill switch CLI command, and surfaces stopped apps in workspace settings. ### Removed - "Install on all workspaces" button (General tab, `SettingsAdminApplicationRegistrationGeneralToggles`), its confirmation modal and tooltip - "Upgrade existing installations" button (`SettingsApplicationRegistrationGeneralStats`), its confirmation modal and batch size input - `backfillApplicationInstallation` and `upgradeRegistrationApplications` admin GraphQL mutations and their frontend documents / generated types - `BackfillApplicationInstallationJob` (its only trigger was the removed mutation); `UpgradeApplicationsJob` is kept since the auto-upgrade flow still enqueues it Per review, the "install on all workspaces" flow is dropped without a CLI replacement for now; a dedicated command will be added when needed. ### application:upgrade command Located in `application-upgrade/commands`, registered in `ApplicationUpgradeModule`: ``` yarn command:prod application:upgrade \ --application-registration-universal-identifier <universalIdentifier> \ [--batch-size 5] \ [--workspace-id <id> --workspace-id <id2>] \ [--workspace-count-limit 10] \ [--dry-run] [--yes] ``` - `--workspace-id` (repeatable) restricts the upgrade to specific workspaces; `--workspace-count-limit` caps how many installations are upgraded (max 50, for canary rollouts) - `--batch-size` and `--workspace-count-limit` are validated as positive integers, max 50 - `--dry-run` reports how many (and which) workspaces would be upgraded, without upgrading - Without `--dry-run`, a confirmation prompt shows the app, target version and impacted workspaces; the run then executes exactly the confirmed set; `--yes` skips the prompt for non-interactive usage The upgrade plan is computed by a new `ApplicationUpgradeService.findApplicationsToUpgrade`, and batches run through a new `upgradeApplications` method — both reused by `upgradeAllApplications`, so the auto-upgrade job path is unchanged. ### Application kill switch (per review) Global mechanism only — a per-workspace stop had no demonstrated operational need and added a Redis key format, execution branching, CLI options and tests; an isolated workspace issue can be handled directly in the DB or Redis with the same effort. - `ApplicationStopService` moved to a dedicated `application-stop/` folder with its own `ApplicationStopModule` (imported and re-exported by `ApplicationModule`) - `stop` / `remove` methods that enable or clear the Redis-backed global kill switch; the logic function executor checks it before executing - `application:kill-switch` command with a positional action, confirmation prompt (shows the installation count) and `--yes` bypass: ``` # Enable the kill switch (stop is the default action) yarn command:prod application:kill-switch stop -u <universalIdentifier> [-y] yarn command:prod application:kill-switch -u <universalIdentifier> # Remove the kill switch yarn command:prod application:kill-switch remove -u <universalIdentifier> [-y] ``` ### Stopped apps surfaced in workspace settings (per review) - Dedicated `isApplicationStopped(applicationUniversalIdentifier)` query backed by the kill switch, fetched with `network-only` policy solely by the application detail page — listing applications triggers no extra Redis reads - Application detail page shows a danger banner when the app is stopped: "We are currently encountering issues with this app, its behavior may be degraded while we work on a fix." ## Test - `npx nx typecheck twenty-server` / `npx nx typecheck twenty-front` pass - `npx nx lint:diff-with-main` passes for both packages - `application-stop.service.spec.ts` covers stop, remove, caching and fail-open behavior - Verified end to end locally: ran the kill switch command on a seeded workspace and confirmed the banner renders on the app detail page (screenshot shared separately) |
||
|
Weiko
|
b036d67ec9 |
Configure async ClickHouse inserts for pageview events (#23274)
## Context Pageview tracking goes through the `trackAnalytics` mutation on the metadata API and is persisted through the unified event pipeline before the mutation resolves. ClickHouse inserts already use: ```text async_insert = 1 wait_for_async_insert = 1 ``` `async_insert` lets ClickHouse buffer and batch small inserts, but `wait_for_async_insert = 1` still keeps the API request open until that buffer is flushed successfully. For sparse pageview inserts, the buffer timeout can therefore account for most of the request duration and contribute to metadata API tail latency. ## What this changes - Adds a named `ClickHouseService.insert` option for overriding `async_insert_busy_timeout_max_ms`. - Caps the pageview buffer wait at 100 ms. - Keeps `wait_for_async_insert = 1`. - Leaves workspace, object, usage, application-log, and other event inserts on the existing default timeout. ## Why this approach This removes the avoidable buffer wait from the pageview request path without changing the delivery guarantees of the event pipeline. In particular, this does **not** use `wait_for_async_insert = 0` or fire-and-forget writes. The API still receives an acknowledgement only after ClickHouse flushes the pageview successfully, and insert/schema errors still propagate through the existing handling. The 100 ms value caps only the batching wait. It does not impose a 100 ms deadline on the complete ClickHouse request. ## Expected impact - Lower ClickHouse span duration for pageview tracking. - Lower tail latency for metadata API requests that emit pageviews. - No behavior or durability change for other event types. The trade-off is that pageviews may be flushed in smaller batches. The setting remains scoped to the pageview table so higher-value event streams keep their current batching behavior. ## Testing - Added coverage for the optional ClickHouse busy-timeout setting. - Added coverage verifying that only pageview inserts receive the 100 ms override. - Existing insert failure/retry behavior remains covered. - `twenty-server` typecheck passes. - Focused test result: 23 tests passed. ## Post-deploy verification - Compare pageview ClickHouse span p95/p99 before and after deployment. - Compare metadata API p95/p99. - Check ClickHouse asynchronous-insert failures. - Watch ClickHouse part creation and merge pressure for unexpected growth. <!-- This is an auto-generated description by cubic. --> <a href="https://cubic.dev/pr/twentyhq/twenty/pull/23274?utm_source=github" target="_blank" rel="noopener noreferrer" data-no-image-dialog="true"><picture><source media="(prefers-color-scheme: dark)" srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source media="(prefers-color-scheme: light)" srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img alt="Review in cubic" src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a> <!-- End of auto-generated description by cubic. --> |
||
|
Paul Rastoin
|
dee653dfa6 |
fix: move message list member backfill command to 2.25 (#23271)
## What
Moves the `backfill-message-list-members-junction-target` workspace
upgrade command from `2.24.0` to `2.25.0`.
Introduced in #23176 (commit
|
||
|
Paul Rastoin
|
ba5cb6ba15 |
fix(server): repair missing keyValuePair.applicationId on 2.23 upgrades (#23272)
Fixes #23254 ## Problem Upgrading a self-hosted instance from `2.23.x` to `2.24.0` leaves `core.keyValuePair` without the `applicationId` column. Database-backed config loading then fails on startup and on every refresh (~every 15s) with: ``` column KeyValuePairEntity.applicationId does not exist ``` The frontend shows "Unable to reach the backend". ## Root cause `AddApplicationIdToKeyValuePairFastInstanceCommand` was added in #23089 (after `2.23.x` shipped) but registered under the already-released `2.23.0` segment: ```ts @RegisteredInstanceCommand('2.23.0', 1784659343818) ``` The upgrade cursor is **positional and forward-only**: - `resolveStartCursor` resumes at `lastAttemptedIndex + 1`. A fully-upgraded `2.23.x` instance has its cursor at the last `2.23` workspace command, which sits *after* this newly-inserted fast command in the sequence. So the runner steps right over it and the DDL never runs. - The upgrade-aware metadata layer decides "applied" the same way (`stepIndex < currentCursor` in `upgrade-aware-entity-metadata.adapter.ts`). Since the step index is below the cursor, the column is considered applied and is **not** hidden from TypeORM SELECTs, so every query references a column that was never created. Fresh `2.24.0` installs replay the whole sequence, so only `2.23.x -> 2.24.0` upgrades are affected. The instance log `1 fast instance ... for 2.24.0` confirms the command landed in the `2.23.0` bundle rather than `2.24.0`. ## Fix - Add `RepairKeyValuePairApplicationIdFastInstanceCommand` under the current version (`2.24.0`) with a fresh timestamp, so it sorts last in the sequence and runs for every existing instance regardless of cursor position. Its DDL mirrors the original command and is fully idempotent (`ADD COLUMN IF NOT EXISTS`, `DROP INDEX IF EXISTS` + recreate, `ADD VALUE IF NOT EXISTS`), so it is a no-op on healthy instances. `down()` is intentionally empty: the column lifecycle is owned by the `2.23.0` introduction command. - Repoint the entity's `@WasIntroducedInUpgrade` to the new command so the column stays hidden from queries until the repair has actually run, eliminating the error window during the migration itself. ## Notes - `2.24.0` (`TWENTY_CURRENT_VERSION`) is the correct target: the upgrade sequence only covers previous + current versions, so a command under `2.25.0` (a next version) would not run. If a version bump lands before this merges, the command should be moved to the new current version. - Follow-up worth considering: nothing currently prevents registering a command under a version in `TWENTY_PREVIOUS_VERSIONS`. A startup validation rejecting that would have caught this at PR time. <!-- This is an auto-generated description by cubic. --> <a href="https://cubic.dev/pr/twentyhq/twenty/pull/23272?utm_source=github" target="_blank" rel="noopener noreferrer" data-no-image-dialog="true"><picture><source media="(prefers-color-scheme: dark)" srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source media="(prefers-color-scheme: light)" srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img alt="Review in cubic" src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a> <!-- End of auto-generated description by cubic. --> |
||
|
Weiko
|
abe4d7491c |
Cap nested relation query concurrency (#23252)
## Context Common API queries load selected relations after fetching the root records. Relation loading is batched: one query pipeline loads a relation for all parent records, so this is not an N+1 problem. However, every sibling relation currently starts concurrently through `Promise.all`. Nested relations repeat the same behavior recursively. A wide selection can therefore submit many independent relation query pipelines at once. Existing query complexity and record limits restrict what can be requested, but they do not limit how much database work starts concurrently. ## What this changes This PR adds a request-local FIFO concurrency limiter for nested relation loading. - At most four `findRelations` pipelines execute concurrently. - One limiter is created for the outer relation-loading call. - The same limiter is shared by every recursive level. - Queued work starts as permits become available. - Permits are released in `finally`, including when a query fails. Conceptually: ```text Before: all sibling relations -> database concurrently nested siblings -> more database work concurrently After: all sibling relations -> FIFO queue -> at most 4 database pipelines nested siblings -> same FIFO queue and same limit ``` Note: Also addressing https://github.com/twentyhq/twenty/pull/23251#discussion_r3644510597 |
||
|
Weiko
|
e5c9fcf058 |
Add PostgreSQL connection pool pressure metrics (#23251)
## Summary - Add pool gauges for total, idle, waiting, and maximum connections - Record PostgreSQL connection acquisition duration and failures - Instrument core, workspace primary, and optional replica data sources - Add unit tests covering gauges, acquisition timing, failures, and deduplication <!-- This is an auto-generated description by cubic. --> <a href="https://cubic.dev/pr/twentyhq/twenty/pull/23251?utm_source=github" target="_blank" rel="noopener noreferrer" data-no-image-dialog="true"><picture><source media="(prefers-color-scheme: dark)" srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source media="(prefers-color-scheme: light)" srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img alt="Review in cubic" src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a> <!-- End of auto-generated description by cubic. --> |
||
|
Abdul Rahman
|
2c79093b74 |
feat: agent roleUniversalIdentifier for manifest-driven role assignment (#23206)
## Summary - Adds optional `roleUniversalIdentifier` on `AgentManifest` / `defineAgent` so apps can declaratively assign a role to an agent (same config shape as `defaultRoleUniversalIdentifier`). - Wires `agentUniversalIdentifier` as a sync many-to-one FK on `roleTarget`, and emits a deterministic `roleTarget` from the agent during app sync (create / update / delete). - Enables app agents (e.g. Slack assistant) to get a role on install without postInstall hooks or manual admin assignment. <!-- This is an auto-generated description by cubic. --> <a href="https://cubic.dev/pr/twentyhq/twenty/pull/23206?utm_source=github" target="_blank" rel="noopener noreferrer" data-no-image-dialog="true"><picture><source media="(prefers-color-scheme: dark)" srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source media="(prefers-color-scheme: light)" srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img alt="Review in cubic" src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a> <!-- End of auto-generated description by cubic. --> |
||
|
neo773
|
59eead238d |
message list member backfill (#23176)
<!-- This is an auto-generated description by cubic. --> <a href="https://cubic.dev/pr/twentyhq/twenty/pull/23176?utm_source=github" target="_blank" rel="noopener noreferrer" data-no-image-dialog="true"><picture><source media="(prefers-color-scheme: dark)" srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source media="(prefers-color-scheme: light)" srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img alt="Review in cubic" src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a> <!-- End of auto-generated description by cubic. --> |
||
|
Paul Rastoin
|
b91c2a6457 |
fix(server): repair missing applicationRegistration.logoFileId on upgraded instances (#23215)
## Problem closes https://github.com/twentyhq/twenty/issues/23210 Self-hosted instances on 2.23.x fail their workspace upgrade with: ``` column ApplicationEntity__ApplicationEntity_applicationRegistration.logoFileId does not exist at UpgradePeopleDataLabsApplicationCommand.runOnWorkspace ``` The `2-21` instance command that adds `core."applicationRegistration"."logoFileId"` was merged ~20 minutes after the 2.22 version bump (PR #22827, `94192a2164`), so it first shipped in 2.22 while registered under `@RegisteredInstanceCommand('2.21.0', ...)`. The upgrade runner resolves its start position from the last recorded command and only moves forward. Any instance that had already run a 2.21.x binary has its cursor past that slot, so the command is skipped permanently and the column is never created. `UpgradeAwareEntityMetadataAdapter` decides column visibility positionally (`index < currentCursor`), not by whether the command actually ran, so it keeps `logoFileId` in the SELECT list and the instance reports "Up to date" while the column is absent. **Affected:** instances that ran 2.21.x, then upgraded to >= 2.22. Instances that went from <= 2.20 straight to >= 2.22 replayed the full sequence and are fine. `logoFileId` is populated lazily by design (NULL is a supported state), so no backfill is added. ## Changes **1. Idempotent DDL guard in the failing workspace command** `2-23-workspace-command-...-upgrade-people-data-labs-application.command.ts` now ensures the column exists at the top of `runOnWorkspace`, before the `findOne` that crashes on affected instances. It uses the core `DataSource` (`@InjectDataSource()`) because `core."applicationRegistration"` is instance-global, guards with a per-process boolean in addition to the SQL-level `IF NOT EXISTS`, and copies the full statement list (column + unique + FK constraints) verbatim from the 2.21 command. In dry-run it probes `information_schema.columns` and returns instead of running the crashing query. **2. Fast instance command in 2.23** New `2-23-instance-command-fast-1784823473532-add-logo-file-id-to-application-registration.ts`, registered at the end of the 2.23 fast segment (highest timestamp), running the same idempotent DDL. This covers the normal 2.22 -> 2.23 path and, critically, instances with zero provisioned workspaces where the workspace command body never executes. The shared DDL lives in `2-23/utils/ensure-application-registration-logo-file-id-column.util.ts` so both paths stay byte-for-byte identical. Class name follows the `Early2_4` / `Early2_5` precedent to avoid colliding with the 2.21 command. The fix lives entirely in 2.23: instances stuck at the failing workspace command retry it every run, and 2.22 -> 2.24 jumps still replay the 2.23 segment. ## Ops note Instances failing right now can be unblocked immediately by running the same `ALTER TABLE` block by hand against their core database (byte-for-byte what the command does). Worth including in the 2.23 patch release note. ## Verification - New fast instance command re-slotted last in the 2.23 fast segment (timestamp `1784823473532` > current max `1784659343818`). - Manual repro path: boot `twentycrm/twenty:v2.21`, seed, stop, run `upgrade` from this branch, assert the column exists and `upgrade:status` reports 0 failed. The default v1.22 baseline does not reproduce it (replays from cursor 0). --- _Generated by [Claude Code](https://claude.ai/code/session_01YAuDR585cx7FyAKoiT32j3)_ <!-- This is an auto-generated description by cubic. --> <a href="https://cubic.dev/pr/twentyhq/twenty/pull/23215?utm_source=github" target="_blank" rel="noopener noreferrer" data-no-image-dialog="true"><picture><source media="(prefers-color-scheme: dark)" srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source media="(prefers-color-scheme: light)" srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img alt="Review in cubic" src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a> <!-- End of auto-generated description by cubic. --> --------- Co-authored-by: Paul Rastoin <paul.rastoin@gmail.com> |
||
|
Thomas Trompette
|
96a2456367 |
feat(workflow): periodic core-consistency check for workflows, versions and triggers (#23103)
Monitoring for the soft-ref migration. The `workflow` /
`workflowVersion` dual-write into core is **best-effort** (async, not
transactional; failures only go to Sentry), so `core.workflow` /
`core.workflowVersion` can silently drift from the workspace source of
truth. This adds a periodic job that detects that drift across **all
three workflow entities** and emits it as metrics.
Supersedes the earlier inline shadow-parity approach that lived on this
branch — that only covered trigger dispatch and added a cache read +
diff to every cron tick and every DB-event batch (too much hot-path
overhead). This is broader and fully off the dispatch path.
## What
A cron (`cron:workflow:core-consistency-check`, every 3 hours, wired
into `cron:register:all`). Per run:
- **Bounded**: `SELECT DISTINCT "workspaceId" FROM core."workflow"` —
only workspaces that actually use workflows (skips the large majority).
- Per such workspace, emit a drift metric per `(entity, driftType)` —
detect-only, plus a triage log:
- **workflow** and **workflowVersion** — `unlinked` / `missingCore` /
`orphanCore` / `fieldMismatch`, via cross-schema `COUNT` aggregates
(core + workspace are the same DB, so indexed joins — no rows pulled
into JS).
- **automated triggers** — the `workflowAutomatedTrigger` table vs the
`workflowAutomatedTriggerMaps` cache: `inTableNotCache` /
`inCacheNotTable` / settings `mismatch`.
- Per-workspace failures are isolated (caught → Sentry) so one bad
workspace does not stop the sweep.
## Why it is efficient
Two central `core.*` queries + a few `COUNT` queries per
*workflow-using* workspace, on a relaxed cadence, off the dispatch path.
Shardable across ticks later if needed.
## Metrics
`workflow-core-consistency/{workflow,version,automated-trigger}/drift`
counters, attribute `driftType`. Dashboards: twentyhq/twenty-infra#800.
## Not in scope
Detect-only — no auto-heal (the existing backfill/rebuild command can
heal). No dispatch or flag changes.
## Test
- The consistency SQL (workflow/version sync counts, orphan counts,
trigger read) validated against a live workspace (it surfaced real drift
there — unlinked versions + an orphan core version). The whole
cron→service→SQL→metric pipeline is proven live: the cron is already
emitting real drift counters on a running server.
- Unit specs for the service (clean → no metric; per-entity drift per
dimension; per-workspace error isolation).
- Command boots and registers via `cron:register:all` (verified).
Typecheck + lint clean.
|
||
|
Weiko
|
66df0ac47c |
Switch application stop/start commands to Redis-backed global kill switch (#23202)
## Context [#23183](https://github.com/twentyhq/twenty/pull/23183) introduced the right enforcement point: every logic-function execution is rejected centrally before consuming the shared workspace throttle when its application is stopped. However, its server-wide path reads PostgreSQL for every execution attempt. A kill switch is most useful while an application is producing abnormal load, potentially while PostgreSQL is already under pressure. The enforcement mechanism should not add more database traffic in that situation. This state is also operational and temporary. It is used to troubleshoot an application, not as durable application configuration. ## What this PR changes - Uses one global Redis key per application universal identifier: ```text module:applications:kill-switch:{applicationUniversalIdentifier} ``` - Keeps the check in `LogicFunctionExecutorService`, before the workspace execution throttle. - Adds a 60-second process-local cache for both present and absent keys. - Deduplicates concurrent cache refreshes, so an execution burst causes at most one Redis read per application and process. - Fails open when Redis cannot be read and caches that result for the same minute, avoiding a Redis retry storm. - Removes the database columns, upgrade command, workspace-cache recomputation, registration lookup, and stop/start CLI commands introduced by #23183. - Keeps disabled queued executions non-retriable, without emitting one warning for every skipped payload. The switch is operated directly in Redis. For example: ```redis SET module:applications:kill-switch:{applicationUniversalIdentifier} 1 EX 3600 DEL module:applications:kill-switch:{applicationUniversalIdentifier} ``` Any value means stopped; deleting or expiring the key means enabled. ## Why this is a better fit | | #23183 | This PR | |---|---|---| | State | Durable PostgreSQL fields | Ephemeral Redis key | | Server-wide hot path | PostgreSQL lookup per execution | At most one Redis lookup per app/process/minute | | Scope | Workspace and application registration | Application universal identifier across all workspaces | | Operational cleanup | Explicit start command | `DEL`, eviction, restart, or operator-selected TTL | | Database dependency during an incident | Required | None | The trade-off is deliberate: a Redis change can take up to 60 seconds to reach every process, and the switch is lost when the cache key disappears. That is acceptable for a temporary troubleshooting control and keeps the normal execution path inexpensive. Existing in-flight functions are not interrupted. New direct or queued executions are rejected when they reach the executor. |
||
|
martmull
|
a3f4acadb1 |
Add workspace and server level stop commands for applications (#23183)
## Context When an installed application misbehaves (e.g. a logic function loop DDoSing the server or the database), we currently have no targeted way to shut it down in production: the only kill switch is `LOGIC_FUNCTION_TYPE=DISABLED`, which disables logic functions for the whole instance. This PR adds an emergency stop mechanism at two levels: - **Workspace level**: stop one installed application in its workspace. - **Server level**: stop every application installed from an `applicationRegistration`, across all workspaces. ## How it works **New nullable `stoppedAt` columns** on `core.application` and `core.applicationRegistration` (fast instance command `2.24.0`, with `up`/`down` and `@WasIntroducedInUpgrade` decorators on the entities). **Enforcement in a single choke point**: `LogicFunctionExecutorService.execute()` is the funnel behind every execution path (public route triggers, server route triggers, cron triggers, database event triggers, workflow actions, agent tool calls, manual GraphQL execution, install hooks). A new `assertApplicationNotStopped` guard runs right after the flat entities are resolved and throws `LOGIC_FUNCTION_DISABLED` (already mapped to a 403 on route triggers and handled by the GraphQL exception handler) when: - `flatApplication.stoppedAt` is set (workspace-level stop, read from the cached flat application maps: zero extra runtime cost), or - the linked registration is stopped (one indexed PK lookup, same pattern as the existing per-execution server-variable query). **Propagation**: the workspace-level stop invalidates and recomputes `flatApplicationMaps` for the workspace, so all server instances pick the flag up within the local cache TTL (100ms). The registration-level flag is read live, so it is effective immediately. ## Ops commands ```bash # Workspace level yarn command:prod application:stop -a <application-id> yarn command:prod application:start -a <application-id> # Server level (all applications of the registration, all workspaces) yarn command:prod application-registration:stop -r <application-registration-id> yarn command:prod application-registration:start -r <application-registration-id> ``` Each command logs what was stopped/started and, for registrations, how many installed applications are affected. ## Notes - Stopped executions fail fast at the guard, so queued trigger jobs (cron/db-event) burn a negligible amount of work while stopped. - The two flags are independent: lifting a registration-level stop does not clear workspace-level stops that were set individually, and vice versa. - Unit tests added for `ApplicationStopService`. --- _Generated by [Claude Code](https://claude.ai/code/session_01CjEnKUACn89aSgK1wEMH2d)_ <!-- This is an auto-generated description by cubic. --> <a href="https://cubic.dev/pr/twentyhq/twenty/pull/23183?utm_source=github" target="_blank" rel="noopener noreferrer" data-no-image-dialog="true"><picture><source media="(prefers-color-scheme: dark)" srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source media="(prefers-color-scheme: light)" srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img alt="Review in cubic" src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a> <!-- End of auto-generated description by cubic. --> |
||
|
Abdul Rahman
|
04d1c2035c |
feat(connections): run a logic function on connection provider connect (#23167)
## What Adds an optional `onConnectLogicFunctionUniversalIdentifier` field to the connection provider manifest. When set, the referenced logic function is dispatched right after an OAuth connection is successfully established for that provider. This gives apps a first-class "on connect" hook — e.g. the Slack app can resolve the workspace's `team_id` via `auth.test` and claim the `team_id -> workspaceId` mapping in the SERVER key-value store immediately on connect, instead of racing against later events. Follow-up to the app key-value store PR (#23089). ## How - **twenty-shared**: add `onConnectLogicFunctionUniversalIdentifier` to `ConnectionProviderManifest`. - **twenty-sdk**: expose the field in `defineConnectionProvider` and validate it is a UUID `universalIdentifier`. - **twenty-server**: - add a nullable `onConnectLogicFunctionUniversalIdentifier` column to `ConnectionProviderEntity` (+ fast instance command / migration). - map the field through the <!-- This is an auto-generated description by cubic. --> <a href="https://cubic.dev/pr/twentyhq/twenty/pull/23167?utm_source=github" target="_blank" rel="noopener noreferrer" data-no-image-dialog="true"><picture><source media="(prefers-color-scheme: dark)" srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source media="(prefers-color-scheme: light)" srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img alt="Review in cubic" src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a> <!-- End of auto-generated description by cubic. --> |
||
|
Paul Rastoin
|
9043ab3091 |
Upgrade to 1.0.9 PDL app (#23172)
<!-- This is an auto-generated description by cubic. --> <a href="https://cubic.dev/pr/twentyhq/twenty/pull/23172?utm_source=github" target="_blank" rel="noopener noreferrer" data-no-image-dialog="true"><picture><source media="(prefers-color-scheme: dark)" srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source media="(prefers-color-scheme: light)" srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img alt="Review in cubic" src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a> <!-- End of auto-generated description by cubic. --> |
||
|
Abdul Rahman
|
4c4a154d31 |
key-value storage for applications (#23089)
## What
Key-value storage for applications, as proposed in
twentyhq/core-team-issues#2391 — built on the existing `keyValuePair`
entity:
- Nullable `applicationId` relation on `keyValuePair` + a new
`APPLICATION_VARIABLE` type (fast instance command included)
- GraphQL CRUD on the metadata schema (`appKeyValue`, `setAppKeyValue`,
`deleteAppKeyValue`), requiring an `APPLICATION_ACCESS` token —
`applicationId` always comes from the token, never from arguments, so
apps can't touch each other's entries
- `kv.get` / `kv.set` / `kv.delete` helpers in
`twenty-sdk/logic-function`
## Scopes
- **`INSTALL`** (default): entries are private to one workspace install;
arbitrary JSON values
- **`GLOBAL`**: entries are shared across every install of the app, with
claim semantics — the value is always the claiming `workspaceId` and
only that workspace can overwrite or delete the key (guarded writes,
race-safe via insert-if-absent)
Since `applicationId` identifies an install (one row per workspace),
GLOBAL entries are stored under the registration owner workspace's
install so all installs of the same app share one namespace.
The GLOBAL scope is what enables cross-workspace webhook routing: e.g.
the Slack app's `serverRoute` resolver (running in the owner workspace)
can resolve `kv.get('slack:team:' + team_id, { scope: 'GLOBAL' })` to
find the workspace that connected that Slack team — without a workspace
being able to hijack another's mapping.
## Follow-ups
- Wire the Slack assistant PR (#22984) to write the claim at connect
time and read it in the events resolver
- `kv.*` access from front components
<!-- This is an auto-generated description by cubic. -->
<a
href="https://cubic.dev/pr/twentyhq/twenty/pull/23089?utm_source=github"
target="_blank" rel="noopener noreferrer"
data-no-image-dialog="true"><picture><source
media="(prefers-color-scheme: dark)"
srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source
media="(prefers-color-scheme: light)"
srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img
alt="Review in cubic"
src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a>
<!-- End of auto-generated description by cubic. -->
|
||
|
Paul Rastoin
|
893db7558e |
Fix instance fast migration, fallback index creation (#23149)
<!-- This is an auto-generated description by cubic. --> <a href="https://cubic.dev/pr/twentyhq/twenty/pull/23149?utm_source=github" target="_blank" rel="noopener noreferrer" data-no-image-dialog="true"><picture><source media="(prefers-color-scheme: dark)" srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source media="(prefers-color-scheme: light)" srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img alt="Review in cubic" src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a> <!-- End of auto-generated description by cubic. --> |
||
|
Etienne
|
5add5ea695 |
fix(billing) - expand billing sub uniqueness constraint (#23123)
fixes : https://discord.com/channels/1130383047699738754/1528956737363771497 <!-- This is an auto-generated description by cubic. --> <a href="https://cubic.dev/pr/twentyhq/twenty/pull/23123?utm_source=github" target="_blank" rel="noopener noreferrer" data-no-image-dialog="true"><picture><source media="(prefers-color-scheme: dark)" srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source media="(prefers-color-scheme: light)" srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img alt="Review in cubic" src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a> <!-- End of auto-generated description by cubic. --> |
||
|
Weiko
|
23cf85f745 |
Fix invalid UUID insert in workflow core-links backfill (#23118)
## Fix invalid UUID insert in workflow core-links backfill ### Problem The `2-23:backfill-workflow-core-links` workspace upgrade command failed with: ``` QueryFailedError: invalid input syntax for type uuid: "" (22P02) ``` The `core."workflow"."lastPublishedVersionId"` column is a `uuid`, but some workspace workflows store an empty string `""` (not `NULL`) for that field. The code used `workflow.lastPublishedVersionId ?? null`, and `??` only falls back on `null`/`undefined` — so `""` was passed straight through and Postgres rejected it. ### Fix Use `|| null` instead of `?? null` so empty strings are normalized to `null` before insertion into the `uuid` column. <!-- This is an auto-generated description by cubic. --> <a href="https://cubic.dev/pr/twentyhq/twenty/pull/23118?utm_source=github" target="_blank" rel="noopener noreferrer" data-no-image-dialog="true"><picture><source media="(prefers-color-scheme: dark)" srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source media="(prefers-color-scheme: light)" srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img alt="Review in cubic" src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a> <!-- End of auto-generated description by cubic. --> |
||
|
Paul Rastoin
|
71a1ff7ac8 |
Cache twenty-client-sdk modules host-side via content-addressed URLs (#22981)
## Context Front component sources are fetched host-side and integrity-verified by the SHA-256 checksum embedded in their URL, cached in Cache Storage — a layer that exists specifically because their download URLs are presigned and rotate. The `twenty-client-sdk` modules (`core` and `metadata`) were re-fetched on every render and could not be cached safely: their URLs carried no checksum and the server exposed no freshness signal. This PR makes the SDK module URLs **content-addressed** and relies on the **browser HTTP cache** for immutability, and it keys the checksums on their real owners: the **application** for `core`, the **instance** for `metadata`. The checksum does double duty: cache invalidation (regeneration changes the checksum → the URL changes → guaranteed cache miss) and a server-side cacheability guard (the server only grants `immutable` when the checksum in the URL matches the authoritative checksum it knows for that module — persisted at generation time for `core`, hashed once at bootstrap for `metadata` — so no per-request hashing of the served bytes). Note this is **not** an end-to-end integrity guarantee: there is no client-side hash verification, and on a fingerprint mismatch the server still serves the current bytes with `no-store` (self-healing for stale URLs) rather than failing. <img width="2412" height="926" alt="image" src="https://github.com/user-attachments/assets/d97935d2-0fdb-4c44-89ac-596b7ca8ca64" /> Closes twentyhq/core-team-issues#2688. ## Routes | Module | URL | Scope | | --- | --- | --- | | `core` | `/rest/sdk-client/{applicationId}/core[/{checksum}]` | Per application (generated bundle) | | `metadata` | `/rest/sdk-client/metadata[/{checksum}]` | **Instance-wide**: no application segment, so every application converges on one URL and the browser downloads the module once per release instead of once per application | The previous application-scoped metadata path (`/rest/sdk-client/{applicationId}/metadata[/{checksum}]`) is **kept for backward compatibility**, new clients just stop generating those URLs. The instance-wide route is declared before the parameterized route so `metadata/{checksum}` is not swallowed as `:applicationId/:moduleName`. ## Caching model | Request | `Cache-Control` | Effect | | --- | --- | --- | | Fingerprinted URL, checksum matches the known module checksum | `immutable` | Cached indefinitely by the browser HTTP cache; a new checksum is a new URL | | Fingerprinted URL, checksum does not match | `no-store` | Current bytes served uncached (self-healing for stale URLs) | | Bare URL (pre-generation fallback, `core` only in practice) | `no-store` | Never cached | - Both responses also set `X-Content-Type-Options: nosniff` and `Content-Type: application/javascript`. - SDK modules are intentionally **not** placed in Cache Storage. That layer stays reserved for the presigned/rotating component-source URLs; SDK modules are served directly and authenticated, so the browser HTTP cache (keyed by the content-addressed URL) is their single cache layer. ## Checksum provenance - **core** — per **application**, persisted on `application.sdkClientCoreChecksum` at generation time and read back from `flatApplicationMaps` (never re-hashed per request). - **metadata** — **instance-wide**, hashed once from the installed `twenty-client-sdk/dist/metadata.mjs` package (warmed at bootstrap, memoized per process) and served straight from that package, so it is fresh from the first request after a release with no archive dependency. ## Server (twenty-server) - Hash `dist/core.mjs` at SDK generation and persist `sdkClientCoreChecksum` via `applicationRepository.update`. Adds the nullable text column to `application.entity.ts` (mirroring `packageJsonChecksum`) plus a fast instance command with up/down; `FlatApplication` picks it up automatically. - New **application-scoped** query `applicationSdkClientChecksums(applicationId: UUID!): SdkClientChecksums` on `ApplicationResolver` (metadata schema, `WorkspaceAuthGuard` + `NoPermissionGuard`). `SdkClientChecksums.core` is **nullable** and stays `null` until the SDK has been generated at least once; `metadata` is **always present** (bootstrap-warmed), so the metadata module is cacheable from the very first render of any app. The query itself returns `null` only for unknown applications. - `SdkClientChecksumsDTO` now lives in the shared `core-modules/sdk-client/dtos/`. `FrontComponentDTO` and the `frontComponent` resolver no longer carry checksums (decoupled from the front-component row). - `sdk-client` controller: instance-wide `metadata[/:checksum]` route (no workspace-cache or application lookup, serves the memoized installed module) + application-scoped `:applicationId/:moduleName[/:checksum]` route (serves `core` from the per-application archive, `metadata` kept for back-compat). Cacheability compares the URL checksum against the **known** checksum — persisted `sdkClientCoreChecksum` for `core`, memoized package hash for `metadata` — instead of hashing the served bytes on every request: `immutable` on match, `no-store` otherwise (bare URL or stale fingerprint), plus `nosniff`. A persisted checksum out of sync with the archive only downgrades to `no-store` until the next regeneration. ## Front (twenty-front) - New metadata query `GetApplicationSdkClientChecksums`, keyed by `applicationId`; removed the `sdkClientChecksums` selection from `FindOneFrontComponent`. - `getSdkClientUrls` builds the two module URLs independently: `/sdk-client/{applicationId}/core/{checksum}` and the **instance-wide** `/sdk-client/metadata/{checksum}` (no application segment → one shared browser cache entry per release across all applications). Each falls back to its bare URL when its checksum is absent — since `core` is nullable, a never-generated app still gets a content-addressed metadata URL and only `core` falls back. The checksum type is sourced from the codegen `SdkClientChecksums` type rather than a hand-maintained duplicate. - `FrontComponentRenderer` is split into a gating outer component (runs `FindOneFrontComponent`, renders nothing while loading) and a content component that receives a guaranteed-non-null `frontComponent`. Following project conventions, the side effects live in dedicated effect components: `FrontComponentLoadErrorSnackBarEffect` (query error → snackbar) and `FrontComponentApplicationTokenPairEffect` (mirrors the query-derived token pair into component state unconditionally, `null` included, so revoked credentials can never be retained or refreshed). The content component fetches checksums via the application-keyed query and **gates the mount of SDK-using components on that query**, so the very first module fetch is always the content-addressed (`immutable`) URL instead of the bare `no-store` one. Non-SDK components skip the query and are never blocked. - **Live invalidation without reload:** SDK regeneration updates the application row, and the server broadcasts an `application` metadata event carrying the new core checksum. `useOnApplicationSdkClientChecksumsUpdated` / `useUpdateSdkClientChecksumsApolloCache` patch the application-keyed checksum query cache (core only; the instance-wide metadata is preserved), so every mounted component of that application picks up the new URL at once. This replaces the previous frontComponent-derived field and closes the earlier "known gap" (a mounted component staying on a session-old checksum until a full reload). The cache-patching callback is memoized (`useCallback`) so the window listener is registered once per application, and the listener is **skipped entirely** for non-SDK components (`useListenToMetadataOperationBrowserEvent` gained a `skip` option) — they register no listener and never refetch a query they don't consume. ## Renderer (twenty-front-component-renderer) - SDK sources are fetched through a dedicated plain authenticated fetch, `fetchJavaScriptModuleSourceText` (Bearer header, `credentials: 'omit'`), instead of the Cache Storage `fetchComponentSource` path; `fetchSdkClientSources` uses it. Execution stays exclusively in the opaque-origin worker via blob URLs; the host only fetches and forwards source strings (no hashing host-side). Staleness self-resolves through the checksum: new checksum → new URL → cache miss. <!-- This is an auto-generated description by cubic. --> <a href="https://cubic.dev/pr/twentyhq/twenty/pull/22981?utm_source=github" target="_blank" rel="noopener noreferrer" data-no-image-dialog="true"><picture><source media="(prefers-color-scheme: dark)" srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source media="(prefers-color-scheme: light)" srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img alt="Review in cubic" src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a> <!-- End of auto-generated description by cubic. --> |
||
|
Félix Malfait
|
3ad3e8bd1a |
feat: kanban, calendar and group-by table layouts for dashboard view widgets (#22963)
## Context Dashboard view widgets previously only rendered flat tables. This PR ships the full feature: **Table with group-by**, **Kanban**, and **Calendar** layouts for dashboard view widgets — server API + frontend, end-to-end. (Originally staged as a 4-PR stack — #22966, #22967, #22968 — consolidated here per review.) ## Server / API - **View typing.** Adds `KANBAN_WIDGET` and `CALENDAR_WIDGET` to `ViewType` (following the `TABLE_WIDGET` precedent) so widget-backing views keep their layout in `view.type` while staying excluded from record-index pickers. Shared `getViewLayoutFromViewType()` maps widget types to their base layout; `isWidgetViewType()` centralizes the exclusions that were previously hardcoded per-site. - **Migrations.** Two fast instance commands (**2.23**): `ALTER TYPE core.view_type_enum ADD VALUE` for both values, and a widened `CHK_VIEW_CALENDAR_INTEGRITY` constraint covering `CALENDAR_WIDGET` (entity `@Check` updated for fresh installs). - **Validation.** `FlatViewValidatorService` keys kanban/calendar validation on the mapped layout, so widget views get the same invariants as index views (kanban needs a groupable group-by field; calendar needs a date field + layout). Calendar widget views default to month; a non-month (DAY/WEEK) layout is rejected at the API level **unless** the `IS_CALENDAR_WEEK_VIEW_ENABLED` feature flag is enabled for the workspace — the same flag that gates day/week on index calendars. - **API.** `upsertViewWidget` (LAYOUTS permission) accepts a nested `view` settings input (`type`, `mainGroupByFieldMetadataId`, `shouldHideEmptyGroups`, kanban aggregate/column-width, calendar layout/fields). Routes through the standard update path, so `viewGroups` auto-generate from SELECT options exactly like index views. Only widget view types accepted; only `RECORD_TABLE` widgets can change view settings. - **AI tools.** `create-complete-dashboard` + `create_view` now use/allow the `*_WIDGET` types (previously they created plain `TABLE` views that leak into index pickers). ## Frontend **Settings panel.** The **Source** (object) row comes first, since which layouts are available depends on it. The **Layout** row below is a working dropdown (Table / Kanban / Calendar); layouts the source object can't support are **disabled with a hint** ("Needs a Select field" / "Needs a Date field") rather than hidden. Group-by row (select fields; searchable) with a **Hide empty groups** toggle while grouped; **Date field** row replaces Group by while Calendar is active, and — when the `IS_CALENDAR_WEEK_VIEW_ENABLED` flag is on — a **Calendar view** row (Day / Week / Month) appears beside it; **Limit** row hidden while grouped (only the flat virtualized loader enforces it). Kanban keeps its group-by locked (no `None` option). **Instant edit-mode preview.** Draft snapshots carry `viewGroups`; picking a group-by synthesizes them client-side (`buildDraftViewGroupsForFieldMetadataItem`, mirroring the server's generation), so grouped tables/boards preview immediately before dashboard save. On save, `upsertViewWidget` responses hand back the server-generated groups, which replace the client-generated ones in the persisted snapshot. **Renderers.** `RecordTableWidgetRendererContent` branches on the backing view's layout: `RecordBoardWidget` (wraps the standard `RecordBoardContainer`) and `RecordCalendarWidget` (mounts the existing `RecordCalendar`, which renders month / day / week) inside the same per-widget provider sandbox the table uses. **Read-only semantics.** Two flags with distinct scopes, each documented on its state: - `isRecordBoardViewSettingsReadOnlyComponentState` — locks the board chrome that edits view settings (add group, column reorder/resize/menu, aggregates); **card drag still updates records** under object permissions. - `isRecordCalendarReadOnlyComponentState` — widget calendars are read-only by default (no drag, no add-new, no in-calendar layout switch); cards open the side panel. The one exception, behind `IS_CALENDAR_WEEK_VIEW_ENABLED`: a **live (non edit-mode) day/week** widget calendar allows drag-to-reschedule and record creation under object permissions. Month calendars and edit-mode previews stay read-only. **Calendar state componentization.** The calendar module's three settings move from global atoms to component states keyed on `RecordCalendarComponentInstanceContext` (same pattern as record-board), so several calendar widgets and an index-page calendar can coexist without leaking state. All readers resolve the ambient instance; calendar unit tests updated. **Multi-instance fixes that also fix index pages:** record drag states were written against a different instance than every reader resolves (now use the ambient instance); the board sticky-header DOM id is namespaced per board; dragged board cards portal to `document.body` while dragging so react-grid-layout's transforms can't offset the clone from the pointer. ## Scope (v1) - Widget calendars are month-only and read-only by default. With `IS_CALENDAR_WEEK_VIEW_ENABLED` enabled, day/week layouts become selectable (UI + API) and live day/week widget calendars support drag-to-reschedule and record creation under object permissions. - Widget group-by offers SELECT fields only (server auto-generates groups from options; widgets have no per-record add-group flow). ## Tests - Integration: `upsert-view-widget-view-settings.integration-spec.ts` (9 tests — group auto-creation, invalid type/field rejections, non-month calendar widget rejected while the week/day flag is off and accepted once it's enabled, combined settings+fields call); pre-existing `upsert-view-widget` suite (20) green. - Front: new suites for draft view-group generation and snapshot clone/build utils; calendar suites componentized; full `twenty-front` jest, typecheck, oxlint green; `twenty-server` typecheck + lint green. - Browser-verified end-to-end (real dev server + seeded workspace): configure → live edit-mode preview → save → reload for all three layouts; measured drag with pointer inside the card; index-page calendar re-verified (with the week/day flag enabled). https://claude.ai/code/session_01E5N87kwwZWhDtEQaP72cMf |
||
|
Paul Rastoin
|
2ef7b7824e |
Upgrade call-recorder, people-data-labs, last-contact and partners apps to twenty-sdk 2.23.0-alpha.1 (#23098)
## What Upgrades the two breaking-change-prone apps to `twenty-sdk` / `twenty-client-sdk` `2.23.0-alpha.1`, and adds the server-side hook that lets the 2.23 upgrade install them: - **people-data-labs** - **partners** Follows up on #22882 (System side effect relations), which re-derived the system relation field universal identifiers name-free and shipped `getSystemRelationFieldUniversalIdentifier` in the SDK. ## How - **people-data-labs**: bump the SDK to `2.23.0-alpha.1`. The enriched views temporarily hardcoded the new system relation identifiers with a TODO because the SDK still embedded the old values; now that the name-free identifiers ship in `2.23`, derive them from `STANDARD_OBJECT_UNIVERSAL_IDENTIFIERS.{company,person}.fields.{noteTargets,taskTargets,attachments,timelineActivities}.universalIdentifier` (identical to the previously pinned values, verified). Engine already pinned `twenty >=2.23.0`; app stays `1.0.7` (manifest unchanged). - **partners**: bump the SDK to `2.23.0-alpha.1`. The partner role references `opportunity.fields.{taskTargets,noteTargets,attachments,timelineActivities}` universal identifiers, which the SDK now resolves to the `2.23` name-free values. Pin `engines.twenty >=2.23.0` and bump the app to `1.3.1`. - **server**: add an opt-in `skipWorkspaceCompatibilityCheck` to the install/upgrade path. The `upgrade-people-data-labs-application` 2.23 command runs mid-upgrade, before the workspace is marked as having completed 2.23, so the workspace-compatibility check would otherwise reject installing `1.0.7` (`engines >=2.23.0`). The server is already on 2.23, so the command passes the flag to install `1.0.7` and close the desync window. Version-progression (downgrade/same-version) checks still run. - **call-recorder** and **last-contact** are intentionally left unchanged (reverted): they don't define custom objects and don't reference the system relation identifiers, so they aren't breaking-change-prone and need no SDK bump. ## Breaking change constraints - **people-data-labs** and **partners** reference system relation identifiers that only exist on a `2.23` server, so both pin `engines.twenty >=2.23.0`. Their `dockerhub-latest` integration leg is red by design until a >=2.23 server image is published (same accepted state as #22882); the `local` leg is green. ## Validation - Regenerated the app lockfiles against the published `2.23.0-alpha.1`. - `people-data-labs` typechecks cleanly against the real `2.23` SDK types. - CI: people-data-labs and partners green on `local`, red on `dockerhub-latest` by design; server/SDK/all other checks green. - Rebased onto latest `main`. |
||
|
Marie
|
e5fc5054cc |
Fix "Go to roles settings" command (#23105)
**Fix the "Go to Roles Settings" command** — it pointed at the non-existent `/settings/roles` route and now navigates to `/settings/members#roles`. **Backfill existing workspaces** — added the `upgrade:2-23:fix-go-to-roles-settings-command-menu-item-path` workspace command, which rewrites the seeded command menu item payload for existing workspaces. It is idempotent and only touches workspaces still holding the legacy path. <!-- This is an auto-generated description by cubic. --> <a href="https://cubic.dev/pr/twentyhq/twenty/pull/23105?utm_source=github" target="_blank" rel="noopener noreferrer" data-no-image-dialog="true"><picture><source media="(prefers-color-scheme: dark)" srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source media="(prefers-color-scheme: light)" srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img alt="Review in cubic" src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a> <!-- End of auto-generated description by cubic. --> |
||
|
Paul Rastoin
|
1be5a0e54a |
System side effect relations (#22882)
Closes twentyhq/core-team-issues#2667 ## What Default relations to the standard relation objects (`timelineActivities`, `attachments`, `noteTargets`, `taskTargets`) are now fully owned by the **metadata side-effect engine**. Neither the API transpilers nor the SDK manifest builder provision them anymore: any object creation, rename or deletion — regardless of the caller — goes through the same engine handlers. ## Why - Provisioning was duplicated across the API path and the SDK manifest builder, with diverging behavior. - Universal identifiers of relation fields were derived from object **names**, so renaming an object mutated them and forced lossy delete+create cycles on manifest sync. ## How ### Engine-owned lifecycle (side-effect handlers) - `objectSystemRelationsOnCreate`: provisions the 8 forward/reverse relation fields (+ join column indexes) when an object is created. - `objectSystemRelationsOnUpdate`: renames the reverse morph fields (`target<ObjectName>`) when their host object is renamed — a lossless `fieldMetadata.update`. - `objectSystemSideEffectsOnDelete`: cascades deletion of engine-owned fields/indexes when the object is deleted. - The API transpilers and the SDK `buildManifest` no longer inject these fields; `isSystemSideEffect: true` marks engine-owned entities, guarded by a granular property allowlist (only `isActive` is user-editable) and excluded from manifest deletion inference. ### Name-free deterministic universal identifiers New `getSystemRelationFieldUniversalIdentifier({ applicationUniversalIdentifier, objectUniversalIdentifier, relationTargetObjectUniversalIdentifier })` in `twenty-shared`, exported from `twenty-sdk/define`. The identifier is keyed on the two **object** identifiers instead of field names (direction encoded by argument order), so object renames never mutate relation field identifiers. It cannot collide with the name-based `getFieldUniversalIdentifier` derivation (field names cannot contain `:`). ### twenty-standard re-owned All 48 forward/reverse system relation field declarations in `STANDARD_OBJECTS` now pin the derived name-free identifiers (computed inline via the shared util) and carry `isSystemSideEffect: true`, with labels/icons declared explicitly (translated via `msg`). `twenty-standard` is projected as if the engine had generated these fields itself. ### 2.23 upgrade commands - `reconcile-system-relation-field-universal-identifier`: structurally matches existing default relation fields per workspace and backfills the derived universal identifiers, `isSystemSideEffect` flags, and standard labels/icons. - `upgrade-people-data-labs-application`: upgrades installed PDL apps to `1.0.7` right after the backfill to close the desync window (its views reference the re-derived identifiers). ### Misc - `people-data-labs` `1.0.7`: views temporarily pin the new derived identifiers (TODO: import from the next released `twenty-sdk`). - `UpgradeStatusModule` split out of `UpgradeModule` so the application module cluster can consume upgrade status/migration services without importing the versioned command bundles (fixes a require cycle that crashed boot). - Docs: `system-fields.mdx` documents the system relation fields and their resolver; `sync-and-recovery.mdx` plan example no longer shows auto-injected relations. ## Known red CI `people-data-labs (dockerhub-latest)` fails by design until the 2.23 server image is published: the app pins the new identifiers which only exist on a 2.23 server. The `local` leg (server built from this branch) is green. ## System fields are no longer manifest-authorable (accepted regression) The manifest converter no longer derives `isSystem` / `isSystemSideEffect` from field names. Reserved-system-named manifest fields (`id`, `createdAt`, `updatedAt`, `deletedAt`, `createdBy`, `updatedBy`, `position`, `searchVector`) are now skipped at conversion time when they carry the exact derived universal identifier (keeps manifests built with older SDKs installable), and rejected with `INVALID_INPUT` when they pin any other identifier. System fields are therefore fully engine-canonical: nothing a manifest carries can produce a system-flagged entity anymore. **Accepted regression**: a manifest can no longer influence system field properties at all. Previously a (legacy) re-declaration could shape them at creation — which actually produced broken system fields, e.g. a nullable, non-unique `id` — and could still toggle the allowlisted `isActive` / `universalSettings` afterwards. We consider this acceptable for now: per-app granularity over system fields will be reintroduced later through the **override framework**, which will also settle update semantics by forbidding direct updates over `isSystemSideEffect: true` entities and expressing divergence as overrides. `isSystemSideEffect`-only entities (the default relation fields provisioned by this PR) still have no engine-level update guard (see Follow-up below); that part is unchanged and also lands with the overrides refactor. ## Follow-up `isSystemSideEffect` field update/delete guards intentionally live at the API layer (`sanitize-raw-update-field-input.ts`, `from-delete-field-input-...util.ts`) rather than in the engine-level `FlatFieldMetadataValidatorService`. Moving them into the validator requires threading operation-origin (direct field mutation vs engine cascade) through the migration matrix, otherwise legitimate object rename/delete cascades (which carry `isSystemBuild=false`) would be rejected. Tracked in twentyhq/core-team-issues#2671. <!-- This is an auto-generated description by cubic. --> <a href="https://cubic.dev/pr/twentyhq/twenty/pull/22882?utm_source=github" target="_blank" rel="noopener noreferrer" data-no-image-dialog="true"><picture><source media="(prefers-color-scheme: dark)" srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source media="(prefers-color-scheme: light)" srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img alt="Review in cubic" src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a> <!-- End of auto-generated description by cubic. --> |
||
|
martmull
|
8d84a0b9f3 |
feat(app): allow non-admin developers to claim and list marketplace apps (#22621)
## Context Follow-up to #22609. Lets a non-admin developer claim ownership of a public Twenty app they published to npm, then request a marketplace listing that a server admin reviews. Marketplace state is per-instance for now. ## Claiming - Developer tab gets a **Claim an application** section: look up an unclaimed npm app by package name or universal identifier. - Ownership is proven with GitHub OAuth against the package's npm provenance (trusted publishing): the connected account must own the GitHub account or organization the package was published from. - Errors from the GitHub callback come back as a code and are shown inline with a link to the relevant documentation. - The old one-click claim stays admin-only. - A **Sync catalog** button triggers a catalog refresh instead of waiting for the hourly cron. - Gated behind the `IS_APP_CLAIMING_ENABLED` feature flag. ## Listing requests - Catalog-synced apps are created **unlisted**; a data migration unlists previously auto-listed unclaimed npm apps (owned or vetted rows are left untouched). - Owners request a listing from the Distribution tab (logo + description required); a server admin approves or rejects it from a **Listing requests** section in the Admin Panel. ## Screenshots <img width="1512" height="829" alt="image" src="https://github.com/user-attachments/assets/788d4362-97c4-4e42-810c-ef1f11517bec"/> <img width="1512" height="829" alt="image" src="https://github.com/user-attachments/assets/d6246190-c82a-4f64-87be-3bb668527645"/> <img width="1512" height="828" alt="image" src="https://github.com/user-attachments/assets/21a8dad4-610b-4d1f-8948-b9acab40d373"/> <img width="1512" height="829" alt="image" src="https://github.com/user-attachments/assets/58246130-41f7-451e-ae7f-57bd21d04bb6"/> --------- Co-authored-by: cubic-dev-ai[bot] <191113872+cubic-dev-ai[bot]@users.noreply.github.com> |
||
|
martmull
|
baa84bb2e0 |
Add auto-upgrade-in-apps (#23001)
We need to auto upgrade application, lets add this column in application entity, and add an admin button to autoupgrade all applications to latest app registrration version manually <img width="1131" height="372" alt="image" src="https://github.com/user-attachments/assets/4e755abc-38ad-4895-a2e8-d55ee1948ac2" /> <img width="906" height="533" alt="image" src="https://github.com/user-attachments/assets/ce002057-0341-4581-bf9a-66ac2bd84a9b" /> |
||
|
Paul Rastoin
|
6b55a6b51c |
Fix duplicate searchFieldMetadata inserts in the 2.16 backfill upgrade command (#23060)
## Context
A self-hosted instance upgrading from 2.0.3 to v2.22.0 got stuck with
one workspace failing at `2.16.0_BackfillSearchFieldMetadataCommand`:
```
[QueryFailedError] duplicate key value violates unique constraint "IDX_SEARCH_FIELD_METADATA_OBJECT_FIELD_UNIQUE"
Detail: Key ("objectMetadataId", "fieldMetadataId")=(...) already exists.
```
The failure happened on a retry after a previous partial run, and
reproduced even though the command already recomputes
`flatSearchFieldMetadataMaps` before deriving the create-set (#22884).
## Root cause
The idempotency dedupe compares `(objectMetadataId, fieldMetadataId)`
pairs across two differently-fresh caches:
- The **existing rows** side comes from `flatSearchFieldMetadataMaps`,
which is recomputed from the database (real current ids).
- The **candidate** side resolves ids through `flatObjectMetadataMaps` /
`flatFieldMetadataMaps`, which are **not** invalidated. During a
cross-version upgrade these can be stale, since the migration runner
only invalidates the cache keys a migration touched.
When a stale map resolves a candidate to an outdated id, the dedupe key
doesn't match the existing row and the row is re-emitted. The migration
runner then re-resolves the universal identifiers against fresh maps at
execution time and inserts with the real current ids — exactly the pair
already committed by the earlier partial run (each per-application
migration commits independently) — tripping the unique constraint and
failing the upgrade.
## Fix
Two independent layers, either of which would have prevented the
failure:
1. **Consistent snapshot for the build phase**: the command now
invalidates and recomputes all three maps the dedupe depends on
(`flatObjectMetadataMaps`, `flatFieldMetadataMaps`,
`flatSearchFieldMetadataMaps`), so candidate resolution, existing-row
keys, and the runner all see the same database state.
2. **Id-churn-proof dedupe**: every row this command creates carries a
deterministic universal identifier (`getSearchFieldUniversalIdentifier`,
derived from application + field universal identifiers, no database ids
involved) and `(workspaceId, universalIdentifier)` is unique. The build
util now also skips any candidate whose deterministic universal
identifier already exists, catching leftovers from a previous partial
run even if objects/fields were recreated under new ids in between.
Deliberately **not** done: `ON CONFLICT DO NOTHING` in the create action
handler — it is shared by all runtime `searchFieldMetadata` creation,
and swallowing a conflict would leave the flat-entity cache holding an
entity id that differs from the row actually in the database.
## Test
Added a regression test reproducing the failure shape: an existing row
with the same deterministic universal identifier but stale metadata ids
must not be re-emitted by the backfill.
Note: `ReconcileSearchFieldMetadataCommand` (2.20) has the same
stale-cache exposure; hardening it is left to a follow-up.
<!-- This is an auto-generated description by cubic. -->
<a
href="https://cubic.dev/pr/twentyhq/twenty/pull/23060?utm_source=github"
target="_blank" rel="noopener noreferrer"
data-no-image-dialog="true"><picture><source
media="(prefers-color-scheme: dark)"
srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source
media="(prefers-color-scheme: light)"
srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img
alt="Review in cubic"
src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a>
<!-- End of auto-generated description by cubic. -->
|
||
|
Thomas Trompette
|
87729a2822 |
feat(workflow): backfill + dual-write for core workflow entity (#22776)
Workflow-side soft-ref sync — the `coreWorkflowId` mirror of the merged version side (#22821 / #22940 / #22944 / #22961). Rebased onto current main; supersedes the original shared-UUID version of this PR. ## What Gives `core.workflow` a per-workspace copy of each workflow (`name`, `lastPublishedVersionId`), soft-reffed from the workspace record via `coreWorkflowId`, so app-shipped workflows have a core home. Does not touch reads/dispatch (that's Phase B). - **Sync service** (`WorkflowCoreSyncService`): core rows get their own id (`uuidv5(workspaceId:recordId)`), the workspace record links via `coreWorkflowId` (written back after the upsert), and the write-back is **guarded** on the `coreWorkflowId` field being present (skips with a warning otherwise — mirrors #22940). Injected repo renamed `coreWorkflowRepository`. - **Dual-write listener** on the `workflow` object: CREATED/UPDATED/RESTORED upsert, DELETED/DESTROYED delete by `coreWorkflowId`. Always-on; failures routed to Sentry so they never break the user write. - **2-20 backfill** (`backfill-workflow-to-core`): reads via the provided `RunOnWorkspaceArgs.dataSource`, upserts each workspace workflow into core. - **2-22 provisioning** (mirrors #22944/#22961): - `add-workflow-core-soft-ref-field`: adds the `coreWorkflowId` system field on existing workspaces (flat-entity legacy migration). - `backfill-workflow-core-links`: full rebuild — per workspace, in one raw-SQL transaction, wipes all `core.workflow` rows, inserts a fresh own-id row per workflow, and re-links every record. (No trigger-map cache to invalidate on `core.workflow`.) Simpler than the version side: `core.workflow` has no one-active-per-workflow index and no trigger-map cache, and it was never backfilled in prod, so there are no legacy shared-id rows. ## Test Fresh `database:reset` + full sequence (2-20 backfill → 2-22 add-field → 2-22 rebuild link): 4/4 workspace records linked via `coreWorkflowId` (id != coreWorkflowId), links resolve, **0 dangling**, no duplicate core rows, names populated. Typecheck + lint + oxfmt clean. |
||
|
neo773
|
c04714b9e0 |
fix(messaging): register and harden webhook subscription renewal cron (#23006)
The renewal cron was never wired into cron:register:all, so Gmail/Calendar/Graph watches were never renewed and went dark ~7 days after connect (the max watch lifetime all three providers allow). Register it, fan renewals out as per-channel queue jobs scoped to active workspaces, retry FAILED channels, and recreate Google Calendar watches before stopping the old one. <!-- This is an auto-generated description by cubic. --> <a href="https://cubic.dev/pr/twentyhq/twenty/pull/23006?utm_source=github" target="_blank" rel="noopener noreferrer" data-no-image-dialog="true"><picture><source media="(prefers-color-scheme: dark)" srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source media="(prefers-color-scheme: light)" srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img alt="Review in cubic" src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a> <!-- End of auto-generated description by cubic. --> |
||
|
Charles Bochet
|
2e671342f5 |
[2/2] Write CREATED at activation, clean stale onboarding workspaces (#22915)
## Context Follow-up to #22904 (merged), which introduced the `CREATED` activation status (schema provisioned, onboarding incomplete — no billing subscription), its enum migration, and the read path. This PR turns the status on and closes the zombie-workspace leak (~60–110 subscription-less ACTIVE workspaces per day since v2 onboarding, 935+ total). ✅ **Deploy gating satisfied**: #22904's slow enum migration shipped with the release deployed to prod on 2026-07-17, so writing `CREATED` is now safe. Rebased on main (clean, no conflicts) — main's #22943/#22955 guarded-transition rework already handles `CREATED` correctly: the webhook suspend switch only suspends `ACTIVE` workspaces, and `reactivateWorkspace` promotes `CREATED`→`ACTIVE`. ## What this PR does 1. **Write path** — `activateWorkspace` sets `CREATED` instead of `ACTIVE` when the workspace has no billing subscription. `hasWorkspaceAnySubscription` returns true when billing is disabled, so self-hosted workspaces keep going straight to `ACTIVE` — no behavior change outside cloud. 2. **Cleanup** — `CREATED` joins `PENDING_CREATION`/`ONGOING_CREATION` in the existing onboarding cleaning flow (cron + `workspace:clean:onboarding` with `--dry-run`): workspaces older than the same seven-day threshold are soft-deleted, then hard-deleted on a later run. **No suspension step and no emails** — an abandoned onboarding is treated as never having completed, exactly like a workspace stuck in creation. A workspace that subscribes before cleanup exits the flow (`CREATED`→`ACTIVE` synchronously via checkout). 3. **Backfill** — slow instance command moving `ACTIVE` workspaces with no `billingSubscription` row, created since v2 onboarding shipped (2026-07-01), to `CREATED`. Gated on `IS_BILLING_ENABLED` so self-hosted instances are untouched. 4. **Resolves #22904's text-cast TODO on the billing activation update** — this PR only deploys after the enum migration, so the `CREATED`→`ACTIVE` promotion is a plain status-scoped update again. The upgrade-path filters (`activationStatusIn`) keep the `::text` cast: upgrade tooling has to run against databases coming from pre-2.22 versions, so its TODO now points at the real removal trigger (dropping pre-2.22 upgrade support). ## Ops note before deploying The backfilled zombies are all older than seven days, so the first cron run after the backfill **soft-deletes them and the next run destroys them (schema and data), with no user-facing communication**. The backfill also catches any post-July-1 cloud workspace that is ACTIVE without a subscription — including intentionally comped/demo/internal ones if any were created since then (verified locally: the seeded demo workspaces matched). **Run the backfill's SELECT as a dry-run against prod and review the list before deploying.** ## CI note ~~`cross-version-upgrade` (and its `ci-server-status-check` aggregate) is red due to a pre-existing regression on main — `Field metadata "coreWorkflowVersionId" is missing in object metadata workflowVersion` on the seed workspaces.~~ Resolved: the rebase picks up main's #22944/#22961 which fixed that regression. ## Verification Server-side (billing-enabled local instance, Stripe test mode) and through the full onboarding UI in both billing modes: - **Billing enabled, UI**: signup → workspace creation → **`activationStatus: CREATED`** in DB mid-onboarding → profile/invite steps work on the CREATED workspace → plan-required page → no-card trial → app loads, workspace **`ACTIVE`** with a `trialing` subscription (exercises #22904's synchronous promotion). - **Billing disabled, UI**: signup → workspace creation → **`ACTIVE` directly**, no plan step anywhere, app loads — self-hosted behavior unchanged. - **Cleanup**: a `CREATED` workspace backdated 8 days is listed by `workspace:clean:onboarding --dry-run`; the real run soft-deletes it silently (no suspension, no email) and the next run hard-deletes it (workspace row and schema gone). - **Backfill**: synthetic `ACTIVE` no-sub workspaces — created 2026-07-05 flips to `CREATED`, created 2026-06-15 stays `ACTIVE`, subscribed workspaces stay `ACTIVE`; billing-disabled short-circuit returns without touching anything. - Lint + typecheck green. |
