bd5d5a3c4e8df77ff434b0f5ed34b07cae728fd3
1189 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Raphaël Bosi
|
56245a35af |
Stop leaking the refresh token in the social SSO redirect URL (#23061)
The Google/Microsoft callback for a sign-in with no target workspace
redirected to `/sign-in-up?tokenPair={...}`, putting a 60-day refresh
token in a query string. Those persist in browser history, `Referer`
headers and access logs.
It now carries a single-use, 5-minute opaque token in the URL fragment,
which the frontend exchanges over POST. Browsers never send the fragment
on the wire, so the token stays out of access logs, proxies and
`Referer` headers entirely. Redemption claims the row with a `DELETE`
guarded on `revokedAt`/`deletedAt` being null, so concurrent requests
cannot each mint a refresh token and a revoked token cannot redeem.
Enterprise SSO (OIDC/SAML) already used a POST exchange and is
unchanged.
```mermaid
sequenceDiagram
participant Browser
participant Server
participant DB
Note over Browser,Server: before, the redirect carried access + 60-day refresh in ?tokenPair
Browser->>Server: GET /auth/google/redirect
Server->>DB: store sha256(token), expires in 5 min
Server-->>Browser: 302 /sign-in-up#ssoExchangeToken=opaque
Note over Browser: fragment never sent back to any server
Browser->>Server: POST getAuthTokensFromSSOExchangeToken
Server->>DB: guarded DELETE, single-use claim
Server-->>Browser: access + refresh token, in the response body
```
Since the token is single-use, the refresh token is minted at redemption
instead of at callback, so an abandoned redirect leaves an inert expired
hash rather than a live credential.
Redemption lives in its own `SignInUpSSOExchangeTokenEffect` +
`useRedeemSSOExchangeToken`, mirroring the existing
`VerifyLoginTokenEffect` + `useVerifyLogin` pair, so
`SignInUpGlobalScopeFormEffect` only loses the vulnerable branch. Like
`useVerifyLogin`, the hook clears any stale token pair before
exchanging. The effect reads `window.location.hash` live and strips it
synchronously, which doubles as the StrictMode double-invocation latch.
Remaining exposure is the browser itself (history until the synchronous
strip, client-side scripts), same as any fragment-based OAuth response.
`loginToken` on the workspace-targeted branch still travels as
`/verify?loginToken=` and is replayable for 15 minutes; moving it to the
fragment too is a separate change.
A fast instance command adds a unique partial index on `("type",
"value")` for live SSO exchange tokens, so redemption is an index lookup
instead of a full scan of the shared token table and at most one row can
ever match.
|
||
|
Weiko
|
a96dc335ab |
fix: apply configured pool size to core database (#23322)
## Context `PG_POOL_MAX_CONNECTIONS` is the server setting for the maximum number of PostgreSQL clients in a connection pool. The workspace primary and replica data sources already apply this setting, but the core TypeORM data source did not. Without an explicit `poolSize`, `node-postgres` uses its default limit of 10. As a result, deployments configured with a larger pool still kept the core pool at 10 connections per server process. During bursts of core database work, requests could therefore wait for a local pool connection even when PostgreSQL itself still had available capacity. That acquisition queue adds latency before the query starts, so database-level utilization alone does not reveal the bottleneck. ## What changes The core data source now applies: ```ts poolSize: Number(process.env.PG_POOL_MAX_CONNECTIONS ?? 10) ``` This makes the core data source consistent with the workspace data sources and with the documented meaning of `PG_POOL_MAX_CONNECTIONS`. ## Expected impact Deployments that configure a value above 10 can use that capacity for core database operations instead of queueing behind the driver's default limit. This targets short acquisition spikes affecting operations backed by the core database. The pool remains lazy, so this changes the maximum number of connections available to each process, it does not eagerly open every configured connection. ## Safety - Deployments without `PG_POOL_MAX_CONNECTIONS` keep the previous limit of 10. - Query behavior, transaction behavior, and timeouts are unchanged. - Workspace pool configuration is unchanged. - Operators remain responsible for choosing a value compatible with their total PostgreSQL connection budget and maximum server replica count. ## Scope This removes an unintended local connection-pool bottleneck. It does not address the source of synchronized database bursts, which should be handled separately by reducing unnecessary work. ## Testing Added a regression test that loads the core data source with `PG_POOL_MAX_CONNECTIONS=40` and verifies that TypeORM receives `poolSize: 40`. <!-- This is an auto-generated description by cubic. --> <a href="https://cubic.dev/pr/twentyhq/twenty/pull/23322?utm_source=github" target="_blank" rel="noopener noreferrer" data-no-image-dialog="true"><picture><source media="(prefers-color-scheme: dark)" srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source media="(prefers-color-scheme: light)" srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img alt="Review in cubic" src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a> <!-- End of auto-generated description by cubic. --> |
||
|
neo773
|
3fb29db28a |
Feat/email settings v2 (#23180)
Settings pages changes - Add `displayName` - Unsubscribers Page <img width="1496" height="844" alt="Screenshot 2026-07-22 at 8 52 15 PM" src="https://github.com/user-attachments/assets/69bc1993-4547-4a64-83a6-b47fef1a4e40" /> <!-- This is an auto-generated description by cubic. --> <a href="https://cubic.dev/pr/twentyhq/twenty/pull/23180?utm_source=github" target="_blank" rel="noopener noreferrer" data-no-image-dialog="true"><picture><source media="(prefers-color-scheme: dark)" srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source media="(prefers-color-scheme: light)" srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img alt="Review in cubic" src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a> <!-- End of auto-generated description by cubic. --> --------- Co-authored-by: Félix Malfait <felix.malfait@gmail.com> |
||
|
Etienne
|
3a1067ec6d |
fix(server): index page-layout FKs to fix workspace cleanup timeout (#23289)
The cleanSuspendedWorkspacesJob cron timed out every run (Sentry monitor
"a timeout check-in was detected"): hard-deleting soft-deleted
workspaces hung on `DELETE FROM core.pageLayout`, hit the 10s query
timeout, rolled back, so those workspaces were never destroyed and got
retried hourly.
Root cause: the FKs in the pageLayout -> pageLayoutTab ->
pageLayoutWidget tree had no usable index on the referencing column. The
existing indexes lead with workspaceId and are partial ("deletedAt" IS
NULL), so ON DELETE CASCADE / SET NULL fell back to full sequential
scans of the shared core tables per deleted row; on layout-heavy
workspaces this exceeded 10s.
- Add non-partial FK-column indexes on pageLayoutTab(pageLayoutId) and
pageLayoutWidget(pageLayoutTabId)
<!-- This is an auto-generated description by cubic. -->
<a
href="https://cubic.dev/pr/twentyhq/twenty/pull/23289?utm_source=github"
target="_blank" rel="noopener noreferrer"
data-no-image-dialog="true"><picture><source
media="(prefers-color-scheme: dark)"
srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source
media="(prefers-color-scheme: light)"
srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img
alt="Review in cubic"
src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a>
<!-- End of auto-generated description by cubic. -->
|
||
|
Paul Rastoin
|
bc7922da7d |
fix: move add-agent-foreign-key-to-role-target instance command to 2.25 (#23285)
## What Moves the `add-agent-foreign-key-to-role-target` fast instance command from `2.24.0` to `2.25.0`. Introduced in #23206, the command was registered under version `2.24.0`. Since `TWENTY_CURRENT_VERSION` is now `2.25.0`, `2.24.0` is an already-released version, so its instance commands do not re-run on upgrade and the foreign-key migration would never execute. This is the same issue #23271 fixed for the message-list-members backfill workspace command. ## Changes - Moved the command file from `upgrade-version-command/2-24/` to `2-25/` (renamed the file prefix). - Updated the decorator from `@RegisteredInstanceCommand('2.24.0', ...)` to `('2.25.0', ...)`. - Updated the import in `instance-commands.constant.ts` to the new relative path and reordered both the import and the array entry to sit after the 2-24 commands. The timestamp (`1784820332810`) and command logic (`up`/`down`) are unchanged. --- _Generated by [Claude Code](https://claude.ai/code/session_01KpCb9BK1eoM1WWXJU6g95e)_ <!-- This is an auto-generated description by cubic. --> <a href="https://cubic.dev/pr/twentyhq/twenty/pull/23285?utm_source=github" target="_blank" rel="noopener noreferrer" data-no-image-dialog="true"><picture><source media="(prefers-color-scheme: dark)" srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source media="(prefers-color-scheme: light)" srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img alt="Review in cubic" src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a> <!-- End of auto-generated description by cubic. --> |
||
|
martmull
|
25b0b2601f |
Replace admin app rollout buttons with upgrade-application CLI command (#23212)
## What Removes the two rollout buttons from the admin application detail page and replaces the upgrade flow with a CLI command that can be run directly from a server or worker pod. Also restructures application stop into its own module with a kill switch CLI command, and surfaces stopped apps in workspace settings. ### Removed - "Install on all workspaces" button (General tab, `SettingsAdminApplicationRegistrationGeneralToggles`), its confirmation modal and tooltip - "Upgrade existing installations" button (`SettingsApplicationRegistrationGeneralStats`), its confirmation modal and batch size input - `backfillApplicationInstallation` and `upgradeRegistrationApplications` admin GraphQL mutations and their frontend documents / generated types - `BackfillApplicationInstallationJob` (its only trigger was the removed mutation); `UpgradeApplicationsJob` is kept since the auto-upgrade flow still enqueues it Per review, the "install on all workspaces" flow is dropped without a CLI replacement for now; a dedicated command will be added when needed. ### application:upgrade command Located in `application-upgrade/commands`, registered in `ApplicationUpgradeModule`: ``` yarn command:prod application:upgrade \ --application-registration-universal-identifier <universalIdentifier> \ [--batch-size 5] \ [--workspace-id <id> --workspace-id <id2>] \ [--workspace-count-limit 10] \ [--dry-run] [--yes] ``` - `--workspace-id` (repeatable) restricts the upgrade to specific workspaces; `--workspace-count-limit` caps how many installations are upgraded (max 50, for canary rollouts) - `--batch-size` and `--workspace-count-limit` are validated as positive integers, max 50 - `--dry-run` reports how many (and which) workspaces would be upgraded, without upgrading - Without `--dry-run`, a confirmation prompt shows the app, target version and impacted workspaces; the run then executes exactly the confirmed set; `--yes` skips the prompt for non-interactive usage The upgrade plan is computed by a new `ApplicationUpgradeService.findApplicationsToUpgrade`, and batches run through a new `upgradeApplications` method — both reused by `upgradeAllApplications`, so the auto-upgrade job path is unchanged. ### Application kill switch (per review) Global mechanism only — a per-workspace stop had no demonstrated operational need and added a Redis key format, execution branching, CLI options and tests; an isolated workspace issue can be handled directly in the DB or Redis with the same effort. - `ApplicationStopService` moved to a dedicated `application-stop/` folder with its own `ApplicationStopModule` (imported and re-exported by `ApplicationModule`) - `stop` / `remove` methods that enable or clear the Redis-backed global kill switch; the logic function executor checks it before executing - `application:kill-switch` command with a positional action, confirmation prompt (shows the installation count) and `--yes` bypass: ``` # Enable the kill switch (stop is the default action) yarn command:prod application:kill-switch stop -u <universalIdentifier> [-y] yarn command:prod application:kill-switch -u <universalIdentifier> # Remove the kill switch yarn command:prod application:kill-switch remove -u <universalIdentifier> [-y] ``` ### Stopped apps surfaced in workspace settings (per review) - Dedicated `isApplicationStopped(applicationUniversalIdentifier)` query backed by the kill switch, fetched with `network-only` policy solely by the application detail page — listing applications triggers no extra Redis reads - Application detail page shows a danger banner when the app is stopped: "We are currently encountering issues with this app, its behavior may be degraded while we work on a fix." ## Test - `npx nx typecheck twenty-server` / `npx nx typecheck twenty-front` pass - `npx nx lint:diff-with-main` passes for both packages - `application-stop.service.spec.ts` covers stop, remove, caching and fail-open behavior - Verified end to end locally: ran the kill switch command on a seeded workspace and confirmed the banner renders on the app detail page (screenshot shared separately) |
||
|
Weiko
|
b036d67ec9 |
Configure async ClickHouse inserts for pageview events (#23274)
## Context Pageview tracking goes through the `trackAnalytics` mutation on the metadata API and is persisted through the unified event pipeline before the mutation resolves. ClickHouse inserts already use: ```text async_insert = 1 wait_for_async_insert = 1 ``` `async_insert` lets ClickHouse buffer and batch small inserts, but `wait_for_async_insert = 1` still keeps the API request open until that buffer is flushed successfully. For sparse pageview inserts, the buffer timeout can therefore account for most of the request duration and contribute to metadata API tail latency. ## What this changes - Adds a named `ClickHouseService.insert` option for overriding `async_insert_busy_timeout_max_ms`. - Caps the pageview buffer wait at 100 ms. - Keeps `wait_for_async_insert = 1`. - Leaves workspace, object, usage, application-log, and other event inserts on the existing default timeout. ## Why this approach This removes the avoidable buffer wait from the pageview request path without changing the delivery guarantees of the event pipeline. In particular, this does **not** use `wait_for_async_insert = 0` or fire-and-forget writes. The API still receives an acknowledgement only after ClickHouse flushes the pageview successfully, and insert/schema errors still propagate through the existing handling. The 100 ms value caps only the batching wait. It does not impose a 100 ms deadline on the complete ClickHouse request. ## Expected impact - Lower ClickHouse span duration for pageview tracking. - Lower tail latency for metadata API requests that emit pageviews. - No behavior or durability change for other event types. The trade-off is that pageviews may be flushed in smaller batches. The setting remains scoped to the pageview table so higher-value event streams keep their current batching behavior. ## Testing - Added coverage for the optional ClickHouse busy-timeout setting. - Added coverage verifying that only pageview inserts receive the 100 ms override. - Existing insert failure/retry behavior remains covered. - `twenty-server` typecheck passes. - Focused test result: 23 tests passed. ## Post-deploy verification - Compare pageview ClickHouse span p95/p99 before and after deployment. - Compare metadata API p95/p99. - Check ClickHouse asynchronous-insert failures. - Watch ClickHouse part creation and merge pressure for unexpected growth. <!-- This is an auto-generated description by cubic. --> <a href="https://cubic.dev/pr/twentyhq/twenty/pull/23274?utm_source=github" target="_blank" rel="noopener noreferrer" data-no-image-dialog="true"><picture><source media="(prefers-color-scheme: dark)" srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source media="(prefers-color-scheme: light)" srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img alt="Review in cubic" src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a> <!-- End of auto-generated description by cubic. --> |
||
|
Paul Rastoin
|
dee653dfa6 |
fix: move message list member backfill command to 2.25 (#23271)
## What
Moves the `backfill-message-list-members-junction-target` workspace
upgrade command from `2.24.0` to `2.25.0`.
Introduced in #23176 (commit
|
||
|
Paul Rastoin
|
ba5cb6ba15 |
fix(server): repair missing keyValuePair.applicationId on 2.23 upgrades (#23272)
Fixes #23254 ## Problem Upgrading a self-hosted instance from `2.23.x` to `2.24.0` leaves `core.keyValuePair` without the `applicationId` column. Database-backed config loading then fails on startup and on every refresh (~every 15s) with: ``` column KeyValuePairEntity.applicationId does not exist ``` The frontend shows "Unable to reach the backend". ## Root cause `AddApplicationIdToKeyValuePairFastInstanceCommand` was added in #23089 (after `2.23.x` shipped) but registered under the already-released `2.23.0` segment: ```ts @RegisteredInstanceCommand('2.23.0', 1784659343818) ``` The upgrade cursor is **positional and forward-only**: - `resolveStartCursor` resumes at `lastAttemptedIndex + 1`. A fully-upgraded `2.23.x` instance has its cursor at the last `2.23` workspace command, which sits *after* this newly-inserted fast command in the sequence. So the runner steps right over it and the DDL never runs. - The upgrade-aware metadata layer decides "applied" the same way (`stepIndex < currentCursor` in `upgrade-aware-entity-metadata.adapter.ts`). Since the step index is below the cursor, the column is considered applied and is **not** hidden from TypeORM SELECTs, so every query references a column that was never created. Fresh `2.24.0` installs replay the whole sequence, so only `2.23.x -> 2.24.0` upgrades are affected. The instance log `1 fast instance ... for 2.24.0` confirms the command landed in the `2.23.0` bundle rather than `2.24.0`. ## Fix - Add `RepairKeyValuePairApplicationIdFastInstanceCommand` under the current version (`2.24.0`) with a fresh timestamp, so it sorts last in the sequence and runs for every existing instance regardless of cursor position. Its DDL mirrors the original command and is fully idempotent (`ADD COLUMN IF NOT EXISTS`, `DROP INDEX IF EXISTS` + recreate, `ADD VALUE IF NOT EXISTS`), so it is a no-op on healthy instances. `down()` is intentionally empty: the column lifecycle is owned by the `2.23.0` introduction command. - Repoint the entity's `@WasIntroducedInUpgrade` to the new command so the column stays hidden from queries until the repair has actually run, eliminating the error window during the migration itself. ## Notes - `2.24.0` (`TWENTY_CURRENT_VERSION`) is the correct target: the upgrade sequence only covers previous + current versions, so a command under `2.25.0` (a next version) would not run. If a version bump lands before this merges, the command should be moved to the new current version. - Follow-up worth considering: nothing currently prevents registering a command under a version in `TWENTY_PREVIOUS_VERSIONS`. A startup validation rejecting that would have caught this at PR time. <!-- This is an auto-generated description by cubic. --> <a href="https://cubic.dev/pr/twentyhq/twenty/pull/23272?utm_source=github" target="_blank" rel="noopener noreferrer" data-no-image-dialog="true"><picture><source media="(prefers-color-scheme: dark)" srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source media="(prefers-color-scheme: light)" srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img alt="Review in cubic" src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a> <!-- End of auto-generated description by cubic. --> |
||
|
Weiko
|
abe4d7491c |
Cap nested relation query concurrency (#23252)
## Context Common API queries load selected relations after fetching the root records. Relation loading is batched: one query pipeline loads a relation for all parent records, so this is not an N+1 problem. However, every sibling relation currently starts concurrently through `Promise.all`. Nested relations repeat the same behavior recursively. A wide selection can therefore submit many independent relation query pipelines at once. Existing query complexity and record limits restrict what can be requested, but they do not limit how much database work starts concurrently. ## What this changes This PR adds a request-local FIFO concurrency limiter for nested relation loading. - At most four `findRelations` pipelines execute concurrently. - One limiter is created for the outer relation-loading call. - The same limiter is shared by every recursive level. - Queued work starts as permits become available. - Permits are released in `finally`, including when a query fails. Conceptually: ```text Before: all sibling relations -> database concurrently nested siblings -> more database work concurrently After: all sibling relations -> FIFO queue -> at most 4 database pipelines nested siblings -> same FIFO queue and same limit ``` Note: Also addressing https://github.com/twentyhq/twenty/pull/23251#discussion_r3644510597 |
||
|
Weiko
|
e5c9fcf058 |
Add PostgreSQL connection pool pressure metrics (#23251)
## Summary - Add pool gauges for total, idle, waiting, and maximum connections - Record PostgreSQL connection acquisition duration and failures - Instrument core, workspace primary, and optional replica data sources - Add unit tests covering gauges, acquisition timing, failures, and deduplication <!-- This is an auto-generated description by cubic. --> <a href="https://cubic.dev/pr/twentyhq/twenty/pull/23251?utm_source=github" target="_blank" rel="noopener noreferrer" data-no-image-dialog="true"><picture><source media="(prefers-color-scheme: dark)" srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source media="(prefers-color-scheme: light)" srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img alt="Review in cubic" src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a> <!-- End of auto-generated description by cubic. --> |
||
|
Abdul Rahman
|
2c79093b74 |
feat: agent roleUniversalIdentifier for manifest-driven role assignment (#23206)
## Summary - Adds optional `roleUniversalIdentifier` on `AgentManifest` / `defineAgent` so apps can declaratively assign a role to an agent (same config shape as `defaultRoleUniversalIdentifier`). - Wires `agentUniversalIdentifier` as a sync many-to-one FK on `roleTarget`, and emits a deterministic `roleTarget` from the agent during app sync (create / update / delete). - Enables app agents (e.g. Slack assistant) to get a role on install without postInstall hooks or manual admin assignment. <!-- This is an auto-generated description by cubic. --> <a href="https://cubic.dev/pr/twentyhq/twenty/pull/23206?utm_source=github" target="_blank" rel="noopener noreferrer" data-no-image-dialog="true"><picture><source media="(prefers-color-scheme: dark)" srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source media="(prefers-color-scheme: light)" srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img alt="Review in cubic" src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a> <!-- End of auto-generated description by cubic. --> |
||
|
neo773
|
59eead238d |
message list member backfill (#23176)
<!-- This is an auto-generated description by cubic. --> <a href="https://cubic.dev/pr/twentyhq/twenty/pull/23176?utm_source=github" target="_blank" rel="noopener noreferrer" data-no-image-dialog="true"><picture><source media="(prefers-color-scheme: dark)" srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source media="(prefers-color-scheme: light)" srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img alt="Review in cubic" src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a> <!-- End of auto-generated description by cubic. --> |
||
|
Paul Rastoin
|
b91c2a6457 |
fix(server): repair missing applicationRegistration.logoFileId on upgraded instances (#23215)
## Problem closes https://github.com/twentyhq/twenty/issues/23210 Self-hosted instances on 2.23.x fail their workspace upgrade with: ``` column ApplicationEntity__ApplicationEntity_applicationRegistration.logoFileId does not exist at UpgradePeopleDataLabsApplicationCommand.runOnWorkspace ``` The `2-21` instance command that adds `core."applicationRegistration"."logoFileId"` was merged ~20 minutes after the 2.22 version bump (PR #22827, `94192a2164`), so it first shipped in 2.22 while registered under `@RegisteredInstanceCommand('2.21.0', ...)`. The upgrade runner resolves its start position from the last recorded command and only moves forward. Any instance that had already run a 2.21.x binary has its cursor past that slot, so the command is skipped permanently and the column is never created. `UpgradeAwareEntityMetadataAdapter` decides column visibility positionally (`index < currentCursor`), not by whether the command actually ran, so it keeps `logoFileId` in the SELECT list and the instance reports "Up to date" while the column is absent. **Affected:** instances that ran 2.21.x, then upgraded to >= 2.22. Instances that went from <= 2.20 straight to >= 2.22 replayed the full sequence and are fine. `logoFileId` is populated lazily by design (NULL is a supported state), so no backfill is added. ## Changes **1. Idempotent DDL guard in the failing workspace command** `2-23-workspace-command-...-upgrade-people-data-labs-application.command.ts` now ensures the column exists at the top of `runOnWorkspace`, before the `findOne` that crashes on affected instances. It uses the core `DataSource` (`@InjectDataSource()`) because `core."applicationRegistration"` is instance-global, guards with a per-process boolean in addition to the SQL-level `IF NOT EXISTS`, and copies the full statement list (column + unique + FK constraints) verbatim from the 2.21 command. In dry-run it probes `information_schema.columns` and returns instead of running the crashing query. **2. Fast instance command in 2.23** New `2-23-instance-command-fast-1784823473532-add-logo-file-id-to-application-registration.ts`, registered at the end of the 2.23 fast segment (highest timestamp), running the same idempotent DDL. This covers the normal 2.22 -> 2.23 path and, critically, instances with zero provisioned workspaces where the workspace command body never executes. The shared DDL lives in `2-23/utils/ensure-application-registration-logo-file-id-column.util.ts` so both paths stay byte-for-byte identical. Class name follows the `Early2_4` / `Early2_5` precedent to avoid colliding with the 2.21 command. The fix lives entirely in 2.23: instances stuck at the failing workspace command retry it every run, and 2.22 -> 2.24 jumps still replay the 2.23 segment. ## Ops note Instances failing right now can be unblocked immediately by running the same `ALTER TABLE` block by hand against their core database (byte-for-byte what the command does). Worth including in the 2.23 patch release note. ## Verification - New fast instance command re-slotted last in the 2.23 fast segment (timestamp `1784823473532` > current max `1784659343818`). - Manual repro path: boot `twentycrm/twenty:v2.21`, seed, stop, run `upgrade` from this branch, assert the column exists and `upgrade:status` reports 0 failed. The default v1.22 baseline does not reproduce it (replays from cursor 0). --- _Generated by [Claude Code](https://claude.ai/code/session_01YAuDR585cx7FyAKoiT32j3)_ <!-- This is an auto-generated description by cubic. --> <a href="https://cubic.dev/pr/twentyhq/twenty/pull/23215?utm_source=github" target="_blank" rel="noopener noreferrer" data-no-image-dialog="true"><picture><source media="(prefers-color-scheme: dark)" srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source media="(prefers-color-scheme: light)" srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img alt="Review in cubic" src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a> <!-- End of auto-generated description by cubic. --> --------- Co-authored-by: Paul Rastoin <paul.rastoin@gmail.com> |
||
|
Thomas Trompette
|
96a2456367 |
feat(workflow): periodic core-consistency check for workflows, versions and triggers (#23103)
Monitoring for the soft-ref migration. The `workflow` /
`workflowVersion` dual-write into core is **best-effort** (async, not
transactional; failures only go to Sentry), so `core.workflow` /
`core.workflowVersion` can silently drift from the workspace source of
truth. This adds a periodic job that detects that drift across **all
three workflow entities** and emits it as metrics.
Supersedes the earlier inline shadow-parity approach that lived on this
branch — that only covered trigger dispatch and added a cache read +
diff to every cron tick and every DB-event batch (too much hot-path
overhead). This is broader and fully off the dispatch path.
## What
A cron (`cron:workflow:core-consistency-check`, every 3 hours, wired
into `cron:register:all`). Per run:
- **Bounded**: `SELECT DISTINCT "workspaceId" FROM core."workflow"` —
only workspaces that actually use workflows (skips the large majority).
- Per such workspace, emit a drift metric per `(entity, driftType)` —
detect-only, plus a triage log:
- **workflow** and **workflowVersion** — `unlinked` / `missingCore` /
`orphanCore` / `fieldMismatch`, via cross-schema `COUNT` aggregates
(core + workspace are the same DB, so indexed joins — no rows pulled
into JS).
- **automated triggers** — the `workflowAutomatedTrigger` table vs the
`workflowAutomatedTriggerMaps` cache: `inTableNotCache` /
`inCacheNotTable` / settings `mismatch`.
- Per-workspace failures are isolated (caught → Sentry) so one bad
workspace does not stop the sweep.
## Why it is efficient
Two central `core.*` queries + a few `COUNT` queries per
*workflow-using* workspace, on a relaxed cadence, off the dispatch path.
Shardable across ticks later if needed.
## Metrics
`workflow-core-consistency/{workflow,version,automated-trigger}/drift`
counters, attribute `driftType`. Dashboards: twentyhq/twenty-infra#800.
## Not in scope
Detect-only — no auto-heal (the existing backfill/rebuild command can
heal). No dispatch or flag changes.
## Test
- The consistency SQL (workflow/version sync counts, orphan counts,
trigger read) validated against a live workspace (it surfaced real drift
there — unlinked versions + an orphan core version). The whole
cron→service→SQL→metric pipeline is proven live: the cron is already
emitting real drift counters on a running server.
- Unit specs for the service (clean → no metric; per-entity drift per
dimension; per-workspace error isolation).
- Command boots and registers via `cron:register:all` (verified).
Typecheck + lint clean.
|
||
|
Weiko
|
66df0ac47c |
Switch application stop/start commands to Redis-backed global kill switch (#23202)
## Context [#23183](https://github.com/twentyhq/twenty/pull/23183) introduced the right enforcement point: every logic-function execution is rejected centrally before consuming the shared workspace throttle when its application is stopped. However, its server-wide path reads PostgreSQL for every execution attempt. A kill switch is most useful while an application is producing abnormal load, potentially while PostgreSQL is already under pressure. The enforcement mechanism should not add more database traffic in that situation. This state is also operational and temporary. It is used to troubleshoot an application, not as durable application configuration. ## What this PR changes - Uses one global Redis key per application universal identifier: ```text module:applications:kill-switch:{applicationUniversalIdentifier} ``` - Keeps the check in `LogicFunctionExecutorService`, before the workspace execution throttle. - Adds a 60-second process-local cache for both present and absent keys. - Deduplicates concurrent cache refreshes, so an execution burst causes at most one Redis read per application and process. - Fails open when Redis cannot be read and caches that result for the same minute, avoiding a Redis retry storm. - Removes the database columns, upgrade command, workspace-cache recomputation, registration lookup, and stop/start CLI commands introduced by #23183. - Keeps disabled queued executions non-retriable, without emitting one warning for every skipped payload. The switch is operated directly in Redis. For example: ```redis SET module:applications:kill-switch:{applicationUniversalIdentifier} 1 EX 3600 DEL module:applications:kill-switch:{applicationUniversalIdentifier} ``` Any value means stopped; deleting or expiring the key means enabled. ## Why this is a better fit | | #23183 | This PR | |---|---|---| | State | Durable PostgreSQL fields | Ephemeral Redis key | | Server-wide hot path | PostgreSQL lookup per execution | At most one Redis lookup per app/process/minute | | Scope | Workspace and application registration | Application universal identifier across all workspaces | | Operational cleanup | Explicit start command | `DEL`, eviction, restart, or operator-selected TTL | | Database dependency during an incident | Required | None | The trade-off is deliberate: a Redis change can take up to 60 seconds to reach every process, and the switch is lost when the cache key disappears. That is acceptable for a temporary troubleshooting control and keeps the normal execution path inexpensive. Existing in-flight functions are not interrupted. New direct or queued executions are rejected when they reach the executor. |
||
|
martmull
|
a3f4acadb1 |
Add workspace and server level stop commands for applications (#23183)
## Context When an installed application misbehaves (e.g. a logic function loop DDoSing the server or the database), we currently have no targeted way to shut it down in production: the only kill switch is `LOGIC_FUNCTION_TYPE=DISABLED`, which disables logic functions for the whole instance. This PR adds an emergency stop mechanism at two levels: - **Workspace level**: stop one installed application in its workspace. - **Server level**: stop every application installed from an `applicationRegistration`, across all workspaces. ## How it works **New nullable `stoppedAt` columns** on `core.application` and `core.applicationRegistration` (fast instance command `2.24.0`, with `up`/`down` and `@WasIntroducedInUpgrade` decorators on the entities). **Enforcement in a single choke point**: `LogicFunctionExecutorService.execute()` is the funnel behind every execution path (public route triggers, server route triggers, cron triggers, database event triggers, workflow actions, agent tool calls, manual GraphQL execution, install hooks). A new `assertApplicationNotStopped` guard runs right after the flat entities are resolved and throws `LOGIC_FUNCTION_DISABLED` (already mapped to a 403 on route triggers and handled by the GraphQL exception handler) when: - `flatApplication.stoppedAt` is set (workspace-level stop, read from the cached flat application maps: zero extra runtime cost), or - the linked registration is stopped (one indexed PK lookup, same pattern as the existing per-execution server-variable query). **Propagation**: the workspace-level stop invalidates and recomputes `flatApplicationMaps` for the workspace, so all server instances pick the flag up within the local cache TTL (100ms). The registration-level flag is read live, so it is effective immediately. ## Ops commands ```bash # Workspace level yarn command:prod application:stop -a <application-id> yarn command:prod application:start -a <application-id> # Server level (all applications of the registration, all workspaces) yarn command:prod application-registration:stop -r <application-registration-id> yarn command:prod application-registration:start -r <application-registration-id> ``` Each command logs what was stopped/started and, for registrations, how many installed applications are affected. ## Notes - Stopped executions fail fast at the guard, so queued trigger jobs (cron/db-event) burn a negligible amount of work while stopped. - The two flags are independent: lifting a registration-level stop does not clear workspace-level stops that were set individually, and vice versa. - Unit tests added for `ApplicationStopService`. --- _Generated by [Claude Code](https://claude.ai/code/session_01CjEnKUACn89aSgK1wEMH2d)_ <!-- This is an auto-generated description by cubic. --> <a href="https://cubic.dev/pr/twentyhq/twenty/pull/23183?utm_source=github" target="_blank" rel="noopener noreferrer" data-no-image-dialog="true"><picture><source media="(prefers-color-scheme: dark)" srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source media="(prefers-color-scheme: light)" srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img alt="Review in cubic" src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a> <!-- End of auto-generated description by cubic. --> |
||
|
Abdul Rahman
|
04d1c2035c |
feat(connections): run a logic function on connection provider connect (#23167)
## What Adds an optional `onConnectLogicFunctionUniversalIdentifier` field to the connection provider manifest. When set, the referenced logic function is dispatched right after an OAuth connection is successfully established for that provider. This gives apps a first-class "on connect" hook — e.g. the Slack app can resolve the workspace's `team_id` via `auth.test` and claim the `team_id -> workspaceId` mapping in the SERVER key-value store immediately on connect, instead of racing against later events. Follow-up to the app key-value store PR (#23089). ## How - **twenty-shared**: add `onConnectLogicFunctionUniversalIdentifier` to `ConnectionProviderManifest`. - **twenty-sdk**: expose the field in `defineConnectionProvider` and validate it is a UUID `universalIdentifier`. - **twenty-server**: - add a nullable `onConnectLogicFunctionUniversalIdentifier` column to `ConnectionProviderEntity` (+ fast instance command / migration). - map the field through the <!-- This is an auto-generated description by cubic. --> <a href="https://cubic.dev/pr/twentyhq/twenty/pull/23167?utm_source=github" target="_blank" rel="noopener noreferrer" data-no-image-dialog="true"><picture><source media="(prefers-color-scheme: dark)" srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source media="(prefers-color-scheme: light)" srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img alt="Review in cubic" src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a> <!-- End of auto-generated description by cubic. --> |
||
|
Paul Rastoin
|
9043ab3091 |
Upgrade to 1.0.9 PDL app (#23172)
<!-- This is an auto-generated description by cubic. --> <a href="https://cubic.dev/pr/twentyhq/twenty/pull/23172?utm_source=github" target="_blank" rel="noopener noreferrer" data-no-image-dialog="true"><picture><source media="(prefers-color-scheme: dark)" srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source media="(prefers-color-scheme: light)" srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img alt="Review in cubic" src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a> <!-- End of auto-generated description by cubic. --> |
||
|
Abdul Rahman
|
4c4a154d31 |
key-value storage for applications (#23089)
## What
Key-value storage for applications, as proposed in
twentyhq/core-team-issues#2391 — built on the existing `keyValuePair`
entity:
- Nullable `applicationId` relation on `keyValuePair` + a new
`APPLICATION_VARIABLE` type (fast instance command included)
- GraphQL CRUD on the metadata schema (`appKeyValue`, `setAppKeyValue`,
`deleteAppKeyValue`), requiring an `APPLICATION_ACCESS` token —
`applicationId` always comes from the token, never from arguments, so
apps can't touch each other's entries
- `kv.get` / `kv.set` / `kv.delete` helpers in
`twenty-sdk/logic-function`
## Scopes
- **`INSTALL`** (default): entries are private to one workspace install;
arbitrary JSON values
- **`GLOBAL`**: entries are shared across every install of the app, with
claim semantics — the value is always the claiming `workspaceId` and
only that workspace can overwrite or delete the key (guarded writes,
race-safe via insert-if-absent)
Since `applicationId` identifies an install (one row per workspace),
GLOBAL entries are stored under the registration owner workspace's
install so all installs of the same app share one namespace.
The GLOBAL scope is what enables cross-workspace webhook routing: e.g.
the Slack app's `serverRoute` resolver (running in the owner workspace)
can resolve `kv.get('slack:team:' + team_id, { scope: 'GLOBAL' })` to
find the workspace that connected that Slack team — without a workspace
being able to hijack another's mapping.
## Follow-ups
- Wire the Slack assistant PR (#22984) to write the claim at connect
time and read it in the events resolver
- `kv.*` access from front components
<!-- This is an auto-generated description by cubic. -->
<a
href="https://cubic.dev/pr/twentyhq/twenty/pull/23089?utm_source=github"
target="_blank" rel="noopener noreferrer"
data-no-image-dialog="true"><picture><source
media="(prefers-color-scheme: dark)"
srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source
media="(prefers-color-scheme: light)"
srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img
alt="Review in cubic"
src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a>
<!-- End of auto-generated description by cubic. -->
|
||
|
Paul Rastoin
|
893db7558e |
Fix instance fast migration, fallback index creation (#23149)
<!-- This is an auto-generated description by cubic. --> <a href="https://cubic.dev/pr/twentyhq/twenty/pull/23149?utm_source=github" target="_blank" rel="noopener noreferrer" data-no-image-dialog="true"><picture><source media="(prefers-color-scheme: dark)" srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source media="(prefers-color-scheme: light)" srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img alt="Review in cubic" src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a> <!-- End of auto-generated description by cubic. --> |
||
|
Etienne
|
5add5ea695 |
fix(billing) - expand billing sub uniqueness constraint (#23123)
fixes : https://discord.com/channels/1130383047699738754/1528956737363771497 <!-- This is an auto-generated description by cubic. --> <a href="https://cubic.dev/pr/twentyhq/twenty/pull/23123?utm_source=github" target="_blank" rel="noopener noreferrer" data-no-image-dialog="true"><picture><source media="(prefers-color-scheme: dark)" srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source media="(prefers-color-scheme: light)" srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img alt="Review in cubic" src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a> <!-- End of auto-generated description by cubic. --> |
||
|
Weiko
|
23cf85f745 |
Fix invalid UUID insert in workflow core-links backfill (#23118)
## Fix invalid UUID insert in workflow core-links backfill ### Problem The `2-23:backfill-workflow-core-links` workspace upgrade command failed with: ``` QueryFailedError: invalid input syntax for type uuid: "" (22P02) ``` The `core."workflow"."lastPublishedVersionId"` column is a `uuid`, but some workspace workflows store an empty string `""` (not `NULL`) for that field. The code used `workflow.lastPublishedVersionId ?? null`, and `??` only falls back on `null`/`undefined` — so `""` was passed straight through and Postgres rejected it. ### Fix Use `|| null` instead of `?? null` so empty strings are normalized to `null` before insertion into the `uuid` column. <!-- This is an auto-generated description by cubic. --> <a href="https://cubic.dev/pr/twentyhq/twenty/pull/23118?utm_source=github" target="_blank" rel="noopener noreferrer" data-no-image-dialog="true"><picture><source media="(prefers-color-scheme: dark)" srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source media="(prefers-color-scheme: light)" srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img alt="Review in cubic" src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a> <!-- End of auto-generated description by cubic. --> |
||
|
Paul Rastoin
|
71a1ff7ac8 |
Cache twenty-client-sdk modules host-side via content-addressed URLs (#22981)
## Context Front component sources are fetched host-side and integrity-verified by the SHA-256 checksum embedded in their URL, cached in Cache Storage — a layer that exists specifically because their download URLs are presigned and rotate. The `twenty-client-sdk` modules (`core` and `metadata`) were re-fetched on every render and could not be cached safely: their URLs carried no checksum and the server exposed no freshness signal. This PR makes the SDK module URLs **content-addressed** and relies on the **browser HTTP cache** for immutability, and it keys the checksums on their real owners: the **application** for `core`, the **instance** for `metadata`. The checksum does double duty: cache invalidation (regeneration changes the checksum → the URL changes → guaranteed cache miss) and a server-side cacheability guard (the server only grants `immutable` when the checksum in the URL matches the authoritative checksum it knows for that module — persisted at generation time for `core`, hashed once at bootstrap for `metadata` — so no per-request hashing of the served bytes). Note this is **not** an end-to-end integrity guarantee: there is no client-side hash verification, and on a fingerprint mismatch the server still serves the current bytes with `no-store` (self-healing for stale URLs) rather than failing. <img width="2412" height="926" alt="image" src="https://github.com/user-attachments/assets/d97935d2-0fdb-4c44-89ac-596b7ca8ca64" /> Closes twentyhq/core-team-issues#2688. ## Routes | Module | URL | Scope | | --- | --- | --- | | `core` | `/rest/sdk-client/{applicationId}/core[/{checksum}]` | Per application (generated bundle) | | `metadata` | `/rest/sdk-client/metadata[/{checksum}]` | **Instance-wide**: no application segment, so every application converges on one URL and the browser downloads the module once per release instead of once per application | The previous application-scoped metadata path (`/rest/sdk-client/{applicationId}/metadata[/{checksum}]`) is **kept for backward compatibility**, new clients just stop generating those URLs. The instance-wide route is declared before the parameterized route so `metadata/{checksum}` is not swallowed as `:applicationId/:moduleName`. ## Caching model | Request | `Cache-Control` | Effect | | --- | --- | --- | | Fingerprinted URL, checksum matches the known module checksum | `immutable` | Cached indefinitely by the browser HTTP cache; a new checksum is a new URL | | Fingerprinted URL, checksum does not match | `no-store` | Current bytes served uncached (self-healing for stale URLs) | | Bare URL (pre-generation fallback, `core` only in practice) | `no-store` | Never cached | - Both responses also set `X-Content-Type-Options: nosniff` and `Content-Type: application/javascript`. - SDK modules are intentionally **not** placed in Cache Storage. That layer stays reserved for the presigned/rotating component-source URLs; SDK modules are served directly and authenticated, so the browser HTTP cache (keyed by the content-addressed URL) is their single cache layer. ## Checksum provenance - **core** — per **application**, persisted on `application.sdkClientCoreChecksum` at generation time and read back from `flatApplicationMaps` (never re-hashed per request). - **metadata** — **instance-wide**, hashed once from the installed `twenty-client-sdk/dist/metadata.mjs` package (warmed at bootstrap, memoized per process) and served straight from that package, so it is fresh from the first request after a release with no archive dependency. ## Server (twenty-server) - Hash `dist/core.mjs` at SDK generation and persist `sdkClientCoreChecksum` via `applicationRepository.update`. Adds the nullable text column to `application.entity.ts` (mirroring `packageJsonChecksum`) plus a fast instance command with up/down; `FlatApplication` picks it up automatically. - New **application-scoped** query `applicationSdkClientChecksums(applicationId: UUID!): SdkClientChecksums` on `ApplicationResolver` (metadata schema, `WorkspaceAuthGuard` + `NoPermissionGuard`). `SdkClientChecksums.core` is **nullable** and stays `null` until the SDK has been generated at least once; `metadata` is **always present** (bootstrap-warmed), so the metadata module is cacheable from the very first render of any app. The query itself returns `null` only for unknown applications. - `SdkClientChecksumsDTO` now lives in the shared `core-modules/sdk-client/dtos/`. `FrontComponentDTO` and the `frontComponent` resolver no longer carry checksums (decoupled from the front-component row). - `sdk-client` controller: instance-wide `metadata[/:checksum]` route (no workspace-cache or application lookup, serves the memoized installed module) + application-scoped `:applicationId/:moduleName[/:checksum]` route (serves `core` from the per-application archive, `metadata` kept for back-compat). Cacheability compares the URL checksum against the **known** checksum — persisted `sdkClientCoreChecksum` for `core`, memoized package hash for `metadata` — instead of hashing the served bytes on every request: `immutable` on match, `no-store` otherwise (bare URL or stale fingerprint), plus `nosniff`. A persisted checksum out of sync with the archive only downgrades to `no-store` until the next regeneration. ## Front (twenty-front) - New metadata query `GetApplicationSdkClientChecksums`, keyed by `applicationId`; removed the `sdkClientChecksums` selection from `FindOneFrontComponent`. - `getSdkClientUrls` builds the two module URLs independently: `/sdk-client/{applicationId}/core/{checksum}` and the **instance-wide** `/sdk-client/metadata/{checksum}` (no application segment → one shared browser cache entry per release across all applications). Each falls back to its bare URL when its checksum is absent — since `core` is nullable, a never-generated app still gets a content-addressed metadata URL and only `core` falls back. The checksum type is sourced from the codegen `SdkClientChecksums` type rather than a hand-maintained duplicate. - `FrontComponentRenderer` is split into a gating outer component (runs `FindOneFrontComponent`, renders nothing while loading) and a content component that receives a guaranteed-non-null `frontComponent`. Following project conventions, the side effects live in dedicated effect components: `FrontComponentLoadErrorSnackBarEffect` (query error → snackbar) and `FrontComponentApplicationTokenPairEffect` (mirrors the query-derived token pair into component state unconditionally, `null` included, so revoked credentials can never be retained or refreshed). The content component fetches checksums via the application-keyed query and **gates the mount of SDK-using components on that query**, so the very first module fetch is always the content-addressed (`immutable`) URL instead of the bare `no-store` one. Non-SDK components skip the query and are never blocked. - **Live invalidation without reload:** SDK regeneration updates the application row, and the server broadcasts an `application` metadata event carrying the new core checksum. `useOnApplicationSdkClientChecksumsUpdated` / `useUpdateSdkClientChecksumsApolloCache` patch the application-keyed checksum query cache (core only; the instance-wide metadata is preserved), so every mounted component of that application picks up the new URL at once. This replaces the previous frontComponent-derived field and closes the earlier "known gap" (a mounted component staying on a session-old checksum until a full reload). The cache-patching callback is memoized (`useCallback`) so the window listener is registered once per application, and the listener is **skipped entirely** for non-SDK components (`useListenToMetadataOperationBrowserEvent` gained a `skip` option) — they register no listener and never refetch a query they don't consume. ## Renderer (twenty-front-component-renderer) - SDK sources are fetched through a dedicated plain authenticated fetch, `fetchJavaScriptModuleSourceText` (Bearer header, `credentials: 'omit'`), instead of the Cache Storage `fetchComponentSource` path; `fetchSdkClientSources` uses it. Execution stays exclusively in the opaque-origin worker via blob URLs; the host only fetches and forwards source strings (no hashing host-side). Staleness self-resolves through the checksum: new checksum → new URL → cache miss. <!-- This is an auto-generated description by cubic. --> <a href="https://cubic.dev/pr/twentyhq/twenty/pull/22981?utm_source=github" target="_blank" rel="noopener noreferrer" data-no-image-dialog="true"><picture><source media="(prefers-color-scheme: dark)" srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source media="(prefers-color-scheme: light)" srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img alt="Review in cubic" src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a> <!-- End of auto-generated description by cubic. --> |
||
|
Félix Malfait
|
3ad3e8bd1a |
feat: kanban, calendar and group-by table layouts for dashboard view widgets (#22963)
## Context Dashboard view widgets previously only rendered flat tables. This PR ships the full feature: **Table with group-by**, **Kanban**, and **Calendar** layouts for dashboard view widgets — server API + frontend, end-to-end. (Originally staged as a 4-PR stack — #22966, #22967, #22968 — consolidated here per review.) ## Server / API - **View typing.** Adds `KANBAN_WIDGET` and `CALENDAR_WIDGET` to `ViewType` (following the `TABLE_WIDGET` precedent) so widget-backing views keep their layout in `view.type` while staying excluded from record-index pickers. Shared `getViewLayoutFromViewType()` maps widget types to their base layout; `isWidgetViewType()` centralizes the exclusions that were previously hardcoded per-site. - **Migrations.** Two fast instance commands (**2.23**): `ALTER TYPE core.view_type_enum ADD VALUE` for both values, and a widened `CHK_VIEW_CALENDAR_INTEGRITY` constraint covering `CALENDAR_WIDGET` (entity `@Check` updated for fresh installs). - **Validation.** `FlatViewValidatorService` keys kanban/calendar validation on the mapped layout, so widget views get the same invariants as index views (kanban needs a groupable group-by field; calendar needs a date field + layout). Calendar widget views default to month; a non-month (DAY/WEEK) layout is rejected at the API level **unless** the `IS_CALENDAR_WEEK_VIEW_ENABLED` feature flag is enabled for the workspace — the same flag that gates day/week on index calendars. - **API.** `upsertViewWidget` (LAYOUTS permission) accepts a nested `view` settings input (`type`, `mainGroupByFieldMetadataId`, `shouldHideEmptyGroups`, kanban aggregate/column-width, calendar layout/fields). Routes through the standard update path, so `viewGroups` auto-generate from SELECT options exactly like index views. Only widget view types accepted; only `RECORD_TABLE` widgets can change view settings. - **AI tools.** `create-complete-dashboard` + `create_view` now use/allow the `*_WIDGET` types (previously they created plain `TABLE` views that leak into index pickers). ## Frontend **Settings panel.** The **Source** (object) row comes first, since which layouts are available depends on it. The **Layout** row below is a working dropdown (Table / Kanban / Calendar); layouts the source object can't support are **disabled with a hint** ("Needs a Select field" / "Needs a Date field") rather than hidden. Group-by row (select fields; searchable) with a **Hide empty groups** toggle while grouped; **Date field** row replaces Group by while Calendar is active, and — when the `IS_CALENDAR_WEEK_VIEW_ENABLED` flag is on — a **Calendar view** row (Day / Week / Month) appears beside it; **Limit** row hidden while grouped (only the flat virtualized loader enforces it). Kanban keeps its group-by locked (no `None` option). **Instant edit-mode preview.** Draft snapshots carry `viewGroups`; picking a group-by synthesizes them client-side (`buildDraftViewGroupsForFieldMetadataItem`, mirroring the server's generation), so grouped tables/boards preview immediately before dashboard save. On save, `upsertViewWidget` responses hand back the server-generated groups, which replace the client-generated ones in the persisted snapshot. **Renderers.** `RecordTableWidgetRendererContent` branches on the backing view's layout: `RecordBoardWidget` (wraps the standard `RecordBoardContainer`) and `RecordCalendarWidget` (mounts the existing `RecordCalendar`, which renders month / day / week) inside the same per-widget provider sandbox the table uses. **Read-only semantics.** Two flags with distinct scopes, each documented on its state: - `isRecordBoardViewSettingsReadOnlyComponentState` — locks the board chrome that edits view settings (add group, column reorder/resize/menu, aggregates); **card drag still updates records** under object permissions. - `isRecordCalendarReadOnlyComponentState` — widget calendars are read-only by default (no drag, no add-new, no in-calendar layout switch); cards open the side panel. The one exception, behind `IS_CALENDAR_WEEK_VIEW_ENABLED`: a **live (non edit-mode) day/week** widget calendar allows drag-to-reschedule and record creation under object permissions. Month calendars and edit-mode previews stay read-only. **Calendar state componentization.** The calendar module's three settings move from global atoms to component states keyed on `RecordCalendarComponentInstanceContext` (same pattern as record-board), so several calendar widgets and an index-page calendar can coexist without leaking state. All readers resolve the ambient instance; calendar unit tests updated. **Multi-instance fixes that also fix index pages:** record drag states were written against a different instance than every reader resolves (now use the ambient instance); the board sticky-header DOM id is namespaced per board; dragged board cards portal to `document.body` while dragging so react-grid-layout's transforms can't offset the clone from the pointer. ## Scope (v1) - Widget calendars are month-only and read-only by default. With `IS_CALENDAR_WEEK_VIEW_ENABLED` enabled, day/week layouts become selectable (UI + API) and live day/week widget calendars support drag-to-reschedule and record creation under object permissions. - Widget group-by offers SELECT fields only (server auto-generates groups from options; widgets have no per-record add-group flow). ## Tests - Integration: `upsert-view-widget-view-settings.integration-spec.ts` (9 tests — group auto-creation, invalid type/field rejections, non-month calendar widget rejected while the week/day flag is off and accepted once it's enabled, combined settings+fields call); pre-existing `upsert-view-widget` suite (20) green. - Front: new suites for draft view-group generation and snapshot clone/build utils; calendar suites componentized; full `twenty-front` jest, typecheck, oxlint green; `twenty-server` typecheck + lint green. - Browser-verified end-to-end (real dev server + seeded workspace): configure → live edit-mode preview → save → reload for all three layouts; measured drag with pointer inside the card; index-page calendar re-verified (with the week/day flag enabled). https://claude.ai/code/session_01E5N87kwwZWhDtEQaP72cMf |
||
|
Paul Rastoin
|
2ef7b7824e |
Upgrade call-recorder, people-data-labs, last-contact and partners apps to twenty-sdk 2.23.0-alpha.1 (#23098)
## What Upgrades the two breaking-change-prone apps to `twenty-sdk` / `twenty-client-sdk` `2.23.0-alpha.1`, and adds the server-side hook that lets the 2.23 upgrade install them: - **people-data-labs** - **partners** Follows up on #22882 (System side effect relations), which re-derived the system relation field universal identifiers name-free and shipped `getSystemRelationFieldUniversalIdentifier` in the SDK. ## How - **people-data-labs**: bump the SDK to `2.23.0-alpha.1`. The enriched views temporarily hardcoded the new system relation identifiers with a TODO because the SDK still embedded the old values; now that the name-free identifiers ship in `2.23`, derive them from `STANDARD_OBJECT_UNIVERSAL_IDENTIFIERS.{company,person}.fields.{noteTargets,taskTargets,attachments,timelineActivities}.universalIdentifier` (identical to the previously pinned values, verified). Engine already pinned `twenty >=2.23.0`; app stays `1.0.7` (manifest unchanged). - **partners**: bump the SDK to `2.23.0-alpha.1`. The partner role references `opportunity.fields.{taskTargets,noteTargets,attachments,timelineActivities}` universal identifiers, which the SDK now resolves to the `2.23` name-free values. Pin `engines.twenty >=2.23.0` and bump the app to `1.3.1`. - **server**: add an opt-in `skipWorkspaceCompatibilityCheck` to the install/upgrade path. The `upgrade-people-data-labs-application` 2.23 command runs mid-upgrade, before the workspace is marked as having completed 2.23, so the workspace-compatibility check would otherwise reject installing `1.0.7` (`engines >=2.23.0`). The server is already on 2.23, so the command passes the flag to install `1.0.7` and close the desync window. Version-progression (downgrade/same-version) checks still run. - **call-recorder** and **last-contact** are intentionally left unchanged (reverted): they don't define custom objects and don't reference the system relation identifiers, so they aren't breaking-change-prone and need no SDK bump. ## Breaking change constraints - **people-data-labs** and **partners** reference system relation identifiers that only exist on a `2.23` server, so both pin `engines.twenty >=2.23.0`. Their `dockerhub-latest` integration leg is red by design until a >=2.23 server image is published (same accepted state as #22882); the `local` leg is green. ## Validation - Regenerated the app lockfiles against the published `2.23.0-alpha.1`. - `people-data-labs` typechecks cleanly against the real `2.23` SDK types. - CI: people-data-labs and partners green on `local`, red on `dockerhub-latest` by design; server/SDK/all other checks green. - Rebased onto latest `main`. |
||
|
Marie
|
e5fc5054cc |
Fix "Go to roles settings" command (#23105)
**Fix the "Go to Roles Settings" command** — it pointed at the non-existent `/settings/roles` route and now navigates to `/settings/members#roles`. **Backfill existing workspaces** — added the `upgrade:2-23:fix-go-to-roles-settings-command-menu-item-path` workspace command, which rewrites the seeded command menu item payload for existing workspaces. It is idempotent and only touches workspaces still holding the legacy path. <!-- This is an auto-generated description by cubic. --> <a href="https://cubic.dev/pr/twentyhq/twenty/pull/23105?utm_source=github" target="_blank" rel="noopener noreferrer" data-no-image-dialog="true"><picture><source media="(prefers-color-scheme: dark)" srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source media="(prefers-color-scheme: light)" srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img alt="Review in cubic" src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a> <!-- End of auto-generated description by cubic. --> |
||
|
Paul Rastoin
|
1be5a0e54a |
System side effect relations (#22882)
Closes twentyhq/core-team-issues#2667 ## What Default relations to the standard relation objects (`timelineActivities`, `attachments`, `noteTargets`, `taskTargets`) are now fully owned by the **metadata side-effect engine**. Neither the API transpilers nor the SDK manifest builder provision them anymore: any object creation, rename or deletion — regardless of the caller — goes through the same engine handlers. ## Why - Provisioning was duplicated across the API path and the SDK manifest builder, with diverging behavior. - Universal identifiers of relation fields were derived from object **names**, so renaming an object mutated them and forced lossy delete+create cycles on manifest sync. ## How ### Engine-owned lifecycle (side-effect handlers) - `objectSystemRelationsOnCreate`: provisions the 8 forward/reverse relation fields (+ join column indexes) when an object is created. - `objectSystemRelationsOnUpdate`: renames the reverse morph fields (`target<ObjectName>`) when their host object is renamed — a lossless `fieldMetadata.update`. - `objectSystemSideEffectsOnDelete`: cascades deletion of engine-owned fields/indexes when the object is deleted. - The API transpilers and the SDK `buildManifest` no longer inject these fields; `isSystemSideEffect: true` marks engine-owned entities, guarded by a granular property allowlist (only `isActive` is user-editable) and excluded from manifest deletion inference. ### Name-free deterministic universal identifiers New `getSystemRelationFieldUniversalIdentifier({ applicationUniversalIdentifier, objectUniversalIdentifier, relationTargetObjectUniversalIdentifier })` in `twenty-shared`, exported from `twenty-sdk/define`. The identifier is keyed on the two **object** identifiers instead of field names (direction encoded by argument order), so object renames never mutate relation field identifiers. It cannot collide with the name-based `getFieldUniversalIdentifier` derivation (field names cannot contain `:`). ### twenty-standard re-owned All 48 forward/reverse system relation field declarations in `STANDARD_OBJECTS` now pin the derived name-free identifiers (computed inline via the shared util) and carry `isSystemSideEffect: true`, with labels/icons declared explicitly (translated via `msg`). `twenty-standard` is projected as if the engine had generated these fields itself. ### 2.23 upgrade commands - `reconcile-system-relation-field-universal-identifier`: structurally matches existing default relation fields per workspace and backfills the derived universal identifiers, `isSystemSideEffect` flags, and standard labels/icons. - `upgrade-people-data-labs-application`: upgrades installed PDL apps to `1.0.7` right after the backfill to close the desync window (its views reference the re-derived identifiers). ### Misc - `people-data-labs` `1.0.7`: views temporarily pin the new derived identifiers (TODO: import from the next released `twenty-sdk`). - `UpgradeStatusModule` split out of `UpgradeModule` so the application module cluster can consume upgrade status/migration services without importing the versioned command bundles (fixes a require cycle that crashed boot). - Docs: `system-fields.mdx` documents the system relation fields and their resolver; `sync-and-recovery.mdx` plan example no longer shows auto-injected relations. ## Known red CI `people-data-labs (dockerhub-latest)` fails by design until the 2.23 server image is published: the app pins the new identifiers which only exist on a 2.23 server. The `local` leg (server built from this branch) is green. ## System fields are no longer manifest-authorable (accepted regression) The manifest converter no longer derives `isSystem` / `isSystemSideEffect` from field names. Reserved-system-named manifest fields (`id`, `createdAt`, `updatedAt`, `deletedAt`, `createdBy`, `updatedBy`, `position`, `searchVector`) are now skipped at conversion time when they carry the exact derived universal identifier (keeps manifests built with older SDKs installable), and rejected with `INVALID_INPUT` when they pin any other identifier. System fields are therefore fully engine-canonical: nothing a manifest carries can produce a system-flagged entity anymore. **Accepted regression**: a manifest can no longer influence system field properties at all. Previously a (legacy) re-declaration could shape them at creation — which actually produced broken system fields, e.g. a nullable, non-unique `id` — and could still toggle the allowlisted `isActive` / `universalSettings` afterwards. We consider this acceptable for now: per-app granularity over system fields will be reintroduced later through the **override framework**, which will also settle update semantics by forbidding direct updates over `isSystemSideEffect: true` entities and expressing divergence as overrides. `isSystemSideEffect`-only entities (the default relation fields provisioned by this PR) still have no engine-level update guard (see Follow-up below); that part is unchanged and also lands with the overrides refactor. ## Follow-up `isSystemSideEffect` field update/delete guards intentionally live at the API layer (`sanitize-raw-update-field-input.ts`, `from-delete-field-input-...util.ts`) rather than in the engine-level `FlatFieldMetadataValidatorService`. Moving them into the validator requires threading operation-origin (direct field mutation vs engine cascade) through the migration matrix, otherwise legitimate object rename/delete cascades (which carry `isSystemBuild=false`) would be rejected. Tracked in twentyhq/core-team-issues#2671. <!-- This is an auto-generated description by cubic. --> <a href="https://cubic.dev/pr/twentyhq/twenty/pull/22882?utm_source=github" target="_blank" rel="noopener noreferrer" data-no-image-dialog="true"><picture><source media="(prefers-color-scheme: dark)" srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source media="(prefers-color-scheme: light)" srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img alt="Review in cubic" src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a> <!-- End of auto-generated description by cubic. --> |
||
|
martmull
|
8d84a0b9f3 |
feat(app): allow non-admin developers to claim and list marketplace apps (#22621)
## Context Follow-up to #22609. Lets a non-admin developer claim ownership of a public Twenty app they published to npm, then request a marketplace listing that a server admin reviews. Marketplace state is per-instance for now. ## Claiming - Developer tab gets a **Claim an application** section: look up an unclaimed npm app by package name or universal identifier. - Ownership is proven with GitHub OAuth against the package's npm provenance (trusted publishing): the connected account must own the GitHub account or organization the package was published from. - Errors from the GitHub callback come back as a code and are shown inline with a link to the relevant documentation. - The old one-click claim stays admin-only. - A **Sync catalog** button triggers a catalog refresh instead of waiting for the hourly cron. - Gated behind the `IS_APP_CLAIMING_ENABLED` feature flag. ## Listing requests - Catalog-synced apps are created **unlisted**; a data migration unlists previously auto-listed unclaimed npm apps (owned or vetted rows are left untouched). - Owners request a listing from the Distribution tab (logo + description required); a server admin approves or rejects it from a **Listing requests** section in the Admin Panel. ## Screenshots <img width="1512" height="829" alt="image" src="https://github.com/user-attachments/assets/788d4362-97c4-4e42-810c-ef1f11517bec"/> <img width="1512" height="829" alt="image" src="https://github.com/user-attachments/assets/d6246190-c82a-4f64-87be-3bb668527645"/> <img width="1512" height="828" alt="image" src="https://github.com/user-attachments/assets/21a8dad4-610b-4d1f-8948-b9acab40d373"/> <img width="1512" height="829" alt="image" src="https://github.com/user-attachments/assets/58246130-41f7-451e-ae7f-57bd21d04bb6"/> --------- Co-authored-by: cubic-dev-ai[bot] <191113872+cubic-dev-ai[bot]@users.noreply.github.com> |
||
|
martmull
|
baa84bb2e0 |
Add auto-upgrade-in-apps (#23001)
We need to auto upgrade application, lets add this column in application entity, and add an admin button to autoupgrade all applications to latest app registrration version manually <img width="1131" height="372" alt="image" src="https://github.com/user-attachments/assets/4e755abc-38ad-4895-a2e8-d55ee1948ac2" /> <img width="906" height="533" alt="image" src="https://github.com/user-attachments/assets/ce002057-0341-4581-bf9a-66ac2bd84a9b" /> |
||
|
Paul Rastoin
|
6b55a6b51c |
Fix duplicate searchFieldMetadata inserts in the 2.16 backfill upgrade command (#23060)
## Context
A self-hosted instance upgrading from 2.0.3 to v2.22.0 got stuck with
one workspace failing at `2.16.0_BackfillSearchFieldMetadataCommand`:
```
[QueryFailedError] duplicate key value violates unique constraint "IDX_SEARCH_FIELD_METADATA_OBJECT_FIELD_UNIQUE"
Detail: Key ("objectMetadataId", "fieldMetadataId")=(...) already exists.
```
The failure happened on a retry after a previous partial run, and
reproduced even though the command already recomputes
`flatSearchFieldMetadataMaps` before deriving the create-set (#22884).
## Root cause
The idempotency dedupe compares `(objectMetadataId, fieldMetadataId)`
pairs across two differently-fresh caches:
- The **existing rows** side comes from `flatSearchFieldMetadataMaps`,
which is recomputed from the database (real current ids).
- The **candidate** side resolves ids through `flatObjectMetadataMaps` /
`flatFieldMetadataMaps`, which are **not** invalidated. During a
cross-version upgrade these can be stale, since the migration runner
only invalidates the cache keys a migration touched.
When a stale map resolves a candidate to an outdated id, the dedupe key
doesn't match the existing row and the row is re-emitted. The migration
runner then re-resolves the universal identifiers against fresh maps at
execution time and inserts with the real current ids — exactly the pair
already committed by the earlier partial run (each per-application
migration commits independently) — tripping the unique constraint and
failing the upgrade.
## Fix
Two independent layers, either of which would have prevented the
failure:
1. **Consistent snapshot for the build phase**: the command now
invalidates and recomputes all three maps the dedupe depends on
(`flatObjectMetadataMaps`, `flatFieldMetadataMaps`,
`flatSearchFieldMetadataMaps`), so candidate resolution, existing-row
keys, and the runner all see the same database state.
2. **Id-churn-proof dedupe**: every row this command creates carries a
deterministic universal identifier (`getSearchFieldUniversalIdentifier`,
derived from application + field universal identifiers, no database ids
involved) and `(workspaceId, universalIdentifier)` is unique. The build
util now also skips any candidate whose deterministic universal
identifier already exists, catching leftovers from a previous partial
run even if objects/fields were recreated under new ids in between.
Deliberately **not** done: `ON CONFLICT DO NOTHING` in the create action
handler — it is shared by all runtime `searchFieldMetadata` creation,
and swallowing a conflict would leave the flat-entity cache holding an
entity id that differs from the row actually in the database.
## Test
Added a regression test reproducing the failure shape: an existing row
with the same deterministic universal identifier but stale metadata ids
must not be re-emitted by the backfill.
Note: `ReconcileSearchFieldMetadataCommand` (2.20) has the same
stale-cache exposure; hardening it is left to a follow-up.
<!-- This is an auto-generated description by cubic. -->
<a
href="https://cubic.dev/pr/twentyhq/twenty/pull/23060?utm_source=github"
target="_blank" rel="noopener noreferrer"
data-no-image-dialog="true"><picture><source
media="(prefers-color-scheme: dark)"
srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source
media="(prefers-color-scheme: light)"
srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img
alt="Review in cubic"
src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a>
<!-- End of auto-generated description by cubic. -->
|
||
|
Thomas Trompette
|
87729a2822 |
feat(workflow): backfill + dual-write for core workflow entity (#22776)
Workflow-side soft-ref sync — the `coreWorkflowId` mirror of the merged version side (#22821 / #22940 / #22944 / #22961). Rebased onto current main; supersedes the original shared-UUID version of this PR. ## What Gives `core.workflow` a per-workspace copy of each workflow (`name`, `lastPublishedVersionId`), soft-reffed from the workspace record via `coreWorkflowId`, so app-shipped workflows have a core home. Does not touch reads/dispatch (that's Phase B). - **Sync service** (`WorkflowCoreSyncService`): core rows get their own id (`uuidv5(workspaceId:recordId)`), the workspace record links via `coreWorkflowId` (written back after the upsert), and the write-back is **guarded** on the `coreWorkflowId` field being present (skips with a warning otherwise — mirrors #22940). Injected repo renamed `coreWorkflowRepository`. - **Dual-write listener** on the `workflow` object: CREATED/UPDATED/RESTORED upsert, DELETED/DESTROYED delete by `coreWorkflowId`. Always-on; failures routed to Sentry so they never break the user write. - **2-20 backfill** (`backfill-workflow-to-core`): reads via the provided `RunOnWorkspaceArgs.dataSource`, upserts each workspace workflow into core. - **2-22 provisioning** (mirrors #22944/#22961): - `add-workflow-core-soft-ref-field`: adds the `coreWorkflowId` system field on existing workspaces (flat-entity legacy migration). - `backfill-workflow-core-links`: full rebuild — per workspace, in one raw-SQL transaction, wipes all `core.workflow` rows, inserts a fresh own-id row per workflow, and re-links every record. (No trigger-map cache to invalidate on `core.workflow`.) Simpler than the version side: `core.workflow` has no one-active-per-workflow index and no trigger-map cache, and it was never backfilled in prod, so there are no legacy shared-id rows. ## Test Fresh `database:reset` + full sequence (2-20 backfill → 2-22 add-field → 2-22 rebuild link): 4/4 workspace records linked via `coreWorkflowId` (id != coreWorkflowId), links resolve, **0 dangling**, no duplicate core rows, names populated. Typecheck + lint + oxfmt clean. |
||
|
neo773
|
c04714b9e0 |
fix(messaging): register and harden webhook subscription renewal cron (#23006)
The renewal cron was never wired into cron:register:all, so Gmail/Calendar/Graph watches were never renewed and went dark ~7 days after connect (the max watch lifetime all three providers allow). Register it, fan renewals out as per-channel queue jobs scoped to active workspaces, retry FAILED channels, and recreate Google Calendar watches before stopping the old one. <!-- This is an auto-generated description by cubic. --> <a href="https://cubic.dev/pr/twentyhq/twenty/pull/23006?utm_source=github" target="_blank" rel="noopener noreferrer" data-no-image-dialog="true"><picture><source media="(prefers-color-scheme: dark)" srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source media="(prefers-color-scheme: light)" srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img alt="Review in cubic" src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a> <!-- End of auto-generated description by cubic. --> |
||
|
Charles Bochet
|
2e671342f5 |
[2/2] Write CREATED at activation, clean stale onboarding workspaces (#22915)
## Context Follow-up to #22904 (merged), which introduced the `CREATED` activation status (schema provisioned, onboarding incomplete — no billing subscription), its enum migration, and the read path. This PR turns the status on and closes the zombie-workspace leak (~60–110 subscription-less ACTIVE workspaces per day since v2 onboarding, 935+ total). ✅ **Deploy gating satisfied**: #22904's slow enum migration shipped with the release deployed to prod on 2026-07-17, so writing `CREATED` is now safe. Rebased on main (clean, no conflicts) — main's #22943/#22955 guarded-transition rework already handles `CREATED` correctly: the webhook suspend switch only suspends `ACTIVE` workspaces, and `reactivateWorkspace` promotes `CREATED`→`ACTIVE`. ## What this PR does 1. **Write path** — `activateWorkspace` sets `CREATED` instead of `ACTIVE` when the workspace has no billing subscription. `hasWorkspaceAnySubscription` returns true when billing is disabled, so self-hosted workspaces keep going straight to `ACTIVE` — no behavior change outside cloud. 2. **Cleanup** — `CREATED` joins `PENDING_CREATION`/`ONGOING_CREATION` in the existing onboarding cleaning flow (cron + `workspace:clean:onboarding` with `--dry-run`): workspaces older than the same seven-day threshold are soft-deleted, then hard-deleted on a later run. **No suspension step and no emails** — an abandoned onboarding is treated as never having completed, exactly like a workspace stuck in creation. A workspace that subscribes before cleanup exits the flow (`CREATED`→`ACTIVE` synchronously via checkout). 3. **Backfill** — slow instance command moving `ACTIVE` workspaces with no `billingSubscription` row, created since v2 onboarding shipped (2026-07-01), to `CREATED`. Gated on `IS_BILLING_ENABLED` so self-hosted instances are untouched. 4. **Resolves #22904's text-cast TODO on the billing activation update** — this PR only deploys after the enum migration, so the `CREATED`→`ACTIVE` promotion is a plain status-scoped update again. The upgrade-path filters (`activationStatusIn`) keep the `::text` cast: upgrade tooling has to run against databases coming from pre-2.22 versions, so its TODO now points at the real removal trigger (dropping pre-2.22 upgrade support). ## Ops note before deploying The backfilled zombies are all older than seven days, so the first cron run after the backfill **soft-deletes them and the next run destroys them (schema and data), with no user-facing communication**. The backfill also catches any post-July-1 cloud workspace that is ACTIVE without a subscription — including intentionally comped/demo/internal ones if any were created since then (verified locally: the seeded demo workspaces matched). **Run the backfill's SELECT as a dry-run against prod and review the list before deploying.** ## CI note ~~`cross-version-upgrade` (and its `ci-server-status-check` aggregate) is red due to a pre-existing regression on main — `Field metadata "coreWorkflowVersionId" is missing in object metadata workflowVersion` on the seed workspaces.~~ Resolved: the rebase picks up main's #22944/#22961 which fixed that regression. ## Verification Server-side (billing-enabled local instance, Stripe test mode) and through the full onboarding UI in both billing modes: - **Billing enabled, UI**: signup → workspace creation → **`activationStatus: CREATED`** in DB mid-onboarding → profile/invite steps work on the CREATED workspace → plan-required page → no-card trial → app loads, workspace **`ACTIVE`** with a `trialing` subscription (exercises #22904's synchronous promotion). - **Billing disabled, UI**: signup → workspace creation → **`ACTIVE` directly**, no plan step anywhere, app loads — self-hosted behavior unchanged. - **Cleanup**: a `CREATED` workspace backdated 8 days is listed by `workspace:clean:onboarding --dry-run`; the real run soft-deletes it silently (no suspension, no email) and the next run hard-deletes it (workspace row and schema gone). - **Backfill**: synthetic `ACTIVE` no-sub workspaces — created 2026-07-05 flips to `CREATED`, created 2026-06-15 stays `ACTIVE`, subscribed workspaces stay `ACTIVE`; billing-disabled short-circuit returns without touching anything. - Lint + typecheck green. |
||
|
Thomas Trompette
|
f8b7ecf680 |
fix(workflow): rebuild core workflowVersion rows in the 2-22 backfill (#22961)
## Problem
Syncing workflowVersion to core fails with `duplicate key value violates
unique constraint "IDX_WORKFLOW_VERSION_ONE_ACTIVE_PER_WORKFLOW"`. The
sync's `INSERT ... ON CONFLICT ("id")` only dedupes the primary key — it
can't dedupe `IDX_WORKFLOW_VERSION_ONE_ACTIVE_PER_WORKFLOW`
(`(workspaceId, workflowId) WHERE status='ACTIVE'`). When a leftover
ACTIVE core row exists for a `(workspaceId, workflowId)` with a stale id
(accumulated across the sync's earlier id schemes), inserting the new
active row collides, and the per-id purge misses it.
## Fix
Rewrite the 2-22 `backfill-workflow-version-core-links` command as a
**full rebuild**. Per workspace, in one raw-SQL transaction (core and
workspace schemas are the same database):
1. `DELETE` all `core.workflowVersion` for the workspace — clears every
stale/leftover row.
2. Insert a fresh own-id core row for **every** workspace version.
3. `UPDATE` `coreWorkflowVersionId` on **every** workspace record to its
new core id.
Because it wipes first and re-links all records, leftover ACTIVE rows
can't collide and no record is left pointing at a deleted core row — so
the dual-write's `linked → update` path stays correct afterward.
## Test (local)
Fresh reset, and a reproduced dirty state (leftover ACTIVE core row with
a stale id + a stale link + an unlinked record):
- rebuild runs with no `ONE_ACTIVE` / duplicate-key error,
- leftover wiped, stale link replaced, every record re-linked to a fresh
own-id row,
- 0 dangling/unlinked, no duplicate core rows, exactly one ACTIVE core
version per workflow.
Typecheck + lint + oxfmt clean.
## Note
The 2-20 `backfill-workflow-version-to-core` can still log per-workspace
conflicts on already-dirty instances, but they're non-fatal (the
iterator continues) and this rebuild corrects the end state. The
dual-write (`upsertToCore`) is unchanged and works on the clean data
this produces.
|
||
|
Thomas Trompette
|
988f8ff900 |
feat(workflow): provision coreWorkflowVersionId on existing workspaces and link them (#22944)
Stacked on the write-back guard hotfix (#22940). Completes the version soft-ref for workspaces that predate the `coreWorkflowVersionId` field. ## Why New standard fields aren't auto-synced to existing workspaces; they're only built at workspace creation or added by an explicit upgrade command. Deployed 2.20/2.21 instances also carry **legacy core rows with `id = record.id`** (the pre-soft-ref shared-UUID model, from the already-run #22663 backfill). The original backfill has already run there and won't re-run (tracking is by command name), so the migration to soft-ref has to be **new appended commands**. ## What (two appended 2-22 workspace commands) 1. `add-workflow-version-core-soft-ref-field` (`1784193206000`): adds the `coreWorkflowVersionId` system field to existing workspaces missing it (flat-entity migration, `add-message-campaign-stat-fields` pattern). Idempotent, dry-run aware, skips workspaces without the `workflowVersion` object. 2. `backfill-workflow-version-core-links` (`1784193207000`): re-runs the sync (`upsertToCore`). For each version, `upsertToCore` **purges the legacy shared-id core row** (`id === record id`) then upserts a deterministic own-id row and writes the link back onto the workspace record. Targeted per-id delete — it does not wipe unrelated core rows. Ordering: both run after the original 2-20 backfill. On instances that run 2-20 fresh (e.g. 2.19 → 2.22) that backfill creates core rows and — via the hotfix guard — skips the write-back until the field exists; command 1 provisions the field; command 2 purges + relinks. On already-migrated 2.21 instances the 2-20 backfill won't re-run, so command 2 is what clears their shared-id rows. The same per-id purge in `upsertToCore` also covers the dual-write path: between the 2.22 deploy and command 2 running, an edited version would otherwise collide with its shared-id row on the one-active-per-workflow index. Scope: version side only. The workflow-side equivalent (`coreWorkflowId`) ships with the workflow-side sync PR; `core.workflow` was never backfilled in prod, so it has no legacy shared-id rows. ## Test - Unit: guard skips write-back when the field is absent; runs it when present. - Happy path (fresh reset): original backfill → add-field no-op → link → 4/4 linked, own ids, 0 duplicates. - Deployed migration (simulated 2.21: shared-id ACTIVE core rows + field removed): add-field re-provisions → link purges the shared-id rows and rebuilds → records linked to own-id rows, 0 duplicates, exactly one ACTIVE core version per workflow (index intact), no collision. - Idempotent re-run: still 4 core rows, links resolve. - Typecheck + lint + oxfmt clean. |
||
|
Charles Bochet
|
8e03921372 |
Add CREATED workspace activation status (read path + enum migration) (#22904)
## Context Since v2 onboarding (#22303), workspaces are activated **before** the billing plan step (now the last onboarding step). Users abandoning at the plan step leave ACTIVE workspaces with a Stripe customer but no subscription (~60–110/day on cloud, 935+ so far), and no cleanup mechanism ever touches them: billing webhooks never fire (no subscription), the suspended-workspaces cron only handles SUSPENDED, the onboarding cron only handles PENDING_CREATION/ONGOING_CREATION. Target lifecycle (across two PRs): `PENDING_CREATION → ONGOING_CREATION → CREATED → ACTIVE → SUSPENDED → deleted`. **`CREATED`** = the workspace schema is provisioned but onboarding is not complete — no billing subscription yet. It is **not** considered active: | Concern | CREATED behavior | |---|---| | Sign-in / invited teammates joining | allowed (invite-team step precedes the plan step) | | Member + metadata loading (app shell) | allowed (user must finish onboarding) | | Permissions | real permission checks (no PENDING-style bypass) | | Version upgrades / workspace migrations | **included** (schema must not drift) | | Messaging/calendar/workflow/etc. crons | **excluded** — no background processing until a plan is chosen | | PLAN_REQUIRED onboarding lock | unchanged (still derived from subscription existence) | ## What this PR does (read path only) The enum addition ships as a **slow** instance command, which can run after deploy — so nothing in this PR ever **writes** `CREATED`. The write path (setting it at activation, the cleanup sweep, the backfill of the existing zombie cohort) is a follow-up PR that ships once this migration has run everywhere. - **twenty-shared**: `CREATED` enum value; `PROVISIONED_WORKSPACE_ACTIVATION_STATUSES` + `isWorkspaceProvisioned` ("schema exists": CREATED | ACTIVE | SUSPENDED), replacing `isWorkspaceActiveOrSuspended` — all call sites (server member loading, access-token workspace-member lookup, front metadata-store gates) meant "has schema/members". - **Slow instance command** (2.22.0): swaps `core.workspace_activationStatus_enum` using the rename→recreate→alter-column idiom. The CHECK constraints on `core.workspace` embed casts to the enum type and would break the swap — the command captures them from `pg_constraint`, drops them, swaps the type, and restores them. - **Pre-migration-safe queries**: Postgres rejects `IN ('CREATED', ...)` when the enum value does not exist yet — even for reads, and the instance-command runner itself queries provisioned workspaces before migrating (a fresh database could never initialize). All provisioned-status filters go through a new `activationStatusIn` util comparing on `"activationStatus"::text`, valid before and after the migration. - **Upgrade path**: workspace iterator, command runner, upgrade-status and workspace-version services iterate CREATED workspaces. Since they now cover more than ACTIVE/SUSPENDED, the stale names were renamed to `ProvisionedWorkspaceCommandRunner`, `hasProvisionedWorkspaces`, `getProvisionedWorkspaceIds`, `loadProvisionedWorkspaces` (the mechanical import rename in old version-command dirs is why this PR carries the `ci:allow-previous-version-upgrade-mutation` label). - **Sign-in**: `throwIfWorkspaceIsNotReadyForSignInUp` accepts CREATED so invited members can join during onboarding (join authorization itself is unchanged — enforced upstream in `checkAccessForSignIn`); `activateWorkspace` idempotent-retry accepts CREATED as a terminal state. - **Transitions out of CREATED** (only write ACTIVE — safe to ship now, dead until the write path lands): the Stripe webhook reactivation branch also promotes CREATED, and `syncSubscriptionToDatabase` promotes synchronously; both gated on `WORKSPACE_ACTIVATING_SUBSCRIPTION_STATUSES` (Active/Trialing — extracted from `shouldReactivateWorkspace`, behavior-preserving) so an `incomplete` subscription created by the payment-intent flow before payment never promotes the workspace. - Deliberately untouched: all background crons, permission guards, JWT strategy, PLAN_REQUIRED logic, admin panel (renders the raw status string). ## Follow-up PR (after this migration has run) 1. `activateWorkspace` sets `hasWorkspaceAnySubscription ? ACTIVE : CREATED` (billing disabled → always ACTIVE, self-hosted unchanged). 2. Cleanup: suspend CREATED workspaces older than N days (config var), handing them to the existing suspended pipeline (warn → soft-delete → destroy). 3. Backfill: cloud-only slow command moving ACTIVE workspaces with no billingSubscription row (created since Jul 1) to CREATED. ## Verification - Migration exercised against a real database via the command class: up → down → up; `enum_range` and `pg_get_constraintdef` checked after each step (constraints restored against the new type, `DEFAULT 'INACTIVE'` preserved). - Pre-migration safety exercised for real: with the migration rolled back (enum without CREATED), `run-instance-commands` — the exact fresh-database CI path that failed before the `::text` fix — completes cleanly. - End-to-end with a workspace manually set to CREATED and the branch server+front running: sign-in issues tokens, `currentUser` loads workspaceMember(s), the full app loads with no console errors; GraphQL returns `activationStatus: CREATED`. - Workspace creation ran end-to-end locally in **both billing modes** on this branch: - billing disabled: signup → workspace creation → ACTIVE immediately → onboarding completes with no plan step → app loads (unchanged behavior); - billing enabled (Stripe test mode): signup creates the Stripe customer eagerly → activation ends ACTIVE → subscription-less workspace is pinned to the plan-required page → no-card trial checkout creates a `trialing` subscription via `createDirectSubscription`/`syncSubscriptionToDatabase` → app loads. - `twenty-shared` unit tests, server specs on touched services, `lint:diff-with-main` and `typecheck` for shared/server/front all green; full CI green. |
||
|
Weiko
|
25bd2897a3 |
Add weekly layout to record calendar (#22819)
## Summary - Add a week layout to record calendar views and persist the selected layout. - Render `DATE` calendars as an all-day week and `DATE_TIME` calendars as an hourly week. - Add an optional end date field across calendar configuration, metadata, persistence, and complete-view upserts. - Use configured end values for ranged and multi-day events, with a one-hour fallback when a `DATE_TIME` end is absent or invalid. - Keep calendar cards consistent with the existing compact view, including checkbox selection and whole-card record opening. - Gate the weekly layout and end-date behavior behind the public Labs `IS_CALENDAR_WEEK_VIEW_ENABLED` workspace feature flag. ## Week interactions - Show overlapping timed events side by side and cap the visible records at two per day. - Display start and end times on timed cards, enforce a readable 30-minute minimum height, and keep today’s text contrast stronger. - Drag timed events between days and times with 30-minute snapping while preserving their duration, including zero-duration events. - Show a create button when hovering a 30-minute slot; keyboard users can focus a day, move the slot with the arrow keys, and reach the same contextual action. - Initialize new records with the selected slot time and a compatible writable end value one hour later. - Show the workspace time zone and current-time indicator in timed weeks; date-only weeks keep the all-day section without an hourly grid. ## Configuration and data loading - Only allow end fields that match the start field type, and prevent selecting the same field for both boundaries. - Load records whose ranges overlap the visible period so month and week layouts display the same relevant records. - Resolve and persist calendar end fields when updating existing views through `upsert_complete_view`. - Fall back to Month and ignore the configured end field while the flag is disabled, without overwriting either persisted setting, so re-enabling restores the previous configuration. - Expose the flag in Labs and keep it default-off for workspaces without a stored value; enable it in the development seeder. <img width="1285" height="808" alt="Screenshot 2026-07-15 at 15 50 17" src="https://github.com/user-attachments/assets/b7e3f7f1-ca77-492f-8cce-cca186ebca0b" /> |
||
|
Abdul Rahman
|
f4ff234db8 |
feat: make record avatar/icon resolution data-driven via a configurable image identifier field (#22644)
## Summary Today the avatar/icon shown for a record is hardcoded per object — Company pulls a favicon from its domain link, Person uses `avatarUrl`, etc. This PR replaces that hardcoding with a generic, data-driven abstraction based on a configurable **image identifier field** on each object's metadata (mirroring the existing **label identifier** concept). An object's image identifier can point to: - a **`FILES`** field → the uploaded image is used directly (rounded avatar), or - a **`LINKS`** field → a favicon is derived from the primary URL via the Twenty icons service (squared avatar), gated by `ALLOW_REQUESTS_TO_TWENTY_ICONS`. This lets any object type (Opportunity, a custom "Listing", etc.) define its own avatar/icon without code changes, and makes the field configurable/overridable for standard objects. ## ❓ Open question: also allow `TEXT` → direct image URL? Right now the image identifier is restricted to `FILES` (uploaded file) and `LINKS` (favicon). We deliberately left out `TEXT` → **direct image URL** (e.g. an imported/synced photo URL stored in a text field). There's precedent for it — Person's avatar was originally a `TEXT` `avatarUrl`, and WorkspaceMember still is — and it's unambiguous (a `TEXT` field has no favicon-vs-image ambiguity, and selecting it as the image identifier is itself the declaration of intent). It's a small, clean extension: - add `TEXT` to the allowed image-identifier types, - add an explicit `TEXT → raw URL` case - `getAvatarType`: `TEXT → rounded`. Caveats: it relies on admin assertion that the text values are image URLs (no data-level guarantee), and external image URLs load third-party content in the browser (IP-leak/hotlinking, same as favicons — a proxy/cache would be the more robust long-term answer). ### ✅ Resolution Decision: **we will not support `TEXT` as an image identifier.** Image identifiers stay restricted to `FILES` and `LINKS`, and any other type fails closed (returns no avatar) on both the frontend and backend. Instead, the legacy items that still rely on a `TEXT` avatar — Person's deprecated `avatarUrl` and WorkspaceMember's `avatarUrl` — will be migrated to `FILE` fields in a follow-up PR. Until then, WorkspaceMember remains an exception (its `avatarUrl` still resolves through the existing CorePicture path), and legacy Person `avatarUrl` values that haven't been migrated will show initials placeholders. <!-- This is an auto-generated description by cubic. --> <a href="https://cubic.dev/pr/twentyhq/twenty/pull/22644?utm_source=github" target="_blank" rel="noopener noreferrer" data-no-image-dialog="true"><picture><source media="(prefers-color-scheme: dark)" srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source media="(prefers-color-scheme: light)" srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img alt="Review in cubic" src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a> <!-- End of auto-generated description by cubic. --> |
||
|
Paul Rastoin
|
a28c3a905a |
Route pre-2.19 upgrade commands through a legacy validate-build path (#22884)
## Problem Since the centralized metadata side-effect engine landed in v2.19, `WorkspaceMigrationValidateBuildAndRunService.validateBuildAndRunWorkspaceMigrationFromRecord` runs `metadataSideEffectEngineService.expandWithSideEffects(...)` before building. As a result every historical upgrade command (`upgrade-version-command/1-21/*` … `2-18/*`), authored before the engine existed, now flows through it. Their operation matrix is no longer applied literally: the engine injects/cascades companions (system fields, `searchVector` field + GIN index, `searchFieldMetadata` rows, unique backing indexes) and can hard-fail on reserved-identifier collisions (`RESERVED_SYSTEM_UNIVERSAL_IDENTIFIER`). Two hazards for already-shipped commands: 1. **Collision → hard failure**: a command declaring a companion the engine now owns collides with the engine's deterministic `universalIdentifier`. 2. **Silent drift**: on object/field create/delete the engine adds/cascades companions the command author never intended, so workspaces upgraded now differ structurally from those upgraded incrementally before 2.19. Suspected real-world impact: a self-hosted user upgrading v2.6.1 → v2.21.0 hit `duplicate key value violates unique constraint "IDX_SEARCH_FIELD_METADATA_OBJECT_FIELD_UNIQUE"` in `upgrade:2-16:backfill-search-field-metadata`, because object-creating commands now cascade and pre-create the deterministic `searchFieldMetadata` rows the standalone backfill then re-inserts. ## Changes - `workspace-migration-validate-build-and-run-service.ts`: extract the shared compute-and-run tail into a private method, and add `validateBuildAndRunLegacyWorkspaceMigration` (marked `@deprecated`) that skips `expandWithSideEffects` and applies the matrix literally. The existing side-effect entry points are unchanged (the live API and application manifests depend on them). - Repoint **all** pre-2.19 upgrade command call sites (1-21 … 2-18, including `2-10 sync-call-recording-standard-objects`) to the legacy method. Only the four `2-20/*` commands (target version ≥ 2.19) remain on the side-effect path. - `2-16 backfill-search-field-metadata`: recompute `flatSearchFieldMetadataMaps` from the database before building the existing-rows dedupe set. The migration runner only invalidates the flat-maps keys a migration touched, so during a cross-version upgrade earlier commands can leave this map stale; a stale map breaks the dedupe and re-inserts rows, tripping `IDX_SEARCH_FIELD_METADATA_OBJECT_FIELD_UNIQUE`. This is the direct fix for the reported failure. - Export `FlatEntityMapsBundle` so the shared tail can be typed. - Document the side-effect vs legacy path and the selection rule in `packages/twenty-server/docs/UPGRADE_COMMANDS.md`. Selection rule: target version **< 2.19** → legacy path; **≥ 2.19** → side-effect path (default). No exceptions. ## Known gap / merge ordering The static twenty-standard definition declares all of `callRecording`'s fields (including the `searchVector` system field) but **not** its `searchVector` GIN index — every other searchable standard object declares its GIN index statically. On the legacy path, workspaces upgrading through `2-10 sync-call-recording-standard-objects` therefore create the `searchVector` column unindexed (`searchFieldMetadata` rows are created later in the same pipeline by the 2-16 backfill). The static GIN index declaration plus a backfill for already-upgraded workspaces land in a follow-up (twentyhq/core-team-issues#2672), which must ship in the same release as this PR. ## Out of scope (separate follow-ups) - `UpgradeMigrationService.getLastAttemptedInstanceCommand()` ordering. - callRecording `searchVector` GIN index static declaration + backfill (twentyhq/core-team-issues#2672, same-release dependency, see above). ## Test plan - `nx typecheck twenty-server` passes. - `nx lint:diff-with-main twenty-server` (oxlint + oxfmt) clean on changed files. - 2-20 command specs (which exercise the unchanged side-effect path) pass. --------- Co-authored-by: twenty <noreply@twenty.com> |
||
|
martmull
|
94192a2164 |
Resolve application registration logo and gallery image urls at query time (#22827)
## Context
Application registration logo and gallery image urls were baked into the
stored manifest and display columns at write time, with each source flow
doing it differently: npm catalog sync baked CDN urls, local dev sync
baked `public-assets` urls, and tarball uploads left raw manifest paths
that never displayed in the UI. The entity also carried a `logoUrl`
getter computed field.
This moves url generation to query time, the same way the workspace logo
works.
## What changed
**Read side**
- `ApplicationRegistrationAssetUrlService` builds display urls when
queried: stored files are served by fileId, absolute urls pass through
untouched, and not-yet-rehosted npm assets fall back to the registry CDN
from `sourcePackage@latestAvailableVersion`.
- The `logoUrl` getter on `ApplicationRegistrationEntity` is replaced by
`logoUrl` and `galleryImages` `@ResolveField`s on the metadata resolver,
the admin panel resolver, and a new resolver for
`ApplicationRegistrationSummary` (used by
`Application.applicationRegistration`).
- The marketplace detail/card DTOs and the public OAuth authorize DTO
(`findApplicationRegistrationByClientId`) go through the same url
builder.
- New public route `GET /file/application-registration/:id` streams
registration server files (these are instance-global marketplace assets,
also shown on the public OAuth authorize page).
`ServerFileStorageService.readServerFileById` now returns the mime type
alongside the stream.
**Write side**
- New `logoFileId` column on `applicationRegistration` (2.21 fast
instance command, constraint names match TypeORM naming), complementing
the fileIds already stored in the `galleryImages` jsonb.
- `ApplicationRegistrationAssetService` copies the manifest logo and
gallery images into instance-global server file storage, so all three
sources behave the same:
- **TARBALL**: from the uploaded package (previously only gallery images
were stored, never the logo).
- **LOCAL**: dev sync reads the already-uploaded public assets from
workspace storage (the CLI uploads files before syncing).
- **NPM**: catalog sync downloads the assets from the registry CDN.
Downloads are skipped when the package version is unchanged and the
files are already stored; failed or pending downloads fall back to CDN
urls at query time.
- Write-time url rewriting is removed
(`ManifestAssetUrlResolverService`, `resolveManifestAssetUrls`);
manifests now keep raw asset paths. Existing rows with baked absolute
urls keep working through the absolute-url passthrough, so no backfill
is needed.
- `updateFromManifest` and `upsertFromCatalog` preserve stored gallery
fileIds for unchanged paths, so installs and the hourly catalog sync no
longer clobber them.
## How it was verified
Against a local Postgres/Redis with the server running:
- Fresh database init runs the new instance command; column and FK/UQ
constraint names match TypeORM's generated names, and the CI
pending-migration check produces no diff.
- `findManyApplicationRegistrations { logoUrl galleryImages }` returns
fileId-served urls for a TARBALL registration (absolute urls passed
through), and null/[] for a LOCAL registration without assets.
- Ran `marketplace:catalog-sync` against the real npm registry: 14
packages synced, logos and gallery images rehosted from unpkg with
fileIds set; a second run re-downloaded nothing (version-unchanged
skip); `findMarketplaceAppDetail` for `twenty-linear` returns
fileId-served urls for the logo and all four gallery images.
- `GET /file/application-registration/:id` serves stored files with the
right content type (png and svg verified), 404s on unknown ids, and the
token-guarded generic `/file/:folder/:id` route still returns 403
without a token.
- Unit tests for the url builder and the assets-stored check; server
unit test suites for the application module pass; typecheck and lint
clean.
|
||
|
Paul Rastoin
|
de75be16e2 |
harden(server): backfill and enforce workspace.databaseSchema invariant with a check constraint (#22855)
## What & why `core.workspace.databaseSchema` is meant to be set for every workspace past the creation phase. It only started being written at creation time in 2.x (dual-write since 2026-03-28, direct write since 2026-04-10); older workspaces relied on the `1-21 backfill-datasource-to-workspace` instance command, which never effectively ran on some instances. On affected rows the column could be left `NULL`. A null value on a post-creation workspace is a real integrity problem — several paths trust the column: - **REST API**: `hydrateRestRequest` throws `No data sources found` for authenticated requests. - **GraphQL API**: `getOrComputeSchemaSDL` returns `null`, so `WorkspaceSchemaFactory` hands back an empty schema. - **GraphQL introspection** (direct execution) returns `null`. This PR makes the invariant impossible to silently violate, and repairs any instance still lagging. ### On the original "No data source, skipping" logs This investigation started from `BackfillActorSourceEnumValuesCommand` logging `No data source for workspace <id>, skipping` at high volume. **That symptom is not explained by this change, and this PR is not a fix for it.** Findings: - The workspace iterator only processes `ACTIVE` + `SUSPENDED` workspaces, and on the affected instance all of those already have `databaseSchema` set (only `PENDING_CREATION` rows are null, and those are never iterated). - `getGlobalWorkspaceDataSource()` never resolves to `undefined` (it returns a value or throws), so a defined-schema workspace should never hit the skip branch. - The upgrade-aware repository proxy was investigated as a possible cause (it can short-circuit `findOne` to `null` for entities marked unavailable during an upgrade) and **exonerated**: `WorkspaceEntity` and its `databaseSchema` column carry no `@WasIntroducedInUpgrade`/`@WasRemovedInUpgrade` decorators, so `resolveEntityShapeAtUpgradeCursor` always reports the entity available and the column visible at every cursor. In other words, current code should emit zero such skips for that instance's data, so the root cause of the observed logs remains undetermined and is tracked separately. See twentyhq/core-team-issues#2666. ## Changes - **Check constraint `workspace_requires_database_schema`** (the core of this PR): enforces `databaseSchema IS NOT NULL` for any workspace past creation (`activationStatus NOT IN ('PENDING_CREATION', 'ONGOING_CREATION')`). Declared on `WorkspaceEntity` and applied in the slow instance command's `up()`. Safe against the creation flow: `databaseSchema` is written in `WorkspaceManagerService.init` (right after schema creation) long before a workspace becomes `ACTIVE`. - **Defensive backfill** (`2-21` slow instance command): repopulates `databaseSchema` where it is `NULL`/empty, deriving the schema name deterministically from the workspace id (`getWorkspaceSchemaName`) and only setting it for workspaces whose schema actually exists in `information_schema.schemata` (so `PENDING_CREATION` rows without a provisioned schema are left untouched, and stay exempt via the constraint). No-op on instances already backfilled. - `runDataMigration` runs before `up()`, so the backfill repairs legacy rows before the constraint is enforced. Keeping both in the same slow command (rather than a standalone fast command) guarantees the constraint is never added ahead of the repair. - `checkSchemaExists` gets an explicit `: Promise<boolean>` return type. ## Notes - Backfill + constraint live in a **slow** instance command, so they only apply on upgrades run with `--include-slow`. - The constraint is added **`NOT VALID`**: the backfill repairs every workspace whose Postgres schema exists, but some legacy active/suspended workspaces (e.g. carried over from very old versions, as reproduced by the cross-version upgrade from v1.22) have a null `databaseSchema` with no schema to point at and are unrepairable. `NOT VALID` enforces the invariant on all future inserts/updates without failing the upgrade on that pre-existing corruption. - No production request path was changed — the iterator and `checkSchemaExists` keep trusting the (now backfilled + constrained) column. ## Test plan - [ ] Run `database:migrate:prod --include-slow` on an instance with null `databaseSchema` rows; verify rows whose schema exists get backfilled and `PENDING_CREATION` rows are left null. - [ ] Verify the `workspace_requires_database_schema` constraint exists on `core.workspace` and rejects nulling `databaseSchema` on an active workspace. - [ ] Verify a fresh workspace creation still succeeds (constraint does not fight the `PENDING_CREATION` → `ACTIVE` transition). |
||
|
Paul Rastoin
|
1b168ac1f7 |
fix(server): gate workspaceDiscoverability behind upgrade decorator (#22818)
## Context Needs to be patched on 2.20, will craft a 2.19 equivalent with fallback asap ( be it won't be merged unlike this one ) Fixes #22662. Follow-up to #22423, which introduced `workspaceDiscoverability`. A clean 2.18.x to 2.19 upgrade breaks on login with: ``` QueryFailedError: column workspaceDiscoverability does not exist ``` `workspaceDiscoverability` was added to `WorkspaceEntity` (in #22423) as a plain, always-selected, non-nullable column, so any workspace query (including the auth-path `findAvailableWorkspacesByEmail` lookup) fails as soon as the ORM selects it, before the `2.19` upgrade command that creates the column has run. Because the Docker entrypoint is fail-open, the API starts even if the upgrade is delayed, and users hit this on their first login. ## Changes - Add `@WasIntroducedInUpgrade` to `workspaceDiscoverability`, referencing the existing `2.19.0` fast instance command that creates the column. The upgrade-aware ORM then skips the column until the command has actually added it, keeping login working during the upgrade. - Keep the GraphQL `@Field` non-nullable and add a `workspaceDiscoverability` `@ResolveField` that falls back to `WorkspaceDiscoverability.PUBLIC` while the column is hidden, so the resolver never returns `null` for the non-nullable field during the upgrade window. This mirrors the existing pattern already applied to `FileEntity.status` and `FileEntity.applicationRegistrationId`, and the resolver-default pattern already used for `fastModel` / `smartModel` / `logo`. ## Cherry-pick This fix needs to be cherry-picked onto both the **2.19** and **2.20** release branches, since affected instances are upgrading into those versions. ## Test - `validate-upgrade-aware-entity-decorators` and `resolve-entity-shape-at-upgrade-cursor` unit tests pass (the referenced upgrade command name resolves correctly). - `upgrade-aware-repository.proxy` and `upgrade-aware-entity-metadata.adapter` specs pass. - `typecheck` and lint pass for `twenty-server` and `twenty-front`. - Regenerating the GraphQL schemas produces no diff (the field stays non-nullable). |
||
|
Paul Rastoin
|
652adc3c03 |
fix(server): backfill isSystemSideEffect on system fields provisioned before 2.15 (#22850)
## Context The `isSystemSideEffect` column was introduced in **2.15** via a fast instance command that added it with `DEFAULT false`. That stamped `false` onto every pre-existing `fieldMetadata` row — including the 8 engine-owned system fields (`id`, `createdAt`, `updatedAt`, `deletedAt`, `createdBy`, `updatedBy`, `position`, `searchVector`) of every object provisioned before 2.15, regardless of the creation path (API metadata **and** manifest sync). The per-workspace backfill that should have re-flagged those existing rows was explicitly deferred as out of scope in #21673 ("PR 2") and never shipped for `fieldMetadata`. Because `isSystemSideEffect` is configured with `toCompare: false`, no later sync ever repaired the stale value either. Since **2.20** the SDK no longer declares system fields in manifests. On an up-to-date instance, `twenty plan` against an unchanged app therefore diffs those stale-`false` system fields as **missing from the manifest**, and they fall through the `isSystemSideEffectFlatEntity` exclusion in `buildAllFlatEntityOperationRecordByMetadataNameFromFromTo`. Deletion inference then emits them as deletes, which the validator rejects: ``` Sync failed with 144 errors fieldMetadata: 144 errors 1..144. FIELD_MUTATION_NOT_ALLOWED: System fields cannot be deleted ``` (144 = 8 system fields × 18 custom objects, as reported on a production 2.20 instance.) ## What this PR does Adds a **2.21 workspace command** (`upgrade:2-21:backfill-system-field-is-system-side-effect`) that iterates active/suspended workspaces and flags the 8 system fields as `isSystemSideEffect: true`. - **Resolution by deterministic universal identifier**: for each object × reserved system field name it recomputes `getFieldUniversalIdentifier(applicationUID, objectUID, name)` and looks the row up in the flat maps. This is safe (and preferable to matching by `name`) because the 2.19 backfill already took over system field UIDs for every application, so an author-declared field reusing a reserved name keeps its own identifier and is never touched. An extra `isSystem` guard warn-and-skips any mismatch. - **All applications** are covered (installed apps, workspace custom app, twenty-standard): the stale flag is a function of *when* a row was provisioned, not *how*. Installed/custom apps are the acute `twenty plan` delete trap; twenty-standard has no trap today but flagging is a zero-diff no-op (`toCompare: false`) and a prerequisite for the end-state ownership invariant. - **`name` is intentionally excluded**: the 2.20 slow instance command deliberately flipped it to `false` (caller-provided default, not engine-owned); re-flagging it would undo that migration. - Supports `--dry-run`, updates only the collected rows, and invalidates the `flatFieldMetadataMaps` workspace cache after the write (a raw repository update does not invalidate it). ## Related - Resolves the pre-2.15 regression tail of twentyhq/core-team-issues#2635 - Follow-up to twentyhq/core-team-issues#2642 (system field side-effect engine migration) <!-- This is an auto-generated description by cubic. --> <a href="https://cubic.dev/pr/twentyhq/twenty/pull/22850?utm_source=github" target="_blank" rel="noopener noreferrer" data-no-image-dialog="true"><picture><source media="(prefers-color-scheme: dark)" srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source media="(prefers-color-scheme: light)" srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img alt="Review in cubic" src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a> <!-- End of auto-generated description by cubic. --> --------- Co-authored-by: Weiko <corentin@twenty.com> |
||
|
Thomas Trompette
|
c1b62334b7 |
fix(workflow): scope one-active-per-workflow index to workspace (#22795)
## Problem
The Phase 0 core index \`IDX_WORKFLOW_VERSION_ONE_ACTIVE_PER_WORKFLOW\`
is on \`(workflowId) WHERE status='ACTIVE'\`, with **no
\`workspaceId\`**. But \`core.workflowVersion\` is a shared multi-tenant
table, so this enforces "one active version per workflowId **globally
across all workspaces**" instead of per workspace.
The version backfill fails on staging with:
\`\`\`
duplicate key value violates unique constraint
"IDX_WORKFLOW_VERSION_ONE_ACTIVE_PER_WORKFLOW"
Detail: Key ("workflowId")=(8b213cac-...) already exists.
\`\`\`
across several different workspaces that share the same workflowId
(seeded/cloned data): workspace A's active version claims the
workflowId, and every other workspace's insert collides. Every other
index on this table includes \`workspaceId\`; this one dropped it when
copied from the per-tenant workspace entity.
## Fix
Index becomes \`(workspaceId, workflowId) WHERE status='ACTIVE'\` — one
active version per workflow **per workspace**, matching the table's
multi-tenant design and the intended invariant. New 2-20 fast instance
command drops and recreates the index (Phase 0's command is
merged/append-only).
## Test
Reset + reproduce the exact scenario against the fixed index:
- two workspaces with the same workflowId, both ACTIVE → **insert
succeeds** (previously collided)
- a second ACTIVE version for the same workflow within one workspace →
**still blocked** (invariant preserved)
Zero \`migrate:generate\` drift, typecheck + lint clean. After this
deploys, re-run \`upgrade:2-20:backfill-workflow-version-to-core\`.
<!-- This is an auto-generated description by cubic. -->
<a
href="https://cubic.dev/pr/twentyhq/twenty/pull/22795?utm_source=github"
target="_blank" rel="noopener noreferrer"
data-no-image-dialog="true"><picture><source
media="(prefers-color-scheme: dark)"
srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source
media="(prefers-color-scheme: light)"
srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img
alt="Review in cubic"
src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a>
<!-- End of auto-generated description by cubic. -->
|
||
|
martmull
|
23cae2040a |
Improve application asset management (#22564)
App manifests could point the logo and screenshots at either external
URLs or public folder paths, and that was handled inconsistently across
install, sync and the marketplace.
This makes assets always bundled files:
- Manifests now use `logo` and `galleryImages` (a `string[]` of public
folder paths) instead of `logoUrl` and `screenshots`. The old fields
still work but are deprecated. Gallery order comes from the array index.
Normalization (deprecated-field migration, and warning about + ignoring
external URLs) happens in `defineApplication`, so the warnings surface
at define time.
- Logo is stored as a File record (`logoFileId`).
- The registration gallery is configured via a `settings` jsonb column
on `applicationRegistration` (`{ galleryImages: string[] }`) — populated
from the manifest, read by the marketplace detail (falling back to the
legacy `screenshots` column, then the manifest). No dedicated gallery
table.
- The marketplace detail DTO and front now use `galleryImages`.
Verified against a local Postgres: the fast instance commands run with
no pending-migration diff, the schema is correct, and the server boots.
Typecheck, lint, codegen and the application unit tests pass.
Not included yet: rehosting assets into storage for npm catalog and
tarball registrations, versioned cache busting on the serving route, and
a backfill for existing installs.
<a
href="https://cubic.dev/pr/twentyhq/twenty/pull/22564?utm_source=github"
rel="nofollow noreferrer noopener" target="_blank">``<img alt="Review
in cubic"
src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg">``</a>
|
||
|
Thomas Trompette
|
b327ab09a7 |
feat(workflow): add universalIdentifier + applicationId to core workflowVersion (#22747)
## workflowVersion core: syncable columns Adds `universalIdentifier` + `applicationId` (nullable) to `core.workflowVersion`, plus the FK to `core.application` and the `(workspaceId, universalIdentifier)` unique index, via a 2.20 add-columns fast command gated with `@WasIntroducedInUpgrade`. Nullable for now: the already-merged Phase A backfill (#22663) inserts version rows without these columns, so `applicationId` can't be NOT NULL yet. Flipping to NOT NULL + `extends SyncableEntity` comes once they're populated (backfill + dual-write follow-ups). Schema captured and verified via `migrate:generate` (zero drift). Independent of the core-workflow PR, but both add 2.20 upgrade commands, so this one (ts `…480`) must merge **after** the core-workflow PR (ts `…479`), or it gets re-timestamped on rebase. <!-- This is an auto-generated description by cubic. --> <a href="https://cubic.dev/pr/twentyhq/twenty/pull/22747?utm_source=github" target="_blank" rel="noopener noreferrer" data-no-image-dialog="true"><picture><source media="(prefers-color-scheme: dark)" srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source media="(prefers-color-scheme: light)" srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img alt="Review in cubic" src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a> <!-- End of auto-generated description by cubic. --> |
||
|
Thomas Trompette
|
6210389221 |
feat(workflow): add core workflow entity (syncable) + create-table (#22746)
## Core `workflow` entity (syncable) Part of the app-workflows work. Adds a core `WorkflowEntity extends SyncableEntity` (`name`, `lastPublishedVersionId`, plus `universalIdentifier`/`applicationId`/workspace from the base class) and its 2.20 create-table fast command, gated with `@WasIntroducedInUpgrade`. Schema was captured and verified via `migrate:generate` (zero drift, FK/index hashes correct), then run against a live DB. Backfill (populate from workspace `workflow` records) and the dual-write listener land in follow-ups. Independent of the version-syncable-columns PR, but note: both add 2.20 upgrade commands, so this one (ts `…479`) must merge **before** the version PR (ts `…480`) to satisfy the append-only guard. <!-- This is an auto-generated description by cubic. --> <a href="https://cubic.dev/pr/twentyhq/twenty/pull/22746?utm_source=github" target="_blank" rel="noopener noreferrer" data-no-image-dialog="true"><picture><source media="(prefers-color-scheme: dark)" srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source media="(prefers-color-scheme: light)" srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img alt="Review in cubic" src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a> <!-- End of auto-generated description by cubic. --> |
||
|
Paul Rastoin
|
60fd322b49 |
Centralize system field side effects + search field metadata (#22594)
## Introduction Closes twentyhq/core-team-issues#2635 and twentyhq/core-team-issues#2642 and twentyhq/core-team-issues#2589 Object system fields (`searchVector` + its GIN index + `searchFieldMetadata`, the reserved system fields, default relations) were provisioned through several scattered, path-specific code paths. As a result the **app-manifest sync path** authored objects with an empty/`NULL` `searchVector` and **zero `searchFieldMetadata`**, so app-owned objects shipped a broken generated search column (see #22657). The generation logic also lived partly in imperative services rather than in the metadata side-effect engine, and relied on non-deterministic (`v4`) universal identifiers that `twenty apply` could not converge, destroying manually backfilled rows. This PR centralizes every object-creation system side effect into the **metadata side-effect engine**, extends the engine to keep search metadata consistent on field delete and object relabel, makes the standard app's search identifiers deterministic, and ships upgrade commands to reconcile existing workspaces. ## What changed ### Side effects moved into the metadata side-effect engine New dedicated, self-contained handlers — so every write path (API and app manifest) gets identical results, and side effects never trigger other side effects. **Object create / delete** (`handlers/object-metadata`) * **`objectSystemFieldsOnCreate`** — generates the 7 reserved system fields (`id`, `createdAt`, `updatedAt`, `deletedAt`, `createdBy`, `updatedBy`, `position`). * **`objectSearchVectorOnCreate`** — provisions the full-text search surface as one unit: the `searchVector` `TS_VECTOR` field, its backing GIN index, and the `searchFieldMetadata` row (for searchable objects whose label identifier is a searchable field) that keeps `searchVector` populated instead of `NULL`. * **`objectSystemSideEffectsOnDelete`** — tears the above down on object deletion. **Search-metadata consistency on relabel / field delete** (new — these are what close the manifest-path gaps) * **`objectSearchVectorOnUpdate`** (`handlers/object-metadata`) — when a searchable object is relabeled onto a new searchable field, provisions the `searchFieldMetadata` row that indexes it. Relabeling is **additive**: existing rows (e.g. the provisioned `name` row) are preserved, so the previous label identifier stays searchable. Mirrors the API update path so a manifest re-sync that changes the label identifier reaches search parity. No-ops for junction objects (`id` label identifier) and non-searchable field types. * **`fieldSearchFieldMetadataOnDelete`** (`handlers/field-metadata`) — when a field is deleted, cascade-deletes every `searchFieldMetadata` row that indexes it. `searchFieldMetadata` is excluded from manifest deletion inference, so this explicit cascade is what covers **both the API and manifest paths** (the object-scoped DB cascade only fires on object deletion). Uses the `searchFieldMetadataUniversalIdentifiers` aggregator on the flat field for an O(k) lookup instead of scanning all rows. The **default `name` field and default relations are now caller-provided default fields** (SDK autocomplete on the manifest path, input transpiler on the API path) rather than system side effects — removing duplicate name generation, the imperative `build-default-*-for-custom-object` utilities, and the ad-hoc system-field integrity validator. ### Deterministic identifiers for the standard app The twenty-standard search GIN index and `searchFieldMetadata` now derive deterministic universal identifiers (`getIndexUniversalIdentifier` / `getSearchFieldUniversalIdentifier`) instead of `v4`, so `twenty apply` converges instead of recreating. ### Upgrade commands (`2-20`) to reconcile existing workspaces **Instance commands** (run once per instance; ordered fast → slow → workspace): 1. **`AddIsSystemSideEffectToSearchFieldMetadata`** (fast) — adds the `isSystemSideEffect` column to `core.searchFieldMetadata`. Defaults to `true`, which also correctly backfills every existing row since `searchFieldMetadata` is always system-derived (never user-authored). 2. **`BackfillNameFieldIsSystemSideEffect`** (slow) — re-flags existing `name` fields from `isSystemSideEffect: true` → `false`, since the default `name` field is now a caller-provided default like any other user-owned field (it was provisioned as `true` in 2.15 → 2.19). This is a pure data backfill, so the bulk `UPDATE` lives in `runDataMigration()` rather than `up()` — keeping it out of the fast schema transaction avoids holding an `ACCESS EXCLUSIVE` lock that could stall reads during the deploy. Slow instance commands still run before every workspace command of the version, so the fresh value is in place before the search-reconcile workspace commands recompute the `fieldMetadata` flat-entity cache. Scoping by name alone is safe (no engine-owned field is named `name`); `down()` is best-effort (pre-2.15 `false` rows are indistinguishable from flipped ones). **Workspace commands** (idempotent, dry-run supported): 1. **`reconcile-search-vector-gin-index-universal-identifier`** — re-owns every searchVector GIN index UID to its deterministic value (all applications), then backfills the missing GIN index for installed-app objects. 2. **`reconcile-search-field-metadata`** — re-owns every `searchFieldMetadata` UID (all applications), then backfills the missing rows for installed-app searchable objects. 3. **`rebuild-installed-app-search-vectors`** — rebuilds the `searchVector` column of every installed-app `TS_VECTOR` field, once the index and rows exist. Design notes: * **Re-own is global** (twenty-standard, workspace-custom, installed) — a UID convergence keyed on each row's own application. * **Backfill is installed-app only** — standard/custom objects already have these rows via the manifest funnel. * Re-own runs **before** backfill and is transaction-guarded; a failure aborts that workspace to avoid a unique-identifier collision. ## Tests * Integration: app manifest sync now asserts system fields + searchable objects (searchVector, GIN index, searchFieldMetadata) are created; a new relabel suite drives three manifest syncs and asserts records stay searchable through the old + new label identifiers and lose searchability when a field is removed; removed the obsolete system-fields-integrity suite/snapshots. * Unit: per-handler side-effect specs (including the new `objectSearchVectorOnUpdate` and `fieldSearchFieldMetadataOnDelete` handlers), and per-util specs for the re-own / backfill operation builders and the GIN-index classifier. ## Upgrade / migration notes * Existing workspaces converge on the next upgrade run via the `2-20` instance + workspace commands (idempotent, dry-run supported). * Backfill and rebuild go through the workspace-migration runner (automatic cache invalidation); the re-own step invalidates only the affected flat-entity maps directly. * The cross-version upgrade CI now flushes the cache before running the upgrade, so the new version recomputes every flat-entity map from the database instead of reading blobs the old version serialized in an older shape. ## Follow-up * `object-metadata.service.ts` still carries a `TODO: remove once default view fields move to the metadata side effect engine` — default view fields are the next candidate to move into the engine. * A single manifest sync cannot yet both create a field and relabel the object onto it, because `objectMetadata.update` is ordered before `fieldMetadata.create` in the migration runner. Tracked in twentyhq/core-team-issues#2655; to be fixed in a follow-up. |
||
|
Thomas Trompette
|
ca90a9358f |
feat(workflow): backfill workspace workflowVersion into core (phase A) (#22663)
## workflowVersion -> core, Phase A Follows #21674 (Phase 0, merged). Base: `main`. Populates core `workflowVersion` and keeps it in sync with the workspace object, so a later phase can switch reads to core. Reads stay on the workspace object in this PR. ### 1. Backfill (upgrade command) `BackfillWorkflowVersionToCoreCommand`, a `@RegisteredWorkspaceCommand('2.20.0', ...)`. Per workspace, reads all workspace `workflowVersion` records and upserts them into core, preserving ids (idempotent), dry-run aware. ### 2. Dual-write (always on, not flag-gated) `WorkflowVersionCoreDualWriteListener` hooks `@OnDatabaseBatchEvent('workflowVersion', CREATED/UPDATED/DELETED)` (same mechanism as the existing workflow-version status listener) and mirrors every mutation into core. Sync failures are logged, never break the user's write; drift is repaired by re-running the backfill command. Dual-write is deliberately not behind a flag: reading from core (next phase) is only safe if core has been continuously in sync since the backfill. An always-on mirror makes "core is fresh" an invariant, so the read switch becomes a plain flag flip. The cost is one extra upsert on infrequent workflowVersion writes. `IS_WORKFLOW_VERSION_IN_CORE_ENABLED` is reserved for the read switch (Phase B): dispatch from the `workflowAutomatedTriggerMaps` cache, runner and builder reading trigger/steps from core. Until then it gates nothing. Both the backfill and the listener go through a single `WorkflowVersionCoreSyncService` (`upsertToCore`/`deleteFromCore`): the workspace-to-core mapping (`trigger` -> `triggers[]`, plus `steps`, `status`, `workflowId`) and `workflowAutomatedTriggerMaps` invalidation live in one place. ### Rollout plan (following phases) - **B, read switch (flag per workspace):** reads move to core; writes keep flowing workspace -> listener -> core. Rollback = flip the flag back, workspace never stopped being source of truth. - **C, contract (code change):** write paths write trigger/steps to core directly; workspace `workflowVersion` stays as a thin shell (nav/relations/search) but drops the trigger/steps columns; listener and flag removed. ### Not in this PR - Reconciliation tooling beyond re-running the backfill. - The read switch (Phase B). |
