cdeebb1a185e4d53186464fc5ffe576dfd3b6fd9
19 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Thomas Trompette
|
9dd9709e97 |
test(workflow): cover the workflow core mirror end to end (#23434)
## What Adds the integration coverage `core.workflow` mirroring never had: create, rename, delete, restore and destroy, asserting the core row appears, follows the rename, disappears and comes back. No production code changes. The async `WorkflowCoreDualWriteListener` stays as the mirror for the workflow entity. ## Why this PR changed shape It originally replaced the listener with query hooks, to remove the last async best-effort write path. That was the wrong call, for three reasons found while reviewing it: 1. **It would have introduced drift, not removed it.** `workflow-trigger.workspace-service.ts` updates `lastPublishedVersionId` via `workflowRepository.update(...)` when a version is activated. That is a repository write, not an API mutation, so query hooks never fire for it and `core.workflow.lastPublishedVersionId` would have gone stale on **every workflow activation**. The consistency cron compares that field, so it would surface as `fieldMismatch` drift. The listener catches it because events are emitted at the ORM layer. 2. **Hooks are no more atomic than the listener here.** Workflow CRUD goes through the generic API, so there is no dedicated mutation to wrap and the generic runner holds no transaction: it commits, then runs post-hooks. Both approaches are post-commit, with the same drift guarantees. 3. **Events carry the full record; hook payloads do not.** `workspace-insert-query-builder` passes the full formatted result to the event while the client response is filtered by `returning`. That partial payload is what forced the re-fetches, the `coreWorkflowId` pre-hook injection, and the destroy pre-commit special case. All three were workarounds for a problem the listener does not have. The principle we settled on: **use the transaction where one exists, use the listener where there isn't one.** Post-hooks were the worst of both here. Version *content* writes keep their transactional mirror, since those go through dedicated mutations that own a transaction. ## Note on the test The mirror is asynchronous, so the assertions poll (20 attempts, 250ms) rather than reading core immediately after the mutation returns. Without that it would be racy. ## Verification - `nx typecheck twenty-server` green - `oxfmt` + `oxlint --type-aware` green |
||
|
Thomas Trompette
|
2c49c4169c |
feat(workflow): remove async workflowVersion core dual-write listener (#23374)
## What Removes `WorkflowVersionCoreDualWriteListener` (and its now-empty module), replacing the last async best-effort core writes for workflow versions with synchronous mirrors. Adds one missing synchronous funnel so nothing is left uncovered. ## Why After #23356, the listener's `handleRestored` / `handleDeleted` / `handleDestroyed` handlers are redundant with the synchronous lifecycle mirror, so the async path (which can silently drift on failure) can go. While removing it I found one path the listener was **not** redundant on: **direct `deleteOneWorkflowVersion` (discard draft)** is an allowed operation (discard a DRAFT version that isn't the only version, via the `DISCARD_DRAFT_WORKFLOW` command) and had **no** synchronous post-hook. The async listener was the sole thing deleting its core row. Removing the listener without a replacement would have drifted on every draft discard. So this PR also adds a `workflowVersion.deleteOne` post-hook that mirrors the deletion to core. ## Coverage after this change | version lifecycle path | synchronous coverage | | --- | --- | | `deleteOneWorkflowVersion` (discard draft) | **new** `workflowVersion.deleteOne` post-hook | | delete via workflow cascade | `handleWorkflowSubEntities` -> `deleteCoreVersionsByWorkflowIds` (#23356) | | restore via workflow cascade | `handleWorkflowSubEntities` -> `recreateCoreVersionsByWorkflowId` (#23356) | | destroy via workflow | `workflow.destroy*` post-hooks (#23356) | | `deleteMany` / `destroyOne|Many` / `restoreOne|Many` version | blocked by pre-hooks ("Method not allowed") | ## Notes - The new post-hook re-fetches the version (`withDeleted`) to resolve its `coreWorkflowVersionId`, because the delete post-hook payload only carries the columns the client selected (the delete `RETURNING` set is built from `selectedFieldsResult.select`), so `coreWorkflowVersionId` is not reliably present. - `deleteCoreVersionsByWorkspaceVersionIds` deletes precisely by `coreWorkflowVersionId` (not by `workflowId`), so discarding one draft does not touch the core rows of the workflow's other versions. - The workflow-side `WorkflowCoreSyncModule` listener is intentionally left in place (separate migration track). ## Verification - `nx typecheck twenty-server` green - `nx lint:diff-with-main twenty-server` green - Added integration test `workflow-version-discard-draft-core-mirror`: activate v1, create a draft, discard it, assert only the draft's core row is removed and the active version's core row remains. - Live run on a dev instance still pending. ## Merge gate Per the migration plan, removing the async backstop should land only after the drift cron reports zero drift over a soak period. <!-- This is an auto-generated description by cubic. --> <a href="https://cubic.dev/pr/twentyhq/twenty/pull/23374?utm_source=github" target="_blank" rel="noopener noreferrer" data-no-image-dialog="true"><picture><source media="(prefers-color-scheme: dark)" srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source media="(prefers-color-scheme: light)" srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img alt="Review in cubic" src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a> <!-- End of auto-generated description by cubic. --> |
||
|
Thomas Trompette
|
79bf20515d |
feat(workflow): mirror version delete/restore/destroy to core transactionally (#23356)
Next step in the workflowVersion -> core soft-ref migration. The transactional mirror (#23243) made **content** writes (create/update) drift-free. This does the same for the **lifecycle** events (delete / restore / destroy), which were still handled only by the async best-effort listener. It's the prerequisite for dropping that listener. ## What changed `delete` and `restore` already soft-delete / restore the workflow's versions inside `handleWorkflowSubEntities` (twenty doesn't cascade soft-deletes, so it does each sub-entity explicitly). So the core delete/recreate just sits next to the existing version write: - **delete** — after `workflowVersionRepository.softDelete({ workflowId })`, `deleteCoreVersionsByWorkflowIds` removes the `core.workflowVersion` rows (`workflowId IN (...)`). (`deactivateVersionOnDelete` no longer re-mirrors the deactivated version — that was recreating the core row it just deleted; it only flips the workspace status to `DEACTIVATED` so a restore comes back deactivated.) - **restore** — after `workflowVersionRepository.restore({ workflowId })`, `recreateCoreVersionsByWorkflowId` re-reads the restored versions and reuses the existing `upsertToCore` (which reuses the stored `coreWorkflowVersionId` soft-ref, so rows come back with their original ids and current status). - **destroy** — version destroy isn't done in `handleWorkflowSubEntities` (it happens via the generic cascade), so there's no existing place to hang the core delete. New `workflow.destroyOne`/`destroyMany` **post**-hooks call `deleteCoreVersionsByWorkflowIds` (batched `IN`) only after the destroy commits, so a rejected destroy can't remove core rows while the workspace versions survive. No new transactional wrappers or raw SQL — the delete/recreate reuse the existing `WorkflowVersionCoreSyncService` methods (`deleteFromCore`-style delete, `upsertToCore`). The async listener stays as an idempotent backstop until the cron soaks zero drift. ## Async listener kept as backstop `handleRestored` / `handleDeleted` / `handleDestroyed` stay for now. Both paths are idempotent (delete-of-deleted is a no-op; upsert converges), so they don't conflict. Those handlers come out in a follow-up once the consistency cron soaks zero drift - which this PR unblocks. ## Verification Lifecycle integration test — creates a workflow, **activates** the version (the active path is where the delete re-mirror bug bit), then asserts the core row: present -> gone after delete -> back after restore (as `DEACTIVATED`, same id) -> gone after destroy. Plus the existing `workflow-resolver` delete/restore suite (regression, since `handleWorkflowSubEntities` is shared). Also verified **live** on a running instance against the real DB: the full active-version lifecycle above, plus a batched `destroyWorkflows` on two workflows removing both core rows in one `IN` delete. `nx typecheck` + oxlint + oxfmt clean. <!-- This is an auto-generated description by cubic. --> <a href="https://cubic.dev/pr/twentyhq/twenty/pull/23356?utm_source=github" target="_blank" rel="noopener noreferrer" data-no-image-dialog="true"><picture><source media="(prefers-color-scheme: dark)" srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source media="(prefers-color-scheme: light)" srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img alt="Review in cubic" src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a> <!-- End of auto-generated description by cubic. --> |
||
|
Thomas Trompette
|
46a3a83866 |
feat(workflow): mirror all workflowVersion writes to core in-transaction, drop async create/update dual-write (#23243)
Switches the workflowVersion -> core dual-write from the async,
best-effort listener to a **transactional mirror**, wires every
content-write funnel through it, and drops the async create/update
handlers. Core can no longer drift from the workspace on any covered
path: the core copy commits or rolls back atomically with the workspace
write.
## Helper (`WorkflowVersionCoreSyncService`)
Two entry points, both writing `core.workflowVersion` and stamping the
`coreWorkflowVersionId` soft-ref on the **caller's transaction
manager**:
- `writeWorkflowVersionAndMirror(workspaceId, write)` - for funnels that
don't own a transaction. Opens a workspace queryRunner, runs the
caller's workspace write on that manager, re-reads the row, mirrors to
core in the same tx, commits, then invalidates the trigger-map cache
post-commit.
- `mirrorWorkflowVersionWrite({ workspaceId, entityManager,
workflowVersion })` - for funnels that already own a queryRunner tx
(activation / deactivation / delete-cascade); they just gain this one
call on their existing manager before commit.
`invalidateAutomatedTriggerMaps` is public so tx-owning callers run it
post-commit.
## Funnels wired (all content writes now mirror in-transaction)
- `updateWorkflowVersionStepsAndTrigger` (central builder step/trigger
edit)
- edge create/delete (4 trigger/step writes)
- `createDraftFromWorkflowVersion` (update + insert),
`duplicateWorkflow` (content update), `updateWorkflowVersionPositions`
- iterator / if-else empty-node step writes
- `workflow.createOne` / `createMany` post-hooks (v1 draft insert)
- AI `create_complete_workflow` tool (v1 insert)
- activation / deactivation status writes (ACTIVE / ARCHIVED /
DEACTIVATED) on their existing tx
- `deactivateVersionOnDelete` cascade (ACTIVE -> DEACTIVATED) on its
existing tx
Direct `workflowVersion.createOne/createMany` is forbidden by a
pre-hook, so there is no un-funneled create path.
## Async listener trimmed
`handleCreated` and `handleUpdated` are removed: every create/update now
mirrors in-transaction, so the post-commit handlers were redundant and
were the source of the rollback-drift (an edit that rolls back still
emitted an `UPDATED` event carrying the uncommitted payload, which the
async listener wrote to core).
`handleRestored`, `handleDeleted`, `handleDestroyed` are **kept**.
Soft-delete/restore go through the generic ORM (not a funnel):
`handleDeleted` drops the core row on soft-delete, and `handleRestored`
recreates it on restore (the restore path just calls
`workflowVersionRepository.restore()`). Removing the restore handler
would leave restored versions with no core row, so the delete/restore
pair stays async.
## Why the core write is raw SQL
`core.workflowVersion` is on the core DataSource, not the workspace
DataSource, so a repository can't be pointed at it from the workspace
queryRunner's manager. But both schemas are one Postgres DB and a
queryRunner is a single connection, so a schema-qualified `INSERT INTO
core."workflowVersion" ... ON CONFLICT` on that manager participates in
the workspace transaction (the prefill util's pattern). The workspace
write and soft-ref write-back go through the ORM's
`repository.update(criteria, data, undefined, queryRunner.manager)`, so
the source-of-truth workspace write keeps its ORM machinery (actor
stamping, search vector, events); only the dumb core mirror is raw,
confined to the helper.
**jsonb:** the raw insert has no entity transformer, so
`triggers`/`steps` are passed as JSON strings (Postgres parses them) -
the inverse of the prefill core insert (the #23204 double-encode trap),
covered by the test. `universalIdentifier` is minted only for a new
link; on conflict only `triggers`/`steps`/`status` are updated.
## Test
In-process integration test: opens a workspace queryRunner, calls the
helper, asserts the core row is visible inside the tx with correct
native jsonb, rolls back, asserts the core row is gone - empirically
confirming one workspace queryRunner writes `core.*` in the same tx.
## Review feedback addressed
- **Duplicate atomicity:** `duplicateWorkflow` now inserts the draft
version with its final steps/trigger inside
`writeWorkflowVersionAndMirror`, so a mirror failure rolls back the
whole version instead of leaving an unmirrored empty draft.
- **Delete cascade:** `deactivateVersionOnDelete` moved its command-menu
cleanup after commit, so a rolled-back deactivation can no longer strip
the menu item from a still-active version.
- **Rollback drift / unlinked rows:** resolved structurally by the
in-transaction mirror + removal of the async CREATED/UPDATED handlers,
and by the soft-ref-field guard that skips mirroring (returns null) on
workspaces without `coreWorkflowVersionId`.
- **Unit suites:** the step-helpers, step-operations and edge suites now
provide a `WorkflowVersionCoreSyncService` mock whose
`writeWorkflowVersionAndMirror` runs the write callback against the test
repo; all three pass (35 tests).
## Verification
Rebased onto `origin/main`. `nx typecheck twenty-server` is green, the
three affected unit suites pass (35 tests), and every changed file
passes type-aware oxlint + oxfmt. Integration suites were failing only
on behind-main DB-init drift (`core.keyValuePair` / data-migration),
which the rebase resolves. Live end-to-end test on a running instance
still pending.
|
||
|
Thomas Trompette
|
cbcfba0de2 |
feat(workflow): dedicated updateWorkflowVersionTrigger mutation + close version CRUD holes (#23207)
Prerequisite for the workflow-core soft-ref migration: every
`workflowVersion` content write must go through a dedicated,
draft-guarded server mutation so it can later be wrapped in a
transactional core mirror. This closes the generic-CRUD holes that let
writes bypass that path.
## Part A - dedicated `updateWorkflowVersionTrigger` mutation
The builder saved a version's trigger through generic
`updateOneWorkflowVersion` - the only content write not going through a
dedicated mutation. Added:
- Server: `updateWorkflowVersionTrigger(input: { workflowVersionId,
trigger })` resolver +
`WorkflowVersionStepWorkspaceService.updateWorkflowVersionTrigger`,
draft-guarded via `getValidatedDraftWorkflowVersion` then
`updateWorkflowVersionStepsAndTrigger` (reuses existing write logic).
- Front: `useUpdateWorkflowVersionTrigger` now calls the dedicated
mutation instead of `useUpdateOneRecord`.
## Part B - restrict generic `updateOneWorkflowVersion`
`validateWorkflowVersionForUpdateOne` previously allowed writing
`trigger`, `position`, `workflowId` (re-parenting),
`coreWorkflowVersionId`, and let `steps: null` slip through on a draft.
It now rejects any update that sets `steps`, `trigger`, `status`,
`workflowId`, or `coreWorkflowVersionId`, or that clears the `name`,
while still allowing a plain rename. (A name-only allowlist was tried
first but blocked legitimate renames - at the pre-hook the generic
update payload is not single-key - so it was replaced by this denylist,
verified live.)
## Part C - close the destroy/restore hole
`workflowVersion` had no `destroyOne/destroyMany/restoreOne/restoreMany`
query hooks, so a caller with object permission could hard-destroy any
version (including active) or resurrect one with no validation. Added
pre-hooks that forbid all four via the API ("Method not allowed"),
matching the existing forbidden generic mutations (`createOne`,
`deleteMany`, ...). Rationale: there is no legitimate API use for
standalone version destroy/restore - retention purging happens through
the trash-cleanup cron (internal, not hook-gated) and restore happens
through the workflow-restore cascade or create-draft-from-version.
## Tests
Integration specs that set a trigger through the generic mutation were
migrated to the new `updateWorkflowVersionTrigger` mutation (new
`update-workflow-version-trigger.util.ts`). Unit test for the front hook
updated.
## Verification
- `twenty-server` + `twenty-front` typecheck: clean.
- oxlint + oxfmt on all changed files: clean.
- `graphql.ts` regenerated for the new mutation; its types match the
server DTOs exactly. Local `graphql:generate` introspects a running
server, so it only succeeds against a server built from this branch - CI
regenerates against the PR server and verifies.
<!-- This is an auto-generated description by cubic. -->
<a
href="https://cubic.dev/pr/twentyhq/twenty/pull/23207?utm_source=github"
target="_blank" rel="noopener noreferrer"
data-no-image-dialog="true"><picture><source
media="(prefers-color-scheme: dark)"
srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source
media="(prefers-color-scheme: light)"
srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img
alt="Review in cubic"
src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a>
<!-- End of auto-generated description by cubic. -->
---------
Co-authored-by: prastoin <paul@twenty.com>
|
||
|
Thomas Trompette
|
cb2e325c4b |
fix(workflow): make prefilled workflow ids unique per workspace (#22800)
## Problem \`prefillWorkflows\` (run for every workspace on \`activateWorkspace\`) inserts workflows and versions with **hardcoded ids** (\`QUICK_LEAD_WORKFLOW_ID = 8b213cac...\`, etc.). So every workspace carries the same workflow/version record ids. Within a workspace schema that's harmless, but it means workspace record ids are **not unique across workspaces**, which: - breaks the workflowVersion backfill on the shared core table (surfaced as the \`IDX_WORKFLOW_VERSION_ONE_ACTIVE_PER_WORKFLOW\` duplicate-key error, since multiple workspaces claim the same active \`workflowId\`), and - collides on \`core.workflow\`/\`core.workflowVersion\` PKs once workflows migrate to core (the core row reuses the workspace record id), causing cross-workspace clobbering. ## Fix Derive the prefill ids **deterministically per workspace**: \`getWorkflowPrefillIds(workspaceId)\` returns \`v5(label:workspaceId, namespace)\` for each of the workflow/version/trigger ids. Deterministic (stable across the idempotent \`orIgnore\` re-runs) but unique per workspace. The command-menu-item prefill uses the same helper so its \`workflowVersionId\` reference stays consistent. Only affects **new** workspaces; existing workspaces keep their current ids (prefill is skipped on re-activation). ## Test Reset seeds two workspaces; both now get a Quick Lead workflow with a **distinct** v5-derived id (not the old \`8b213cac\`), and internal references stay consistent (\`version.workflowId == workflow.id\`, \`lastPublishedVersionId == version.id\`). Typecheck + lint clean. Companion to #22795 (which scopes the active index to workspace). Together they fix the backfill duplicate-id failures. <!-- This is an auto-generated description by cubic. --> <a href="https://cubic.dev/pr/twentyhq/twenty/pull/22800?utm_source=github" target="_blank" rel="noopener noreferrer" data-no-image-dialog="true"><picture><source media="(prefers-color-scheme: dark)" srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source media="(prefers-color-scheme: light)" srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img alt="Review in cubic" src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a> <!-- End of auto-generated description by cubic. --> |
||
|
neo773
|
f14e62ec0c |
Run integration tests against the real BullMQ driver (#22551)
Migrate whole suite to real BullMQ Shard times unchanged, still 5-6 min. |
||
|
Félix Malfait
|
2abf9c2930 |
feat(workflow): Pick Record load balanced strategy (3/3) (#21902)
## Overview Final PR in the Pick Record stack. Adds the **Load Balanced** strategy: pick the candidate that currently has the *fewest related records*. This is the "fair assignment" mode — e.g. assign a new company to the account owner who currently owns the fewest companies, or route a lead to the rep with the fewest open opportunities. **Stacked on #21900** (which is stacked on #21899) — merge in order. This PR's diff against `main` includes PRs 1 & 2 until they merge. ## What changed - Widened the `strategy` enum to add `LOAD_BALANCED`, and added an optional `loadBalance: { objectNameSingular, fieldName }` to the action input. - Editor: selecting **Load balanced** reveals a **Balance by** object picker and a **Count by** field picker (the related object's many-to-one relation fields). - Executor: for each candidate, counts records of the chosen related object whose chosen relation points at that candidate, then selects the least-loaded one. ## How it works Given pool = workspace members and config `{ objectNameSingular: "opportunity", fieldName: "pointOfContact" }`, the executor counts, per member, the opportunities whose `pointOfContact` is that member, and picks the member with the lowest count. ## Design decisions & tradeoffs 1. **No persistent state — computed live each run.** Unlike round robin, load balancing reads current data, so there's no cursor to store. Correct by construction even under concurrency (each run recomputes counts); the only caveat is two simultaneous runs can both see the same "least loaded" candidate before either assignment lands (a small, self-correcting skew), which is inherent to load-balancing and acceptable. 2. **Count via per-candidate queries.** One filtered count per candidate (`{ [relationField]: { id: { eq: candidateId } } }`), run in parallel. For the realistic pool sizes this targets (a team), this is simple and clear. A single `group_by` aggregate would scale better for very large pools — noted as a future optimization, deliberately not done to keep the logic obvious. 3. **Deterministic tie-break.** Candidates are pre-sorted by id (shared with round robin), and the first minimum wins — so equal-load ties resolve deterministically rather than arbitrarily. 4. **`Count by` lists all many-to-one relations of the chosen object** (not filtered to those targeting the pool object). Keeps the editor simple; picking an unrelated field just yields zero counts, which is visibly wrong. Filtering options to relations that target the pool object is a nice follow-up. 5. **Filter on the counted set** (e.g. only *open* opportunities) is intentionally out of scope for this first cut — documented as a follow-up. ## Testing Added `pick-record-load-balanced-workflow.integration-spec.ts`: creates two fresh companies (0 related opportunities each), attaches one opportunity to the second, configures `LOAD_BALANCED` counting opportunities by `company`, and asserts the step picks the **first** company (0 < 1). Passes locally alongside the random and round-robin tests (3 suites / 4 tests). `typecheck` + `lint:diff-with-main` green for shared/server/front. ## The full stack 1. #21899 — Random (the action + the whole scaffold) 2. #21900 — Round robin (atomic Redis cursor) 3. this — Load balanced Together these enable round-robin / load-balanced / random **assignment workflows** in Twenty, composed via the standard variable picker (assign the chosen record downstream with `{{step.<id>.id}}`). https://claude.ai/code/session_01MuPWZsqf2bmQSevk6fRbX8 --- _Generated by [Claude Code](https://claude.ai/code/session_01MuPWZsqf2bmQSevk6fRbX8)_ <!-- This is an auto-generated description by cubic. --> <a href="https://cubic.dev/pr/twentyhq/twenty/pull/21902?utm_source=github" target="_blank" rel="noopener noreferrer" data-no-image-dialog="true"><picture><source media="(prefers-color-scheme: dark)" srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source media="(prefers-color-scheme: light)" srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img alt="Review in cubic" src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a> <!-- End of auto-generated description by cubic. --> |
||
|
Félix Malfait
|
fa6d1394af |
feat(workflow): Pick Record round robin strategy (2/3) (#21900)
## Overview Second PR in the Pick Record stack. Adds a **Round Robin** selection strategy alongside Random, so an assignment workflow can distribute records *evenly* across a candidate pool (e.g. rotate company ownership across a set of workspace members) rather than just randomly. **Stacked on #21899** — review/merge that one first. This PR's diff against `main` includes PR 1's commits until #21899 merges. ## What changed - Widened the `strategy` enum (`RANDOM` → `RANDOM | ROUND_ROBIN`) in the shared schema and the server input type. - Editor now shows a **Strategy** selector (Random / Round robin). The candidate-pool label changed from "Pick at random from" to the neutral "Pick from" since random is no longer the only mode. - Executor implements round robin. ## Design decisions & tradeoffs 1. **State store: Redis `incrBy` (atomic), keyed `pick-record:round-robin:{workspaceId}:{stepId}`.** Round robin needs a persistent cursor, and workflow runs are **not** serialized — two runs can execute the same step concurrently — so the increment must be atomic. `CacheStorageService.incrBy` (workflow cache namespace) is a single atomic Redis op, needs no schema change, and is already injectable. Index = `(cursor - 1) % poolSize`. **Tradeoff — durability:** a Redis flush/eviction resets the cursor, which restarts the cycle from an offset. That causes a one-time *fairness drift*, never a *correctness* bug (no double-assignment, since each increment is atomic). If strict durability is ever required, the cursor can move to a Postgres counter table with `INSERT … ON CONFLICT … DO UPDATE SET cursor = cursor + 1 RETURNING cursor` (atomic + durable) — deliberately **not** done here to avoid a migration for what is, in practice, an acceptable reset. 2. **Deterministic pool ordering.** The resolved pool is sorted by `id` before the cursor is applied, so position→record mapping is stable run-to-run regardless of fetch order. Without this, round robin wouldn't reliably cycle. 3. **Cursor key uses `stepId`.** Stable across runs of a published version. Republishing a version may mint new step ids, which resets the cursor — acceptable and documented here. 4. **Slot-on-increment.** The cursor increments when the step runs (reserving a position); if a later step in the run fails, that position is effectively skipped. Minor, acceptable unfairness — flagged rather than adding cross-step compensation. ## Testing Added `pick-record-round-robin-workflow.integration-spec.ts`: builds a workflow with a 3-record pool and `ROUND_ROBIN`, runs it 4 times sequentially, and asserts the picks are exactly `[p0, p1, p2, p0]` (full cycle + wraparound) against the deterministically-ordered pool. Passes locally alongside PR 1's random test (2 suites / 3 tests). `typecheck` + `lint:diff-with-main` green for shared/server/front. ## Follow-up - PR 3: `LOAD_BALANCED` (fewest related records wins). https://claude.ai/code/session_01MuPWZsqf2bmQSevk6fRbX8 --- _Generated by [Claude Code](https://claude.ai/code/session_01MuPWZsqf2bmQSevk6fRbX8)_ <!-- This is an auto-generated description by cubic. --> <a href="https://cubic.dev/pr/twentyhq/twenty/pull/21900?utm_source=github" target="_blank" rel="noopener noreferrer" data-no-image-dialog="true"><picture><source media="(prefers-color-scheme: dark)" srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source media="(prefers-color-scheme: light)" srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img alt="Review in cubic" src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a> <!-- End of auto-generated description by cubic. --> |
||
|
Félix Malfait
|
573fd00ea7 |
feat(workflow): add Pick Record action (1/3 — random selection) (#21899)
## Overview
Adds a new workflow action, **Pick Record**, that selects **one** record
from a configured candidate pool and exposes the chosen record as the
step's output. Downstream steps can then reference it through the normal
variable picker — e.g. assign an owner in an _Update Record_ step by
setting **Account Owner = `{{step.<pickRecordId>.id}}`**.
This is the foundation for building **assignment workflows**
(round-robin / load-balanced owner assignment, reviewer rotation, etc.)
in Twenty.
## This is PR 1 of a 3-PR stack
| PR | Strategy | Adds |
|----|----------|------|
| **1 (this one)** | `RANDOM` | The whole `PICK_RECORD` action,
end-to-end, stateless |
| 2 | `ROUND_ROBIN` | A persistent, atomically-incremented per-step
cursor + the strategy selector UI |
| 3 | `LOAD_BALANCED` | "fewest related records wins" via an aggregate
count |
Each PR widens the `strategy` enum (a backward-compatible change), so no
data migration is needed between them.
## How it works
- **Editor**: pick an Object, then pick the candidate records (a
multi-record selector). A random record is selected from that pool at
run time.
- **Output**: a single record of the chosen object — the same output
shape as `CREATE_RECORD`/`UPDATE_RECORD` — so it drills into
`{{step.x.id}}`, `{{step.x.name}}`, … in the variable picker.
- **Execution**: reuses `FindRecordsService` to fetch the pool (`id IN
(recordIds)`, which also transparently drops any deleted candidates),
then returns one at random.
## Design decisions & tradeoffs
1. **Standalone step that outputs a variable, not an inline "random"
mode on the relation field.** This mirrors Attio's round-robin block.
The decisive reason is composition: the chosen record is almost always
reused (assign owner **and** create a follow-up task for them **and**
email them). A variable is chosen once and reused everywhere; an inline
per-field value would re-roll independently in each place. It also keeps
the (stateful) round-robin/load-balanced logic out of the field inputs.
Tradeoff: one extra step to wire up vs. an inline control — accepted for
the composability win. An inline "Assign automatically" entry point can
still be layered on later as sugar that inserts this step.
2. **Co-located in the `record-crud` action module and reuses
`FindRecordsService`.** Avoids duplicating module wiring (auth context,
permissions, object-metadata resolution) and the data-access path.
Tradeoff: "Pick" is a selection rather than a CRUD op, so the folder
name is slightly broad; chose reuse + low risk over a separate module.
Can be extracted if the family grows.
3. **`strategy` exists in the schema (defaulted `RANDOM`) but the
selector is hidden in this PR.** A dropdown with a single option would
be UX slop, and adding the field only in PR 2 would force a data
backfill for any `PICK_RECORD` steps created in between. Keeping the
field now (hidden) avoids both. PR 2 introduces the selector once
there's a real choice.
4. **Pool is an explicit static list (`recordIds`) for v1.** Matches the
most common assignment case ("rotate among these N people") and reuses
the existing `FormMultiRecordPicker`. A filter-based pool (reusing the
Find Records filter UI) and a list-from-a-previous-step pool are natural
follow-ups, intentionally out of scope here to keep the stack focused on
the three strategies.
5. **Output schema is computed on the frontend** (like `CREATE_RECORD`),
derived from `input.objectName` — so it is **not** added to
`PERSISTED_OUTPUT_SCHEMA_TYPES` and needs no server-side schema
computation.
6. **Validation**: `PICK_RECORD` is added to object-name metadata
validation (so a deleted/invalid target object is flagged) via a
dedicated `OBJECT_TARGETING_ACTION_TYPES` set — deliberately **not** to
`VARIABLE_CONSUMING_ACTION_TYPES`, because a static pool legitimately
references no upstream variable and would otherwise raise a spurious "no
variable reference" warning.
7. **Empty pool → step error** at run time (respecting the step's
error-handling options) rather than a silent no-op, since an empty pool
is a misconfiguration or fully-deleted set.
8. **`Math.random`** is used for selection — no cryptographic guarantee
is needed for assignment fairness.
## Testing
Per our testing convention (integration test over service/`.spec`
tests): added `pick-record-workflow.integration-spec.ts`, which builds a
workflow with a manual trigger + a `PICK_RECORD` step, configures a
known two-record pool, runs it, and asserts the run completes and the
picked record is **always** within the configured pool (verifying the
pool filter) across repeated runs.
Local verification (typecheck + lint for shared/server/front) is green;
running the integration suite and attaching editor screenshots in a
follow-up comment.
## Follow-ups
- PR 2: `ROUND_ROBIN` + persistent atomic cursor (Redis `incrBy` vs. a
Postgres counter table — tradeoff to be documented on that PR) +
strategy selector.
- PR 3: `LOAD_BALANCED`.
- Later (not in this stack): filter-based / variable-list pools, an
inline "Assign automatically" entry point on relation fields, OOO-skip /
weighting.
https://claude.ai/code/session_01MuPWZsqf2bmQSevk6fRbX8
---
_Generated by [Claude
Code](https://claude.ai/code/session_01MuPWZsqf2bmQSevk6fRbX8)_
<!-- This is an auto-generated description by cubic. -->
<a
href="https://cubic.dev/pr/twentyhq/twenty/pull/21899?utm_source=github"
target="_blank" rel="noopener noreferrer"
data-no-image-dialog="true"><picture><source
media="(prefers-color-scheme: dark)"
srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source
media="(prefers-color-scheme: light)"
srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img
alt="Review in cubic"
src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a>
<!-- End of auto-generated description by cubic. -->
|
||
|
Félix Malfait
|
a0689d1577 |
feat(workflow): condition filter on database-event triggers (#21868)
## Problem
Connecting a mailbox bulk-creates contacts via the email/calendar sync,
and each `person.upserted` fires the seeded **"Create company when
adding a new person"** workflow. The trigger enqueues one run per record
(no batching) and each run bills several `WORKFLOW_NODE_RUN` events — so
a single mailbox connect can rack up tens of thousands of runs and
exhaust credits on a brand-new workspace. The workflow is also redundant
on that path: the sync already creates the company from the email domain
and links the person to it.
## What this does
Adds an optional, user-defined **filter** to database-event (listener)
triggers, evaluated in the listener **before a run is enqueued**.
Non-matching events never create a run, so they consume zero execution
credits. This is the Filter node's capability, lifted to the trigger
level, and available for all event types (created / updated / upserted /
deleted).
The seeded "Create company when adding a new person" workflow now
carries a visible trigger filter — `Created by → Source is not Email`
**and** `is not Calendar` — so it no longer runs for sync-created
contacts, while still running for manually / API / CSV-added people.
## How (reuse)
- **Backend:** extracted `evaluateStepFilters()`, shared by the Filter
action and the trigger listener's new `eventMatchesRecordFilter` gate.
The record is exposed under the `trigger` key so filters reference it
exactly like steps do (`{{trigger.properties.after.…}}`).
- **Shared:** one optional `filter` added to the database-event trigger
zod schema; the front-end type derives from it (settings stay JSON — no
codegen).
- **Frontend:** extracted `WorkflowStepFilterBuilder` from the Filter
action's body; both the Filter action and the trigger editor render it.
The field picker needed no changes — at the trigger it already resolves
to the record's own fields via `TRIGGER_STEP_ID`.
## Scope / decisions
- **No migration for existing workspaces** (by request) — only newly
created workspaces get the filtered default; already-created workspaces
keep the always-on workflow.
- Deliberately did **not** add relation-enrichment to the upsert path
(it would add a DB lookup to the very bulk-sync path we're relieving).
Trigger filters work on the record's own scalar/composite fields (e.g.
`createdBy.source`); relation-based filters work on created/updated
where enrichment already runs.
## Verification
- Typecheck: `twenty-shared`, `twenty-server`, `twenty-front` all green.
- Lint (diff, autofix): 0 warnings / 0 errors across all three.
- Unit tests: a new `evaluate-step-filters` spec exercising the exact
`createdBy.source IS_NOT` seed mechanism, plus new listener specs
proving non-matching events are not enqueued. All backend
filter/listener suites pass.
- Not run here: integration tests (need a DB) and Storybook.
https://claude.ai/code/session_013k36vfekDppwRCgM6Ha7De
---
_Generated by [Claude
Code](https://claude.ai/code/session_013k36vfekDppwRCgM6Ha7De)_
<!-- This is an auto-generated description by cubic. -->
<a
href="https://cubic.dev/pr/twentyhq/twenty/pull/21868?utm_source=github"
target="_blank" rel="noopener noreferrer"
data-no-image-dialog="true"><picture><source
media="(prefers-color-scheme: dark)"
srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source
media="(prefers-color-scheme: light)"
srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img
alt="Review in cubic"
src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a>
<!-- End of auto-generated description by cubic. -->
---------
Co-authored-by: Claude <noreply@anthropic.com>
|
||
|
Thomas Trompette
|
b36c0c51c3 |
fix(server): keep workflow command menu item label in sync with workflow name (#21490)
## Summary Fixes #20766 — manual-trigger workflows showed `Manual Trigger` in the command menu instead of the workflow's name. Root cause (confirmed against a live instance): the command menu item's `label` is written **only at activation** in `createOrUpdateCommandMenuItem`, from `workflow.name`, with a hardcoded `'Manual Trigger'` fallback. So: - a workflow activated while unnamed gets the misleading `Manual Trigger` label, and - renaming the workflow afterwards never updates the label (`workflow.updateOne` had no label-related hook). Changes: - Add `getWorkflowCommandMenuItemLabel` helper and use it in activation; the empty-name fallback is now `Untitled Workflow` (consistent with the rest of the UI) instead of `Manual Trigger`. - Add `WorkflowCommandMenuSyncWorkspaceService` that updates the active version's command menu item label/shortLabel from the workflow name (idempotent, no-op for non-manual / inactive workflows). - Add `workflow.updateOne` and `workflow.updateMany` post-query hooks that call the sync service, registered in `WorkflowQueryHookModule`. Out of scope (separate follow-up): the activation create path can produce duplicate command items for one `workflowVersionId`; recommend making it idempotent / adding a unique constraint. ## Test plan - [x] `oxlint --type-aware` + `oxfmt` clean on changed files - [x] Editor TS diagnostics clean (full `nx typecheck` was starved by local dev servers) - [ ] New integration test `workflow-command-menu-label.integration-spec.ts`: - labels the command menu item with the workflow name on activation - updates the label when the workflow is renamed - falls back to `Untitled Workflow` when the name is cleared <!-- This is an auto-generated description by cubic. --> <a href="https://cubic.dev/pr/twentyhq/twenty/pull/21490?utm_source=github" target="_blank" rel="noopener noreferrer" data-no-image-dialog="true"><picture><source media="(prefers-color-scheme: dark)" srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source media="(prefers-color-scheme: light)" srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img alt="Review in cubic" src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a> <!-- End of auto-generated description by cubic. --> |
||
|
Charles Bochet
|
58907b733c |
feat(logic-function): add LIVE / PREBUILT execution modes (#20873)
## Summary
### Why
1. Sending the code to the lambda (~1Mb usually) is heavy on network and
results to a constant traffic of ~30Mb/s on AWS which results into TB of
network data every month
2. eval(1MB of code) is not that fast, it's heavy on memory and CPU on
lambda side
### High level
Adds two execution modes for logic functions, gated behind the new
`IS_LOGIC_FUNCTION_PREBUILT_MODE_ENABLED` workspace feature flag (off
everywhere by default):
- **LIVE** (current behavior, preserved bit-for-bit): the compiled
bundle is read from object storage and shipped in every Lambda invoke
payload. Used for fast iteration in the workflow editor / Settings test
runs.
- **PREBUILT** (new): the bundle is installed onto the per-function
Lambda alongside the unified executor, and invocations carry only `{
params, env, handlerName }` — saving JSON payload egress and warm-start
`import()` cost on every call.
### Key design choices
- **Unified Lambda handler** (`constants/executor/index.mjs`) dispatches
at runtime: `event.code` present ? LIVE (write to `/tmp`, dynamic
import) : `import('./prebuilt-logic-function.mjs')`. Both code paths
always coexist on the deployment package, so the same Lambda can serve
either mode without redeploying.
- **Install runs inside the `validateBuildAndRun` migration pipeline**,
not at execute time. `Create/UpdateLogicFunctionActionHandlerService`
calls `driver.installPrebuiltBundle` when `executionMode` flips
LIVE?PREBUILT or `checksum` changes while PREBUILT, gated on
`isBuildUpToDate=true` and a fresh checksum.
- **Strict execute, no reconciliation**:
`LogicFunctionExecutorService.execute` resolves `effectiveExecutionMode`
(caller override > feature flag > entity column). For PREBUILT it asks
the driver `getInstalledBundleChecksum` (Lambda `twenty:bundle-checksum`
tag for AWS, sidecar file locally) and throws
`LOGIC_FUNCTION_PREBUILT_BUNDLE_NOT_INSTALLED` on mismatch.
- **Feature flag gates every side effect**: with the flag off the
executor forces LIVE, the action-handler install hooks bail before AWS,
and workflow activation does not flip the mode. Rollback is just turning
the flag off.
### Lifecycle
- New workflow CODE step ? `LIVE`, no install.
- Workflow activated ? build + activation flips `executionMode=PREBUILT`
? action-handler installs the bundle + sets the Lambda tag.
- Draft from active version ? duplicated logic function reset to `LIVE`.
- App install ? manifest converter sets `PREBUILT`, create-action
handler installs.
- Test runs (`executeOneFromSource`, workflow editor) pass
`executionMode=LIVE` explicitly.
### Observability
`[lambda-timing]` log lines now include `effectiveExecutionMode` and
`payloadBytes`; the action handler logs `install_duration_ms` for each
install.
## Test plan
- [x] `npx nx typecheck twenty-server` ? passes
- [x] `npx oxlint --type-aware` on all changed files ? 0 warnings, 0
errors
- [x] `npx nx test twenty-server` ? 588 suites / 5009 tests pass (no
regressions vs main)
- [x] New unit suite `flat-logic-function-validator.service.spec.ts` ?
9/9
- [x] Existing
`workflow-version-step-operations.workspace-service.spec.ts` ? 8/8
(verified the new token-based DI avoids a circular-import regression)
- [x] Snapshot for
`ALL_UNIVERSAL_FLAT_ENTITY_PROPERTIES_TO_COMPARE_AND_STRINGIFY` updated
to include `executionMode`
- [x] Integration suite `logic-function-execution.integration-spec.ts`
extended to assert `executionMode=LIVE` on newly-created functions and
continues to exercise the LIVE happy path
- [ ] Manual staging rollout: flip
`IS_LOGIC_FUNCTION_PREBUILT_MODE_ENABLED` per workspace, observe
`[lambda-timing]` `payloadBytes` drop + `install_duration_ms`, then ramp
in prod.
|
||
|
Paul Rastoin
|
3d49c17e34 |
[CONNECTED_ACCOUNT_BREAKING_CHANGE] Unify connected account permissions (#20732)
# Introduction This PR is a followup of https://github.com/twentyhq/twenty/pull/20673 It aims to unify the authentication/permissions layer with all the connectedAccount interactions across the application ## Deprecate - findAll - findById ## Email sync An user can only sync the message of his own connected account ## Workflow email - Related https://github.com/twentyhq/private-issues/issues/478 - Only reauthorize owned account |
||
|
Félix Malfait
|
291ce5ccdb |
fix(filters): make filter dispatcher own relation-target resolution (#20670)
## Summary Two relation-traversal bugs surfaced post-merge of #20533, both rooted in the same architectural smell: the GraphQL filter dispatcher took a flat `fields: FieldShared[]` array and silently dropped any filter whose `relationTargetFieldMetadataId` wasn't in that array. Callers had to remember to pre-augment the list with relation targets — and 16+ call sites did not all know this. This PR fixes both bugs and removes the smell. ### Bug 1 — Save as new view loses the relation target `useCreateViewFromCurrentView` built the create-filter input without `relationTargetFieldMetadataId`. The saved view's filter persisted without the traversal — on reload the chip showed "Company contains 'air'" instead of "Company → Name contains 'air'". Discarded at save time, not at read time. Fix: include `relationTargetFieldMetadataId` in the create input. (Commit 1.) ### Bug 2 — Workflow Search Records drops one-hop traversals `FindRecordsWorkflowAction` built its fields list from `flatObjectMetadata.fieldIds` only (source object's fields). The shared dispatcher then couldn't resolve the relation target field on the related object and silently dropped the filter — a configured "People where Company → Name Contains 'Airbnb'" came through as `{ and: [] }`. This was the same shape as bugs already fixed in 5 other call sites (chart filters, view filters, record table, etc.). The pattern was: caller forgets to augment fields → dispatcher silently drops the filter. Fix (commit 2): change the dispatcher to take a `findFieldMetadataItemById: (id) => FieldShared | undefined` resolver callback. Both source-field and relation-target-field lookups go through the same resolver, so callers no longer need to know about the augmentation requirement. Frontend callers pass a workspace-wide resolver built from `flattenedFieldMetadataItemsSelector`; server callers wrap `findFlatEntityByIdInFlatEntityMaps` on `flatFieldMetadataMaps`. In both cases relation-target lookups just work, because the resolver can see fields on related objects. ## Why this matters Before: "if you call the dispatcher, pre-augment your fields list with relation targets, or filters get silently dropped." An invariant only enforceable by code review, broken often enough to ship two user-visible bugs in one week. After: the dispatcher resolves field ids itself. There's no list to forget to augment. The failure mode (filter silently dropped) becomes structurally impossible at the dispatcher boundary. Net diff: 240 insertions, 319 deletions. Removed `augmentFieldsWithRelationTargets` (frontend) and the workflow whack-a-mole code (server). ## Test plan - [ ] Save view: create an advanced filter using a one-hop relation traversal, click "Save as new view", reload, confirm the chip still reads "Source → Target operator value" - [ ] Workflow: configure a Search Records action with a relation-traversal filter, run the workflow, confirm the filter is actually applied - [ ] Dashboard chart: configure a chart with a relation-traversal filter, confirm the chart data respects it - [ ] Record table, group-by, calendar, total count, footer aggregates: all continue to work with both plain and relation-traversal filters |
||
|
Abdul Rahman
|
10de7acef3 | If else node tests (#16916) | ||
|
Félix Malfait
|
fb41b116a4 |
Add Quick Lead workflow integration tests (#16862)
## Description This PR adds integration tests for the Quick Lead workflow, including a complete end-to-end test with full workflow execution. ### Key Changes 1. **Enabled SyncDriver for integration tests** - Jobs are now processed synchronously in tests - Modified `create-app.ts` to use `SyncDriver` instead of `BullMQ` - Added `MessageQueueExplorer` to discover and register workflow job handlers - This enables complete workflow execution in integration tests 2. **Added integration tests for Quick Lead workflow**: - Verify workflow exists and is active - Verify workflow version has correct structure (MANUAL trigger, FORM step, CREATE_RECORD steps) - Test workflow triggering creates workflow run with correct initial state - Test stop workflow run on a running workflow - **Full end-to-end test**: trigger → submit form → verify Company and Person records created ### Test Coverage The complete end-to-end test verifies: - Workflow triggers and is in RUNNING status (waiting on FORM step) - Form submission with test data succeeds - Workflow completes successfully with all steps in SUCCESS status - Company record is created with correct name and domain - Person record is created with correct name and email - Records are properly cleaned up after test ### How to Run Tests ```bash npx nx run twenty-server:test:integration -- --testPathPattern="quick-lead-workflow" ``` Or with database reset: ```bash npx nx run twenty-server:test:integration:with-db-reset -- --testPathPattern="quick-lead-workflow" ``` |
||
|
Charles Bochet
|
7e419337b5 |
Delete userWorkspace when removed from workspace (#13131)
Fixes https://github.com/twentyhq/twenty/issues/13024 |
||
|
martmull
|
cb010d90fe |
998 workflow restore (#12417)
Add a post hook to restore workflow sub-entities |
