feat(workflow): periodic core-consistency check for workflows, versions and triggers (#23103)

Monitoring for the soft-ref migration. The `workflow` /
`workflowVersion` dual-write into core is **best-effort** (async, not
transactional; failures only go to Sentry), so `core.workflow` /
`core.workflowVersion` can silently drift from the workspace source of
truth. This adds a periodic job that detects that drift across **all
three workflow entities** and emits it as metrics.

Supersedes the earlier inline shadow-parity approach that lived on this
branch — that only covered trigger dispatch and added a cache read +
diff to every cron tick and every DB-event batch (too much hot-path
overhead). This is broader and fully off the dispatch path.

## What
A cron (`cron:workflow:core-consistency-check`, every 3 hours, wired
into `cron:register:all`). Per run:
- **Bounded**: `SELECT DISTINCT "workspaceId" FROM core."workflow"` —
only workspaces that actually use workflows (skips the large majority).
- Per such workspace, emit a drift metric per `(entity, driftType)` —
detect-only, plus a triage log:
- **workflow** and **workflowVersion** — `unlinked` / `missingCore` /
`orphanCore` / `fieldMismatch`, via cross-schema `COUNT` aggregates
(core + workspace are the same DB, so indexed joins — no rows pulled
into JS).
- **automated triggers** — the `workflowAutomatedTrigger` table vs the
`workflowAutomatedTriggerMaps` cache: `inTableNotCache` /
`inCacheNotTable` / settings `mismatch`.
- Per-workspace failures are isolated (caught → Sentry) so one bad
workspace does not stop the sweep.

## Why it is efficient
Two central `core.*` queries + a few `COUNT` queries per
*workflow-using* workspace, on a relaxed cadence, off the dispatch path.
Shardable across ticks later if needed.

## Metrics
`workflow-core-consistency/{workflow,version,automated-trigger}/drift`
counters, attribute `driftType`. Dashboards: twentyhq/twenty-infra#800.

## Not in scope
Detect-only — no auto-heal (the existing backfill/rebuild command can
heal). No dispatch or flag changes.

## Test
- The consistency SQL (workflow/version sync counts, orphan counts,
trigger read) validated against a live workspace (it surfaced real drift
there — unlinked versions + an orphan core version). The whole
cron→service→SQL→metric pipeline is proven live: the cron is already
emitting real drift counters on a running server.
- Unit specs for the service (clean → no metric; per-entity drift per
dimension; per-workspace error isolation).
- Command boots and registers via `cron:register:all` (verified).
Typecheck + lint clean.
This commit is contained in:
Thomas Trompette
2026-07-23 14:30:12 +02:00
committed by GitHub
parent 66df0ac47c
commit 96a2456367
9 changed files with 571 additions and 0 deletions
@@ -53,6 +53,7 @@ import { CalendarEventImportManagerModule } from 'src/modules/calendar/calendar-
import { MessagingImportManagerModule } from 'src/modules/messaging/message-import-manager/messaging-import-manager.module';
import { WorkflowRunQueueModule } from 'src/modules/workflow/workflow-runner/workflow-run-queue/workflow-run-queue.module';
import { AutomatedTriggerModule } from 'src/modules/workflow/workflow-trigger/automated-trigger/automated-trigger.module';
import { WorkflowCoreConsistencyModule } from 'src/modules/workflow/workflow-core-consistency/workflow-core-consistency.module';
@Module({
imports: [
@@ -64,6 +65,7 @@ import { AutomatedTriggerModule } from 'src/modules/workflow/workflow-trigger/au
CalendarEventImportManagerModule,
WebhookSubscriptionModule,
AutomatedTriggerModule,
WorkflowCoreConsistencyModule,
FileModule,
WorkspaceModule,
WorkflowRunQueueModule,