4ab36a630bbe3a1a2d2ff58d7b15cfbc3bc7737a
92 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 neo773
|
674de0056b |
feat(messaging): message campaign delivery stats + views (#22661)
Re-land of #22452 (reverted in #22627). Rebuilt on fresh main with upgrade commands isolated to 2-20 only; no other version's commands touched. <!-- This is an auto-generated description by cubic. --> <a href="https://cubic.dev/pr/twentyhq/twenty/pull/22661?utm_source=github" target="_blank" rel="noopener noreferrer" data-no-image-dialog="true"><picture><source media="(prefers-color-scheme: dark)" srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source media="(prefers-color-scheme: light)" srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img alt="Review in cubic" src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a> <!-- End of auto-generated description by cubic. --> |
||
|
Paul Rastoin
|
bd8bf89653 |
Revert "feat(messaging): message campaign delivery stats + views" (#22452) (#22627)
Revert "feat(messaging): message campaign delivery stats + views
(#22452)"
This reverts commit
|
||
|
neo773
|
2e1117d442 |
feat(messaging): message campaign delivery stats + views (#22452)
<!-- This is an auto-generated description by cubic. --> <a href="https://cubic.dev/pr/twentyhq/twenty/pull/22452?utm_source=github" target="_blank" rel="noopener noreferrer" data-no-image-dialog="true"><picture><source media="(prefers-color-scheme: dark)" srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source media="(prefers-color-scheme: light)" srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img alt="Review in cubic" src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a> <!-- End of auto-generated description by cubic. --> |
||
|
Félix Malfait
|
3a21089e0a |
feat(server): abort ai stream jobs that outlive the shutdown drain budget (#22517)
## Why #22514 makes SIGTERM drain workers, but `worker.close()` waits for active jobs with **no upper bound** (BullMQ semantics). An AI stream job can run for 10 minutes; a deploy would either hang the rollout or hit the pod's termination grace deadline and get SIGKILLed anyway — back to the frozen-stream + stalled-rerun failure this series eliminates. ## What On shutdown, `aiStreamQueue` gets a bounded drain: active stream jobs have `AI_STREAM_SHUTDOWN_DRAIN_MS` (60s) to finish naturally; stragglers are then aborted and terminate exactly like a stream failure — `lastStreamError` persisted with the new typed `AiExceptionCode.STREAM_INTERRUPTED`, the pinned `stream-error` → `queue-updated` terminal sequence published, claim released. The client shows the interrupted state with Retry (#22434) within seconds instead of a stream frozen mid-sentence. This is deliberately **not** the user-cancel path, which resolves cleanly and persists no error. Mechanism — evaluated BullMQ 5.78's native cancellation vs a parallel in-process registry, and picked native: - The driver's processor now declares the 3-arg signature, which makes BullMQ create a per-job `AbortController` (`processor.length >= 3` is the trigger), and the signal is handed to job handlers as an optional `MessageQueueJobContext`. - `worker.cancelAllJobs()` is purely cooperative: it aborts the signal and nothing else, so the job's own persist/publish/cleanup still runs to completion and `worker.close()` still waits for it — no force-fail race, no second signaling channel to maintain, and the timer lives inside the same `closeWorker()` call so there is no dependence on Nest module-destroy ordering. - The stream job maps the shutdown signal onto its **existing** AbortController (the one already wired through the AI SDK for user cancel), with an `AiException(STREAM_INTERRUPTED)` reason to tell the two apart. One abort path end to end, no new infrastructure. Error-type choice: the job throws a plain `AiException`, not BullMQ's `UnrecoverableError`. Stream jobs are enqueued with `attempts: 1` (no `retryLimit`), so there is no BullMQ retry to suppress — retryability for this queue lives at the app layer (`lastStreamError` + client Retry), and an `UnrecoverableError` would only obscure the typed exception. `STREAM_INTERRUPTED` also replaces the string constant introduced on the base branch (#22482's reap now uses the same enum member) — one code, two producers (reap for dead workers, abort for live shutdowns), identical client behavior. Notes: - 60s is a static constant mirroring `AI_STREAM_LOCK_DURATION_MS` rather than an env var — it has to move in lockstep with the worker's `terminationGracePeriodSeconds` (120s, twenty-infra PR) anyway, and we ship multiple releases a day. Happy to lift it into a config variable if you want runtime tunability. - The `onModuleDestroy` scaffolding (drain logs, workers-close-before-queues) deliberately matches #22514; whichever lands second rebases clean. Stacked on #22482 (needs the heartbeat/reap base). Merge order: #22482 → this. Depends on #22514 for SIGTERM to reach the driver at all. ## Validation - `stream-agent-chat.job.spec.ts`: shutdown-abort persists `STREAM_INTERRUPTED`, publishes `stream-error` before `queue-updated`, releases the claim, skips the queued-message flush; user-cancel semantics unchanged with a wired-but-idle shutdown signal (60/60 ai-chat tests green). - Local end-to-end: real AI stream mid-flight, SIGTERM the worker → drain window → abort → interrupted state persisted, process exits on its own. (Transcript in the PR conversation.) <!-- This is an auto-generated description by cubic. --> <a href="https://cubic.dev/pr/twentyhq/twenty/pull/22517?utm_source=github" target="_blank" rel="noopener noreferrer" data-no-image-dialog="true"><picture><source media="(prefers-color-scheme: dark)" srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source media="(prefers-color-scheme: light)" srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img alt="Review in cubic" src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a> <!-- End of auto-generated description by cubic. --> |
||
|
Félix Malfait
|
e5fc1b3702 |
feat(server): drain queue workers gracefully on SIGTERM (#22514)
## Why Deploys kill workers mid-job: neither the queue worker nor the API server ever calls `enableShutdownHooks()`, so NestJS never listens for SIGTERM and the graceful close in `BullMQDriver.onModuleDestroy` is dead code. On every rollout a worker dies instantly — an in-flight 10-minute AI stream freezes for the watching user, and BullMQ silently re-runs the half-executed job on another worker ~10 minutes later. This is the root cause, not a symptom: the correct drain semantics already exist in the driver (`worker.close()` waits for active jobs and stops picking new ones, per the BullMQ graceful-shutdown docs) — the process just never received the signal. ## What - `queue-worker.ts` + `main.ts`: enable shutdown hooks. SIGTERM now runs `onModuleDestroy` across providers: the BullMQ driver drains active jobs, `RedisClientService` and the AI cancel subscriber quit their Redis connections, TypeORM closes its pools, then the process exits on its own. - BullMQ close order: workers drain before queues close, so a job finishing during the drain can still enqueue follow-ups (e.g. the AI queue flushing the next queued message). - API server: `forceCloseConnections` so long-lived subscription sockets don't hold `close()` open until the pod is force-killed. They were dropped abruptly on every deploy before this PR too — clients already recover. - Drain start/completion logs so pod terminations are debuggable. ## User impact Deploys stop corrupting in-flight background work. Follow-ups build on this: bounded drain-then-abort for AI stream jobs, and eliminating the stalled-job zombie re-run. ## Validation - Local: SIGTERM'd a running worker mid-job — drain log appears, the active job completes, "Message queue shutdown complete" is logged, process exits by itself. (Also verified with `LOGGER_IS_BUFFER_ENABLED=true` that final logs are not swallowed.) - The k8s side (termination grace period ≥ drain budget, exec'ing `node` directly so PID 1 receives SIGTERM) lands separately in twenty-infra. <!-- This is an auto-generated description by cubic. --> <a href="https://cubic.dev/pr/twentyhq/twenty/pull/22514?utm_source=github" target="_blank" rel="noopener noreferrer" data-no-image-dialog="true"><picture><source media="(prefers-color-scheme: dark)" srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source media="(prefers-color-scheme: light)" srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img alt="Review in cubic" src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a> <!-- End of auto-generated description by cubic. --> --------- Co-authored-by: cubic-dev-ai[bot] <191113872+cubic-dev-ai[bot]@users.noreply.github.com> |
||
|
Raphaël Bosi
|
2e6077383b |
Add install your first apps onboarding V2 step (#22347)
https://github.com/user-attachments/assets/5326d48f-1842-4db1-bc7c-94852145c035 <img width="838" height="754" alt="CleanShot 2026-06-30 at 16 25 05@2x" src="https://github.com/user-attachments/assets/5c7d53d7-4d65-4e35-aed1-edf0c104e140" /> Adds an "Install your first apps" step to the V2 onboarding, shown right after import-contacts. It lets users opt into installing marketplace apps (Call recorder and People Data Labs for now) during onboarding. - New backend `OnboardingStatus.APPS_INSTALLATION` (between SYNC_EMAIL and PROFILE_CREATION); V1 auto-skips it. - The primary button sends the selected app ids to the server via `triggerInstallAppsOnboardingStep`, which enqueues a dedicated job that installs them asynchronously so onboarding isn't blocked. Skip continues without installing. - The workspace is credited per app on successful installation. Credits are env-driven via `ONBOARDING_INSTALL_APPS_CREDITS_REWARD_PER_APP`, shown as "Earn +N free credits (1 per tool)". <!-- This is an auto-generated description by cubic. --> <a href="https://cubic.dev/pr/twentyhq/twenty/pull/22347?utm_source=github" target="_blank" rel="noopener noreferrer" data-no-image-dialog="true"><picture><source media="(prefers-color-scheme: dark)" srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source media="(prefers-color-scheme: light)" srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img alt="Review in cubic" src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a> <!-- End of auto-generated description by cubic. --> |
||
|
martmull
|
e747bc3e42 |
Add backfill installation feature for pre-installed apps (#22199)
## After <img width="1070" height="514" alt="image" src="https://github.com/user-attachments/assets/9cbd2ff5-1678-4c2f-84da-074568f37c51" /> ## Summary Adds the ability to backfill application installations across all existing workspaces. This allows admins to retroactively install a pre-registered application on every active and suspended workspace through a background job, making the feature idempotent and non-blocking. ## Key Changes - **Backend Service**: Added `backfillApplicationOnAllWorkspaces()` method to `PreInstalledAppsService` that: - Validates the application registration exists - Iterates through all workspaces using `WorkspaceIteratorService` - Installs the app on each workspace - Swallows `APP_ALREADY_INSTALLED` errors for idempotency - Logs success/failure counts - **Background Job**: Created `BackfillApplicationInstallationJob` to process backfill requests asynchronously via the message queue - **GraphQL Mutation**: Added `backfillApplicationInstallation` mutation to `AdminPanelResolver` that: - Validates the application registration exists - Enqueues the background job - Returns immediately without blocking the request - **UI Components**: Enhanced `SettingsAdminApplicationRegistrationGeneralToggles` with: - New "Pre-install on new workspaces" toggle for the `isPreInstalled` flag - "Backfill on all workspaces" button with confirmation modal - Loading state and success/error snack bar feedback - **Data Model**: Added `isPreInstalled` field to `UpdateApplicationRegistrationPayload` input type - **Tests**: Added comprehensive unit tests for `PreInstalledAppsService.backfillApplicationOnAllWorkspaces()` covering: - Missing registration validation - Successful multi-workspace installation - Idempotent handling of already-installed errors - Proper error propagation for unexpected failures ## Implementation Details The backfill operation is designed to be: - **Idempotent**: Already-installed apps are skipped without error - **Non-blocking**: Runs as a background job via message queue - **Resilient**: Per-workspace failures don't block other installations - **Observable**: Logs aggregated success/failure counts for monitoring <!-- This is an auto-generated description by cubic. --> <a href="https://cubic.dev/pr/twentyhq/twenty/pull/22199?utm_source=github" target="_blank" rel="noopener noreferrer" data-no-image-dialog="true"><picture><source media="(prefers-color-scheme: dark)" srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source media="(prefers-color-scheme: light)" srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img alt="Review in cubic" src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a> <!-- End of auto-generated description by cubic. --> --------- Co-authored-by: cubic-dev-ai[bot] <191113872+cubic-dev-ai[bot]@users.noreply.github.com> |
||
|
Félix Malfait
|
da6a2ee300 |
fix(ai-chat): keep streams alive on silent SSE death + make the stream job idempotent (#22201)
## Problem In production, an AI-chat assistant response sometimes freezes mid-stream (partial text, looks hung), then "picks up again on its own" later without the user resending and without a known worker restart. Root cause: the **agent-chat SSE subscription has no keepalive and no silent-death detection**. - Delivery is fire-and-forget Redis pub/sub (`SubscriptionService.publishToAgentChat`) and the resolver returns the **raw** iterator — unlike `EventStreamResolver`, which heartbeats every 30s via `wrapAsyncIteratorWithLifecycle`. - During a quiet model/tool gap the connection sends no bytes, so a proxy/LB/NAT can silently drop it mid-stream. `graphql-sse` neither surfaces an error nor resumes with `Last-Event-ID`, and **nothing re-pulls the existing Redis chunk catch-up on reconnect** (it only runs on thread (re)mount / `message-persisted` refetch). - So the live view freezes; recovery only happens when the terminal `message-persisted` fires a full refetch from the DB — the observed "self-recovery". This is the **same silent-SSE-death class fixed for the DB event stream in #21061**, which was never applied to the agent-chat path. The symptom also matches #21096 (worker logs the job finishing, client never updates, reload shows the message). It is **not** queue prioritization, and it is **not** addressed by #22193 (which only stabilizes the assistant message id and removes end-of-stream flicker). A secondary, independent self-recovery path also existed: BullMQ stalled-job re-run (default 30s `lockDuration`, no idempotency guard) re-streaming the whole turn → duplicate assistant messages / double billing. ## Changes ### Commit 1 — keepalive + silent-death recovery (ports the #21061 pattern to agent chat) - **Shared:** new `keepalive` variant on `AgentChatSubscriptionEvent`. - **Server:** wrap the agent-chat subscription iterator with `wrapAsyncIteratorWithLifecycle` — emit a `keepalive` on connect and every `APPLICATION_KEEPALIVE_INTERVAL_MS` (30s) so the connection keeps flushing bytes and a dead connection becomes detectable. - **Client:** track the last received event timestamp (refreshed on every chunk/keepalive in the SSE `next` sink); new `AgentChatStreamKeepAliveEffect` forces a resubscribe + messages refetch after 90s of silence, so the durable Redis chunk list backfills the gap (`firstLiveSeq` is reset on resubscribe). ### Commit 2 — stream-job idempotency + lockDuration - Thread a `lockDuration` option through `MessageQueueWorkerOptions` + the BullMQ driver; set `aiStreamQueue` to 10 min so long streams aren't falsely stalled. - Guard `StreamAgentChatJob.handle` with a `streamId`-scoped Redis lock (`SET NX PX` + compare-and-delete release) so a stalled re-run is skipped instead of double-processing. ## Verification ⚠️ I could **not run typecheck/lint locally** — `yarn install` could not complete in this environment (transient registry network aborts before the link step, so `node_modules` never populated). **Please rely on CI for type/lint verification.** The changes are written to match existing conventions; the points most worth a reviewer's eye are the resolver's iterator typing and the ioredis `set(..., 'PX', ttl, 'NX')` overload. How to confirm the root cause in prod: a frozen client with the worker logging `StreamAgentChatJob processed in …ms` and no `[AI_CHAT_NO_TEXT]` is the silent-death signature (check reverse-proxy idle/buffering). For the secondary path, watch `aiStreamQueue` `stalled`/re-processed metrics and duplicate turns around worker restarts. ## Notes / trade-offs - The 10-min `lockDuration` means a genuinely crashed worker's job isn't reclaimed for up to 10 min; the client-side keepalive/catch-up recovers the view independently, and the idempotency lock prevents duplicates. Faster dead-worker recovery could be a follow-up. - Touches `useAgentChatSubscription.ts` / `AgentChatRuntimeEffects.tsx` / `stream-agent-chat.job.ts`, which #22193 also touches — trivial rebase expected. Opened as **draft** pending CI. https://claude.ai/code/session_018dF82A1VcsuWMxPLmdY3dm --- _Generated by [Claude Code](https://claude.ai/code/session_018dF82A1VcsuWMxPLmdY3dm)_ <!-- This is an auto-generated description by cubic. --> <a href="https://cubic.dev/pr/twentyhq/twenty/pull/22201?utm_source=github" target="_blank" rel="noopener noreferrer" data-no-image-dialog="true"><picture><source media="(prefers-color-scheme: dark)" srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source media="(prefers-color-scheme: light)" srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img alt="Review in cubic" src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a> <!-- End of auto-generated description by cubic. --> |
||
|
Félix Malfait
|
ea9e11581c |
feat(billing): replace Stripe trial emails with fair, well-timed reminders (#22186)
## Why We currently rely on Stripe's automated trial-ending email. It misfires: the global "remind 7 days before trial ends" setting lands the reminder on **signup day** for the 7‑day no‑card trial, and the "your card will be charged" copy makes no sense for a trial with no card. This replaces it with our own honest, well‑timed, Twenty‑branded emails. ## 🔒 Safety — these emails are OFF by default Because these reach real customers, the whole feature is gated behind a kill‑switch that **defaults to `false`**: - **`BILLING_REMINDER_EMAILS_ENABLED` (default `false`)** — checked **both** at cron registration **and** on every job run (defense in depth), so the emails can never be sent inadvertently (not on deploy, not in staging, not via a stray trigger). They only go out once an operator explicitly opts in. - Also gated on `IS_BILLING_ENABLED` (cloud‑only; self‑hosters unaffected). - In non‑prod the email driver is typically `logger`, so even if enabled there, nothing is actually sent. A unit test asserts that with the switch off, **zero** emails are produced. ## What it does A daily cron (`0 8 * * *`) sends three honest, Twenty‑branded emails: | Plan | Email | When | |---|---|---| | No‑card trial (7d) | "Add a card to keep your data" | **1 day before** trial ends | | Card‑on‑file trial (30d) | Upcoming‑charge heads‑up (cancel in one click) | **7 days before** first charge | | Yearly subscription | Renewal reminder (no surprise) | **7 days before** each renewal | - **Monthly renewals get no reminder** (avoids noise) — only the first charge and annual renewals do. - Branches no‑card vs with‑card on the customer's payment‑method flag (with a trial‑duration fallback), so someone who adds a card mid‑trial correctly gets the charge heads‑up instead of the add‑a‑card one. - **Idempotent** per `(workspace, boundary date)` via workspace‑level user vars — yearly reminders re‑fire each period, but the daily cron never double‑sends. - Offsets are configurable via new `BILLING_*_REMINDER_DAYS_BEFORE` variables. Also **warms up the tone** of the existing suspended / deleted workspace emails (less robotic, fair, loss‑aversion framing) — these already act as the "come back or lose your data" win‑back, so no extra win‑back email was added. ## Rollout 1. Merge. 2. Disable Stripe's automated trial/renewal customer emails in the Stripe dashboard. 3. Review copy/timing, then set `BILLING_REMINDER_EMAILS_ENABLED=true` to turn the cron on. ## Notes for reviewers - **i18n:** new English strings render via Lingui's msgid fallback; translation catalogs are intentionally **not** included to keep the diff focused (the repo extracts translations via its standard periodic `lingui extract` sync — `main` already carries catalog drift). Diff is 18 code files. - **Recipients:** reminders go to all workspace members, consistent with the existing suspension emails. Happy to scope the charge‑related ones to billing admins if preferred. - **Follow‑ups discussed:** in‑app trial banner, loss‑aversion with real record counts, and failed‑payment dunning are the higher‑leverage conversion levers beyond this. ## Test plan - [x] `typecheck` (twenty-server, twenty-emails) - [x] oxlint type‑aware + oxfmt - [x] Unit tests: no‑card path, with‑card path, idempotency, yearly renewal, billing‑disabled, **kill‑switch off → no send** (6/6 green) - [ ] Manual: set the flag on a staging instance with `logger` driver and confirm the right email is logged at each boundary https://claude.ai/code/session_0147ujzHv1X4vzimf4iGbnT4 --- _Generated by [Claude Code](https://claude.ai/code/session_0147ujzHv1X4vzimf4iGbnT4)_ <!-- This is an auto-generated description by cubic. --> <a href="https://cubic.dev/pr/twentyhq/twenty/pull/22186?utm_source=github" target="_blank" rel="noopener noreferrer" data-no-image-dialog="true"><picture><source media="(prefers-color-scheme: dark)" srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source media="(prefers-color-scheme: light)" srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img alt="Review in cubic" src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a> <!-- End of auto-generated description by cubic. --> |
||
|
Félix Malfait
|
35c2a24afb |
perf(onboarding): compute invite suggestions on-demand (#21696)
## Summary Follow-up to #21640. In production, invite suggestions took ~1 minute to appear because `FetchOnboardingInviteSuggestionsJob` ran on the shared `calendarQueue` behind heavy calendar-sync jobs. - **Drop the background job entirely.** `getInviteSuggestions` now resolves the connected account from the authenticated `@AuthUserWorkspaceId()` and computes suggestions on demand: cache-first, with a bounded calendar fetch + cache write on a miss. Removes the Google/Microsoft enqueues, the `shouldComputeInviteSuggestions` threading through the auth controllers, and the now-unused `shouldComputeInviteSuggestionsOnConnect` / `isOnboardingConnectAccountPending` helpers. - **Prefetch one step earlier.** New `usePrefetchInviteSuggestions` hook fires the query from `CreateProfile` so the server cache is warm by the time the invite step renders. `InviteTeam` switches from `network-only` → `cache-first`. If the profile step is skipped, the invite step still computes on-demand (~1–3s, no queue) — no more minute-long waits. No GraphQL schema change. ## Test plan - [ ] Connect Google calendar in onboarding → invite step renders prefilled teammates with no perceivable wait - [ ] Connect Microsoft calendar in onboarding → same - [ ] Onboard with workspace name already set so profile step is skipped → invite step still prefills (just with a brief on-demand fetch instead of 1 min) - [ ] Connect a non-work-email account → invite step renders empty form (no suggestions) - [ ] `npx nx typecheck twenty-server` ✅ - [ ] `npx nx lint:diff-with-main twenty-server` ✅ - [ ] `npx nx lint:diff-with-main twenty-front` ✅ (changed files clean) - [ ] `google-apis.service.spec.ts` + `microsoft-apis.service.spec.ts` pass https://claude.ai/code/session_019MyY3bfAEij4AwSXMCLtWY --- _Generated by [Claude Code](https://claude.ai/code/session_019MyY3bfAEij4AwSXMCLtWY)_ <!-- This is an auto-generated description by cubic. --> <a href="https://cubic.dev/pr/twentyhq/twenty/pull/21696?utm_source=github" target="_blank" rel="noopener noreferrer" data-no-image-dialog="true"><picture><source media="(prefers-color-scheme: dark)" srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source media="(prefers-color-scheme: light)" srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img alt="Review in cubic" src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a> <!-- End of auto-generated description by cubic. --> Co-authored-by: Claude <noreply@anthropic.com> |
||
|
Félix Malfait
|
61309c45e6 |
feat(onboarding): prefill the invite step with teammates from the connected calendar (#21640)
## What & why Implements core-team-issues#1414: move the calendar/email connection earlier in onboarding and use the freshly connected calendar to prefill the **Invite your team** step with likely teammates, so users don't start from an empty form. ## Approach Everything is behind the feature flag `IS_ONBOARDING_INVITE_SUGGESTIONS_ENABLED` (off by default). **Reorder** — onboarding becomes `Workspace activation → Connect account → Create profile → Invite team`. Connecting before profile gives the calendar sync a head start; connecting *before* the workspace exists isn't possible (a connected account requires an activated workspace + workspace member + OAuth transient token). Gated in both `OnboardingService.getOnboardingStatus` (backend) and `useSetNextOnboardingStatus` (frontend) so the two agree. **Fast teammate lookup** — on Google/Microsoft connect *during onboarding*, a background job (`FetchOnboardingInviteSuggestionsJob`) runs a single bounded calendar fetch (recent events, attendees inline), keeps same-work-email-domain colleagues (excludes self + aliases; personal mailboxes yield nothing), ranks by meeting frequency, and caches the top 5. The invite step reads the cache via a new `getInviteSuggestions` query and prefills the form — polling briefly while the cache warms, and never overwriting input the user has already typed. Providers: **Google** (Calendar `events.list`) and **Microsoft** (Graph `calendarView`), routed by a `CalendarAttendeesService` dispatcher (mirrors the existing `CalendarGetCalendarEventsService`). Any fetch failure (missing scope, API error) degrades to today's empty form via the orchestrator's best-effort catch. ## How to enable Turn on `IS_ONBOARDING_INVITE_SUGGESTIONS_ENABLED` for a workspace (admin panel). ## Notes - New-workspace creators only (invitees never see the connect/invite steps). Skipping the connect step, or signing up with a personal email, falls back to the current empty form. - The "We found teammates from your calendar" subtitle only shows once suggestions are actually prefilled. - i18n: the new `<Trans>` strings are extracted on merge to `main` by the existing Crowdin workflow. ## Testing - Frontend unit tests for the reorder state machine (both flag states). - `npx nx typecheck` and `npx nx lint:diff-with-main` green for `twenty-front` and `twenty-server`. - Server boots with the new DI wiring (no circular dependency); `getInviteSuggestions` / `InviteSuggestion` present in the live metadata schema. - Not exercised in CI: live Google/Microsoft OAuth end-to-end (requires real accounts + calendar data). https://claude.ai/code/session_019MyY3bfAEij4AwSXMCLtWY --- _Generated by [Claude Code](https://claude.ai/code/session_019MyY3bfAEij4AwSXMCLtWY)_ <!-- This is an auto-generated description by cubic. --> <a href="https://cubic.dev/pr/twentyhq/twenty/pull/21640?utm_source=github" target="_blank" rel="noopener noreferrer" data-no-image-dialog="true"><picture><source media="(prefers-color-scheme: dark)" srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source media="(prefers-color-scheme: light)" srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img alt="Review in cubic" src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a> <!-- End of auto-generated description by cubic. --> --------- Co-authored-by: neo773 <62795688+neo773@users.noreply.github.com> Co-authored-by: neo773 <neo773@protonmail.com> Co-authored-by: claude[bot] <41898282+claude[bot]@users.noreply.github.com> |
||
|
neo773
|
5d892bdfd0 |
[WIP] Feat/marketing emails (#21173)
Marketing/campaign emails on top of the emailing-domain (SES) feature:
send a broadcast to a hand-picked list, with per-customer-domain
unsubscribe links and opt-out-only **unsubscribe topics**.
## Model
Standard objects (workspace schema, flat-metadata):
- `messageCampaign` — a campaign send (subject, body template, from
address, status, list, optional unsubscribe topic).
- `messageList` + `messageListMember` — the hand-picked audience (person
↔ list join). A campaign's recipients are its list's members; everyone
is sendable unless suppressed.
Core entities (`core` schema, workspace-scoped — readable by the public
unsubscribe flow without a workspace context):
- `unsubscribeTopic` — an opt-out-only category (name, description,
visibility). There is no opt-in subscription state.
- `messageSuppression` — the single consent store: a row with
`unsubscribeTopicId` NULL is a global block; a row with an
`unsubscribeTopicId` and reason `UNSUBSCRIBE` is a per-topic opt-out.
Two partial unique indexes dedupe global vs per-topic rows (Postgres
treats NULLs as distinct).
- `emailingDomain` — the workspace's SES sending domain,
auto-provisioned when an email channel is added (and cleaned up when its
last channel is removed), with verification status + DNS records.
Campaign messages reuse the existing `message` / `messageThread` /
`messageParticipant` model — one outbound `message` per recipient with a
`deliveryStatus` state machine.
## Sending
- `sendMessageCampaign` resolves the audience **under the caller's
permissions**, creates the campaign, and enqueues a single fan-out job
(the request never materializes per-recipient rows or jobs).
- The fan-out job materializes one QUEUED message per recipient
(deterministic ids → idempotent re-runs, reconciles crash-orphaned rows)
and fans out per-recipient send jobs carrying **only ids**.
- Each send job renders per-recipient `{{variable}}` merge fields and
sends via `EmailingDomainSenderService`, which applies suppression
(global + per-topic) and the unsubscribe footer/headers. Suppressed
recipients are recorded `SKIPPED`.
- The campaign finalizes `SENT`, or `SENT_WITH_ERRORS` if any recipient
terminally failed.
- `previewMessageCampaignAudience` returns a pre-send breakdown (total /
without-email / duplicate / globally-unsubscribed / topic-unsubscribed /
sendable), shown as a hint under the composer pickers.
## Unsubscribe
- Encrypted (AES-256-GCM) token carrying workspaceId, address, optional
`unsubscribeTopicId`, `issuedAt`, and a `preview` flag.
- One-click POST (RFC 8058) + `mailto:` — topic-scoped when the token
carries a topic, global otherwise.
- Preferences page: a checkbox per visible topic (checked = still
receiving); submitting creates per-topic opt-outs for unchecked topics
and lifts re-checked ones (UNSUBSCRIBE only — never
`BOUNCE`/`COMPLAINT`, never a global block).
- A **Preview** action in settings opens the live page via a
preview-claim token; opt-out POSTs are no-ops for preview tokens, so
previewing never mutates state.
- SES webhooks: inbound unsubscribe + outbound bounce/complaint →
suppression (race-safe against at-least-once delivery, with reason
escalation that never downgrades).
- Per-customer unsubscribe hostname (Cloudflare DNS); sends are gated on
it being active, except in LOG/demo mode.
## Architecture
Campaign orchestration, suppression, the sender, the unsubscribe
controller, and the SES webhook handlers live in `src/modules/emailing`
+ `src/modules/messaging-webhooks` (the workspace-feature layer).
`core-modules/emailing-domain` keeps the SES driver, domain
provisioning, the `unsubscribeTopic` / `messageSuppression` core
entities, and the unsubscribe token/hostname plumbing. Domain creation
is validated (`CreateEmailingDomainInput` — domain-format regex,
lowercased) before any value reaches SES or the unsubscribe hostname.
## Frontend
- Campaign composer side panel (from / list / unsubscribe topic /
subject / body) with a live audience-preview hint.
- Email settings: email channels each showing their auto-provisioned
sending domain in a single section (status + DNS records + a "Check
verification" action), plus an **Unsubscribe Topics** section to
create/manage topics and preview the recipient page. A demo-mode banner
is shown when the LOG driver is active.
---------
Co-authored-by: Félix Malfait <felix@twenty.com>
Co-authored-by: Félix Malfait <felix.malfait@gmail.com>
|
||
|
Thomas Trompette
|
e9086b49f6 |
Increase logicFunctionQueue worker concurrency to 10 (#21364)
## Summary - Increase BullMQ worker concurrency for `logicFunctionQueue` from 1 (default) to 10 - Logic function executions are I/O-bound Lambda calls — the worker just holds an HTTP connection open, so higher concurrency doesn't add CPU/memory pressure - With 13 worker pods, this goes from 13 to ~130 concurrent slots, which should resolve the ~3h average queue latency observed in Grafana ## Test plan - [ ] Monitor `avg_latency_ms` for `logic-function-queue` in the Grafana job queue dashboard after deploy - [ ] Verify worker pod CPU/memory remains stable |
||
|
Thomas Trompette
|
16db47fb70 |
fix: add queue attribute to jobs waiting gauge metric (#21324)
## Summary
- The `twenty_queue_jobs_waiting_total` gauge was summing all queues
into a single value without a `queue` label, making the Grafana "Jobs
Waiting by Queue" panel show a single aggregated line instead of
per-queue breakdown.
- Uses `getMeter()` directly to call `observableResult.observe(count, {
queue: queueName })` per queue, matching the `by (queue)` grouping the
dashboard already expects.
## Test plan
- [x] Deploy and verify the Grafana "Jobs Waiting by Queue" panel
displays separate series per queue
- [x] Confirm Prometheus scrape returns
`twenty_queue_jobs_waiting_total{queue="..."}` with distinct queue
labels
|
||
|
Félix Malfait
|
91f2f08995 |
feat(server): unify workspace-event ingestion behind one EventSink pipeline (#21197)
## Why The five event-log streams (`workspaceEvent`, `pageview`, `objectEvent`, `usageEvent`, `applicationLog`) each wrote to ClickHouse through their own fire-and-forget writer (`AuditService`, `UsageEventWriterService`, and the `application-logs` driver), with the per-type knowledge (table names, normalization, access rules) spread across several modules. Three of them reimplemented the same ClickHouse insert, and the read side, the live stream, and the producers lived in different modules under two different names. This consolidates them into one `core-modules/event-logs/` subsystem (emit, write, live, read), with the per-type config in a single registry so adding an event type is roughly one file. The base Logs settings tab and free application logs shipped separately in #21180 (merged). This PR adds the unified backend, the registry, and the viewer's live mode and entitlement gating. ## Pipeline ```mermaid flowchart TB subgraph PROD["Producers"] A["auth, billing, impersonation,<br/>webhook, custom-domain"] U["usage listener"] F["logic-function executor (app logs)"] R["record CRUD (entity events)"] end EM["EventLogEmitterService<br/>createContext().insert* / dispatch()"] EQ(["entityEventsToDbQueue<br/>(existing, shared with timeline)"]) CIE["CreateEventLogFromInternalEvent"] SINK["WorkspaceEventSinkService.ingest()"] C1["ClickHouseEventSink"] C2["ConsoleEventSink"] LIVE["EventLogLiveService.publishWatched()<br/>(presence-gated)"] CH[("ClickHouse, 5 tables, async_insert")] CHAN(["WORKSPACE_EVENTS_CHANNEL"]) RS["EventLogsService (registry-driven read)"] LR["EventLogsLiveResolver"] UI["Settings > Logs"] A --> EM U --> EM F --> EM EM -->|direct| SINK R --> EQ --> CIE -->|ingest| SINK SINK --> C1 --> CH SINK --> C2 SINK --> LIVE -.->|if a viewer is watching| CHAN --> LR --> UI CH --> RS --> UI ``` ## What it does - Producers call `EventLogEmitterService.createContext().insert*()`, which builds a typed `WorkspaceEventEnvelope` and writes it through `WorkspaceEventSinkService` to the configured sinks (ClickHouse, Console) plus a presence-gated live fan-out. Record/CRUD events reach the same sink through the existing `entityEventsToDbQueue`. There is no dedicated queue; ClickHouse `async_insert` batches server-side. Writes are best-effort, as on main today. - `EVENT_LOG_TYPES[table]` is the per-type source of truth: the ClickHouse table, the required entitlement, the free-text filter column, and the row-to-GraphQL mapping. Read row shapes derive from the write rows. - Four modules along their dependency boundaries: `EventLogEmitterModule` (producer API), `EventLogIngestionModule` (sink layer), `EventLogLiveModule` (fan-out), and `EventLogsViewerModule` (the entitlement-gated GraphQL read, which is where billing/enterprise/permissions stay so producers stay light). - Logs viewer: per-table columns, filters (text, date, record), live mode, and an upgrade card that points to Billing on Cloud or the Admin Panel on self-hosted. Application logs are free on every plan; the other four require the `AUDIT_LOGS` entitlement (with a `NO_ENTITLEMENT` fallback to the upgrade card). - Renames `AuditService` to `EventLogEmitterService`, and the generic `Monitoring` event to a typed `Impersonation` event (`level` + `action`). - Removes `UsageEventWriterService`, the `application-logs` driver/module, and `AuditService`'s direct inserts. ## Durability Writes are best-effort, the same as main today (the old writers were fire-and-forget). A dedicated queue was tried mid-PR and removed: `async_insert` already batches server-side, so the queue only added durability, which isn't a requirement right now. The `EventSink` seam keeps a durable transport (e.g. a Redis-Streams buffer) easy to add later without touching producers. ## Out of scope S3 peer sink (seam only), Postgres or any second read path, `ReplicatedMergeTree`, ClickHouse table-schema changes, and the record-data `EVENT_STREAM_CHANNEL` (unchanged, separate concern). ## Testing Unit tests cover the registry definitions and row normalization, the entitlement gating, the envelope builders, and the producers. Integration tests cover the write paths (record create produces an `objectEvent`; the track mutation produces a `workspaceEvent`) and the read/query path across all five tables. Verified with typecheck, lint, a server boot, and GraphQL/SDK codegen. |
||
|
Thomas Trompette
|
51e3eddb29 |
Add job queue latency histogram metric (#21260)
## Summary - Records the time each job spends waiting in the queue (enqueue → processing start) as an OpenTelemetry histogram - Metric is broken down by `queue` and `job_name` attributes, enabling per-queue p50/p95/p99 latency analysis - Uses BullMQ's native `job.timestamp` for accurate measurement ## Test plan - [x] Deploy to staging and verify `job/latency-ms` metric appears in ClickHouse `otel_metrics_histogram` table - [x] Confirm Grafana dashboard can query the histogram data |
||
|
Félix Malfait
|
c3dd6b25a6 |
fix: use canonical oxlint rule id in lint-disable directives (#21253)
## What Many `oxlint-disable` / `eslint-disable` directives across the repo carry a corrupted rule id — `@typescripttypescript/<rule>` — most likely a find-and-replace accident that mangled the eslint-era `@typescript-eslint/` prefix. oxlint matches disable directives **loosely by rule name**, so these still suppress in practice (not a silent no-op), but the id is malformed and misleading. ## Change Replace them with the **canonical oxlint id** `typescript/<rule>` — matching the plugin name and rule keys declared in `.oxlintrc.json` — **127 files, 262 directives**: | rule | count | | --- | ----- | | `typescript/no-explicit-any` | 250 | | `typescript/ban-ts-comment` | 6 | | `typescript/no-misused-promises` | 4 | | `typescript/no-empty-object-type` | 2 | - `twenty-server`: 122 files - `twenty-front`: 5 files Comment-only — no code or runtime changes. ## Verification `oxlint --type-aware -c .oxlintrc.json` reports **0 warnings / 0 errors** for both `twenty-server` and `twenty-front`. Every changed line is exactly the id correction inside a disable directive (262 insertions / 262 deletions, no collateral edits). > Addresses the cubic review, which flagged that the canonical oxlint id is `typescript/...` (no `@`). Worth noting the original `@typescripttypescript/` was not actually a silent no-op — oxlint matches these directives loosely by rule name — but `typescript/` is the correct, config-aligned id. |
||
|
neo773
|
c5606212f2 |
Ses outbound followup (#20610)
This pull request unifies outbound with inbound under the new feature and the new email groups feature. These are workspace level shared inboxes that are shared between all workspace members. outbound sending with SES works, we only listen for tenant status events, rest is managed by AWS PR refactors old code and webhook to be split for outbound and inbound for proper separation | Area | Change | |---|---| | AWS SES driver | Split into `AwsSesRegisterDomainService` (tenant + identity + DKIM + MAIL FROM + configuration-set + EventBridge dest + contact list) and `AwsSesSendEmailService` (SendEmail). | | Reputation webhook | New `/webhooks/messaging/ses/outbound` route. SES → EventBridge (`Sending Status Enabled/Disabled` on default bus) → SNS → router → `SesOutboundSendingStateHandlerService` updates `emailing_domain.tenantStatus`. | | Inbound webhook | Refactored into `SesInboundWebhookRouterService` + `SesInboundMailHandlerService`. Shared `SnsSignatureVerifierService` + `SnsSubscriptionConfirmerService` across both routes. | | Global uniqueness | New migration + instance command: `emailing_domain.domain` is now globally unique (one tenant per domain across workspaces). | | Tenant status | New `emailing_domain.tenantStatus` column (`ACTIVE` / `PAUSED`) + `EmailingDomainTenantStatusService`. | | Send-email mutation | New `sendEmailViaDomain` GraphQL mutation + DTOs. | | Cleanup | `EmailingDomainWorkspaceCleanupJob` wired into `WorkspaceService.deleteWorkspace` — tears down SES tenant association + identity on workspace delete. | | Settings UI | Rewritten around reusable `SettingsTableListSection`. "Email Group" → "Email Handle" rename. New cells for status/source/forwarding. Outbound domains surfaced on workspace settings page. | ### Env vars (new) All in `config-variables.ts`, group `AWS_SES_SETTINGS`, all optional: - `AWS_SES_REGION` — `@IsAWSRegion`, consumed by `AwsSesClientProvider` + driver factory - `AWS_SES_ACCOUNT_ID` — used for ARN construction in driver factory - `SES_SNS_TOPIC_ARN_ALLOWLIST` — **shared** by inbound + outbound webhook routers, comma-separated list of accepted SNS topic ARNs (verified via `sns-payload-validator`) ### Migrations - `1778862608620-add-emailing-domain-tenant-status` (fast) — adds `tenantStatus` column. - `1778865501791-unique-emailing-domain-globally` (slow, idempotent) — enforces global uniqueness on `domain`. - Instance commands bumped to `2.5`. ### Infra dependency Two coupled twenty-infra PRs: - `ses-inbound-email` — receipt-rule + inbound SNS topic + S3 bucket policy + KMS grant + `email_group_*` outputs. - `ses-outbound-tf` — EventBridge rule + outbound SNS topic + SES IAM policy + outbound `webhook_url` subscription. **Based on `ses-inbound-email`.** Merge order: inbound first, then outbound. Outbound PR's chart edit owns the comma-joined `SES_SNS_TOPIC_ARN_ALLOWLIST` value (both ARNs). Features lives under `/settings/general` <img width="1496" height="845" alt="SCR-20260519-ofhi-2" src="https://github.com/user-attachments/assets/a025485a-09f7-4131-91cd-0067690ff18d" /> --------- Co-authored-by: Félix Malfait <felix.malfait@gmail.com> Co-authored-by: claude[bot] <41898282+claude[bot]@users.noreply.github.com> Co-authored-by: Félix Malfait <FelixMalfait@users.noreply.github.com> |
||
|
Etienne
|
eda41b4eba |
feat(ai) - add observability (#20850)
**AI Chat - Tool Executions (counters, tagged with model)** ai-chat/tool-execution-succeeded: number of tool calls invoked by the AI that completed without error ai-chat/tool-execution-failed: number of tool calls invoked by the AI that threw an error **AI Chat - Token Usage (counters, tagged with model)** ai-chat/input-tokens: total input tokens sent to the model across all turns ai-chat/output-tokens: total output tokens generated by the model ai-chat/cache-read-tokens: input tokens served from the model's prompt cache (cheaper) ai-chat/cache-write-tokens: input tokens written into the prompt cache for future reuse **AI Chat - Latency (histograms in ms, tagged with model)** ai-chat/turn-latency-ms: total duration of a full chat turn (from stream start to stream end) ai-chat/step-latency-ms: duration of a single reasoning/tool-call step within a turn ai-chat/ttft-ms: time-to-first-token, i.e. how long until the model starts streaming output **MCP - Tool Executions (counters)** mcp/tool-execution-succeeded: number of MCP tool calls that completed successfully mcp/tool-execution-failed: number of MCP tool calls that threw an error |
||
|
Etienne
|
fcd2d586ee |
chore(billing) - remove feature flag (#20531)
- remove feature flag - remove old enforce cap usage logic |
||
|
Paul Rastoin
|
a159a68e2c |
[twenty-server] no floating promises lint rule (#20499)
## Introduction
That's an audit + RFC
## Fire-and-forget (`void`) -- Intentional, correct
These are telemetry, metrics, and audit logging in hot paths or
non-critical contexts. `void` is the right choice.
| File | What was voided |
|---|---|
| `sign-in-up.service.ts` | `metricsService.incrementCounter` (sign-up
metric) + `auditService.insertWorkspaceEvent` (workspace created) |
| `use-graphql-error-handler.hook.ts` | 5x
`metricsService.incrementCounter` (GraphQL operation metrics) |
| `bullmq.driver.ts` | 2x `metricsService.incrementCounter` (job
completed/failed metrics) |
| `call-webhook.job.ts` | 2x `auditService.insertWorkspaceEvent` + 1x
`metricsService.incrementCounter` |
| `custom-domain-manager.service.ts` | `analytics.insertWorkspaceEvent`
(domain activation event) |
| `logic-function-executor.service.ts` |
`auditService.insertWorkspaceEvent` (function execution) |
| `workflow-runner.workspace-service.ts` |
`metricsService.incrementCounter` (throttle metric) |
| `cleaner.workspace-service.ts` | `metricsService.incrementCounter`
(deleted workspace metric) |
| `stream-agent-chat.job.ts` | Detached IIFE for streaming chunks
(intentional concurrent pipeline) |
| `workspace-auth-context.middleware.ts` |
`withWorkspaceAuthContext(...)` (AsyncLocalStorage, returns void anyway)
|
## Top-level script entry points (`void bootstrap()`)
These are module-level calls where the promise has no consumer. `void`
makes the lint rule happy and documents the intent.
| File | What changed |
|---|---|
| `main.ts` | `void bootstrap()` |
| `command.ts` | `void bootstrap()` |
| `queue-worker.ts` | `void bootstrap()` |
| `truncate-db.ts` | `void dropSchemasSequentially()` |
| `codegen/index.ts` | `void generateTests(forceArg)` |
## Now properly awaited -- Real bug fixes
These were floating promises that could silently fail, lose data, or
cause race conditions.
| File | What was fixed |
|---|---|
| `billing-sync-plans-data.command.ts` | `meters.map(async ...)` wrapped
in `Promise.all` -- was returning before upserts finished |
| `cache-storage.service.ts` | `setAdd` and `setPop` had `.then()`
chains that weren't returned/awaited |
| `create-audit-log-from-internal-event.ts` | 4x
`auditService.createObjectEvent` now awaited inside a job |
| `cleaner.workspace-service.ts` | 2x `emailService.send(...)` now
awaited -- emails could silently fail |
| `agent-async-executor.service.ts` | `calculateAndBillUsage` +
`billNativeWebSearchUsage` in `finally` block now awaited |
| `repair-tool-call.util.ts` | `calculateAndBillUsage` now awaited |
| `agent-title-generation.service.ts` | `calculateAndBillUsage` now
awaited |
| `chat-execution.service.ts` | `billNativeWebSearchUsage` now awaited |
| `ai-generate-text.controller.ts` | `calculateAndBillUsage` in
`finally` block now awaited |
| `agent-turn.resolver.ts` | `messageQueueService.add(...)` now awaited
|
| `command.ts` | `app.close()` now awaited (was exiting before graceful
shutdown) |
| `i18n.service.ts` | `loadTranslations()` in `onModuleInit` now awaited
|
| `workspace-query-hook.explorer.ts` | `explore()` in `onModuleInit` now
awaited |
| `message-queue.explorer.ts` | `handleProcessorGroupCollection` in
`onModuleInit` now awaited |
| `ai-billing.service.spec.ts` | Test now properly `await`s the async
call |
| `messaging-messages-import.service.spec.ts` | `expect(...)` now
properly `await`ed for async assertion |
| `archive.finalize()` (3 files) | Voided -- promise resolution already
handled by `pipeline()` / `on('end')` |
## Impersonation & security audit trail -- Upgraded from `void` to
`await`
These were previously fire-and-forget but are
security/compliance-critical events that must be reliably persisted.
| File | What was fixed |
|---|---|
| `impersonation.service.ts` | 4x `auditService.insertWorkspaceEvent`
now awaited (impersonation attempt, token generation
attempt/success/failure) |
| `auth.resolver.ts` | 5x `auditService.insertWorkspaceEvent` now
awaited (impersonation token exchange attempt/success/failure at server
and workspace levels) |
| `auth.service.ts` | 2x `analytics.insertWorkspaceEvent` now awaited
(impersonation attempted/issued) |
## Billing audit -- Upgraded from `void` to `await`
Payment events should be reliably persisted for financial/compliance
reporting.
| File | What was fixed |
|---|---|
| `billing-webhook-invoice.service.ts` |
`auditService.insertWorkspaceEvent(PAYMENT_RECEIVED_EVENT)` now awaited
inside Stripe webhook handler |
## Fire-and-forget with proper error handling -- Upgraded from bare
`void`
These remain non-blocking but now catch and log errors instead of
risking unhandled rejections.
| File | What was fixed |
|---|---|
| `logic-function-executor.service.ts` |
`applicationLogsService.writeLogs` now uses `.catch()` instead of bare
`void` -- user-facing logs should surface errors |
## Systemic infrastructure fixes
| File | What was fixed |
|---|---|
| `metrics.service.ts` | `incrementCounter`: Redis cache write
(`metricsCacheService.updateCounter`) now uses `.catch()` internally
instead of raw `await` -- prevents unhandled rejections across all `void
metricsService.incrementCounter(...)` call sites when Redis is unhealthy
|
| `audit.service.ts` | `preventIfDisabled`: made properly `async` with
`await` and consistent `Promise<{ success: boolean }>` return type.
Removed broken `catch` that returned an `AuditException` as a value
(wrong constructor args, unreachable dead code). Removed unused
`AuditException` import |
## Fixed in this session (beyond original PR)
| File | What changed |
|---|---|
| `telemetry.listener.ts` | Removed misleading `Promise.all` + `void`
combo; replaced with simple `for...of` + `void` |
| `message-queue.explorer.ts` | Changed from `void` to `await` so
startup crashes on registration failure |
|
||
|
Etienne
|
9fc5be1c4c |
Billing - Migrate from Stripe metering (#20298)
**Overall strategy** **1. Introduce “Billing V2” behind a workspace flag** Gate the new model with FeatureFlagKey.IS_BILLING_V2_ENABLED so existing workspaces stay on the old behavior until they’re migrated or explicitly on V2. **2. Replace workflow metered SKUs with a resource-credit product** Conceptually, billable “workflow execution” usage is not the primary subscription line item anymore. Add a RESOURCE_CREDIT product (and keep WORKFLOW_NODE_EXECUTION as deprecated for the transition). Usage and limits are expressed through credit buckets (e.g. price metadata like credit_amount), so one product can represent pooled credits instead of a narrow workflow-only meter. **3. Migrate subscriptions in two layers** Schema/catalog: persist extra price metadata (instance upgrade) so the server knows credit amounts and can match Stripe prices to the new model. Per workspace: the registered workspace command upgrade:2-2:migrate-to-billing-v2 finds subscriptions that still have WORKFLOW_NODE_EXECUTION, swaps those items to the right RESOURCE_CREDIT prices (using existing Stripe schedule + BillingSubscriptionUpdateService stack), then treats the workspace as V2 (flag). Workspaces without that legacy item or without a subscription are skipped. **4. Unify subscription lifecycle + usage on the server** **5. Refresh the product surface in Settings** Test : - [x] Subscribe v1 + Update subscribe + Migrate - [x] Subscribe v2 + Update subscribe |
||
|
Charles Bochet
|
3c7c62c79f |
fix(server): deduplicate @opentelemetry/api to fix NoopMeterProvider (#20231)
## Summary **All OTel metrics in twenty-server have been silently dropped since April 30.** ### Root cause PR #20149 (`bump @sentry/profiling-node 10.27→10.51`) pulled in `@sentry/node@10.51.0`, which declares `@opentelemetry/api: ^1.9.1` as a **dependency** (not peer). Yarn installed it as a **nested** copy at `1.9.1`, while the hoisted copy stayed at `1.9.0`. At startup in `instrument.ts`: 1. `Sentry.init()` uses the **nested `1.9.1`** to register `trace`, `propagation`, `context` on the OTel global → global version becomes **`1.9.1`** 2. `setGlobalMeterProvider()` uses the **hoisted `1.9.0`** → `registerGlobal` sees version mismatch (`1.9.1` ≠ `1.9.0`) → **silently returns `false`** 3. Global stays `NoopMeterProvider` → every counter, gauge, and histogram in the server is a no-op ### What this PR does 1. **Reverts three troubleshooting PRs** that are no longer needed now that the root cause is identified: - #20230 — heartbeat gauge - #20228 — OTLP export lifecycle logs - #20221 — Sentry revert to 10.27 (which never actually downgraded in `yarn.lock` since `^10.27.0` resolved to `10.51.0`) 2. **Fixes the root cause**: - Root Yarn resolution pinning `@opentelemetry/api` to `1.9.1` → single copy in the entire tree, Sentry and Twenty share the same instance - Named import in `instrument.ts` (`import { metrics as otelMetrics }` instead of default import) as defense-in-depth against CJS interop issues ### Verified on dev cluster Exec'd into the running pod and confirmed: - `@sentry/node` nests `@opentelemetry/api@1.9.1`, hoisted is `1.9.0` - `Sentry.init()` → global version `1.9.1` → `setGlobalMeterProvider` with VERSION `1.9.0` → returns `false` → `NoopMeterProvider` - Same-version registration returns `true` → `MeterProvider` ✓ ## Test plan - [ ] CI passes (lint, typecheck, build) - [ ] Deploy to dev cluster and verify metrics flow to collector - [ ] Confirm `node_modules/@opentelemetry/api/package.json` shows `1.9.1` with no nested copy under `@sentry/` --------- Co-authored-by: Cursor <cursoragent@cursor.com> |
||
|
Charles Bochet
|
ff22988caf |
revert: Sentry #20064 + @sentry 10.27 (prod bisect) (#20221)
## Summary Reverts **#20064** (`feat(sentry): propagate workspace context to all spans`) and downgrades **@sentry** packages from **10.51** back to **10.27** (reversing **#20149**), to validate in production whether recent Sentry/instrumentation changes correlate with OTLP/metrics issues. ## Changes 1. **Revert #20064** — removes `beforeSendSpan` from `instrument.ts`, restores `WorkspaceAuthContextMiddleware` / `BullMQDriver` behavior, and deletes the three `apply-workspace-sentry-*` utils added in that PR. 2. **Sentry versions** — `packages/twenty-server` (`@sentry/nestjs`, `@sentry/node`, `@sentry/profiling-node`) and `packages/twenty-front` (`@sentry/react`) set to `^10.27.0`; `yarn.lock` regenerated via `yarn install`. --------- Co-authored-by: Cursor <cursoragent@cursor.com> |
||
|
Félix Malfait
|
875795cc30 |
feat(sentry): propagate workspace context to all spans (#20064)
## Summary After the 2026-04-26 token-usage incident, identifying the responsible workspace from a Sentry trace required a Postgres scavenger hunt — Vercel AI SDK auto-instrumentation captures token counts and model name but no twenty-specific identifiers, and that same gap exists for every other auto-instrumented span (HTTP outbound, Postgres queries, GraphQL resolvers, Redis, etc.). This PR plugs that gap globally, not just for AI: - A small utility (`packages/twenty-server/src/engine/core-modules/sentry/utils/sentry-workspace-context.util.ts`) that writes workspace identifiers onto Sentry's active isolation scope as a `twenty` context block plus filterable tags and a `Sentry.setUser` call. - Two hook points covering all server traffic: - **`WorkspaceAuthContextMiddleware`** — already runs after token hydration on the GraphQL, metadata, admin-panel, and REST routes. It now calls the utility once per authenticated request, before delegating to `withWorkspaceAuthContext`. - **`BullMQDriver.work` and `SyncDriver.processJob`** — every queue job now runs inside `Sentry.withIsolationScope` and applies workspace context from `job.data.workspaceId` (skipping silently for system jobs that don't carry one). - A `beforeSendSpan` hook in `instrument.ts` that reads the scope's `twenty` context block back and projects it onto every span as `twenty.workspace.id` and (when available) `twenty.user_workspace.id` — dotted-namespace naming consistent with OTel/Sentry conventions like `user.id` and `http.response.status_code`. Spans without a workspace context (unauthenticated traffic) pass through untouched. ## Why this shape Sentry's docs position `beforeSendSpan` as a per-span hook. The previous iteration set context only at AI-specific call sites, which left non-AI spans (DB queries, outbound HTTP, regular GraphQL queries, workflow steps not touching AI) entirely unenriched. Hooking the two existing global boundaries — auth middleware for HTTP/GraphQL/REST, and the queue driver `work()` callback for background jobs — covers every authenticated span across the app with no per-handler instrumentation. ## What's not in this PR AI-specific identifiers (`twenty.agent.id`, `twenty.thread.id`, `twenty.turn.id`, `twenty.workflow_run.id`) are out of scope here. They're useful additions but require either propagating the IDs through the call stack or a more fine-grained scope (per-step, per-turn) than the request/job boundary, which is best handled in follow-up PRs that target those specific call sites. ## Test plan - [ ] Make any authenticated GraphQL request locally and confirm the resulting span(s) in Sentry carry `twenty.workspace.id` and (for user-authenticated routes) `twenty.user_workspace.id`. - [ ] Make any authenticated REST request and confirm the same. - [ ] Trigger a queue job (chat stream, agent turn evaluation, workflow run, etc.) and confirm spans produced inside the worker carry `twenty.workspace.id`. - [ ] Confirm that DB and outbound HTTP spans produced under the request/job also carry the workspace tag — these previously had no twenty-specific identifiers. - [ ] In the Sentry UI, filter events by the `twenty.workspace.id` tag and confirm matching events appear. ## Notes for review - Sentry init lives in `instrument.ts`, loaded before Nest bootstraps, so `beforeSendSpan` runs outside Nest DI and reads context off the isolation scope rather than holding a service reference. - The middleware change is three lines; the BullMQ wrap is a single `Sentry.withIsolationScope` around the existing job handler body; the SyncDriver wrap mirrors it for the dev/test path. No new modules or DI providers. - The previous iteration's `AiCallContextService` and per-handler `setContext` / `withContext` calls have been removed in favor of these two hooks. 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
Etienne
|
aecbc89a3f |
Fix slow db query issue (#19770)
https://github.com/twentyhq/twenty/pull/19586#discussion_r3074136617 |
||
|
Thomas Trompette
|
446a3923f2 |
Add workspace id in job logs (#19764)
Cannot currently investigate spikes |
||
|
Félix Malfait
|
455022f652 |
Add ClickHouse-backed metered credit cap enforcement (#19586)
## Summary Implements a ClickHouse-backed polling system to enforce metered-credit caps for workflow executions, replacing reliance on Stripe billing alerts. The system re-evaluates tier caps against live pricing on every poll cycle, allowing price/tier changes to propagate immediately without recreating Stripe alert objects. ## Key Changes - **BillingUsageCapService**: New service that queries ClickHouse for current-period credit usage and evaluates whether a subscription has reached its metered-credit allowance (tier cap + credit balance) - `isClickHouseEnabled()`: Checks if ClickHouse is configured - `getCurrentPeriodCreditsUsed()`: Sums creditsUsedMicro from usageEvent table for a workspace within a billing period - `evaluateCap()`: Determines if usage has reached the allowance by reading live pricing from the subscription - **EnforceUsageCapJob**: Cron job that polls all active subscriptions and updates `hasReachedCurrentPeriodCap` on metered items - Runs every 2 minutes to keep cap enforcement in sync with live usage - Supports shadow mode (log-only) via `BILLING_USAGE_CAP_CLICKHOUSE_ENABLED` flag for safe rollout - Continues processing after per-subscription errors with detailed logging - **EnforceUsageCapCronCommand**: CLI command to register the enforcement cron job - **MeteredCreditService**: Extracted `extractMeteredPricingInfoFromSubscription()` as a pure function for callers that already hold the subscription with pricing loaded, avoiding redundant DB queries - **Configuration**: Added `BILLING_USAGE_CAP_CLICKHOUSE_ENABLED` flag to control enforcement mode (active vs. shadow) - **Constants**: Added `METERED_OPERATION_TYPES` to define which operation types count toward the metered product's credit cap ## Implementation Details - The service queries ClickHouse for the sum of `creditsUsedMicro` in the current billing period, matching Stripe meter semantics - Pricing is re-read on every evaluation, so tier changes propagate within one poll cycle without Stripe alert recreation - The cron job only updates the database when the cap state actually changes (no-op if already in the correct state) - Shadow mode allows safe validation before enabling enforcement; transitions are logged but not persisted - Comprehensive test coverage for both the service and cron job, including error handling and state transitions https://claude.ai/code/session_01VksTSrYLXJVCPVBQhQdBTe --------- Co-authored-by: Claude <noreply@anthropic.com> |
||
|
Charles Bochet
|
b284c8323c |
Remove Favorite and FavoriteFolder from workspace schema (#19536)
## Summary - Removes all workspace schema definitions for `Favorite` and `FavoriteFolder` entities, which have been fully migrated to `NavigationMenuItems` - Deletes 26 standalone files including workspace entities, NestJS modules, services, listeners, jobs, standard application builders (field metadata, views, view fields, view field groups, indexes, page layouts), mocks, and integration tests - Cleans up ~40 modified files: removes `favorites` relation from 10 workspace entities and their field metadata utils, removes entries from all builder maps, shared constants (`STANDARD_OBJECTS`, `CoreObjectNameSingular`, `DEFAULT_RELATIONS_OBJECTS_STANDARD_IDS`), SDK default relations, AI tool filtering, and standard object icons |
||
|
Charles Bochet
|
f6423f5925 |
Remove DataSourceService and clean up datasource migration logic (#19532)
## Summary - **Drop the `objectMetadata.dataSourceId` foreign key and index** via a 1-22 fast instance command — column kept nullable for data preservation - **Delete `DataSourceService`, `DataSourceModule`, and `DataSourceException`** — all code now uses `workspace.databaseSchema` directly - **Remove `IS_DATASOURCE_MIGRATED` feature flag** from default flags and all branching logic - **Simplify workspace/object creation pipelines** — `WorkspaceManagerService`, `DevSeederService`, and the object creation action handler no longer route through `DataSourceService` - **Keep `DataSourceEntity` and the `dataSource` table** for historical data — entity stripped of all ORM relations |
||
|
Paul Rastoin
|
a53b9cb68a |
On workspace creation generate sdk client through a job (#19271)
Avoid overloading the workspace creation with the standard and custom
app sdk generation, do through a job
```ts
[Nest] 90397 - 04/02/2026, 4:44:20 PM LOG [CoreEntityCacheService] Cache stats: localCacheSize=0 memoizerCacheSize=0 memoizerPendingSize=0 entriesByKey={} versionsByKey={}
[Nest] 90397 - 04/02/2026, 4:45:03 PM LOG [BullMQDriver] Processing job 4325 with name CallWebhookJobsForMetadataJob on queue webhook-queue
[Nest] 90397 - 04/02/2026, 4:45:03 PM LOG [BullMQDriver] Processing job 1 with name GenerateSdkClientJob on queue workspace-queue
[Nest] 90397 - 04/02/2026, 4:45:03 PM LOG [BullMQDriver] Job 1 with name GenerateSdkClientJob processed on queue workspace-queue in 0.64ms
[Nest] 90397 - 04/02/2026, 4:45:03 PM LOG [BullMQDriver] Processing job 2 with name GenerateSdkClientJob on queue workspace-queue
[Nest] 90397 - 04/02/2026, 4:45:03 PM LOG [BullMQDriver] Job 2 with name GenerateSdkClientJob processed on queue workspace-queue in 0.06ms
[Nest] 90397 - 04/02/2026, 4:45:03 PM LOG [BullMQDriver] Job 4325 with name CallWebhookJobsForMetadataJob processed on queue webhook-queue in 17.47ms
[Nest] 90397 - 04/02/2026, 4:45:03 PM LOG [BullMQDriver] Processing job 4326 with name CallWebhookJobsForMetadataJob on queue webhook-queue
```
|
||
|
Félix Malfait
|
8fa3962e1c |
feat: add resumable stream support for agent chat (#19107)
## Overview Add resumable stream support for agent chat to allow clients to reconnect and resume streaming responses if the connection is interrupted (e.g., during page refresh). ## Changes ### Backend (Twenty Server) - Add `activeStreamId` column to `AgentChatThreadEntity` to track ongoing streams - Create `AgentChatResumableStreamService` to manage Redis-backed resumable streams using the `resumable-stream` library with ioredis - Extend `AgentChatController` with: - `GET /:threadId/stream` endpoint to resume an existing stream - `DELETE /:threadId/stream` endpoint to stop an active stream - Update `AgentChatStreamingService` to store streams in Redis and track active stream IDs - Add `resumable-stream@^2.2.12` dependency to package.json ### Frontend (Twenty Front) - Update `useAgentChat` hook to: - Use a persistent transport with `prepareReconnectToStreamRequest` for resumable streams - Export `resumeStream` function from useChat - Add `handleStop` callback to clear active stream on DELETE endpoint - Use thread ID as stable message ID instead of including message count - Add stream resumption logic in `AgentChatAiSdkStreamEffect` component to automatically call `resumeStream()` when switching threads ## Database Migration New migration `1774003611071-add-active-stream-id-to-agent-chat-thread` adds the `activeStreamId` column to store the current resumable stream identifier. --------- Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com> |
||
|
Marie
|
c1da7be6d7 |
Billing for self-hosts (#18075)
## Summary Implements enterprise licensing and per-seat billing for self-hosted environments, with Stripe as the single source of truth for subscription data. ### Components - **twenty-website** hosts the private key to sign `ENTERPRISE_KEY` and `ENTERPRISE_VALIDITY_TOKEN`. It communicates with Stripe to emit the daily `ENTERPRISE_VALIDITY_TOKEN` if the subscription is active, based on the user's Stripe subscription ID stored in `ENTERPRISE_KEY`. - **Stripe** is the single source of truth for subscription data (status, seats, billing). - **The client** (twenty-server + DB + workers) saves `ENTERPRISE_KEY` in the `keyValuePair` table (or `.env` if `IS_CONFIG_VARIABLES_IN_DB_ENABLED` is false) and the daily-renewed `ENTERPRISE_VALIDITY_TOKEN` in the `appToken` table. `ENTERPRISE_VALIDITY_TOKEN` is verified client-side using a public key to grant access to enterprise features (RLS, SSO, audit logs, etc.). ### Flow 1. When requesting an upgrade to an enterprise plan (from **Enterprise** in settings), the user is shown a modal to choose monthly/yearly billing, then redirected to Stripe to enter payment details. After checkout, they land on twenty-website where they are exposed to their `ENTERPRISE_KEY`, which they paste in the UI. It is saved in the `keyValuePair` table. On activation, a first `ENTERPRISE_VALIDITY_TOKEN` with 30-day validity is stored in the `appToken` table. 2. **Every day**, a cron job runs and does two things: - **Refreshes the validity token**: communicates with twenty-website to get a new `ENTERPRISE_VALIDITY_TOKEN` with 30-day validity if the Stripe subscription is still active. If the subscription is in cancellation, the emitted token has a validity equal to the cancellation date. If it's no longer valid, the token is not replaced. The cron only needs to run every 30 days in practice, but runs daily so it's resilient to occasional failures. - **Reports seat count**: counts active (non-soft-deleted) `UserWorkspace` entries and sends the count to twenty-website, which updates the Stripe subscription quantity with proration. Seats are also reported on first activation. If the subscription is canceled or scheduled for cancellation, the seat update is skipped. 3. `ENTERPRISE_VALIDITY_TOKEN` is verified server-side via a public key to grant access to enterprise features. ### Key concepts Three distinct checks are exposed as GraphQL fields on `Workspace`: | Field | Meaning | |---|---| | `hasValidEnterpriseKey` | Has any valid enterprise key (signed JWT **or** legacy plain string) | | `hasValidSignedEnterpriseKey` | `ENTERPRISE_KEY` is a properly signed JWT (billing portal makes sense) | | `hasValidEnterpriseValidityToken` | `ENTERPRISE_VALIDITY_TOKEN` is present and not expired (expiration depends on signed token payload, not on "expiresAt" on appToken table which is only indicative) | Feature access is gated by `isValid()` = `hasValidEnterpriseValidityToken || hasValidEnterpriseKey` (to support both new and legacy keys during transition). After transition isValid() = hasValidEnterpriseValidityToken ### Frontend states The Enterprise settings page handles multiple states: - **No key**: show "Get Enterprise" with checkout modal - **Orphaned validity token** (token valid but no signed key): prompt user to set a valid enterprise key - **Active/trialing but no validity token**: show subscription status with a "Reload validity token" action - **Active/trialing**: show full subscription info, billing portal access, cancel option - **Cancellation scheduled**: show cancellation date, billing portal - **Canceled**: show billing history link and option to start a new subscription - **Past due / Incomplete**: prompt to update payment or restart ### Temporary retro-compatibility: legacy plain-text keys Previously, enterprise features were gated by a simple check: any non-empty string in `ENTERPRISE_KEY` granted access. With this PR, we transition to a controlled system relying on signed JWTs. To avoid breaking existing self-hosted users: - **Legacy plain-text keys still grant access** to enterprise features. `hasValidEnterpriseKey` returns `true` for both signed JWTs and plain strings, and `isValid()` checks `hasValidEnterpriseKey` as a fallback when no validity token is present. - **A deprecation banner** is shown at the top of the app when `hasValidEnterpriseKey` is `true` but `hasValidSignedEnterpriseKey` is `false`, informing the user that their key format is deprecated and they should activate a new signed key. - **No billing portal or subscription management** is available for legacy keys since there is no Stripe subscription to manage. This retro-compatibility will be removed in a future version. At that point, `isValid()` will only check `hasValidEnterpriseValidityToken`. ### Edge cases - **Air-gapped / production environments**: for self-hosted clients that block external traffic (or for our own production), provide a long-lived `ENTERPRISE_VALIDITY_TOKEN` (e.g. 99 years) directly in the `appToken` table, with no `ENTERPRISE_KEY`. The daily cron will skip the refresh (no enterprise key to authenticate with), but the pre-seeded validity token will be used to grant feature access. No billing or seat reporting occurs in this mode. - **`IS_CONFIG_VARIABLES_IN_DB_ENABLED` is false**: if the user tries to activate an enterprise key but DB config writes are disabled, the backend returns a clear error asking them to add `ENTERPRISE_KEY` to their `.env` file manually. - **Canceled subscriptions**: the `/seats` endpoint skips Stripe updates for canceled or cancellation-scheduled subscriptions to avoid Stripe API errors. ### How to test - launch twenty-website on a different url (eg localhost:1002) - add ENTERPRISE_API_URL=http://localhost:3002/api/enterprise (or else) in your server .env - ask me for twenty-website's .env file content (STRIPE_SECRET_KEY; STRIPE_ENTERPRISE_MONTHLY_PRICE_ID;STRIPE_ENTERPRISE_YEARLY_PRICE_ID; ENTERPRISE_JWT_PRIVATE_KEY; ENTERPRISE_JWT_PUBLIC_KEY; NEXT_PUBLIC_WEBSITE_URL) - visit Admin panel / enterprise |
||
|
Charles Bochet
|
9d57bc39e5 |
Migrate from ESLint to OxLint (#18443)
## Summary Fully replaces ESLint with OxLint across the entire monorepo: - **Replaced all ESLint configs** (`eslint.config.mjs`) with OxLint configs (`.oxlintrc.json`) for every package: `twenty-front`, `twenty-server`, `twenty-emails`, `twenty-ui`, `twenty-shared`, `twenty-sdk`, `twenty-zapier`, `twenty-docs`, `twenty-website`, `twenty-apps/*`, `create-twenty-app` - **Migrated custom lint rules** from ESLint plugin format to OxLint JS plugin system (`@oxlint/plugins`), including `styled-components-prefixed-with-styled`, `no-hardcoded-colors`, `sort-css-properties-alphabetically`, `graphql-resolvers-should-be-guarded`, `rest-api-methods-should-be-guarded`, `max-consts-per-file`, and Jotai-related rules - **Migrated custom rule tests** from ESLint `RuleTester` + Jest to `oxlint/plugins-dev` `RuleTester` + Vitest - **Removed all ESLint dependencies** from `package.json` files and regenerated lockfiles - **Updated Nx targets** (`lint`, `lint:diff-with-main`, `fmt`) in `nx.json` and per-project `project.json` to use `oxlint` commands with proper `dependsOn` for plugin builds - **Updated CI workflows** (`.github/workflows/ci-*.yaml`) — no more ESLint executor - **Updated IDE setup**: replaced `dbaeumer.vscode-eslint` with `oxc.oxc-vscode` extension, configured `source.fixAll.oxc` and format-on-save with Prettier - **Replaced all `eslint-disable` comments** with `oxlint-disable` equivalents across the codebase - **Updated docs** (`twenty-docs`) to reference OxLint instead of ESLint - **Renamed** `twenty-eslint-rules` package to `twenty-oxlint-rules` ### Temporarily disabled rules (tracked in `OXLINT_MIGRATION_TODO.md`) | Rule | Package | Violations | Auto-fixable | |------|---------|-----------|-------------| | `twenty/sort-css-properties-alphabetically` | twenty-front | 578 | Yes | | `typescript/consistent-type-imports` | twenty-server | 3814 | Yes | | `twenty/max-consts-per-file` | twenty-server | 94 | No | ### Dropped plugins (no OxLint equivalent) `eslint-plugin-project-structure`, `lingui/*`, `@stylistic/*`, `import/order`, `prefer-arrow/prefer-arrow-functions`, `eslint-plugin-mdx`, `@next/eslint-plugin-next`, `eslint-plugin-storybook`, `eslint-plugin-react-refresh`. Partial coverage for `jsx-a11y` and `unused-imports`. ### Additional fixes (pre-existing issues exposed by merge) - Fixed `EmailThreadPreview.tsx` broken import from main rename (`useOpenEmailThreadInSidePanel`) - Restored truthiness guard in `getActivityTargetObjectRecords.ts` - Fixed `AgentTurnResolver` return types to match entity (virtual `fileMediaType`/`fileUrl` are resolved via `@ResolveField()`) ## Test plan - [x] `npx nx lint twenty-front` passes - [x] `npx nx lint twenty-server` passes - [x] `npx nx lint twenty-docs` passes - [x] Custom oxlint rules validated with Vitest: `npx nx test twenty-oxlint-rules` - [x] `npx nx typecheck twenty-front` passes - [x] `npx nx typecheck twenty-server` passes - [x] CI workflows trigger correctly with `dependsOn: ["twenty-oxlint-rules:build"]` - [x] IDE linting works with `oxc.oxc-vscode` extension |
||
|
Charles Bochet
|
37b9a55382 |
Migrate metrics to prometheus (#17810)
<img width="1316" height="611" alt="image" src="https://github.com/user-attachments/assets/277a63ed-2a8b-41ff-be78-281de8891579" /> |
||
|
Thomas Trompette
|
04b8f2aaf5 |
Add gauge for awaiting jobs (#17747)
<img width="774" height="333" alt="Capture d’écran 2026-02-05 à 16 18 06" src="https://github.com/user-attachments/assets/ec938a39-801d-4d71-a2de-7ced1795a0bd" /> |
||
|
Charles Bochet
|
4a770eafa1 |
Rework logic function module (#17588)
core-modules/logic-function/
├── logic-function.module.ts
├── logic-function-executor/
│ ├── logic-function-executor.module.ts
│ ├── commands/
│ │ └── add-packages.command.ts
│ ├── constants/
│ │ └── logic-function-executor.constants.ts
│ ├── factories/
│ │ └── logic-function-module.factory.ts
│ ├── interfaces/
│ │ └── logic-function-executor.interface.ts
│ └── services/
│ └── logic-function-executor.service.ts
├── logic-function-build/
│ ├── logic-function-build.module.ts
│ ├── services/
│ │ └── logic-function-build.service.ts
│ └── utils/
│ └── get-logic-function-base-folder-path.util.ts
├── logic-function-drivers/
│ ├── logic-function-drivers.module.ts
│ ├── constants/
│ │ └── ...
│ ├── drivers/
│ │ ├── disabled.driver.ts
│ │ ├── lambda.driver.ts
│ │ └── local.driver.ts
│ ├── interfaces/
│ │ └── logic-function-executor-driver.interface.ts
│ ├── layers/
│ │ └── ...
│ └── utils/
│ └── ...
├── logic-function-layer/
│ ├── logic-function-layer.module.ts
│ └── services/
│ └── logic-function-layer.service.ts
└── logic-function-trigger/
├── logic-function-trigger.module.ts
├── jobs/
│ └── logic-function-trigger.job.ts
└── triggers/
├── cron/
├── database-event/
└── route/
├── exceptions/
├── services/
│ └── route-trigger.service.ts
└── utils/
|
||
|
Charles Bochet
|
da6f1bbef3 |
Rename serverlessFunction to logicFunction (#17494)
## Summary Rename "Serverless Function" to "Logic Function" across the codebase for clearer naming. ### Environment Variable Changes | Old | New | |-----|-----| | `SERVERLESS_TYPE` | `LOGIC_FUNCTION_TYPE` | | `SERVERLESS_LAMBDA_REGION` | `LOGIC_FUNCTION_LAMBDA_REGION` | | `SERVERLESS_LAMBDA_ROLE` | `LOGIC_FUNCTION_LAMBDA_ROLE` | | `SERVERLESS_LAMBDA_SUBHOSTING_URL` | `LOGIC_FUNCTION_LAMBDA_SUBHOSTING_URL` | | `SERVERLESS_LAMBDA_ACCESS_KEY_ID` | `LOGIC_FUNCTION_LAMBDA_ACCESS_KEY_ID` | | `SERVERLESS_LAMBDA_SECRET_ACCESS_KEY` | `LOGIC_FUNCTION_LAMBDA_SECRET_ACCESS_KEY` | ### Breaking Changes - Environment variables must be updated in production deployments - Database migration renames `serverlessFunction` → `logicFunction` tables |
||
|
Charles Bochet
|
bc7791871f |
Introduce webhook v2 (#17456)
Migrate webhook to v2 entity |
||
|
Abdul Rahman
|
2a8f834377 |
Integrate NavigationMenuItem with feature flag support (#17268)
## Implement Navigation Menu Items Frontend Implements the frontend for navigation menu items, the new system replacing favorites. ### Changes - Added GraphQL fragments and queries for navigation menu items - Added hooks for managing navigation menu items (create, update, delete, sorting, filtering) - Updated components to use navigation menu items instead of favorites - Added test coverage for utility functions ### Migration Note The favorites and navigation menu item modules currently exist in parallel. The favorites code will be removed once all data has been migrated to navigation menu items. <!-- CURSOR_SUMMARY --> --- > [!NOTE] > Replaces Favorites with feature-flagged `NavigationMenuItem` across frontend and backend, while keeping Favorites as fallback until migration completes. > > - UI: new `navigation-menu-item` components (folders, orphan items, drag provider/droppable, icons, skeleton), dispatcher components to switch from Favorites, and updated “Add to favorites” action to create `NavigationMenuItem` when `IS_NAVIGATION_MENU_ITEM_ENABLED` > - DnD: shared `validateAndExtractFolderId` and droppable id utils moved to `ui/layout/draggable-list`; favorites DnD updated to use shared utils > - GraphQL (client): add fragments, queries, mutations, hooks (create/update/delete/find), and generated types; added `RecordIdentifier` and `targetRecordIdentifier` on `NavigationMenuItem` > - Prefetch: new prefetch state/effect for navigation menu items; skip favorites prefetch when flag enabled > - Backend: add DTOs (`NavigationMenuItem`, `RecordIdentifier`), resolver `targetRecordIdentifier` field, service logic to fetch record identifiers with permission-aware access and image signing, `getRecordImageIdentifier` util, entity relation to `view`, and migration adding FK on `viewId` > - Feature flags & seeding: add `IS_NAVIGATION_MENU_ITEM_ENABLED` to enums, dev seeder enables it; standard app seeds workspace navigation menu items instead of favorites when flag on > - Tests: add unit tests for sorting/labels/folder id and related utils > > <sup>Written by [Cursor Bugbot](https://cursor.com/dashboard?tab=bugbot) for commit c99746f08b9f84fc8cec4fcc3a7d7afb8ea92db7. This will update automatically on new commits. Configure [here](https://cursor.com/dashboard?tab=bugbot).</sup> <!-- /CURSOR_SUMMARY --> --------- Co-authored-by: Félix Malfait <felix.malfait@gmail.com> Co-authored-by: Aman Raj <92664006+araj00@users.noreply.github.com> Co-authored-by: Félix Malfait <felix@twenty.com> Co-authored-by: Paul Rastoin <45004772+prastoin@users.noreply.github.com> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: github-actions <github-actions@twenty.com> |
||
|
Thomas Trompette
|
6043edd53f |
Add metrics for completed and failed jobs (#16969)
Add a counter for completed and failed jobs |
||
|
Paul Rastoin
|
98a9ae2a0e |
Migrate view filter group to v2 (#16876)
## Introduction On the side hanlded PR with a // agents on another repo Made several iterations to fix behavior and direction Find below auto-generated PR description closes https://github.com/twentyhq/core-team-issues/issues/2037 Created generic tooling for entity circular dep checking, will be useful for permissions validation too @Weiko ## Migrate `viewFilterGroup` entity to v2 flat architecture ### Summary Migrates the `viewFilterGroup` entity from v1 to the v2 flat entity architecture, following the established patterns for other v2 entities like `viewFilter`, `view`, and `viewField`. ### Changes **Types & Constants** - Added `FlatViewFilterGroup` and `FlatViewFilterGroupMaps` types - Added editable properties constant for `viewFilterGroup` - Registered `viewFilterGroup` in `ALL_METADATA_NAME`, `ALL_METADATA_RELATION_PROPERTIES`, `ALL_METADATA_MANY_TO_ONE_RELATIONS`, and related constants **Cache Service** - Created `WorkspaceFlatViewFilterGroupMapCacheService` with proper relation loading for `viewFilters` and `childViewFilterGroups` - Updated `WorkspaceFlatViewMapCacheService` to load `viewFilterGroups` relation **Builder & Validator** - Created `WorkspaceMigrationV2ViewFilterGroupActionsBuilderService` - Created `FlatViewFilterGroupValidatorService` with creation, update, and deletion validation - Integrated validation into the orchestrator service (runs before `viewFilter` validation) **Action Handlers** - Created create, update, and delete action handlers for `viewFilterGroup` **Service Migration** - Rewrote `ViewFilterGroupService` to use v2 migration pattern with `WorkspaceMigrationValidateBuildAndRunService` - Created utility functions for transforming DTOs to flat entities **Database Migration** - Added migration to make `parentViewFilterGroupId` foreign key deferrable (handles self-referential parent/child insertions) **ViewFilter Integration** - Added `viewFilterGroupId` validation in `FlatViewFilterValidatorService` - Updated `viewFilter` many-to-one relations to include `viewFilterGroup` **Tests** - Added integration tests for successful creation, update, deletion, and destruction - Added failing test cases for non-existent entities and invalid references - Added failing test for `viewFilter` creation with non-existent `viewFilterGroupId` ### Breaking Changes None - existing API contracts are preserved. |
||
|
Abdul Rahman
|
f248b3f7f4 | refactor: move agent evaluation to background jobs for non-blocking execution (#16234) | ||
|
Abdullah.
|
0cab2b49fc |
(breaking change) Allow users with a single workspace to update their email. (#15736)
- Users with a single workspace are allowed to update their email across `core.user` and `workspace_xyz.workspaceMember`. - The latter happens asynchronously (built it like this for non-blocking with multiple workspaces), but since we restrict the email update functionality to a single user, we can also update the email in workspaceMember synchronously - I left asynchronous there to receive feedback on whether we should move to synchronous or not. - Merged main and resolved conflicts to ensure we use the `SettingsPermissionGuard` and the updated `workspace.service.ts` code. One edge-case that I was trying to communicate on Discord: Say that an admin is a member of multiple workspaces. Therefore, they can allow roles with PROFILE_INFORMATION permission to update their email. <p align="center"> <img width="553" height="115" alt="image" src="https://github.com/user-attachments/assets/80382b1f-a9e3-4dac-b606-c2defeb2c330" /> </p> However, since the admin is part of multiple workspaces, he/she cannot even update own email - the field stays disabled, leading to some confusion. <p align="center"> <img width="545" height="255" alt="image" src="https://github.com/user-attachments/assets/5e6d27db-c9a8-4d5e-9ab6-65c77beae5b4" /> </p> However, the workspace can have another member with admin role or some other role that has PROFILE_INFORMATION permission flag. That user will be and should be allowed to update email, so we cannot hide `email` from dropdown options. <p align="center"> <img width="585" height="283" alt="image" src="https://github.com/user-attachments/assets/a670d3ac-cf48-4865-a425-b909093d8420" /> </p> The behavior is fine imo, just a little confusing for members with more than one workspace. I have also tested the flow by signing up to YC workspace with my org google account (twenty.com), then changing email to my personal address. - After changing, I need to login using Google with my personal account to access YC workspace again. - If I login using Google with org google account (twenty.com), a new user account is created. This behavior is consistent with Notion and Linear. Finally, as for the verification of email, the user is asked to verify email while they're logged in, but just in case they logout without verifying, the next login would force them to verify their email in the email/password flow. However, for Social/SSO, they must verify before they logout or else they'd have to contact support for assistance. I have not looked into how to show verification screen while logging in via Social/SSO yet, but if that's something critical for completeness here, I shall revisit it. --------- Co-authored-by: Félix Malfait <felix@twenty.com> |
||
|
Félix Malfait
|
c5564d9bd0 |
[BREAKING CHANGE] refactor: Add Entity suffix to TypeORM entity classes (#15239)
## Summary This PR refactors all TypeORM entity classes in the Twenty codebase to include an 'Entity' suffix (e.g., User → UserEntity, Workspace → WorkspaceEntity) to improve code clarity and follow TypeORM naming conventions. ## Changes ### Entity Renaming - ✅ Renamed **57 core TypeORM entities** with 'Entity' suffix - ✅ Updated all related imports, decorators, and type references - ✅ Fixed Repository<T>, @InjectRepository(), and TypeOrmModule.forFeature() patterns - ✅ Fixed @ManyToOne/@OneToMany/@OneToOne decorator references ### Backward Compatibility - ✅ Preserved GraphQL schema names using @ObjectType('OriginalName') decorators - ✅ **No breaking changes** to GraphQL API - ✅ **No database migrations** required - ✅ File names unchanged (user.entity.ts remains as-is) ### Code Quality - ✅ Fixed **497 TypeScript errors** (82% reduction from 606 to 109) - ✅ **All linter checks passing** - ✅ Improved type safety across the codebase ## Entities Renamed ``` User → UserEntity Workspace → WorkspaceEntity ApiKey → ApiKeyEntity AppToken → AppTokenEntity UserWorkspace → UserWorkspaceEntity Webhook → WebhookEntity FeatureFlag → FeatureFlagEntity ApprovedAccessDomain → ApprovedAccessDomainEntity TwoFactorAuthenticationMethod → TwoFactorAuthenticationMethodEntity WorkspaceSSOIdentityProvider → WorkspaceSSOIdentityProviderEntity EmailingDomain → EmailingDomainEntity KeyValuePair → KeyValuePairEntity PublicDomain → PublicDomainEntity PostgresCredentials → PostgresCredentialsEntity ...and 43 more entities ``` ## Impact ### Files Changed - **400 files** modified - **2,575 insertions**, **2,191 deletions** ### Progress - ✅ **82% complete** (497/606 errors fixed) - ⚠️ **109 TypeScript errors** remain (18% of original) ## Remaining Work The 109 remaining TypeScript errors are primarily: 1. **Function signature mismatches** (~15 errors) - Test mocks with incorrect parameter counts 2. **Entity type mismatches** (~25 errors) - UserEntity vs UserWorkspaceEntity confusion 3. **Pre-existing issues** (~50 errors) - Null safety and DTO compatibility (unrelated to refactoring) 4. **Import type issues** (~10 errors) - Entities imported with 'import type' but used as values 5. **Minor decorator issues** (~9 errors) - onDelete property configurations These can be addressed in follow-up PRs without blocking this refactoring. ## Testing Checklist - [x] Linter passing - [ ] Unit tests should be run (CI will verify) - [ ] Integration tests should be run (CI will verify) - [ ] Manual testing recommended for critical user flows ## Breaking Changes **None** - This is a pure refactoring with full backward compatibility: - GraphQL API unchanged (uses original entity names) - Database schema unchanged - External APIs unchanged ## Notes - Created comprehensive `REFACTORING_STATUS.md` documenting the entire process - All temporary scripts have been cleaned up - Branch: `refactor/add-entity-suffix-to-typeorm-entities` ## Reviewers Please review especially: - Entity renaming patterns - GraphQL backward compatibility - Any areas where entity types are confused (UserEntity vs UserWorkspaceEntity) --------- Co-authored-by: Charles Bochet <charles@twenty.com> |
||
|
Félix Malfait
|
f7421c5fc0 |
Add queue management dashboard (#15202)
Adds a comprehensive queue management interface to the admin panel for viewing and managing background jobs. **Features:** - Queue detail pages showing paginated job lists (50 per page) - Filter jobs by state: completed, failed, active, waiting, delayed, paused - Checkbox selection with bulk actions (delete jobs, retry failed jobs) - Per-job dropdown menu for individual retry/delete - Expandable rows showing error messages, stack traces, and job data - Relative timestamps with hover tooltips - Display attempt counts on failed jobs - Dynamic retention policy info from backend **Changes:** - Backend: New AdminPanelQueueService with GraphQL endpoints for job listing, retry, and delete - Frontend: Queue detail page with QueueJobsTable component - Updated retention policy: completed jobs kept 4 hours, failed jobs kept 7 days (max 1000 each) - Added JobState enum for type safety <img width="634" height="696" alt="Screenshot_2025-10-20_at_11 45 25" src="https://github.com/user-attachments/assets/c67bcd27-26cf-47f5-9575-3cd5684d006b" /> <img width="484" height="680" alt="Screenshot_2025-10-20_at_11 45 14" src="https://github.com/user-attachments/assets/68725cc6-b3ec-4098-99ca-f9a717d6f8f1" /> <img width="490" height="643" alt="Screenshot_2025-10-20_at_11 45 05" src="https://github.com/user-attachments/assets/b68a5809-33ff-4452-b48b-741aff7f1dd6" /> <img width="685" height="662" alt="Screenshot 2025-10-20 at 13 15 01" src="https://github.com/user-attachments/assets/eeb5207b-de5c-4b18-bdde-392892053dab" /> |
||
|
Harshit Singh
|
4e5783eaf4 |
feat: workflow delay action (Pause - Wait/Sleep/Delay) (#14915)
## Description - This PR focuses on issue https://github.com/orgs/twentyhq/projects/1/views/33?pane=issue&itemId=93150683&issue=twentyhq%7Ccore-team-issues%7C20 - added Workflow delay as a Flow action - for V1 added Type 1: Resume at a specific date or time ## Visual Appearance <img width="1792" height="1038" alt="Screenshot 2025-10-09 at 5 46 18 PM" src="https://github.com/user-attachments/assets/e62980e9-59c7-4e5a-b8ec-1e848a462d3f" /> <img width="1792" height="1037" alt="Screenshot 2025-10-09 at 5 46 35 PM" src="https://github.com/user-attachments/assets/7c3f4e39-ab0a-40ed-97a8-4f0cdb86f295" /> --------- Co-authored-by: Thomas Trompette <thomas.trompette@sfr.fr> |
||
|
Antoine Moreaux
|
d4160bf064 |
feat(redis): add support for dedicated queue client configuration (#14840)
Fix https://github.com/twentyhq/core-team-issues/issues/452 Fix https://github.com/twentyhq/core-team-issues/issues/923 --------- Co-authored-by: Félix Malfait <felix@twenty.com> |
||
|
Weiko
|
68b21d2942 |
Add db event trigger and cron trigger to migration v2 (#14772)
## Context Add DB events triggers and Cron triggers to migration v2 builder/runner (and adding corresponding services/resolvers) |
||
|
Charles Bochet
|
71216b4db8 |
Add logs to debug worker performance on production (#14712)
As per title, just adding worker job execution time |
