## Context
A production customer was stuck on the billing settings page: their
workspace was `SUSPENDED` (with `suspendedAt` set) while their
subscription was `active` in the database.
## Problem
Stripe webhook events can be delivered out of order or processed
concurrently ([Stripe explicitly does not guarantee
ordering](https://docs.stripe.com/webhooks#events-ordering)), and
`BillingWebhookSubscriptionService.processStripeEvent` made its
suspend/reactivate decision from stale and incomplete data:
- The decision used the **event payload's** subscription status, while
the subscription row was upserted from a **live Stripe fetch** — so the
two could diverge. Around trial end, Stripe emits `active → past_due`
then (once the customer pays) `past_due → active` within a short window.
If the stale `past_due` event is processed last, it suspends the
workspace while writing an `active` subscription to the DB. Nothing
self-heals from that state: the workspace stays suspended while the
cleanup cron warns and eventually soft-deletes it.
- The decision only looked at the **event's own subscription**, but
suspension is a workspace-level decision and a Stripe customer can hold
several subscriptions (plan switch, cancel-then-resubscribe). A
`customer.subscription.deleted` event for the old subscription is
*genuinely* canceled — only the sibling subscription proves the customer
is still paying.
- The workspace snapshot was read at the top of the handler, before
several slow awaits, so a concurrent event could change it mid-flight.
## Fix
Every event now converges the workspace to the current Stripe state,
regardless of delivery order:
- **Live input**: fetch the customer's not-ended subscriptions from
Stripe (`subscriptions.list` without a `status` param excludes the
unbounded canceled history server-side; an explicit
`NOT_ENDED_SUBSCRIPTION_STATUSES` filter additionally drops
`incomplete_expired`, which would otherwise block suspension forever
since it is neither suspend-worthy nor activating). The event's
subscription is taken from that list, or fetched directly by id when
absent (deletion events, deleted customers) — same retrieve the code
used before. The DB upsert uses this live object, never the payload.
- **Workspace-level decision over all live subscriptions**: suspend only
when **every** subscription warrants it, reactivate as soon as **one**
is activating (`active`/`trialing`), and deliberately do nothing in
between — e.g. a `past_due` subscription in its payment-retry grace
period blocks suspension without triggering reactivation.
- **Guarded transitions (compare-and-swap)**:
`WorkspaceService.suspendWorkspace`/`reactivateWorkspace` now apply
their UPDATE only when the workspace is still in a state the transition
is valid from, and return whether they applied. Repeated suspensions
keep the first `suspendedAt` so the cleanup countdown stays anchored to
the original suspension date; the deletion-warning cleanup job is only
enqueued when a reactivation actually applied. The suspend path re-reads
the workspace right before deciding and switches exhaustively on
`activationStatus` (`assertUnreachable` in `default`), preserving the
previous behavior including suspend-over-reactivate precedence.
## Tests
Unit tests cover the incident scenario and its neighbors: a stale
`past_due` event after payment reactivates instead of suspending; a live
`unpaid` state suspends even when the payload says `active`; a canceled
subscription event does not suspend (and reactivates) when a sibling
`active`/`trialing` subscription exists; a sibling in `past_due` grace
blocks suspension without reactivating; the direct-retrieve fallback
handles subscriptions absent from the customer list; the guarded
reactivation skips the cleanup job when it did not apply; and
soft-deleted workspaces are never transitioned.
Paul Rastoin
217cf650aa