- enrich SSE events with relations
- remove queries from sse metadata events
- on sse event, manage store
- on object/field metadata changes, manage store
TODO left:
- fix other metadata items
---------
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: github-actions <github-actions@twenty.com>
Co-authored-by: Charles Bochet <charles@twenty.com>
## Summary
Replace the single `metadataVersion` integer with per-entity-type
**collection hashes** for granular metadata staleness detection. The
backend already generates a UUID per flat entity map on each cache
recompute (`crypto.randomUUID()` in `WorkspaceCacheService`); we now
expose these via the minimal metadata endpoint and SSE events so the
frontend can compare and know exactly which entity types are stale.
### Key changes
**Backend:**
- `WorkspaceCacheService.getCacheHashes()` — new public method that
reads only `:hash` keys from Redis without fetching full data
- `MinimalMetadataDTO` — added `collectionHashes: Record<string,
string>` (JSON scalar mapping `AllMetadataName` → collection hash),
removed `metadataVersion`
- `MetadataEventDTO` — added optional `updatedCollectionHash` field to
SSE events
- `MetadataEventsToDbListener` — reads the collection hash for the
affected entity type after cache invalidation and attaches it to the SSE
event before publishing
- `MinimalMetadataService` — no longer queries the workspace table; uses
`getCacheHashes()` for all flat entity maps and maps cache keys to
`AllMetadataName` locally
**Frontend:**
- `metadataCollectionHashesState` — new Jotai atom with
`atomWithStorage` + `getOnInit: true` storing
`Partial<Record<MetadataEntityKey, string>>`
- `mapAllMetadataNameToEntityKey()` — explicit mapping from backend
`AllMetadataName` to frontend `MetadataEntityKey` (23 entries)
- `useLoadMinimalMetadata` — stores `collectionHashes` from server,
computes `staleEntityKeys` by comparing local vs server hashes
- `patchMetadataStoreFromSSEEvent()` — accepts optional
`updatedCollectionHash` and updates `metadataCollectionHashesState`
- All 11 SSE effect components — pass
`eventDetail.updatedCollectionHash` through to the patch function
- `useStaleMetadataEntities` — new hook returning entity keys missing
from collection hashes (not yet loaded/synced)
- `resetMetadataStore()` — also clears collection hashes
- Deleted `metadataVersionState` (superseded by collection hashes)
### Design decisions
- **No change to hash generation** — existing `crypto.randomUUID()` is
sufficient. Hashes are persisted in Redis, survive server restarts, and
change only on `invalidateAndRecompute`.
- **"Collection hash" naming** — used consistently to clarify the hash
represents an entire entity collection (e.g., all views), not a single
record.
- **Mapping localized** — backend `WorkspaceCacheKeyName` →
`AllMetadataName` mapping lives in the minimal metadata service.
Frontend `AllMetadataName` → `MetadataEntityKey` mapping lives in a
local utility. Nothing in `twenty-shared`.
- **Backward compatible** — `collectionHashes` is additive;
`updatedCollectionHash` is nullable.
Create the necessary tooling to listen to metadata events and plug it to
the front components. Now we have a hot reload like experience when we
edit a component in an app.
## Backend
- Split `EventWithQueryIds` into `ObjectRecordEventWithQueryIds` and
`MetadataEventWithQueryIds`
- Publish metadata event batches to active SSE streams in
`MetadataEventsToDbListener`
## Frontend
- Create a metadata event dispatching pipeline: SSE metadata events are
grouped by metadata name, transformed into
`MetadataOperationBrowserEventDetail` objects, and dispatched as browser
`CustomEvents`
- Add `useListenToMetadataOperationBrowserEvent` hook for consuming
metadata operation events filtered by metadata name and operation type
- Rename `useListenToObjectRecordEventsForQuery` to
`useListenToEventsForQuery`, now accepting both
`RecordGqlOperationSignature` and `MetadataGqlOperationSignature`
- Implement `useOnFrontComponentUpdated` which subscribes to front
component metadata events and updates the Apollo cache when the
component is modified
- Add `builtComponentChecksum` to the front component query and appends
it to the component URL for browser cache invalidation
- new channel EVENT_STREAM_CHANNEL based on event stream id
- on event, perform the matching and publish only to the right streams
- store a list of active streams per workspace
- store the user id along with the queries for each stream
Bonus:
- remove onSubscriptionMatch
This PR is a follow-up of https://github.com/twentyhq/twenty/pull/16966
and implements a new mechanism to handle SSE events.
It creates only one event stream per browser tab, then use mutations to
tell the backend which query to listen to, without re-mounting the event
stream connexion.
Then each event that comes from this unique subscription is then
dispatched in a new JavaScript CustomEvent, per queryId, on which
specific hooks add an event listener.
This PR introduces the generic tooling as well as the handling of update
events on table.
- moved a few gql types to twenty shared to re-use in server
- added a new endpoint, onEventSubscription that expect a streamId to
create a connection
- two new endpoints to store queries in Redis
- updated the existing batch channel to directly use object record type
related to https://github.com/twentyhq/core-team-issues/issues/601
## Done
- add a `onDbEvent` `Subscription` graphql endpoint to listen to
database_event using what we have done with webhooks:
- you can subscribe to any `action` (created, updated, ...) for any
`objectNameSingular` or a specific `recordId`. Parameters are nullable
and treated as wildcards when null.
- returns events with following shape
```typescript
@Field(() => String)
eventId: string;
@Field()
emittedAt: string;
@Field(() => DatabaseEventAction)
action: DatabaseEventAction;
@Field(() => String)
objectNameSingular: string;
@Field(() => GraphQLJSON)
record: ObjectRecord;
@Field(() => [String], { nullable: true })
updatedFields?: string[];
```
- front provide a componentEffect `<ListenRecordUpdatesEffect />` that
listen for an `objectNameSingular`, a `recordId` and a list of
`listenedFields`. It subscribes to record updates and updates its apollo
cached value for specified `listenedFields`
- subscription is protected with credentials
## Result
Here is an application with `workflowRun`
https://github.com/user-attachments/assets/c964d857-3b54-495f-bf14-587ba26c5a8c
---------
Co-authored-by: prastoin <paul@twenty.com>
Thomas Trompette