houseassassin/twenty
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
Files
fdab89ae02f75255ccd82249bb5f5eeb6f3a3df7
twenty/packages/twenty-front/package.json
T
Charles Bochet 88b9294afd feat(front): persist metadata store cache in IndexedDB instead of localStorage (#21586) 
## Problem

The metadata store cache (object/field metadata, views, page layouts,
command menu items, …) is persisted client-side to power **cache-first
boot**: the app renders instantly from the cache, then
`MinimalMetadataLoadEffect` revalidates per-collection hashes and only
refetches what's stale.

It was persisted to **localStorage**, which Safari/WebKit caps at **~5
MB per origin, counted in UTF-16 (2 bytes/char)** → an effective ceiling
of ~2.5 M characters. Measured on the seeded demo workspace (33 objects,
612 fields):

| Bucket | Safari quota (UTF-16) |
|---|---|
| `metadataStoreState__*` (26 keys) | **1.9 MB — 37%** |
| Whole origin | **2.47 MB — 48%** |

A workspace ~2.5× the demo's schema blows past 5 MB, and there is **no
`QuotaExceededError` handling** — `setItem` throws and breaks the app.
This is what large-workspace users on Safari have been hitting.

## Fix

Move **only the metadata store** to **IndexedDB** (multi-GB, disk-based
quota), keeping a **fully synchronous read path** so the ~24 consumers
that read these atoms with `useAtomValue` never suspend. The auth/UI
atoms (incl. the synchronously-read `tokenPair`) stay on localStorage —
intentionally scoped.

- **`createIndexedDbBackedJotaiStorage.ts`** — a synchronous Jotai
storage facade backed by an in-memory map, hydrated once from IndexedDB
at boot and written through on every set. IndexedDB access uses the
**`idb-keyval`** library (by the IndexedDB spec co-author, ~0.6 KB)
rather than a hand-rolled wrapper. Each cache gets its own database +
BroadcastChannel (`twenty-front-<cacheName>`), so it's safely reusable.
Swallowed errors are surfaced via `logError`. When IndexedDB is
unavailable the cache stays in memory only (re-fetched each boot).
- **`createAtomFamilyState`** — gains an optional `storage` param;
`metadataStoreState` uses the IndexedDB-backed storage.
- **`index.tsx`** — awaits hydration before mounting so atoms
(`getOnInit: true`) read the persisted snapshot synchronously →
cache-first boot preserved.
- **No migration**: the facade does not touch localStorage at all.
Pre-existing localStorage snapshots are ignored — on first boot of the
new code the IndexedDB cache is empty and atoms re-fetch from the
network (a one-time reconnect). Old `metadataStoreState__*` localStorage
keys are left in place (cleared by the existing logout/reset cleanup);
new writes only ever go to IndexedDB.
- **Cross-tab sync**: the old localStorage atoms synced across tabs for
free via `storage` events; the IndexedDB facade had no equivalent, so a
schema change in one tab left others stale until reload. Restored by
implementing the Jotai storage `subscribe` contract over a
**`BroadcastChannel`** — writes broadcast to other tabs, which update
their in-memory map and notify `atomWithStorage` subscribers so mounted
atoms re-render live. (BroadcastChannel doesn't echo to the sender, so
no feedback loop; guarded for environments without it.)

## Why a synchronous facade (not async `atomWithStorage`)

Consumers use `useAtomValue` directly; an async storage would make the
atoms resolve to Promises and **suspend** every reader. The in-memory
facade keeps reads synchronous (zero ripple on consumers) and confines
the async part to a single bulk read at boot, which the existing
`MinimalMetadataGater` loader already covers.

## Tests

### Automated
- Unit test (10 cases) for the storage facade: synchronous read/write,
IndexedDB write-through, hydration from IndexedDB, `removeItem`/`clear`,
per-cache DB namespacing, persist-failure logging, in-memory-only
behaviour when IndexedDB is unavailable, distinguishing a stored
`undefined` from a missing key, and cross-tab subscriber registration.
- Existing metadata-store tests (`useIsLayoutCustomizationDirty`,
`useDefaultHomePagePath`) still pass.
- `nx typecheck twenty-front` and `nx lint:diff-with-main twenty-front`
clean.

### Manual (local seeded workspace, two tabs, Playwright)
Storage:
- After login the metadata cache lives in **IndexedDB (24 keys, ~945
KB)** and **localStorage drops 48% → 11%** of the Safari quota (the
remainder is `currentUserState` + auth, out of scope).
- Reload boots from the cache (no heavy refetch).

Scenarios:

| Scenario | Result |
|---|---|
| **Sign out** | auth cleared, redirect to sign-in, no leftover
localStorage, no errors |
| **Sign back in** | metadata `up-to-date`, company table renders, token
restored |
| **Add object** (`Gadget`) | write-through to IndexedDB; survives
reload via cache-first hydration |
| **Add view** (`QA Cross Tab View`, TABLE) | persisted to the `views`
collection (`up-to-date`) |
| **Two tabs open** | second tab boots cleanly from the shared IndexedDB
— no lock/crash under concurrent access |
| **Cross-tab live sync** | creating an object in tab A makes it appear
in tab B's open settings object list **without a reload** |

Verified by design (no regression):
- Runtime sign-out (`clearSession`) clears session keys and does a full
`window.location.assign` reload; the metadata-clearing path
(`resetJotaiStore`) is test-only, so there's no
async-`clear()`-vs-sign-in race. Metadata persisting across sign-out is
unchanged from the old localStorage behavior (it's schema, revalidated
by hash on next login).

## Notes / follow-ups (not in this PR)

- **IndexedDB query capabilities** are not used yet: the cache stores
one blob per collection (as it did in localStorage), so this is still a
pure key-value use (`idb-keyval`). If we later want to query individual
metadata records — e.g. fields by `objectMetadataId` via an
index/cursor, or partial hydration — that means record-level storage and
a richer wrapper (`idb` for a thin near-native layer, or **Dexie** for a
full query API + reactive `liveQuery` that could also replace the
BroadcastChannel sync).
- IndexedDB still has a (large) quota and Safari ITP eviction applies to
both stores — the cache-first design already tolerates eviction by
revalidating.
- Complementary "load less" wins remain: the denormalized per-field
`relation` block (~700 chars/field of pure duplication) and persisting
`currentUser.workspaceMembers` (the ~0.5 MB still in localStorage).

<!-- This is an auto-generated description by cubic. -->
<a
href="https://cubic.dev/pr/twentyhq/twenty/pull/21586?utm_source=github"
target="_blank" rel="noopener noreferrer"
data-no-image-dialog="true"><picture><source
media="(prefers-color-scheme: dark)"
srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source
media="(prefers-color-scheme: light)"
srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img
alt="Review in cubic"
src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a>
<!-- End of auto-generated description by cubic. -->
2026-06-15 13:25:53 +00:00

207 lines
6.6 KiB
JSON
Raw Blame History

{
"name": "twenty-front",
"private": true,
"type": "module",
"scripts": {
"build": "NODE_ENV=production NODE_OPTIONS=--max-old-space-size=8192 npx vite build",
"build:sourcemaps": "NODE_ENV=production VITE_BUILD_SOURCEMAP=true NODE_OPTIONS=--max-old-space-size=8192 npx vite build",
"start:prod": "NODE_ENV=production npx serve -s build",
"tsup": "npx tsup"
},
"engines": {
"node": "^24.5.0",
"npm": "please-use-yarn",
"yarn": "^4.0.2"
},
"browserslist": {
"production": [
">0.2%",
"not dead",
"not op_mini all"
],
"development": [
"last 1 chrome version",
"last 1 firefox version",
"last 1 safari version"
]
},
"msw": {
"workerDirectory": "public"
},
"dependencies": {
"@apollo/client": "^4.0.0",
"@blocknote/mantine": "^0.51.4",
"@blocknote/react": "^0.51.4",
"@blocknote/xl-docx-exporter": "^0.51.4",
"@blocknote/xl-pdf-exporter": "^0.51.4",
"@calcom/embed-react": "^1.5.3",
"@cyntler/react-doc-viewer": "^1.17.0",
"@dagrejs/dagre": "^1.1.8",
"@dnd-kit/react": "^0.3.2",
"@dr.pogodin/react-helmet": "^3.2.2",
"@floating-ui/react": "^0.24.3",
"@graphiql/plugin-explorer": "^5.1.2",
"@graphiql/react": "^0.37.6",
"@hello-pangea/dnd": "^18.0.1",
"@hookform/resolvers": "^5.2.2",
"@lingui/core": "^5.9.5",
"@lingui/detect-locale": "^5.9.5",
"@lingui/react": "^5.9.5",
"@mantine/core": "^8.3.11",
"@mantine/hooks": "^8.3.11",
"@mantine/utils": "^6.0.22",
"@monaco-editor/react": "^4.7.0",
"@nivo/core": "^0.99.0",
"@nivo/line": "^0.99.0",
"@nivo/pie": "^0.99.0",
"@nivo/radial-bar": "^0.99.0",
"@react-email/components": "^0.5.3",
"@react-pdf/renderer": "^4.1.6",
"@scalar/api-reference-react": "^0.9.42",
"@sentry/react": "^10.51.0",
"@sniptt/guards": "^0.2.0",
"@tiptap/core": "3.4.2",
"@tiptap/extension-bold": "3.4.2",
"@tiptap/extension-document": "3.4.2",
"@tiptap/extension-hard-break": "3.4.2",
"@tiptap/extension-heading": "3.4.2",
"@tiptap/extension-image": "3.4.4",
"@tiptap/extension-italic": "3.4.2",
"@tiptap/extension-link": "3.4.2",
"@tiptap/extension-list": "3.4.2",
"@tiptap/extension-paragraph": "3.4.2",
"@tiptap/extension-strike": "3.4.2",
"@tiptap/extension-text": "3.4.2",
"@tiptap/extension-underline": "3.4.2",
"@tiptap/extensions": "3.4.2",
"@tiptap/react": "3.4.2",
"@types/marked": "^6.0.0",
"@xyflow/react": "^12.4.2",
"ai": "6.0.97",
"apollo-link-rest": "^0.10.0-rc.2",
"apollo-upload-client": "^19.0.0",
"buffer": "^6.0.3",
"country-flag-icons": "^1.5.11",
"cron-parser": "5.1.1",
"d3-shape": "^3.2.0",
"date-fns": "^4.4.0",
"date-fns-tz": "^3.2.0",
"deep-equal": "^2.2.2",
"file-saver": "^2.0.5",
"framer-motion": "^11.18.0",
"fuse.js": "^7.1.0",
"graphiql": "^5.2.3",
"graphql": "16.8.1",
"graphql-sse": "^2.5.4",
"graphql-tag": "^2.12.6",
"idb-keyval": "^6.2.5",
"immer": "^10.1.1",
"input-otp": "^1.4.2",
"jotai": "^2.17.1",
"js-cookie": "^3.0.5",
"json-2-csv": "^5.4.0",
"json-logic-js": "^2.0.5",
"jwt-decode": "^4.0.0",
"libphonenumber-js": "^1.10.26",
"linkify-react": "^4.1.3",
"linkifyjs": "^4.1.3",
"lodash.camelcase": "^4.3.0",
"lodash.groupby": "^4.6.0",
"lodash.isempty": "^4.4.0",
"lodash.omit": "^4.5.0",
"lodash.uniqby": "^4.7.0",
"marked": "^17.0.1",
"microdiff": "^1.3.2",
"papaparse": "^5.4.1",
"pluralize": "^8.0.0",
"qs": "^6.15.2",
"react": "^19.2.0",
"react-data-grid": "7.0.0-beta.59",
"react-datepicker": "^9.1.0",
"react-dom": "^19.2.0",
"react-dropzone": "^14.2.3",
"react-error-boundary": "^4.0.11",
"react-grid-layout": "^1.5.2",
"react-hook-form": "^7.45.1",
"react-hotkeys-hook": "^4.4.4",
"react-imask": "^7.6.0",
"react-intersection-observer": "^9.15.1",
"react-loading-skeleton": "^3.3.1",
"react-markdown": "^10.1.0",
"react-phone-number-input": "patch:react-phone-number-input@npm%3A3.4.5#../../.yarn/patches/react-phone-number-input-npm-3.4.5-dc2895c306.patch",
"react-qr-code": "^2.0.18",
"react-responsive": "^9.0.2",
"react-router-dom": "^6.4.4",
"react-textarea-autosize": "^8.4.1",
"remark-gfm": "^4.0.1",
"rxjs": "^7.2.0",
"temporal-polyfill": "^0.3.0",
"transliteration": "^2.3.5",
"ts-key-enum": "^2.0.12",
"twenty-front-component-renderer": "workspace:*",
"twenty-shared": "workspace:*",
"twenty-ui": "workspace:*",
"twenty-ui-deprecated": "workspace:*",
"type-fest": "4.10.1",
"use-debounce": "^10.0.0",
"uuid": "^11.1.1",
"xlsx-ugnis": "^0.19.3",
"zod": "^4.1.11"
},
"devDependencies": {
"@argos-ci/storybook": "^6.0.6",
"@babel/core": "^7.14.5",
"@babel/preset-typescript": "^7.24.6",
"@graphql-codegen/cli": "^6.3.1",
"@graphql-codegen/typed-document-node": "^6.1.8",
"@graphql-codegen/typescript": "^5.0.10",
"@graphql-codegen/typescript-operations": "^5.1.0",
"@lingui/cli": "^5.9.5",
"@lingui/swc-plugin": "^5.11.0",
"@lingui/vite-plugin": "^5.9.5",
"@playwright/test": "^1.60.0",
"@storybook-community/storybook-addon-cookie": "^5.0.0",
"@storybook/addon-coverage": "^3.0.0",
"@storybook/addon-docs": "^10.3.3",
"@storybook/addon-links": "^10.3.3",
"@storybook/react-vite": "^10.3.3",
"@swc/core": "^1.15.11",
"@swc/jest": "^0.2.39",
"@testing-library/dom": "^10.4.0",
"@testing-library/jest-dom": "^6.6.3",
"@testing-library/react": "^16.3.0",
"@tiptap/suggestion": "3.4.2",
"@types/deep-equal": "^1.0.1",
"@types/file-saver": "^2.0.7",
"@types/jest": "^30.0.0",
"@types/js-cookie": "^3.0.3",
"@types/json-logic-js": "^2",
"@types/react-grid-layout": "^1",
"@types/uuid": "^9.0.2",
"@vitejs/plugin-react-swc": "^4.3.1",
"@vitest/coverage-istanbul": "^4.1.0",
"@wyw-in-js/vite": "^1.1.0",
"dotenv-cli": "^7.4.4",
"esbuild": "^0.28.1",
"jest": "29.7.0",
"jest-environment-jsdom": "30.0.0-beta.3",
"jest-fetch-mock": "^3.0.3",
"monaco-editor": "^0.51.0",
"monaco-editor-auto-typings": "^0.4.5",
"msw": "^2.12.7",
"msw-storybook-addon": "^2.0.6",
"optionator": "^0.9.1",
"oxlint": "^1.51.0",
"oxlint-tsgolint": "^0.16.0",
"playwright": "^1.60.0",
"prettier": "^3.1.1",
"rollup-plugin-node-polyfills": "^0.2.1",
"rollup-plugin-visualizer": "^5.14.0",
"storybook-addon-mock-date": "2.0.0",
"storybook-addon-pseudo-states": "^10.3.3",
"ts-jest": "^29.1.1",
"vite-plugin-svgr": "^4.3.0",
"vite-tsconfig-paths": "^4.2.1"
}
}
Reference in New Issue View Git Blame Copy Permalink