Paul Rastoin
6b55a6b51c
## Context
A self-hosted instance upgrading from 2.0.3 to v2.22.0 got stuck with
one workspace failing at `2.16.0_BackfillSearchFieldMetadataCommand`:
```
[QueryFailedError] duplicate key value violates unique constraint "IDX_SEARCH_FIELD_METADATA_OBJECT_FIELD_UNIQUE"
Detail: Key ("objectMetadataId", "fieldMetadataId")=(...) already exists.
```
The failure happened on a retry after a previous partial run, and
reproduced even though the command already recomputes
`flatSearchFieldMetadataMaps` before deriving the create-set (#22884).
## Root cause
The idempotency dedupe compares `(objectMetadataId, fieldMetadataId)`
pairs across two differently-fresh caches:
- The **existing rows** side comes from `flatSearchFieldMetadataMaps`,
which is recomputed from the database (real current ids).
- The **candidate** side resolves ids through `flatObjectMetadataMaps` /
`flatFieldMetadataMaps`, which are **not** invalidated. During a
cross-version upgrade these can be stale, since the migration runner
only invalidates the cache keys a migration touched.
When a stale map resolves a candidate to an outdated id, the dedupe key
doesn't match the existing row and the row is re-emitted. The migration
runner then re-resolves the universal identifiers against fresh maps at
execution time and inserts with the real current ids — exactly the pair
already committed by the earlier partial run (each per-application
migration commits independently) — tripping the unique constraint and
failing the upgrade.
## Fix
Two independent layers, either of which would have prevented the
failure:
1. **Consistent snapshot for the build phase**: the command now
invalidates and recomputes all three maps the dedupe depends on
(`flatObjectMetadataMaps`, `flatFieldMetadataMaps`,
`flatSearchFieldMetadataMaps`), so candidate resolution, existing-row
keys, and the runner all see the same database state.
2. **Id-churn-proof dedupe**: every row this command creates carries a
deterministic universal identifier (`getSearchFieldUniversalIdentifier`,
derived from application + field universal identifiers, no database ids
involved) and `(workspaceId, universalIdentifier)` is unique. The build
util now also skips any candidate whose deterministic universal
identifier already exists, catching leftovers from a previous partial
run even if objects/fields were recreated under new ids in between.
Deliberately **not** done: `ON CONFLICT DO NOTHING` in the create action
handler — it is shared by all runtime `searchFieldMetadata` creation,
and swallowing a conflict would leave the flat-entity cache holding an
entity id that differs from the row actually in the database.
## Test
Added a regression test reproducing the failure shape: an existing row
with the same deterministic universal identifier but stale metadata ids
must not be re-emitted by the backfill.
Note: `ReconcileSearchFieldMetadataCommand` (2.20) has the same
stale-cache exposure; hardening it is left to a follow-up.
<!-- This is an auto-generated description by cubic. -->
<a
href="https://cubic.dev/pr/twentyhq/twenty/pull/23060?utm_source=github"
target="_blank" rel="noopener noreferrer"
data-no-image-dialog="true"><picture><source
media="(prefers-color-scheme: dark)"
srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source
media="(prefers-color-scheme: light)"
srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img
alt="Review in cubic"
src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a>
<!-- End of auto-generated description by cubic. -->
The #1 Open-Source CRM
Website · 
Documentation · 
Roadmap · 
Discord · 
Figma
Why Twenty
Twenty gives technical teams the building blocks for a custom CRM that meets complex business needs and quickly adapts as the business evolves. Twenty is the CRM you build, ship, and version like the rest of your stack.
Learn more about why we built Twenty
Installation
Cloud
The fastest way to get started. Sign up at twenty.com and spin up a workspace in under a minute, with no infrastructure to manage and always up to date.
Build an app
Scaffold a new app with the Twenty CLI:
npx create-twenty-app my-app
Define objects, fields, and views as code:
import { defineObject, FieldType } from 'twenty-sdk/define';
export default defineObject({
nameSingular: 'deal',
namePlural: 'deals',
labelSingular: 'Deal',
labelPlural: 'Deals',
fields: [
{ name: 'name', label: 'Name', type: FieldType.TEXT },
{ name: 'amount', label: 'Amount', type: FieldType.CURRENCY },
{ name: 'closeDate', label: 'Close Date', type: FieldType.DATE_TIME },
],
});
Then ship it to your workspace:
npx twenty app:publish --private
See the app development guide for objects, views, agents, and logic functions.
Self-hosting
Run Twenty on your own infrastructure with Docker Compose, or contribute locally via the local setup guide.
Everything you need
Twenty gives you the building blocks of a modern CRM (objects, views, workflows, and agents) and lets you extend them as code. Here's a tour of what's in the box.
Want to go deeper? Read the 
User Guide for product walkthroughs, or the Documentation for developer reference.
|
|
|
|
|
|
Learn more about version control in doc
Stack
TypeScript
Nx
NestJS, with BullMQ, 
PostgreSQL, 
Redis
React, with Jotai, Linaria and Lingui
Thanks
Thanks to these amazing services that we use and recommend for code review (Greptile), catching bugs (Sentry) and translating (Crowdin).
Join the Community
Star the repo · Discord · 
Feature requests · Releases · 
X · 
LinkedIn · 
Crowdin · 
Contribute