Etienne
7475b5f16f
## Summary
Fixes a performance-correctness bug in the createMany upsert path where
the
existing-record lookup built an overly broad WHERE clause, causing full
table
scans and multi-second latency on bulk upserts.
Reported via Sentry: a 100-record create*(upsert: true) request on
_sdWorkspace
completed with HTTP 200 but took ~14s. The candidate-lookup SELECT alone
took
~6.7s because it fetched a huge superset of rows before matching in
memory.
## Root cause
buildWhereConditions created one IN(...) per conflicting column across
the whole
batch, and findExistingRecords OR-ed them together:
WHERE "cbCustomerId" IN ($1..$100) OR "environment" IN ($101..$200)
For a composite unique index (cbCustomerId, environment), this is
semantically
wrong: it OR's the columns instead of matching them as a tuple. Because
environment is low-cardinality (prod/staging/dev/test), the second IN
alone
matched almost the entire table, forcing a Seq Scan and shipping the
whole
result set to the app for in-memory filtering.
## Fix
buildWhereConditions now generates targeted lookup conditions:
- Single-column unique keys collapse into one `column IN
(distinctValues)`
condition (instead of N OR-ed equalities).
- Composite unique keys produce one `(colA = ? AND colB = ?)` condition
per
input record — the columns are ANDed as a tuple, and separate unique
indexes
are still OR-ed together.
- Conditions are deduplicated (robust JSON-based key, no separator
collisions)
to avoid redundant OR branches.
- Values are now typed as string | number | boolean instead of being
implicitly
coerced to string.
## Benchmark
Reproduced the incident with a table mirroring _sdWorkspace:
high-cardinality
cbCustomerId + low-cardinality environment (4 values), composite unique
index on
(cbCustomerId, environment), 100-record upsert batch. EXPLAIN (ANALYZE,
BUFFERS)
on a warm 500k-row / 392 MB table:
Metric | OLD (colA IN OR colB IN) | NEW (targeted) | Improvement
----------------------|--------------------------|----------------|------------
Scan type | Seq Scan (full table) | Index Scan | index vs full scan
Rows returned to app | 500,000 | 100 | ~5,000x fewer
Buffers touched | ~45,455 (~355 MB) | 400 (~3.2 MB) | ~110x fewer
Execution time | 224 ms | 11.9 ms | ~19x faster
Data shipped to app | ~322 MB | ~64 KB | ~5,000x less
The cache-independent facts are the proof: the old query returned the
entire
table for a 100-record batch (the "overly broad record set" from the
report),
while the new query returns exactly the matching rows via the composite
index.
## Test plan
- [x] Unit tests for buildWhereConditions (single-column IN batching,
nested
paths, composite AND tuples, dedup incl. separator-collision safety,
numeric values, mixed single-column + composite OR) — 22 passing across
build-where-conditions, get-matching-record-id, get-value-from-path.
- [x] Upsert integration suites pass (upsert +
composite-unique-index-upsert,
10 tests).
- [x] EXPLAIN ANALYZE benchmark confirms Index Scan and bounded row
counts.
<!-- This is an auto-generated description by cubic. -->
<a
href="https://cubic.dev/pr/twentyhq/twenty/pull/22721?utm_source=github"
target="_blank" rel="noopener noreferrer"
data-no-image-dialog="true"><picture><source
media="(prefers-color-scheme: dark)"
srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source
media="(prefers-color-scheme: light)"
srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img
alt="Review in cubic"
src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a>
<!-- End of auto-generated description by cubic. -->
The #1 Open-Source CRM
Website · 
Documentation · 
Roadmap · 
Discord · 
Figma
Why Twenty
Twenty gives technical teams the building blocks for a custom CRM that meets complex business needs and quickly adapts as the business evolves. Twenty is the CRM you build, ship, and version like the rest of your stack.
Learn more about why we built Twenty
Installation
Cloud
The fastest way to get started. Sign up at twenty.com and spin up a workspace in under a minute, with no infrastructure to manage and always up to date.
Build an app
Scaffold a new app with the Twenty CLI:
npx create-twenty-app my-app
Define objects, fields, and views as code:
import { defineObject, FieldType } from 'twenty-sdk/define';
export default defineObject({
nameSingular: 'deal',
namePlural: 'deals',
labelSingular: 'Deal',
labelPlural: 'Deals',
fields: [
{ name: 'name', label: 'Name', type: FieldType.TEXT },
{ name: 'amount', label: 'Amount', type: FieldType.CURRENCY },
{ name: 'closeDate', label: 'Close Date', type: FieldType.DATE_TIME },
],
});
Then ship it to your workspace:
npx twenty app:publish --private
See the app development guide for objects, views, agents, and logic functions.
Self-hosting
Run Twenty on your own infrastructure with Docker Compose, or contribute locally via the local setup guide.
Everything you need
Twenty gives you the building blocks of a modern CRM (objects, views, workflows, and agents) and lets you extend them as code. Here's a tour of what's in the box.
Want to go deeper? Read the 
User Guide for product walkthroughs, or the Documentation for developer reference.
|
|
|
|
|
|
Learn more about version control in doc
Stack
TypeScript
Nx
NestJS, with BullMQ, 
PostgreSQL, 
Redis
React, with Jotai, Linaria and Lingui
Thanks
Thanks to these amazing services that we use and recommend for code review (Greptile), catching bugs (Sentry) and translating (Crowdin).
Join the Community
Star the repo · Discord · 
Feature requests · Releases · 
X · 
LinkedIn · 
Crowdin · 
Contribute