fix(server): auto-index target<X>Id join columns on polymorphic standard objects (#20820)

Closes #20726

## The bug

`timelineActivity` (and the three other polymorphic standard objects —
`attachment`, `noteTarget`, `taskTarget`) store relations as N nullable
`target<X>Id` columns, one per related object. Each one is a join key
queried as `WHERE target<X>Id IN (...) AND deletedAt IS NULL`.

For **built-in** related objects (Person, Company, Opportunity, …), each
`target<X>Id` column gets a BTREE index, declared statically in
`compute-{timelineActivity,attachment,noteTarget,taskTarget}-standard-flat-index-metadata.util.ts`.

For **custom** related objects, the same `target<CustomObject>Id` column
was added — **without an index**. On a `timelineActivity` table at
issue-reporter scale (~21.9M rows, 7.1 GB), this turned record loads
into 20–40s sequential scans and produced `QueryFailedError: Query read
timeout` for end users.

## Diagnosis

The morph/relation field generator
(`generateMorphOrRelationFlatFieldMetadataPair`) already creates a BTREE
index for the field that owns the join column and returns it alongside
the field metadata pair. The two user-driven entry points
(`fromRelationCreateFieldInput…`, `fromMorphRelationCreateFieldInput…`)
correctly destructure and propagate that index.

But the **custom-object creation path** —
`buildDefaultRelationFlatFieldMetadatasForCustomObject`, called when a
user creates a new custom object — destructured only `{
flatFieldMetadatas }` and threw away `indexMetadatas`. So every
`target<CustomObject>Id` column added to the four polymorphic standard
objects has been shipping unindexed since custom morph relations went
in.

## The fix

Three commits.

### 1. `fix(server): index target<CustomObject>Id columns on standard
polymorphic objects`

13 lines across 2 files.

-
`build-default-relation-flat-field-metadatas-for-custom-object.util.ts`
— also destructure `indexMetadatas` from the pair generator and
accumulate them into the returned record (new field
`standardTargetFlatIndexMetadatas`).
-
`from-create-object-input-to-flat-object-metadata-and-flat-field-metadatas-to-create.util.ts`
— append the accumulated indexes to `flatIndexMetadataToCreate`. The
migration pipeline at `object-metadata.service.ts:559–562` already
passes `flatIndexMetadataToCreate` to the migration runner, so no
further wiring is needed.

From now on, creating a custom object also creates the four BTREE
indexes — one per polymorphic standard object's new
`target<CustomObject>Id` column — atomically with the rest of the
migration.

### 2. `feat(server): backfill workspace command for relation join
column indexes`

For existing workspaces whose custom objects were created before the
forward-fix.

`upgrade:2-8:backfill-relation-join-column-indexes` is a
`@RegisteredWorkspaceCommand('2.8.0', 1798100000000)` matching the
pattern from
`2-7-workspace-command-…-drop-connected-account-standard-object.command.ts`.

Per workspace:
1. Load `flatObjectMetadataMaps`, `flatFieldMetadataMaps`,
`flatIndexMaps` from the workspace cache.
2. Resolve the four polymorphic standard object IDs by `nameSingular`
against `DEFAULT_RELATIONS_OBJECTS_STANDARD_IDS`.
3. Collect every field ID that's already covered by any existing index.
4. Filter `flatFieldMetadataMaps` to MORPH_RELATION fields on those four
objects whose `settings.relationType === MANY_TO_ONE` (i.e. owns a join
column) and whose ID isn't in the indexed set.
5. Generate a BTREE `UniversalFlatIndexMetadata` for each via
`generateIndexForFlatFieldMetadata` (same helper the forward-fix uses).
6. Create the indexes in the workspace schema with **CONCURRENTLY** (see
commit 3).
7. Submit the metadata through
`WorkspaceMigrationValidateBuildAndRunService` so it lands in
`indexMetadata` and the cache — same pipeline as a normal metadata
change. The pipeline's own `CREATE INDEX IF NOT EXISTS` no-ops because
the index already exists.

Properties:
- **Idempotent.** Re-running is a no-op once indexes exist.
- **Scoped.** Only the four polymorphic standard objects, only their
MANY_TO_ONE morph relation fields, only those with no covering index.
- **Same code path as the forward-fix.** The backfill produces exactly
the indexes the forward-fix would have created at custom-object creation
time.
- **`--dry-run` supported** via the base
`ActiveOrSuspendedWorkspaceCommandRunner`.

### 3. `feat(server): create index CONCURRENTLY in relation join column
backfill`

Adds an opt-in `concurrently` flag to
`WorkspaceSchemaIndexManagerService.createIndex` (threaded through
`createIndexInWorkspaceSchema`). When `true`, emits `CREATE INDEX
CONCURRENTLY IF NOT EXISTS …`. Defaults to `false` — every existing
caller keeps the current transactional `CREATE INDEX` behavior.

The backfill command opts in. It creates a QueryRunner **without**
`startTransaction()`, issues the CONCURRENTLY indexes one-by-one (each
waits for the previous to finish), then submits the metadata through the
normal migration pipeline whose own `CREATE INDEX IF NOT EXISTS` is now
a no-op.

Why not flip the default for the helper:
- `CREATE INDEX CONCURRENTLY` cannot run inside a transaction — Postgres
errors out. The migration pipeline calls `createIndex` from inside a
transactional schema migration.
- CONCURRENTLY doesn't roll back with the transaction. If the
surrounding migration fails, the index remains and you end up with
metadata/schema drift.
- Failed CONCURRENTLY builds leave an INVALID index behind that needs
manual `DROP`.
- UNIQUE indexes have different failure semantics under CONCURRENTLY
(deferred, not immediate).

So CONCURRENTLY is opt-in, used only where it's the right tool (post-hoc
backfills on populated tables).

## Decisions / tradeoffs

- **Single-column BTREE vs partial `WHERE deletedAt IS NULL` vs
composite.** Twenty's queries always include `deletedAt IS NULL`. A
partial index would be slightly better than a plain BTREE (smaller, no
wasted seeks on soft-deleted rows). This PR ships single-column to match
the existing built-in target index pattern, which already covers >95% of
the available speedup (the 20s→4ms drop the reporter saw comes from
having any index — composite/partial is a second-order effect).
Switching all relation indexes to partial is a separate, broader change.
- **CONCURRENTLY operator caveat.** If a CONCURRENTLY build is
interrupted (kill, connection drop, OOM), Postgres leaves the index as
INVALID. We deliberately don't probe `pg_index` for invalid leftovers on
every create — catalog-table queries can be slow at multi-tenant scale
and the failure mode is rare. Recovery is manual: `DROP INDEX <name>`
and re-run the backfill.
- **Forward-fix is not gated** behind a feature flag. The change is
metadata-pipeline-internal; before, custom-object creation silently
produced a degraded state. After, it produces the correct state. No new
public API, no behavioural change for end users besides the indexes
existing.

## Risk

- Forward-fix: changes only the metadata produced during custom-object
creation. New objects get four extra `FlatIndexMetadata` rows and four
extra `CREATE INDEX` statements during their creation migration. Tables
are empty at that point so the index builds in microseconds.
- Helper change: API-compatible, default behavior unchanged. The new
`concurrently` parameter is optional.
- Backfill: read-only state probe → CONCURRENTLY index creation (no
write blocking) → metadata insert via the normal migration pipeline.
Idempotent. Reverting is `DROP INDEX`.

## Test plan

- [ ] Verify forward-fix: create a custom object, confirm four new BTREE
indexes appear on `timelineActivity`, `attachment`, `noteTarget`,
`taskTarget` for the new `target<CustomObject>Id` columns, and that
`flatIndexMaps` has matching entries.
- [ ] Verify backfill on a workspace that had custom objects created
before the fix: run `--dry-run` first, confirm the expected indexes are
listed; then run for real, confirm the indexes appear in pg (and as
`indisvalid = true` in `pg_index`) and in `flatIndexMaps`. Re-run;
confirm no-op.
- [ ] Verify backfill on a clean workspace: should log "no missing
indexes" and exit.
- [ ] Verify CONCURRENTLY behavior under load: run backfill against a
workspace with active writes on `timelineActivity`; confirm
inserts/updates keep working during index build (no `ShareLock` waits in
`pg_stat_activity`).
- [ ] On the affected reporter-scale workspace, confirm `EXPLAIN
ANALYZE` switches from sequential scan to index scan and timeline
activity timeouts go away.

packages/twenty-server/src/engine/metadata-modules/flat-object-metadata/utils/from-create-object-input-to-flat-object-metadata-and-flat-field-metadatas-to-create.util.ts

@@ -101,6 +101,7 @@ export const fromCreateObjectInputToFlatObjectMetadataAndFlatFieldMetadatasToCre
     const {
       standardSourceFlatFieldMetadatas,
       standardTargetFlatFieldMetadatas,
+      standardTargetFlatIndexMetadatas,
     } = buildDefaultRelationFlatFieldMetadatasForCustomObject({
       existingFlatObjectMetadataMaps,
       sourceFlatObjectMetadata: universalFlatObjectMetadataToCreate,
@@ -120,9 +121,10 @@ export const fromCreateObjectInputToFlatObjectMetadataAndFlatFieldMetadatasToCre
 
     return {
       flatObjectMetadataToCreate: universalFlatObjectMetadataToCreate,
-      flatIndexMetadataToCreate: Object.values(
-        defaultIndexesForCustomObject.indexes,
-      ),
+      flatIndexMetadataToCreate: [
+        ...Object.values(defaultIndexesForCustomObject.indexes),
+        ...standardTargetFlatIndexMetadatas,
+      ],
       relationTargetFlatFieldMetadataToCreate: standardTargetFlatFieldMetadatas,
       flatFieldMetadataToCreateOnObject: objectFlatFieldMetadatas,
     };

packages/twenty-server/src/engine/metadata-modules/flat-object-metadata/utils/from-delete-object-input-to-flat-field-metadatas-to-delete.util.ts

@@ -77,13 +77,20 @@ export const fromDeleteObjectInputToFlatFieldMetadatasToDelete = ({
     },
   );
 
+  const fieldIdsToDelete = new Set(
+    flatFieldMetadatasToDelete.map((flatField) => flatField.id),
+  );
+
   // TODO We should maintain a idsByObjectMetadataId in the flatIndexMaps
   const flatIndexMetadataToDelete = Object.values(
     flatIndexMaps.byUniversalIdentifier,
   ).filter(
     (flatIndex): flatIndex is FlatIndexMetadata =>
       isDefined(flatIndex) &&
-      flatIndex.objectMetadataId === flatObjectMetadataToDelete.id,
+      (flatIndex.objectMetadataId === flatObjectMetadataToDelete.id ||
+        flatIndex.flatIndexFieldMetadatas.some((flatIndexField) =>
+          fieldIdsToDelete.has(flatIndexField.fieldMetadataId),
+        )),
   );
 
   return {

packages/twenty-server/src/engine/metadata-modules/object-metadata/utils/build-default-relation-flat-field-metadatas-for-custom-object.util.ts

@@ -19,6 +19,7 @@ import {
 } from 'src/engine/metadata-modules/object-metadata/object-metadata.exception';
 import { STANDARD_OBJECT_ICONS } from 'src/engine/workspace-manager/workspace-migration/constant/standard-object-icons';
 import { type UniversalFlatFieldMetadata } from 'src/engine/workspace-manager/workspace-migration/universal-flat-entity/types/universal-flat-field-metadata.type';
+import { type UniversalFlatIndexMetadata } from 'src/engine/workspace-manager/workspace-migration/universal-flat-entity/types/universal-flat-index-metadata.type';
 import { type UniversalFlatObjectMetadata } from 'src/engine/workspace-manager/workspace-migration/universal-flat-entity/types/universal-flat-object-metadata.type';
 
 const morphIdByRelationObjectNameSingular = {
@@ -41,11 +42,13 @@ export type BuildDefaultRelationFieldsForCustomObjectArgs = {
 type SourceAndTargetFlatFieldMetadatasRecord = {
   standardSourceFlatFieldMetadatas: UniversalFlatFieldMetadata[];
   standardTargetFlatFieldMetadatas: UniversalFlatFieldMetadata[];
+  standardTargetFlatIndexMetadatas: UniversalFlatIndexMetadata[];
 };
 const EMPTY_SOURCE_AND_TARGET_FLAT_FIELD_METADATAS_RECORD: SourceAndTargetFlatFieldMetadatasRecord =
   {
     standardSourceFlatFieldMetadatas: [],
     standardTargetFlatFieldMetadatas: [],
+    standardTargetFlatIndexMetadatas: [],
   };
 
 export const buildDefaultRelationFlatFieldMetadatasForCustomObject = ({
@@ -107,7 +110,7 @@ export const buildDefaultRelationFlatFieldMetadatasForCustomObject = ({
         const morphId =
           morphIdByRelationObjectNameSingular[objectMetadataNameSingular];
 
-        const { flatFieldMetadatas } =
+        const { flatFieldMetadatas, indexMetadatas } =
           generateMorphOrRelationFlatFieldMetadataPair({
             sourceFlatObjectMetadata,
             targetFlatObjectMetadata,
@@ -144,6 +147,10 @@ export const buildDefaultRelationFlatFieldMetadatasForCustomObject = ({
             ...sourceAndTargetFlatFieldMetadatasRecord.standardTargetFlatFieldMetadatas,
             flatFieldMetadatas[1],
           ],
+          standardTargetFlatIndexMetadatas: [
+            ...sourceAndTargetFlatFieldMetadatasRecord.standardTargetFlatIndexMetadatas,
+            ...indexMetadatas,
+          ],
         };
       },
       EMPTY_SOURCE_AND_TARGET_FLAT_FIELD_METADATAS_RECORD,