perf: cap to-many relation records per parent and inline chips in table (#22206)

## Problem Record table views that show a to-many relation column (e.g. Workflows with a "Runs" column) get slow and janky to scroll when some records have many related records. Two root causes, found by profiling the page live: 1. **Backend over-fetch + unfairness.** Nested one-to-many relations were loaded with a single flat limit of `QUERY_MAX_RECORDS_FROM_RELATION * parentCount` shared across *all* parents in the page (`WHERE parentColumn IN (ids) LIMIT 60*N`, no per-parent cap). A single hot parent can consume the entire budget — returning thousands of rows for one cell, and potentially starving sibling parents of records they actually have. The `limit * parentCount` shape shows the original intent *was* a per-parent budget; it was just implemented as a global limit. 2. **Frontend DOM explosion.** `ExpandableList` mounts the *entire* child array inline (clipped with `overflow: hidden`) when unfocused, and mounts all children for measurement when focused. A cell with 2,000+ relation chips mounts ~14k DOM nodes — one observed page reached ~55k nodes for 43 rows, producing 100–300 ms main-thread long tasks on every scroll. ## Fix - **Backend:** load one-to-many relations with a true **per-parent** cap via a `LATERAL` join — each parent runs its own indexed, `LIMIT`-ed scan that stops after the per-parent budget. This is `O(perParentLimit × parentCount)` and never reads or sorts a parent's full relation set. The per-parent query is built through the workspace query builder (so it stays schema-qualified and keeps the soft-delete predicate) and wrapped as a `FROM` subquery; read/row-level permissions are enforced when records are hydrated by id, as elsewhere in the relation loader. Many-to-one is unchanged. - **Frontend:** add an opt-in `maxInlineCount` to `ExpandableList` so to-many relation cells mount only a small inline preview; the expand dropdown still renders the full fetched set. Fully backward compatible (no cap → identical behavior). ## Why LATERAL over a window function A windowed `ROW_NUMBER() OVER (PARTITION BY parent) <= limit` is correct and fair too, but a window function **cannot stop early within a partition** — it must read every matching row (and sort it). Measured on skewed data (one parent with ~4k children, on the existing single-column join index, PG16): | Approach | Time | Buffers | Rows read from the hot partition | |---|---|---|---| | Pre-PR (`LIMIT 60×N`) | 1.6 ms | 91 | ~180 total, early-stops, but **unfair** (starves siblings) | | Window (`ROW_NUMBER`) | 3.7 ms | 128 | **all ~4k + sort** | | **LATERAL (`per-parent LIMIT`)** | **0.5 ms** | **57** | **~60, index early-stop** | LATERAL matches the pre-PR read cost while being fair, needs no new index, and scales independently of how large any single relation is. ## Verification - Backend integration test (`nested-relation-per-parent-limit`): a parent with 65 children is capped at 60 while a sibling with 3 keeps all 3 — passes. - `EXPLAIN ANALYZE` on the generated SQL: Index Scan with the `LIMIT` pushed into the per-parent lateral (early-stop). - Frontend unit test for the `ExpandableList` cap. - Manual check on a table cell with 40 related records: exactly 10 chips mount inline (down from 40), no console errors, chips still clickable and the overflow count reflects the true total.
2026-06-27 15:12:43 +02:00
parent 4840233a1c
commit 2a21eb46c0
7 changed files with 406 additions and 23 deletions
@@ -1,7 +1,7 @@
 import { Injectable } from '@nestjs/common';

 import { FieldMetadataType, type ObjectRecord } from 'twenty-shared/types';
-import { isDefined } from 'twenty-shared/utils';
+import { isDefined, isValidUuid } from 'twenty-shared/utils';
 import { type FindOptionsRelations, type ObjectLiteral } from 'typeorm';

 import { computeMorphOrRelationFieldJoinColumnName } from 'src/engine/metadata-modules/field-metadata/utils/compute-morph-or-relation-field-join-column-name.util';
@@ -26,10 +26,14 @@ import {
 } from 'src/engine/metadata-modules/flat-field-metadata/utils/build-field-maps-from-flat-object-metadata.util';
 import { FlatObjectMetadata } from 'src/engine/metadata-modules/flat-object-metadata/types/flat-object-metadata.type';
 import { GlobalWorkspaceDataSource } from 'src/engine/twenty-orm/global-workspace-datasource/global-workspace-datasource';
+import { type WorkspaceRepository } from 'src/engine/twenty-orm/repository/workspace.repository';
 import { type WorkspaceSelectQueryBuilder } from 'src/engine/twenty-orm/repository/workspace-select-query-builder';
 import { type RolePermissionConfig } from 'src/engine/twenty-orm/types/role-permission-config';
 import { isFieldMetadataEntityOfType } from 'src/engine/utils/is-field-metadata-of-type.util';

+const EMPTY_RELATION_SENTINEL_RECORD_ID =
+  '00000000-0000-0000-0000-000000000000';
+
@Injectable()
 export class ProcessNestedRelationsV2Helper {
  constructor() {}
@@ -228,12 +232,15 @@ export class ProcessNestedRelationsV2Helper {
    const { relationResults, relationAggregatedFieldsResult } =
      await this.findRelations({
        referenceQueryBuilder: targetObjectQueryBuilder,
+        targetObjectRepository,
        column:
          relationType === RelationType.ONE_TO_MANY
            ? `"${fieldMetadataTargetRelationColumnName}"`
            : 'id',
        ids: relationIds,
-        limit: limit * parentObjectRecords.length,
+        relationType,
+        perParentLimit: limit,
+        parentRecordsCount: parentObjectRecords.length,
        aggregate,
        sourceFieldName,
        targetObjectNameSingular,
@@ -340,19 +347,25 @@ export class ProcessNestedRelationsV2Helper {

  private async findRelations({
    referenceQueryBuilder,
+    targetObjectRepository,
    column,
    ids,
-    limit,
+    relationType,
+    perParentLimit,
+    parentRecordsCount,
    aggregate,
    sourceFieldName,
    targetObjectNameSingular,
  }: {
    // oxlint-disable-next-line typescript/no-explicit-any
    referenceQueryBuilder: WorkspaceSelectQueryBuilder<any>;
+    targetObjectRepository: WorkspaceRepository<ObjectLiteral>;
    column: string;
    // oxlint-disable-next-line typescript/no-explicit-any
    ids: any[];
-    limit: number;
+    relationType: RelationType;
+    perParentLimit: number;
+    parentRecordsCount: number;
    // oxlint-disable-next-line typescript/no-explicit-any
    aggregate: Record<string, any>;
    sourceFieldName: string;
@@ -401,20 +414,95 @@ export class ProcessNestedRelationsV2Helper {
    const queryBuilderOptions = referenceQueryBuilder.getFindOptions();
    const columnWithoutQuotes = column.replace(/["']/g, '');

-    const result = await referenceQueryBuilder
-      .setFindOptions({
-        ...queryBuilderOptions,
-        select: { ...queryBuilderOptions.select, [columnWithoutQuotes]: true },
-      })
-      .where(`${column} IN (:...ids)`, {
+    const findOptionsWithJoinColumn = {
+      ...queryBuilderOptions,
+      select: { ...queryBuilderOptions.select, [columnWithoutQuotes]: true },
+    };
+
+    if (relationType !== RelationType.ONE_TO_MANY) {
+      const result = await referenceQueryBuilder
+        .setFindOptions(findOptionsWithJoinColumn)
+        .where(`${column} IN (:...ids)`, { ids })
+        .take(perParentLimit * parentRecordsCount)
+        .getMany();
+
+      return { relationResults: result, relationAggregatedFieldsResult };
+    }
+
+    const allowedRelationRecordIds =
+      await this.findRelationRecordIdsLimitedPerParent({
+        targetObjectRepository,
+        targetObjectNameSingular,
+        column,
        ids,
+        perParentLimit,
+      });
+
+    const recordIdsToHydrate =
+      allowedRelationRecordIds.length > 0
+        ? allowedRelationRecordIds
+        : [EMPTY_RELATION_SENTINEL_RECORD_ID];
+
+    const result = await referenceQueryBuilder
+      .setFindOptions(findOptionsWithJoinColumn)
+      .where(`id IN (:...recordIdsToHydrate)`, {
+        recordIdsToHydrate,
      })
-      .take(limit)
      .getMany();

    return { relationResults: result, relationAggregatedFieldsResult };
  }

+  private async findRelationRecordIdsLimitedPerParent({
+    targetObjectRepository,
+    targetObjectNameSingular,
+    column,
+    ids,
+    perParentLimit,
+  }: {
+    targetObjectRepository: WorkspaceRepository<ObjectLiteral>;
+    targetObjectNameSingular: string;
+    column: string;
+    ids: string[];
+    perParentLimit: number;
+  }): Promise<string[]> {
+    const sanitizedIds = ids.filter(isValidUuid);
+
+    if (sanitizedIds.length === 0) {
+      return [];
+    }
+
+    const perParentRecordIdsSql = targetObjectRepository
+      .createQueryBuilder(targetObjectNameSingular)
+      .select('id', 'id')
+      .where(`${column} = "lateralParents"."parentId"`)
+      .limit(perParentLimit)
+      .getQuery();
+
+    const parentValues = sanitizedIds.map((id) => `('${id}'::uuid)`).join(', ');
+
+    const lateralFromSubquery =
+      `(SELECT "lateralRecords"."id" AS "id" ` +
+      `FROM (VALUES ${parentValues}) AS "lateralParents"("parentId") ` +
+      `CROSS JOIN LATERAL (${perParentRecordIdsSql}) AS "lateralRecords")`;
+
+    const limitedRecordsQueryBuilder = targetObjectRepository
+      .createQueryBuilder()
+      .from(lateralFromSubquery, 'limited_relation_records')
+      .select('limited_relation_records.id', 'id');
+
+    limitedRecordsQueryBuilder.expressionMap.aliases =
+      limitedRecordsQueryBuilder.expressionMap.aliases.filter((alias) =>
+        isDefined(alias.subQuery),
+      );
+
+    const limitedRecords = await limitedRecordsQueryBuilder.getRawMany<{
+      id: string;
+    }>();
+
+    return limitedRecords.map((limitedRecord) => limitedRecord.id);
+  }
+
  private assignRelationResults({
    parentRecords,
    parentObjectRecordsAggregatedValues,