Improve performance on metadata computation (#12785)

In this PR:

## Improve recompute metadata cache performance. We are aiming for
~100ms

Deleting relationMetadata table and FKs pointing on it
Fetching indexMetadata and indexFieldMetadata in a separate query as
typeorm is suboptimizing

## Remove caching lock

As recomputing the metadata cache is lighter, we try to stop preventing
multiple concurrent computations. This also simplifies interfaces

## Introduce self recovery mecanisms to recompute cache automatically if
corrupted

Aka getFreshObjectMetadataMaps

## custom object resolver performance improvement:  1sec to 200ms

Double check queries and indexes used while creating a custom object
Remove the queries to db to use the cached objectMetadataMap

## reduce objectMetadataMaps to 500kb
<img width="222" alt="image"
src="https://github.com/user-attachments/assets/2370dc80-49b6-4b63-8d5e-30c5ebdaa062"
/>

We used to stored 3 fieldMetadataMaps (byId, byName, byJoinColumnName).
While this is great for devXP, this is not great for performances.
Using the same mecanisme as for objectMetadataMap: we only keep byIdMap
and introduce two otherMaps to idByName, idByJoinColumnName to make the
bridge

## Add dataloader on IndexMetadata (aka indexMetadataList in the API)

## Improve field resolver performances too

## Deprecate ClientConfig
This commit is contained in:
Charles Bochet
2025-06-23 21:06:17 +02:00
committed by GitHub
parent 6aee42ab22
commit d5c974054d
145 changed files with 1485 additions and 2245 deletions
@@ -11,19 +11,19 @@ import {
GraphqlQueryRunnerException,
GraphqlQueryRunnerExceptionCode,
} from 'src/engine/api/graphql/graphql-query-runner/errors/graphql-query-runner.exception';
import { buildCursorCompositeFieldWhereCondition } from 'src/engine/api/utils/build-cursor-composite-field-where-condition.utils';
import { computeOperator } from 'src/engine/api/utils/compute-operator.utils';
import { isAscendingOrder } from 'src/engine/api/utils/is-ascending-order.utils';
import { validateAndGetOrderByForScalarField } from 'src/engine/api/utils/validate-and-get-order-by.utils';
import { isCompositeFieldMetadataType } from 'src/engine/metadata-modules/field-metadata/utils/is-composite-field-metadata-type.util';
import { FieldMetadataMap } from 'src/engine/metadata-modules/types/field-metadata-map';
import { buildCursorCompositeFieldWhereCondition } from 'src/engine/api/utils/build-cursor-composite-field-where-condition.utils';
import { ObjectMetadataItemWithFieldMaps } from 'src/engine/metadata-modules/types/object-metadata-item-with-field-maps';
type BuildCursorWhereConditionParams = {
cursorKey: keyof ObjectRecord;
cursorValue:
| ObjectRecordCursorLeafScalarValue
| ObjectRecordCursorLeafCompositeValue;
fieldMetadataMapByName: FieldMetadataMap;
objectMetadataItemWithFieldMaps: ObjectMetadataItemWithFieldMaps;
orderBy: ObjectRecordOrderBy;
isForwardPagination: boolean;
isEqualityCondition?: boolean;
@@ -32,12 +32,15 @@ type BuildCursorWhereConditionParams = {
export const buildCursorWhereCondition = ({
cursorKey,
cursorValue,
fieldMetadataMapByName,
objectMetadataItemWithFieldMaps,
orderBy,
isForwardPagination,
isEqualityCondition = false,
}: BuildCursorWhereConditionParams): Record<string, unknown> => {
const fieldMetadata = fieldMetadataMapByName[cursorKey];
const fieldMetadataId =
objectMetadataItemWithFieldMaps.fieldIdByName[cursorKey];
const fieldMetadata =
objectMetadataItemWithFieldMaps.fieldsById[fieldMetadataId];
if (!fieldMetadata) {
throw new GraphqlQueryRunnerException(