Improve performance on metadata computation (#12785)

In this PR:

## Improve recompute metadata cache performance. We are aiming for
~100ms

Deleting relationMetadata table and FKs pointing on it
Fetching indexMetadata and indexFieldMetadata in a separate query as
typeorm is suboptimizing

## Remove caching lock

As recomputing the metadata cache is lighter, we try to stop preventing
multiple concurrent computations. This also simplifies interfaces

## Introduce self recovery mecanisms to recompute cache automatically if
corrupted

Aka getFreshObjectMetadataMaps

## custom object resolver performance improvement:  1sec to 200ms

Double check queries and indexes used while creating a custom object
Remove the queries to db to use the cached objectMetadataMap

## reduce objectMetadataMaps to 500kb
<img width="222" alt="image"
src="https://github.com/user-attachments/assets/2370dc80-49b6-4b63-8d5e-30c5ebdaa062"
/>

We used to stored 3 fieldMetadataMaps (byId, byName, byJoinColumnName).
While this is great for devXP, this is not great for performances.
Using the same mecanisme as for objectMetadataMap: we only keep byIdMap
and introduce two otherMaps to idByName, idByJoinColumnName to make the
bridge

## Add dataloader on IndexMetadata (aka indexMetadataList in the API)

## Improve field resolver performances too

## Deprecate ClientConfig
This commit is contained in:
Charles Bochet
2025-06-23 21:06:17 +02:00
committed by GitHub
parent 6aee42ab22
commit d5c974054d
145 changed files with 1485 additions and 2245 deletions
@@ -28,6 +28,7 @@ export const graphqlQueryRunnerExceptionHandler = (
case GraphqlQueryRunnerExceptionCode.RELATION_SETTINGS_NOT_FOUND:
case GraphqlQueryRunnerExceptionCode.RELATION_TARGET_OBJECT_METADATA_NOT_FOUND:
case GraphqlQueryRunnerExceptionCode.INVALID_POST_HOOK_PAYLOAD:
case GraphqlQueryRunnerExceptionCode.MISSING_SYSTEM_FIELD:
throw error;
default: {
const _exhaustiveCheck: never = error.code;
@@ -1,5 +1,5 @@
import { QueryFailedError } from 'typeorm';
import { isDefined } from 'twenty-shared/utils';
import { QueryFailedError } from 'typeorm';
import { WorkspaceQueryRunnerOptions } from 'src/engine/api/graphql/workspace-query-runner/interfaces/query-runner-option.interface';
@@ -14,14 +14,14 @@ export const handleDuplicateKeyError = (
if (indexNameMatch) {
const indexName = indexNameMatch[1];
const deletedAtFieldMetadata =
context.objectMetadataItemWithFieldMaps.fieldsByName['deletedAt'];
const deletedAtFieldMetadataId =
context.objectMetadataItemWithFieldMaps.fieldIdByName['deletedAt'];
const affectedColumns =
context.objectMetadataItemWithFieldMaps.indexMetadatas
.find((index) => index.name === indexName)
?.indexFieldMetadatas?.filter(
(field) => field.fieldMetadataId !== deletedAtFieldMetadata?.id,
(field) => field.fieldMetadataId !== deletedAtFieldMetadataId,
)
.map((indexField) => {
const fieldMetadata =