Refactor twenty client sdk provisioning for logic function and front-component (#18544)

## 1. The `twenty-client-sdk` Package (Source of Truth)

The monorepo package at `packages/twenty-client-sdk` ships with:
- A **pre-built metadata client** (static, generated from a fixed
schema)
- A **stub core client** that throws at runtime (`CoreApiClient was not
generated...`)
- Both ESM (`.mjs`) and CJS (`.cjs`) bundles in `dist/`
- A `package.json` with proper `exports` map for
`twenty-client-sdk/core`, `twenty-client-sdk/metadata`, and
`twenty-client-sdk/generate`

## 2. Generation & Upload (Server-Side, at Migration Time)

**When**: `WorkspaceMigrationRunnerService.run()` executes after a
metadata schema change.

**What happens in `SdkClientGenerationService.generateAndStore()`**:
1. Copies the stub `twenty-client-sdk` package from the server's assets
(resolved via `SDK_CLIENT_PACKAGE_DIRNAME` — from
`dist/assets/twenty-client-sdk/` in production, or from `node_modules`
in dev)
2. Filters out `node_modules/` and `src/` during copy — only
`package.json` + `dist/` are kept (like an npm publish)
3. Calls `replaceCoreClient()` which uses `@genql/cli` to introspect the
**application-scoped** GraphQL schema and generates a real
`CoreApiClient`, then compiles it to ESM+CJS and overwrites
`dist/core.mjs` and `dist/core.cjs`
4. Archives the **entire package** (with `package.json` + `dist/`) into
`twenty-client-sdk.zip`
5. Uploads the single archive to S3 under
`FileFolder.GeneratedSdkClient`
6. Sets `isSdkLayerStale = true` on the `ApplicationEntity` in the
database

## 3. Invalidation Signal

The `isSdkLayerStale` boolean column on `ApplicationEntity` is the
invalidation mechanism:
- **Set to `true`** by `generateAndStore()` after uploading a new client
archive
- **Checked** by both logic function drivers before execution — if
`true`, they rebuild their local layer
- **Set back to `false`** by `markSdkLayerFresh()` after the driver has
successfully consumed the new archive

Default is `false` so existing applications without a generated client
aren't affected.

## 4a. Logic Functions — Local Driver

**`ensureSdkLayer()`** is called before every execution:
1. Checks if the local SDK layer directory exists AND `isSdkLayerStale`
is `false` → early return
2. Otherwise, cleans the local layer directory
3. Calls `downloadAndExtractToPackage()` which streams the zip from S3
directly to disk and extracts the full package into
`<tmpdir>/sdk/<workspaceId>-<appId>/node_modules/twenty-client-sdk/`
4. Calls `markSdkLayerFresh()` to set `isSdkLayerStale = false`

**At execution time**, `assembleNodeModules()` symlinks everything from
the deps layer's `node_modules/` **except** `twenty-client-sdk`, which
is symlinked from the SDK layer instead. This ensures the logic
function's `import ... from 'twenty-client-sdk/core'` resolves to the
generated client.

## 4b. Logic Functions — Lambda Driver

**`ensureSdkLayer()`** is called during `build()`:
1. Checks if `isSdkLayerStale` is `false` and an existing Lambda layer
ARN exists → early return
2. Otherwise, deletes all existing layer versions for this SDK layer
name
3. Calls `downloadArchiveBuffer()` to get the raw zip from S3 (no disk
extraction)
4. Calls `reprefixZipEntries()` which streams the zip entries into a
**new zip** with the path prefix
`nodejs/node_modules/twenty-client-sdk/` — this is the Lambda layer
convention path. All done in memory, no disk round-trip
5. Publishes the re-prefixed zip as a new Lambda layer via
`publishLayer()`
6. Calls `markSdkLayerFresh()`

**At function creation**, the Lambda is created with **two layers**:
`[depsLayerArn, sdkLayerArn]`. The SDK layer is listed last so it
overwrites the stub `twenty-client-sdk` from the deps layer (later
layers take precedence in Lambda's `/opt` merge).

## 5. Front Components

Front components are built by `app:build` with `twenty-client-sdk/core`
and `twenty-client-sdk/metadata` as **esbuild externals**. The stored
`.mjs` in S3 has unresolved bare import specifiers like `import {
CoreApiClient } from 'twenty-client-sdk/core'`.

SDK import resolution is split between the **frontend host** (fetching &
caching SDK modules) and the **Web Worker** (rewriting imports):

**Server endpoints**:
- `GET /rest/front-components/:id` —
`FrontComponentService.getBuiltComponentStream()` returns the **raw
`.mjs`** directly from file storage. No bundling, no SDK injection.
- `GET /rest/sdk-client/:applicationId/:moduleName` —
`SdkClientController` reads a single file (e.g. `dist/core.mjs`) from
the generated SDK archive via
`SdkClientGenerationService.readFileFromArchive()` and serves it as
JavaScript.

**Frontend host** (`FrontComponentRenderer` in `twenty-front`):
1. Queries `FindOneFrontComponent` which returns `applicationId`,
`builtComponentChecksum`, `usesSdkClient`, and `applicationTokenPair`
2. If `usesSdkClient` is `true`, renders
`FrontComponentRendererWithSdkClient` which calls the
`useApplicationSdkClient` hook
3. `useApplicationSdkClient({ applicationId, accessToken })` checks the
Jotai atom family cache for existing blob URLs. On cache miss, fetches
both SDK modules from `GET /rest/sdk-client/:applicationId/core` and
`/metadata`, creates **blob URLs** for each, and stores them in the atom
family
4. Once the blob URLs are cached, passes them as `sdkClientUrls`
(already blob URLs, not server URLs) to `SharedFrontComponentRenderer` →
`FrontComponentWorkerEffect` → worker's `render()` call via
`HostToWorkerRenderContext`

**Worker** (`remote-worker.ts` in `twenty-sdk`):
1. Fetches the raw component `.mjs` source as text
2. If `sdkClientUrls` are provided and the source contains SDK import
specifiers (`twenty-client-sdk/core`, `twenty-client-sdk/metadata`),
**rewrites** the bare specifiers to the blob URLs received from the host
(e.g. `'twenty-client-sdk/core'` → `'blob:...'`)
3. Creates a blob URL for the rewritten source and `import()`s it
4. Revokes only the component blob URL after the module is loaded — the
SDK blob URLs are owned and managed by the host's Jotai cache

This approach eliminates server-side esbuild bundling on every request,
caches SDK modules per application in the frontend, and keeps the
worker's job to a simple string rewrite.

## Summary Diagram

```
app:build (SDK)
  └─ twenty-client-sdk stub (metadata=real, core=stub)
       │
       ▼
WorkspaceMigrationRunnerService.run()
  └─ SdkClientGenerationService.generateAndStore()
       ├─ Copy stub package (package.json + dist/)
       ├─ replaceCoreClient() → regenerate core.mjs/core.cjs
       ├─ Zip entire package → upload to S3
       └─ Set isSdkLayerStale = true
              │
     ┌────────┴────────────────────┐
     ▼                             ▼
Logic Functions               Front Components
     │                             │
     ├─ Local Driver               ├─ GET /rest/sdk-client/:appId/core
     │   └─ downloadAndExtract     │    → core.mjs from archive
     │      → symlink into         │
     │        node_modules         ├─ Host (useApplicationSdkClient)
     │                             │    ├─ Fetch SDK modules
     └─ Lambda Driver              │    ├─ Create blob URLs
         └─ downloadArchiveBuffer  │    └─ Cache in Jotai atom family
            → reprefixZipEntries   │
            → publish as Lambda    ├─ GET /rest/front-components/:id
              layer                │    → raw .mjs (no bundling)
                                   │
                                   └─ Worker (browser)
                                        ├─ Fetch component .mjs
                                        ├─ Rewrite imports → blob URLs
                                        └─ import() rewritten source
```

## Next PR
- Estimate perf improvement by implementing a redis caching for front
component client storage ( we don't even cache front comp initially )
- Implem frontent blob invalidation sse event from server

---------

Co-authored-by: Charles Bochet <charlesBochet@users.noreply.github.com>
This commit is contained in:
Paul Rastoin
2026-03-24 19:10:25 +01:00
committed by GitHub
parent 16451ee2ee
commit 4ea2e32366
189 changed files with 12487 additions and 9856 deletions
@@ -4,7 +4,7 @@ import { ScalarsExplorerService } from 'src/engine/api/graphql/services/scalars-
import { WorkspaceResolverBuilderModule } from 'src/engine/api/graphql/workspace-resolver-builder/workspace-resolver-builder.module';
import { WorkspaceSchemaBuilderModule } from 'src/engine/api/graphql/workspace-schema-builder/workspace-schema-builder.module';
import { FeatureFlagModule } from 'src/engine/core-modules/feature-flag/feature-flag.module';
import { MetadataEngineModule } from 'src/engine/metadata-modules/metadata-engine.module';
import { DataSourceModule } from 'src/engine/metadata-modules/data-source/data-source.module';
import { WorkspaceManyOrAllFlatEntityMapsCacheModule } from 'src/engine/metadata-modules/flat-entity/services/workspace-many-or-all-flat-entity-maps-cache.module';
import { WorkspaceCacheStorageModule } from 'src/engine/workspace-cache-storage/workspace-cache-storage.module';
@@ -12,7 +12,7 @@ import { WorkspaceSchemaFactory } from './workspace-schema.factory';
@Module({
imports: [
MetadataEngineModule,
DataSourceModule,
WorkspaceSchemaBuilderModule,
WorkspaceResolverBuilderModule,
WorkspaceCacheStorageModule,
@@ -23,6 +23,10 @@ import { twentyORMGraphqlApiExceptionHandler } from 'src/engine/twenty-orm/utils
import { WorkflowQueryValidationException } from 'src/modules/workflow/common/exceptions/workflow-query-validation.exception';
import { workflowQueryValidationGraphqlApiExceptionHandler } from 'src/modules/workflow/common/utils/workflow-query-validation-graphql-api-exception-handler.util';
export interface QueryFailedErrorWithCode extends QueryFailedError {
code: string;
}
export const workspaceQueryRunnerGraphqlApiExceptionHandler = (
error: Error | QueryFailedError,
) => {
@@ -4,7 +4,6 @@ import { TypeOrmModule } from '@nestjs/typeorm';
import { TelemetryListener } from 'src/engine/api/graphql/workspace-query-runner/listeners/telemetry.listener';
import { WorkspaceQueryHookModule } from 'src/engine/api/graphql/workspace-query-runner/workspace-query-hook/workspace-query-hook.module';
import { AuditModule } from 'src/engine/core-modules/audit/audit.module';
import { AuthModule } from 'src/engine/core-modules/auth/auth.module';
import { FeatureFlagEntity } from 'src/engine/core-modules/feature-flag/feature-flag.entity';
import { FileModule } from 'src/engine/core-modules/file/file.module';
import { RecordPositionModule } from 'src/engine/core-modules/record-position/record-position.module';
@@ -17,7 +16,6 @@ import { EntityEventsToDbListener } from './listeners/entity-events-to-db.listen
@Module({
imports: [
AuthModule,
WorkspaceDataSourceModule,
WorkspaceQueryHookModule,
TypeOrmModule.forFeature([FeatureFlagEntity]),
@@ -0,0 +1,36 @@
import { type MessageDescriptor } from '@lingui/core';
import { assertUnreachable } from 'twenty-shared/utils';
import { STANDARD_ERROR_MESSAGE } from 'src/engine/api/common/common-query-runners/errors/standard-error-message.constant';
import { CustomException } from 'src/utils/custom-exception';
export enum WorkspaceGraphQLSchemaExceptionCode {
QUERY_TYPE_NOT_FOUND = 'QUERY_TYPE_NOT_FOUND',
MUTATION_TYPE_NOT_FOUND = 'MUTATION_TYPE_NOT_FOUND',
}
const getWorkspaceGraphQLSchemaExceptionUserFriendlyMessage = (
code: WorkspaceGraphQLSchemaExceptionCode,
) => {
switch (code) {
case WorkspaceGraphQLSchemaExceptionCode.QUERY_TYPE_NOT_FOUND:
case WorkspaceGraphQLSchemaExceptionCode.MUTATION_TYPE_NOT_FOUND:
return STANDARD_ERROR_MESSAGE;
default:
assertUnreachable(code);
}
};
export class WorkspaceGraphQLSchemaException extends CustomException<WorkspaceGraphQLSchemaExceptionCode> {
constructor(
message: string,
code: WorkspaceGraphQLSchemaExceptionCode,
{ userFriendlyMessage }: { userFriendlyMessage?: MessageDescriptor } = {},
) {
super(message, code, {
userFriendlyMessage:
userFriendlyMessage ??
getWorkspaceGraphQLSchemaExceptionUserFriendlyMessage(code),
});
}
}
@@ -6,20 +6,13 @@ import {
} from 'twenty-shared/types';
import { isDefined } from 'twenty-shared/utils';
import { CompositeFieldMetadataGqlEnumTypeGenerator } from 'src/engine/api/graphql/workspace-schema-builder/graphql-type-generators/enum-types/composite-field-metadata-gql-enum-type.generator';
import { EnumFieldMetadataGqlEnumTypeGenerator } from 'src/engine/api/graphql/workspace-schema-builder/graphql-type-generators/enum-types/enum-field-metadata-gql-enum-type.generator';
import { CompositeFieldMetadataGqlInputTypeGenerator } from 'src/engine/api/graphql/workspace-schema-builder/graphql-type-generators/input-types/composite-field-metadata-gql-input-type.generator';
import { GroupByDateGranularityInputTypeGenerator } from 'src/engine/api/graphql/workspace-schema-builder/graphql-type-generators/input-types/group-by-input/group-by-date-granularity-gql-input-type.generator';
import { ObjectMetadataGqlInputTypeGenerator } from 'src/engine/api/graphql/workspace-schema-builder/graphql-type-generators/input-types/object-metadata-gql-input-type.generator';
import { RelationConnectGqlInputTypeGenerator } from 'src/engine/api/graphql/workspace-schema-builder/graphql-type-generators/input-types/relation-connect-gql-input-type.generator';
import { CompositeFieldMetadataGqlObjectTypeGenerator } from 'src/engine/api/graphql/workspace-schema-builder/graphql-type-generators/object-types/composite-field-metadata-gql-object-type.generator';
import { ConnectionGqlObjectTypeGenerator } from 'src/engine/api/graphql/workspace-schema-builder/graphql-type-generators/object-types/connection-gql-object-type.generator';
import { EdgeGqlObjectTypeGenerator } from 'src/engine/api/graphql/workspace-schema-builder/graphql-type-generators/object-types/edge-gql-object-type.generator';
import { GroupByConnectionGqlObjectTypeGenerator } from 'src/engine/api/graphql/workspace-schema-builder/graphql-type-generators/object-types/group-by-connection-gql-object-type.generator';
import { ObjectMetadataGqlObjectTypeGenerator } from 'src/engine/api/graphql/workspace-schema-builder/graphql-type-generators/object-types/object-metadata-gql-object-type.generator';
import { ObjectMetadataWithRelationsGqlObjectTypeGenerator } from 'src/engine/api/graphql/workspace-schema-builder/graphql-type-generators/object-types/object-metadata-with-relations-gql-object-type.generator';
import { MutationTypeGenerator } from 'src/engine/api/graphql/workspace-schema-builder/graphql-type-generators/root-types/mutation-type.generator';
import { QueryTypeGenerator } from 'src/engine/api/graphql/workspace-schema-builder/graphql-type-generators/root-types/query-type.generator';
import { WorkspaceResolverBuilderService } from 'src/engine/api/graphql/workspace-resolver-builder/workspace-resolver-builder.service';
import {
type TypeGenerators,
instantiateTypeGenerators,
} from 'src/engine/api/graphql/workspace-schema-builder/graphql-type-generators/type-generators';
import { TypeMapperService } from 'src/engine/api/graphql/workspace-schema-builder/services/type-mapper.service';
import { GqlTypesStorage } from 'src/engine/api/graphql/workspace-schema-builder/storages/gql-types.storage';
import { type SchemaGenerationContext } from 'src/engine/api/graphql/workspace-schema-builder/types/schema-generation-context.type';
import { getFlatFieldsFromFlatObjectMetadata } from 'src/engine/api/graphql/workspace-schema-builder/utils/get-flat-fields-for-flat-object-metadata.util';
import { FlatFieldMetadata } from 'src/engine/metadata-modules/flat-field-metadata/types/flat-field-metadata.type';
@@ -30,54 +23,54 @@ export class GqlTypeGenerator {
private readonly logger = new Logger(GqlTypeGenerator.name);
constructor(
private readonly objectMetadataGqlObjectTypeGenerator: ObjectMetadataGqlObjectTypeGenerator,
private readonly objectMetadataGqlInputTypeGenerator: ObjectMetadataGqlInputTypeGenerator,
private readonly compositeFieldMetadataGqlObjectTypeGenerator: CompositeFieldMetadataGqlObjectTypeGenerator,
private readonly enumFieldMetadataGqlEnumTypeGenerator: EnumFieldMetadataGqlEnumTypeGenerator,
private readonly compositeFieldMetadataGqlEnumTypeGenerator: CompositeFieldMetadataGqlEnumTypeGenerator,
private readonly compositeFieldMetadataGqlInputTypeGenerator: CompositeFieldMetadataGqlInputTypeGenerator,
private readonly edgeGqlObjectTypeGenerator: EdgeGqlObjectTypeGenerator,
private readonly connectionGqlObjectTypeGenerator: ConnectionGqlObjectTypeGenerator,
private readonly groupByConnectionGqlObjectTypeGenerator: GroupByConnectionGqlObjectTypeGenerator,
private readonly objectMetadataWithRelationsGqlObjectTypeGenerator: ObjectMetadataWithRelationsGqlObjectTypeGenerator,
private readonly relationConnectGqlInputTypeGenerator: RelationConnectGqlInputTypeGenerator,
private readonly groupByDateGranularityInputTypeGenerator: GroupByDateGranularityInputTypeGenerator,
private readonly queryTypeGenerator: QueryTypeGenerator,
private readonly mutationTypeGenerator: MutationTypeGenerator,
private readonly typeMapperService: TypeMapperService,
private readonly workspaceResolverBuilderService: WorkspaceResolverBuilderService,
) {}
async buildAndStore(context: SchemaGenerationContext) {
async buildAndStore(
context: SchemaGenerationContext,
): Promise<GqlTypesStorage> {
const gqlTypesStorage = new GqlTypesStorage();
const generators = instantiateTypeGenerators(
gqlTypesStorage,
this.typeMapperService,
this.workspaceResolverBuilderService,
);
const compositeTypeCollection = [...compositeTypeDefinitions.values()];
this.buildAndStoreCompositeFieldMetadataGqlTypes(compositeTypeCollection);
this.buildAndStoreDateFieldMetadataGroupByGqlTypes();
this.buildAndStoreObjectMetadataGqlTypes(context);
await this.queryTypeGenerator.buildAndStore(context);
this.mutationTypeGenerator.buildAndStore(context);
this.buildAndStoreCompositeFieldMetadataGqlTypes(
compositeTypeCollection,
generators,
);
generators.groupByDateGranularityInputTypeGenerator.buildAndStore();
this.buildAndStoreObjectMetadataGqlTypes(context, generators);
await generators.queryTypeGenerator.buildAndStore(context);
generators.mutationTypeGenerator.buildAndStore(context);
return gqlTypesStorage;
}
private buildAndStoreCompositeFieldMetadataGqlTypes(
compositeTypes: CompositeType[],
generators: TypeGenerators,
) {
for (const compositeType of compositeTypes) {
this.compositeFieldMetadataGqlEnumTypeGenerator.buildAndStore(
generators.compositeFieldMetadataGqlEnumTypeGenerator.buildAndStore(
compositeType,
);
this.compositeFieldMetadataGqlObjectTypeGenerator.buildAndStore(
generators.compositeFieldMetadataGqlObjectTypeGenerator.buildAndStore(
compositeType,
);
this.compositeFieldMetadataGqlInputTypeGenerator.buildAndStore(
generators.compositeFieldMetadataGqlInputTypeGenerator.buildAndStore(
compositeType,
);
}
}
private buildAndStoreDateFieldMetadataGroupByGqlTypes() {
this.groupByDateGranularityInputTypeGenerator.buildAndStore();
}
private buildAndStoreObjectMetadataGqlTypes(
context: SchemaGenerationContext,
generators: TypeGenerators,
) {
const { flatObjectMetadataMaps, flatFieldMetadataMaps } = context;
const objectMetadataCollection = Object.values(
@@ -96,35 +89,35 @@ export class GqlTypeGenerator {
flatFieldMetadataMaps,
);
this.enumFieldMetadataGqlEnumTypeGenerator.buildAndStore(
generators.enumFieldMetadataGqlEnumTypeGenerator.buildAndStore(
flatObjectMetadata,
fields,
);
this.objectMetadataGqlObjectTypeGenerator.buildAndStore(
generators.objectMetadataGqlObjectTypeGenerator.buildAndStore(
flatObjectMetadata,
fields,
);
this.edgeGqlObjectTypeGenerator.buildAndStore(flatObjectMetadata);
this.connectionGqlObjectTypeGenerator.buildAndStore(
generators.edgeGqlObjectTypeGenerator.buildAndStore(flatObjectMetadata);
generators.connectionGqlObjectTypeGenerator.buildAndStore(
flatObjectMetadata,
fields,
);
this.groupByConnectionGqlObjectTypeGenerator.buildAndStore(
generators.groupByConnectionGqlObjectTypeGenerator.buildAndStore(
flatObjectMetadata,
);
this.relationConnectGqlInputTypeGenerator.buildAndStore(
generators.relationConnectGqlInputTypeGenerator.buildAndStore(
flatObjectMetadata,
fields,
context,
);
this.objectMetadataGqlInputTypeGenerator.buildAndStore(
generators.objectMetadataGqlInputTypeGenerator.buildAndStore(
flatObjectMetadata,
fields,
context,
);
if (this.objectContainsRelationOrMorphField(fields)) {
this.objectMetadataWithRelationsGqlObjectTypeGenerator.buildAndStore(
generators.objectMetadataWithRelationsGqlObjectTypeGenerator.buildAndStore(
flatObjectMetadata,
fields,
context,
@@ -1,6 +1,7 @@
import { Injectable, Logger } from '@nestjs/common';
import {
GraphQLEnumType,
GraphQLInputFieldConfigMap,
GraphQLInputObjectType,
isObjectType,
@@ -71,7 +72,7 @@ export class CompositeFieldMetadataCreateGqlInputTypeGenerator {
const typeOptions = computeCompositeFieldTypeOptions(property);
const type = isEnumFieldMetadataType(property.type)
? this.gqlTypesStorage.getGqlTypeByKey(key)
? this.gqlTypesStorage.getGqlTypeByKey<GraphQLEnumType>(key)
: this.typeMapperService.mapToPreBuiltGraphQLInputType({
fieldMetadataType: property.type,
typeOptions,
@@ -1,6 +1,7 @@
import { Injectable, Logger } from '@nestjs/common';
import {
GraphQLEnumType,
GraphQLInputFieldConfigMap,
GraphQLInputObjectType,
isObjectType,
@@ -72,7 +73,7 @@ export class CompositeFieldMetadataUpdateGqlInputTypeGenerator {
const typeOptions = computeCompositeFieldTypeOptions(property);
const type = isEnumFieldMetadataType(property.type)
? this.gqlTypesStorage.getGqlTypeByKey(key)
? this.gqlTypesStorage.getGqlTypeByKey<GraphQLEnumType>(key)
: this.typeMapperService.mapToPreBuiltGraphQLInputType({
fieldMetadataType: property.type,
typeOptions,
@@ -1,3 +1,4 @@
import { WorkspaceResolverBuilderService } from 'src/engine/api/graphql/workspace-resolver-builder/workspace-resolver-builder.service';
import { ArgsTypeGenerator } from 'src/engine/api/graphql/workspace-schema-builder/graphql-type-generators/args-type/args-type.generator';
import { CompositeFieldMetadataGqlEnumTypeGenerator } from 'src/engine/api/graphql/workspace-schema-builder/graphql-type-generators/enum-types/composite-field-metadata-gql-enum-type.generator';
import { EnumFieldMetadataGqlEnumTypeGenerator } from 'src/engine/api/graphql/workspace-schema-builder/graphql-type-generators/enum-types/enum-field-metadata-gql-enum-type.generator';
@@ -26,42 +27,152 @@ import { GroupByConnectionGqlObjectTypeGenerator } from 'src/engine/api/graphql/
import { ObjectMetadataGqlObjectTypeGenerator } from 'src/engine/api/graphql/workspace-schema-builder/graphql-type-generators/object-types/object-metadata-gql-object-type.generator';
import { ObjectMetadataWithRelationsGqlObjectTypeGenerator } from 'src/engine/api/graphql/workspace-schema-builder/graphql-type-generators/object-types/object-metadata-with-relations-gql-object-type.generator';
import { RelationFieldMetadataGqlObjectTypeGenerator } from 'src/engine/api/graphql/workspace-schema-builder/graphql-type-generators/object-types/relation-field-metadata-gql-object-type.generator';
import { OrphanedTypesGenerator } from 'src/engine/api/graphql/workspace-schema-builder/graphql-type-generators/orphaned-types.generator';
import { MutationTypeGenerator } from 'src/engine/api/graphql/workspace-schema-builder/graphql-type-generators/root-types/mutation-type.generator';
import { QueryTypeGenerator } from 'src/engine/api/graphql/workspace-schema-builder/graphql-type-generators/root-types/query-type.generator';
import { RootTypeGenerator } from 'src/engine/api/graphql/workspace-schema-builder/graphql-type-generators/root-types/root-type.generator';
import { TypeMapperService } from 'src/engine/api/graphql/workspace-schema-builder/services/type-mapper.service';
import { GqlTypesStorage } from 'src/engine/api/graphql/workspace-schema-builder/storages/gql-types.storage';
export const workspaceSchemaBuilderTypeGenerators = [
CompositeFieldMetadataGqlEnumTypeGenerator,
EnumFieldMetadataGqlEnumTypeGenerator,
CompositeFieldMetadataGqlInputTypeGenerator,
CompositeFieldMetadataCreateGqlInputTypeGenerator,
CompositeFieldMetadataUpdateGqlInputTypeGenerator,
CompositeFieldMetadataFilterGqlInputTypeGenerator,
CompositeFieldMetadataOrderByGqlInputTypeGenerator,
CompositeFieldMetadataGroupByGqlInputTypeGenerator,
RelationFieldMetadataGqlInputTypeGenerator,
RelationFieldMetadataGqlObjectTypeGenerator,
ObjectMetadataGqlInputTypeGenerator,
ObjectMetadataCreateGqlInputTypeGenerator,
ObjectMetadataUpdateGqlInputTypeGenerator,
ObjectMetadataFilterGqlInputTypeGenerator,
ObjectMetadataOrderByGqlInputTypeGenerator,
ObjectMetadataGroupByGqlInputTypeGenerator,
ObjectMetadataOrderByWithGroupByGqlInputTypeGenerator,
ObjectMetadataOrderByBaseGenerator,
GroupByDateGranularityInputTypeGenerator,
CompositeFieldMetadataGqlObjectTypeGenerator,
ObjectMetadataGqlObjectTypeGenerator,
RelationConnectGqlInputTypeGenerator,
ConnectionGqlObjectTypeGenerator,
EdgeGqlObjectTypeGenerator,
GroupByConnectionGqlObjectTypeGenerator,
ObjectMetadataWithRelationsGqlObjectTypeGenerator,
AggregationObjectTypeGenerator,
ArgsTypeGenerator,
RootTypeGenerator,
QueryTypeGenerator,
MutationTypeGenerator,
OrphanedTypesGenerator,
];
export const instantiateTypeGenerators = (
gqlTypesStorage: GqlTypesStorage,
typeMapperService: TypeMapperService,
workspaceResolverBuilderService: WorkspaceResolverBuilderService,
) => {
const argsTypeGenerator = new ArgsTypeGenerator(gqlTypesStorage);
const relationFieldMetadataGqlObjectTypeGenerator =
new RelationFieldMetadataGqlObjectTypeGenerator(typeMapperService);
const relationFieldMetadataGqlInputTypeGenerator =
new RelationFieldMetadataGqlInputTypeGenerator(
typeMapperService,
gqlTypesStorage,
);
const compositeFieldMetadataCreateGqlInputTypeGenerator =
new CompositeFieldMetadataCreateGqlInputTypeGenerator(
gqlTypesStorage,
typeMapperService,
);
const compositeFieldMetadataUpdateGqlInputTypeGenerator =
new CompositeFieldMetadataUpdateGqlInputTypeGenerator(
gqlTypesStorage,
typeMapperService,
);
const compositeFieldMetadataFilterGqlInputTypeGenerator =
new CompositeFieldMetadataFilterGqlInputTypeGenerator(
gqlTypesStorage,
typeMapperService,
);
const compositeFieldMetadataOrderByGqlInputTypeGenerator =
new CompositeFieldMetadataOrderByGqlInputTypeGenerator(
gqlTypesStorage,
typeMapperService,
);
const compositeFieldMetadataGroupByGqlInputTypeGenerator =
new CompositeFieldMetadataGroupByGqlInputTypeGenerator(
gqlTypesStorage,
typeMapperService,
);
const objectMetadataOrderByBaseGenerator =
new ObjectMetadataOrderByBaseGenerator(
gqlTypesStorage,
relationFieldMetadataGqlInputTypeGenerator,
typeMapperService,
);
const rootTypeGenerator = new RootTypeGenerator(
gqlTypesStorage,
argsTypeGenerator,
workspaceResolverBuilderService,
);
return {
enumFieldMetadataGqlEnumTypeGenerator:
new EnumFieldMetadataGqlEnumTypeGenerator(gqlTypesStorage),
compositeFieldMetadataGqlEnumTypeGenerator:
new CompositeFieldMetadataGqlEnumTypeGenerator(gqlTypesStorage),
compositeFieldMetadataGqlObjectTypeGenerator:
new CompositeFieldMetadataGqlObjectTypeGenerator(
gqlTypesStorage,
typeMapperService,
),
compositeFieldMetadataGqlInputTypeGenerator:
new CompositeFieldMetadataGqlInputTypeGenerator(
compositeFieldMetadataCreateGqlInputTypeGenerator,
compositeFieldMetadataUpdateGqlInputTypeGenerator,
compositeFieldMetadataFilterGqlInputTypeGenerator,
compositeFieldMetadataOrderByGqlInputTypeGenerator,
compositeFieldMetadataGroupByGqlInputTypeGenerator,
),
objectMetadataGqlObjectTypeGenerator:
new ObjectMetadataGqlObjectTypeGenerator(
relationFieldMetadataGqlObjectTypeGenerator,
gqlTypesStorage,
typeMapperService,
),
objectMetadataGqlInputTypeGenerator:
new ObjectMetadataGqlInputTypeGenerator(
new ObjectMetadataCreateGqlInputTypeGenerator(
gqlTypesStorage,
relationFieldMetadataGqlInputTypeGenerator,
typeMapperService,
),
new ObjectMetadataUpdateGqlInputTypeGenerator(
gqlTypesStorage,
relationFieldMetadataGqlInputTypeGenerator,
typeMapperService,
),
new ObjectMetadataFilterGqlInputTypeGenerator(
gqlTypesStorage,
relationFieldMetadataGqlInputTypeGenerator,
typeMapperService,
),
new ObjectMetadataOrderByGqlInputTypeGenerator(
gqlTypesStorage,
objectMetadataOrderByBaseGenerator,
),
new ObjectMetadataOrderByWithGroupByGqlInputTypeGenerator(
gqlTypesStorage,
relationFieldMetadataGqlInputTypeGenerator,
objectMetadataOrderByBaseGenerator,
typeMapperService,
),
new ObjectMetadataGroupByGqlInputTypeGenerator(
gqlTypesStorage,
relationFieldMetadataGqlInputTypeGenerator,
typeMapperService,
),
),
edgeGqlObjectTypeGenerator: new EdgeGqlObjectTypeGenerator(gqlTypesStorage),
connectionGqlObjectTypeGenerator: new ConnectionGqlObjectTypeGenerator(
new AggregationObjectTypeGenerator(),
gqlTypesStorage,
),
groupByConnectionGqlObjectTypeGenerator:
new GroupByConnectionGqlObjectTypeGenerator(gqlTypesStorage),
objectMetadataWithRelationsGqlObjectTypeGenerator:
new ObjectMetadataWithRelationsGqlObjectTypeGenerator(
argsTypeGenerator,
gqlTypesStorage,
),
relationConnectGqlInputTypeGenerator:
new RelationConnectGqlInputTypeGenerator(
typeMapperService,
gqlTypesStorage,
),
groupByDateGranularityInputTypeGenerator:
new GroupByDateGranularityInputTypeGenerator(gqlTypesStorage),
queryTypeGenerator: new QueryTypeGenerator(
rootTypeGenerator,
gqlTypesStorage,
),
mutationTypeGenerator: new MutationTypeGenerator(
rootTypeGenerator,
gqlTypesStorage,
),
};
};
export type TypeGenerators = ReturnType<typeof instantiateTypeGenerators>;
@@ -1,36 +1,19 @@
import { Injectable, Scope } from '@nestjs/common';
import { type GraphQLNamedType } from 'graphql';
import {
type GraphQLEnumType,
type GraphQLInputObjectType,
type GraphQLObjectType,
} from 'graphql';
// Must be scoped on REQUEST level, because we need to recreate it for each workspaces
// TODO: Implement properly durable by workspace
@Injectable({ scope: Scope.REQUEST, durable: true })
export class GqlTypesStorage {
private readonly gqlTypes = new Map<
string,
GraphQLEnumType | GraphQLInputObjectType | GraphQLObjectType
>();
private readonly gqlTypes = new Map<string, GraphQLNamedType>();
addGqlType(
key: string,
type: GraphQLEnumType | GraphQLInputObjectType | GraphQLObjectType,
) {
addGqlType(key: string, type: GraphQLNamedType) {
this.gqlTypes.set(key, type);
}
getGqlTypeByKey(
getGqlTypeByKey<T extends GraphQLNamedType = GraphQLNamedType>(
key: string,
): GraphQLEnumType | GraphQLInputObjectType | GraphQLObjectType | undefined {
return this.gqlTypes.get(key);
): T | undefined {
return this.gqlTypes.get(key) as T | undefined;
}
getAllGqlTypesExcept(
keysToExclude: string[],
): (GraphQLEnumType | GraphQLInputObjectType | GraphQLObjectType)[] {
getAllGqlTypesExcept(keysToExclude: string[]): GraphQLNamedType[] {
return Array.from(this.gqlTypes.entries())
.filter(([key]) => !keysToExclude.includes(key))
.map(([, value]) => value);
@@ -1,34 +1,53 @@
import { Injectable } from '@nestjs/common';
import { GraphQLSchema } from 'graphql';
import { GraphQLObjectType, GraphQLSchema } from 'graphql';
import { isDefined } from 'twenty-shared/utils';
import { GqlOperation } from 'src/engine/api/graphql/workspace-schema-builder/enums/gql-operation.enum';
import {
WorkspaceGraphQLSchemaException,
WorkspaceGraphQLSchemaExceptionCode,
} from 'src/engine/api/graphql/workspace-schema-builder/exceptions/workspace-graphql-schema.exception';
import { GqlTypeGenerator } from 'src/engine/api/graphql/workspace-schema-builder/graphql-type-generators/gql-type.generator';
import { OrphanedTypesGenerator } from 'src/engine/api/graphql/workspace-schema-builder/graphql-type-generators/orphaned-types.generator';
import { MutationTypeGenerator } from 'src/engine/api/graphql/workspace-schema-builder/graphql-type-generators/root-types/mutation-type.generator';
import { QueryTypeGenerator } from 'src/engine/api/graphql/workspace-schema-builder/graphql-type-generators/root-types/query-type.generator';
import { type SchemaGenerationContext } from 'src/engine/api/graphql/workspace-schema-builder/types/schema-generation-context.type';
@Injectable()
export class WorkspaceGraphQLSchemaGenerator {
constructor(
private readonly gqlTypeGenerator: GqlTypeGenerator,
private readonly queryTypeGenerator: QueryTypeGenerator,
private readonly mutationTypeGenerator: MutationTypeGenerator,
private readonly orphanedTypesGenerator: OrphanedTypesGenerator,
) {}
constructor(private readonly gqlTypeGenerator: GqlTypeGenerator) {}
async generateSchema(
context: SchemaGenerationContext,
): Promise<GraphQLSchema> {
await this.gqlTypeGenerator.buildAndStore(context);
const gqlTypesStorage = await this.gqlTypeGenerator.buildAndStore(context);
// Assemble schema
const schema = new GraphQLSchema({
query: this.queryTypeGenerator.fetchQueryType(),
mutation: this.mutationTypeGenerator.fetchMutationType(),
types: this.orphanedTypesGenerator.fetchOrphanedTypes(),
const queryType = gqlTypesStorage.getGqlTypeByKey<GraphQLObjectType>(
GqlOperation.Query,
);
const mutationType = gqlTypesStorage.getGqlTypeByKey<GraphQLObjectType>(
GqlOperation.Mutation,
);
if (!isDefined(queryType)) {
throw new WorkspaceGraphQLSchemaException(
'Query type not found in GqlTypesStorage',
WorkspaceGraphQLSchemaExceptionCode.QUERY_TYPE_NOT_FOUND,
);
}
if (!isDefined(mutationType)) {
throw new WorkspaceGraphQLSchemaException(
'Mutation type not found in GqlTypesStorage',
WorkspaceGraphQLSchemaExceptionCode.MUTATION_TYPE_NOT_FOUND,
);
}
return new GraphQLSchema({
query: queryType,
mutation: mutationType,
types: gqlTypesStorage.getAllGqlTypesExcept([
GqlOperation.Query,
GqlOperation.Mutation,
]),
});
return schema;
}
}
@@ -2,23 +2,13 @@ import { Module } from '@nestjs/common';
import { WorkspaceResolverBuilderModule } from 'src/engine/api/graphql/workspace-resolver-builder/workspace-resolver-builder.module';
import { GqlTypeGenerator } from 'src/engine/api/graphql/workspace-schema-builder/graphql-type-generators/gql-type.generator';
import { workspaceSchemaBuilderTypeGenerators } from 'src/engine/api/graphql/workspace-schema-builder/graphql-type-generators/type-generators';
import { GqlTypesStorage } from 'src/engine/api/graphql/workspace-schema-builder/storages/gql-types.storage';
import { TypeMapperService } from 'src/engine/api/graphql/workspace-schema-builder/services/type-mapper.service';
import { WorkspaceGraphQLSchemaGenerator } from 'src/engine/api/graphql/workspace-schema-builder/workspace-graphql-schema.factory';
import { FeatureFlagModule } from 'src/engine/core-modules/feature-flag/feature-flag.module';
import { ObjectMetadataModule } from 'src/engine/metadata-modules/object-metadata/object-metadata.module';
import { TypeMapperService } from './services/type-mapper.service';
@Module({
imports: [
ObjectMetadataModule,
WorkspaceResolverBuilderModule,
FeatureFlagModule,
],
imports: [WorkspaceResolverBuilderModule],
providers: [
GqlTypesStorage,
TypeMapperService,
...workspaceSchemaBuilderTypeGenerators,
GqlTypeGenerator,
WorkspaceGraphQLSchemaGenerator,
],