martmull
65155fe50c
Closes twentyhq/core-team-issues#2742 A logic function run is capped by its own `timeoutSeconds` (900s max), so anything that can't finish in one run — a full re-sync, a per-record fan-out, a rate-limited third-party API — had no way to continue. This adds a way to hand that work to the workers. ## What it looks like for an app author ```ts import { enqueueJob } from 'twenty-sdk/logic-function'; await enqueueJob({ logicFunctionUniversalIdentifier: '9f1c3d7e-51b8-4a29-8f0d-7c4e2a6b1d33', payload: { cursor: nextCursor }, retryLimit: 3, priority: 2, delayMs: 60_000, }); ``` The target runs in its own process with its own timeout budget. The classic shape is a function that enqueues *itself* with the next cursor until there is nothing left. ## Changes **twenty-shared** — `EnqueueJobInput` / `EnqueueJobOptions` / `EnqueueJobResult` in `application`. **twenty-server** — new `application-job` module under `core-modules/application`, following the `application-key-value` pattern: - `enqueueJob` mutation on the metadata API, `@AuthApplication`-scoped - the lookup is scoped to `applicationId` + `workspaceId` — that's the authorization boundary, an app can only enqueue its own logic functions, anything else is `LOGIC_FUNCTION_NOT_FOUND` - pushes a `LogicFunctionTriggerJob` onto the existing `logicFunctionQueue`, so the enqueued run goes through the same executor (and the same execution throttling) as every other trigger - the queued run inherits the caller's `userId`/`userWorkspaceId`, so its app access token carries the same permissions as the function that queued it **Job options** are range-checked via `ResolverValidationPipe`, since the values come from application code and an unbounded delay or retry count would let an app pin work in the shared queue: | Option | Default | Range | |--------|---------|-------| | `retryLimit` | `0` | `0`–`10` | | `priority` | queue default | `1`–`10` (lower first) | | `delayMs` | `0` | `0`–7 days | `retryLimit` defaults to `0` rather than inheriting the server-route path's `3`: retries re-run the whole handler, so opting in should be the author's explicit choice. **twenty-sdk** — `enqueueJob` in `twenty-sdk/logic-function`, same shape as `runAgent`/`kv`. **Docs** — new "Background Jobs" page under Extend → Apps → Logic, plus nav and overview entries. **Generated** — regenerated `twenty-front/src/generated-metadata` and `twenty-client-sdk/src/metadata/generated` for the new mutation. ## Tests - `application-job.service.spec.ts` — 5 unit tests: job options mapping, defaults, acting-user propagation, application-scoped lookup, not-found - `enqueue-job.integration-spec.ts` — 5 integration tests: rejects a non-`APPLICATION_ACCESS` token, enqueues a function the app owns, rejects a function owned by another application, rejects an unknown identifier, rejects out-of-range options All green locally, along with `typecheck` for `twenty-server`/`twenty-sdk` and oxlint/oxfmt on the touched files. ## Notes for review - The target is addressed by `universalIdentifier`, matching `runAgent({ agentUniversalIdentifier })` and `ServerRouteDispatchResult.targetLogicFunctionUniversalIdentifier`. Addressing by `name` would be friendlier, but logic function names aren't validated for uniqueness within an app — happy to add it as a convenience if you'd rather. - `enqueueJob` returns as soon as the job is accepted; it can't return the target's result, since the queue driver's `add` returns void. Documented, with a pointer to the KV store for handing results back. --- _Generated by [Claude Code](https://claude.ai/code/session_01QrYvGonS3HMdeuMAVjs5hR)_ <!-- This is an auto-generated description by cubic. --> <a href="https://cubic.dev/pr/twentyhq/twenty/pull/23527?utm_source=github" target="_blank" rel="noopener noreferrer" data-no-image-dialog="true"><picture><source media="(prefers-color-scheme: dark)" srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source media="(prefers-color-scheme: light)" srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img alt="Review in cubic" src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a> <!-- End of auto-generated description by cubic. --> --------- Co-authored-by: github-actions <github-actions@twenty.com>
104 lines
5.2 KiB
TypeScript
104 lines
5.2 KiB
TypeScript
import { Module } from '@nestjs/common';
|
|
import { APP_FILTER } from '@nestjs/core';
|
|
|
|
import { AiAgentMonitorModule } from 'src/engine/metadata-modules/ai/ai-agent-monitor/ai-agent-monitor.module';
|
|
import { AiAgentModule } from 'src/engine/metadata-modules/ai/ai-agent/ai-agent.module';
|
|
import { AiChatModule } from 'src/engine/metadata-modules/ai/ai-chat/ai-chat.module';
|
|
import { AiGenerateTextModule } from 'src/engine/metadata-modules/ai/ai-generate-text/ai-generate-text.module';
|
|
import { AiWorkspaceStatsModule } from 'src/engine/metadata-modules/ai/ai-workspace-stats/ai-workspace-stats.module';
|
|
import { ApplicationConnectionsModule } from 'src/engine/core-modules/application/connection-provider/connections/application-connections.module';
|
|
import { ApplicationJobModule } from 'src/engine/core-modules/application/application-job/application-job.module';
|
|
import { ApplicationKeyValueModule } from 'src/engine/core-modules/application/application-key-value/application-key-value.module';
|
|
import { CalendarChannelMetadataModule } from 'src/engine/metadata-modules/calendar-channel/calendar-channel-metadata.module';
|
|
import { ConnectedAccountMetadataModule } from 'src/engine/metadata-modules/connected-account/connected-account-metadata.module';
|
|
import { CommandMenuItemModule } from 'src/engine/metadata-modules/command-menu-item/command-menu-item.module';
|
|
import { FieldMetadataModule } from 'src/engine/metadata-modules/field-metadata/field-metadata.module';
|
|
import { FlatEntityMapsGraphqlApiExceptionFilter } from 'src/engine/metadata-modules/flat-entity/filters/flat-entity-maps-graphql-api-exception.filter';
|
|
import { PermissionsGraphqlApiExceptionFilter } from 'src/engine/metadata-modules/permissions/utils/permissions-graphql-api-exception.filter';
|
|
import { FrontComponentModule } from 'src/engine/metadata-modules/front-component/front-component.module';
|
|
import { LogicFunctionLayerModule } from 'src/engine/metadata-modules/logic-function-layer/logic-function-layer.module';
|
|
import { LogicFunctionModule } from 'src/engine/metadata-modules/logic-function/logic-function.module';
|
|
import { MessageChannelMetadataModule } from 'src/engine/metadata-modules/message-channel/message-channel-metadata.module';
|
|
import { MessageFolderMetadataModule } from 'src/engine/metadata-modules/message-folder/message-folder-metadata.module';
|
|
import { NavigationMenuItemModule } from 'src/engine/metadata-modules/navigation-menu-item/navigation-menu-item.module';
|
|
import { ObjectMetadataModule } from 'src/engine/metadata-modules/object-metadata/object-metadata.module';
|
|
import { PermissionFlagModule } from 'src/engine/metadata-modules/permission-flag/permission-flag.module';
|
|
import { PermissionsModule } from 'src/engine/metadata-modules/permissions/permissions.module';
|
|
import { MinimalMetadataModule } from 'src/engine/metadata-modules/minimal-metadata/minimal-metadata.module';
|
|
import { ServerRouteTriggerModule } from 'src/engine/core-modules/server-route-trigger/server-route-trigger.module';
|
|
import { RoleModule } from 'src/engine/metadata-modules/role/role.module';
|
|
import { RouteTriggerModule } from 'src/engine/metadata-modules/route-trigger/route-trigger.module';
|
|
import { SearchFieldMetadataModule } from 'src/engine/metadata-modules/search-field-metadata/search-field-metadata.module';
|
|
import { SkillModule } from 'src/engine/metadata-modules/skill/skill.module';
|
|
import { ViewModule } from 'src/engine/metadata-modules/view/view.module';
|
|
import { WebhookModule } from 'src/engine/metadata-modules/webhook/webhook.module';
|
|
import { WorkspaceMetadataVersionModule } from 'src/engine/metadata-modules/workspace-metadata-version/workspace-metadata-version.module';
|
|
|
|
@Module({
|
|
imports: [
|
|
FieldMetadataModule,
|
|
FrontComponentModule,
|
|
ObjectMetadataModule,
|
|
SearchFieldMetadataModule,
|
|
LogicFunctionModule,
|
|
LogicFunctionLayerModule,
|
|
SkillModule,
|
|
CommandMenuItemModule,
|
|
NavigationMenuItemModule,
|
|
AiAgentModule,
|
|
AiAgentMonitorModule,
|
|
AiChatModule,
|
|
AiGenerateTextModule,
|
|
AiWorkspaceStatsModule,
|
|
ApplicationConnectionsModule,
|
|
ApplicationJobModule,
|
|
ApplicationKeyValueModule,
|
|
MinimalMetadataModule,
|
|
ViewModule,
|
|
WorkspaceMetadataVersionModule,
|
|
RoleModule,
|
|
PermissionsModule,
|
|
PermissionFlagModule,
|
|
RouteTriggerModule,
|
|
ServerRouteTriggerModule,
|
|
WebhookModule,
|
|
ConnectedAccountMetadataModule,
|
|
MessageChannelMetadataModule,
|
|
CalendarChannelMetadataModule,
|
|
MessageFolderMetadataModule,
|
|
],
|
|
providers: [
|
|
{
|
|
provide: APP_FILTER,
|
|
useClass: FlatEntityMapsGraphqlApiExceptionFilter,
|
|
},
|
|
{
|
|
provide: APP_FILTER,
|
|
useClass: PermissionsGraphqlApiExceptionFilter,
|
|
},
|
|
],
|
|
exports: [
|
|
FieldMetadataModule,
|
|
FrontComponentModule,
|
|
ObjectMetadataModule,
|
|
SearchFieldMetadataModule,
|
|
LogicFunctionModule,
|
|
SkillModule,
|
|
CommandMenuItemModule,
|
|
NavigationMenuItemModule,
|
|
AiAgentModule,
|
|
AiChatModule,
|
|
MinimalMetadataModule,
|
|
ViewModule,
|
|
RoleModule,
|
|
PermissionsModule,
|
|
PermissionFlagModule,
|
|
WebhookModule,
|
|
ConnectedAccountMetadataModule,
|
|
MessageChannelMetadataModule,
|
|
CalendarChannelMetadataModule,
|
|
MessageFolderMetadataModule,
|
|
],
|
|
})
|
|
export class MetadataEngineModule {}
|