feat(app): infrastructure for pre-installed apps (#19973)

**PR 1 of 2.** Follow-up PR ships the Exa app, sets it as a default pre-installed app, and removes the current `WebSearchTool` / `WebSearchService` / `ExaDriver`. This PR adds the plumbing; no user-visible change yet. ## Summary - Server admins can declare a list of npm app packages to auto-install on every new workspace and backfill onto existing workspaces via CLI. - Server-level secrets (like Exa's API key) live on the `ApplicationRegistration` (one row per server, encrypted) and are injected into logic function execution env at runtime. No more per-workspace storage of global secrets. - A generic `POST /app/billing/charge` endpoint lets app logic functions emit workspace usage events for metered features. Exa uses it in PR 2; future apps (call recorder, etc.) reuse it. - `LogicFunctionToolProvider` tool name prefix changes `logic_function_` → `app_`. Shorter, accurate (they come from installed apps). ## What's in this PR **Logic function executor — server-level variables** - `LogicFunctionExecutorService.getExecutionEnvVariables` now resolves env vars in the order: hardcoded defaults → `ApplicationRegistrationVariable[]` (server-level) → `ApplicationVariable[]` (workspace-level override). The manifest `serverVariables` schema has existed; this closes the loop. **Config** - `PRE_INSTALLED_APPS` — comma-separated list of npm packages. Default: empty. **\`PreInstalledAppsService\`** (new module) - \`onApplicationBootstrap()\` — fetches each package's manifest from the app registry CDN, upserts an \`ApplicationRegistration\`, and seeds declared \`serverVariables\` from matching env vars (e.g. \`EXA_API_KEY\` env → encrypted registration variable). - \`installOnWorkspace(workspaceId)\` — installs all pre-installed apps on a single workspace. Tolerates per-app failures. **Auto-install on new workspace activation** - \`WorkspaceService.prefillCreatedWorkspaceRecords\` invokes \`installOnWorkspace\` after prefilling standard records. Non-blocking on failure. **Backfill CLI command** - \`install-pre-installed-apps\` — iterates active and suspended workspaces, installs pre-installed apps that aren't yet installed. Idempotent. Run after changing \`PRE_INSTALLED_APPS\`. **App billing endpoint** - \`POST /app/billing/charge\`. Authenticated via \`APPLICATION_ACCESS\` token (already injected into logic function execution env as \`DEFAULT_APP_ACCESS_TOKEN\`). Body: \`{ creditsUsedMicro, quantity, unit, operationType, resourceContext? }\`. Emits \`USAGE_RECORDED\` with \`applicationId\` as \`resourceId\`. Generic — reusable by any app. **Tool name prefix** - \`LogicFunctionToolProvider.buildLogicFunctionToolName\` now produces \`app_<name>\` instead of \`logic_function_<name>\`. Only affects tools sourced from logic functions; other tool providers unchanged. ## Stats - 16 files, +501 / −2 - 7 new files (1 command, 1 service × 2, 1 controller, 1 DTO, 2 modules) - Typecheck: 7 pre-existing errors, zero new - Prettier clean ## Behavior deltas - **\`PRE_INSTALLED_APPS\` default = empty**: existing servers see no change on merge. - **\`ApplicationRegistrationVariable\` is now read by the executor**: apps that were using manifest \`serverVariables\` but expecting them to be ignored by the executor will now see them injected. No apps ship with \`isTool: true\` logic functions today, so this is latent — first consumer is Exa in PR 2. - **Tool prefix**: currently no logic-function tools are named \`logic_function_*\` in any production flow. The prefix change affects only future tools emitted by \`LogicFunctionToolProvider\`. ## Risks - **CDN unavailability at startup**: if the app registry CDN is down, \`ensureRegistrationsExist\` logs warnings but doesn't block server start. Installation on new workspaces during this window will find no registrations and log a non-blocking error. Backfill command can retry after CDN recovers. - **Cold-start overhead**: \`ensureRegistrationsExist\` is called once per process on bootstrap. Current configurable default is empty, so zero overhead. When an admin sets \`PRE_INSTALLED_APPS\`, they accept one HTTP call per package at boot. - **Server-level variables flow**: \`ApplicationRegistrationVariable.encryptedValue\` is shared by all workspaces of a server. Appropriate for a single-tenant Exa key. Not appropriate for per-tenant keys — those go in workspace-level \`ApplicationVariable\` and override. ## Test plan - [ ] \`npx nx typecheck twenty-server\` passes (verified: 7 pre-existing unrelated errors, zero new) - [ ] Set \`PRE_INSTALLED_APPS=@twenty-apps/hello-world\` (or any real npm-published app), \`HELLO_WORLD_API_KEY=xxx\`, restart server: \`ApplicationRegistration\` row is upserted, \`ApplicationRegistrationVariable\` for HELLO_WORLD_API_KEY is populated (encrypted). - [ ] Create a new workspace: the app is auto-installed, \`ApplicationEntity\` row created, \`LogicFunctionEntity\` rows created. - [ ] Existing workspace: run \`yarn nx run twenty-server:command install-pre-installed-apps\`: apps install across all workspaces, idempotent on re-run. - [ ] Trigger a logic function that reads \`process.env.HELLO_WORLD_API_KEY\`: value resolves from the server-level \`ApplicationRegistrationVariable\`. - [ ] Log a charge from the handler: \`POST /app/billing/charge\` with \`Authorization: Bearer \$DEFAULT_APP_ACCESS_TOKEN\` body \`{creditsUsedMicro: 1000, quantity: 1, unit: "INVOCATION", operationType: "WEB_SEARCH"}\` → returns \`{success: true}\`, \`USAGE_RECORDED\` event emitted with correct \`resourceId=applicationId\`. - [ ] Tool name generated by \`LogicFunctionToolProvider\` starts with \`app_\`. ## What's NOT in this PR (PR 2 scope) - The Exa app itself (\`packages/twenty-apps/...\` directory) - Removing \`WebSearchTool\`, \`WebSearchService\`, \`ExaDriver\`, \`web-search\` module - Removing \`WEB_SEARCH_DRIVER\` config var - Removing the current \`exa_web_search\` entry in \`ActionToolProvider\` - Chat preload list updated to \`app_exa_web_search\` - Frontend \`getToolDisplayMessage\` branch for \`app_exa_web_search\` - Setting \`PRE_INSTALLED_APPS\` default to include \`@twenty-apps/exa\` --------- Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-23 21:59:00 +02:00
parent ec98130def
commit 4f938aa097
71 changed files with 5546 additions and 648 deletions
@@ -0,0 +1,77 @@
+/* @license Enterprise */
+
+import {
+  Body,
+  Controller,
+  ForbiddenException,
+  HttpCode,
+  HttpStatus,
+  NotFoundException,
+  Post,
+  Req,
+  UseGuards,
+  UsePipes,
+  ValidationPipe,
+} from '@nestjs/common';
+
+import { Request } from 'express';
+import { isDefined } from 'twenty-shared/utils';
+
+import { AppBillingService } from 'src/engine/core-modules/billing/app-billing/app-billing.service';
+import { ChargeDto } from 'src/engine/core-modules/billing/app-billing/dtos/charge.dto';
+import { ThrottlerService } from 'src/engine/core-modules/throttler/throttler.service';
+import { TwentyConfigService } from 'src/engine/core-modules/twenty-config/twenty-config.service';
+import { JwtAuthGuard } from 'src/engine/guards/jwt-auth.guard';
+import { NoPermissionGuard } from 'src/engine/guards/no-permission.guard';
+import { WorkspaceAuthGuard } from 'src/engine/guards/workspace-auth.guard';
+
+// Belt-and-suspenders on top of LogicFunctionExecutorService's execution
+// throttle: application-access tokens are JWTs usable outside the runtime.
+const APP_BILLING_CHARGE_THROTTLE_LIMIT = 1000;
+const APP_BILLING_CHARGE_THROTTLE_TTL_MS = 60_000;
+
+@Controller('app/billing')
+@UseGuards(JwtAuthGuard, WorkspaceAuthGuard, NoPermissionGuard)
+export class AppBillingController {
+  constructor(
+    private readonly appBillingService: AppBillingService,
+    private readonly throttlerService: ThrottlerService,
+    private readonly twentyConfigService: TwentyConfigService,
+  ) {}
+
+  @Post('charge')
+  @HttpCode(HttpStatus.NO_CONTENT)
+  @UsePipes(new ValidationPipe({ whitelist: true, forbidNonWhitelisted: true }))
+  async charge(
+    @Req() request: Request,
+    @Body() charge: ChargeDto,
+  ): Promise<void> {
+    // Billing disabled: no listener consumes the event — fail fast so apps
+    // don't silently discard charges on Community instances.
+    if (!this.twentyConfigService.get('IS_BILLING_ENABLED')) {
+      throw new NotFoundException();
+    }
+
+    // Reject user-access / api-key tokens — only application-access tokens
+    // populate `request.application`.
+    if (!isDefined(request.application) || !isDefined(request.workspace)) {
+      throw new ForbiddenException(
+        'App billing endpoint requires an APPLICATION_ACCESS token.',
+      );
+    }
+
+    await this.throttlerService.tokenBucketThrottleOrThrow(
+      `${request.workspace.id}-${request.application.id}-app-billing-charge`,
+      1,
+      APP_BILLING_CHARGE_THROTTLE_LIMIT,
+      APP_BILLING_CHARGE_THROTTLE_TTL_MS,
+    );
+
+    this.appBillingService.emitChargeEvent({
+      workspaceId: request.workspace.id,
+      applicationId: request.application.id,
+      userWorkspaceId: request.userWorkspaceId,
+      charge,
+    });
+  }
+}
@@ -0,0 +1,25 @@
+/* @license Enterprise */
+
+import { Module } from '@nestjs/common';
+
+import { AppBillingController } from 'src/engine/core-modules/billing/app-billing/app-billing.controller';
+import { AppBillingService } from 'src/engine/core-modules/billing/app-billing/app-billing.service';
+import { AuthModule } from 'src/engine/core-modules/auth/auth.module';
+import { ThrottlerModule } from 'src/engine/core-modules/throttler/throttler.module';
+import { TwentyConfigModule } from 'src/engine/core-modules/twenty-config/twenty-config.module';
+import { WorkspaceCacheStorageModule } from 'src/engine/workspace-cache-storage/workspace-cache-storage.module';
+import { WorkspaceEventEmitterModule } from 'src/engine/workspace-event-emitter/workspace-event-emitter.module';
+
+@Module({
+  imports: [
+    AuthModule,
+    ThrottlerModule,
+    TwentyConfigModule,
+    WorkspaceCacheStorageModule,
+    WorkspaceEventEmitterModule,
+  ],
+  controllers: [AppBillingController],
+  providers: [AppBillingService],
+  exports: [AppBillingService],
+})
+export class AppBillingModule {}
@@ -0,0 +1,63 @@
+/* @license Enterprise */
+
+import { Injectable, Logger } from '@nestjs/common';
+
+import { type ChargeDto } from 'src/engine/core-modules/billing/app-billing/dtos/charge.dto';
+import { USAGE_RECORDED } from 'src/engine/core-modules/usage/constants/usage-recorded.constant';
+import { UsageOperationType } from 'src/engine/core-modules/usage/enums/usage-operation-type.enum';
+import { UsageResourceType } from 'src/engine/core-modules/usage/enums/usage-resource-type.enum';
+import { UsageUnit } from 'src/engine/core-modules/usage/enums/usage-unit.enum';
+import { type UsageEvent } from 'src/engine/core-modules/usage/types/usage-event.type';
+import { WorkspaceEventEmitter } from 'src/engine/workspace-event-emitter/workspace-event-emitter';
+
+// Each operation type has one canonical counting unit — matches how
+// `ai-billing.service.ts` emits native usage events.
+const USAGE_UNIT_BY_OPERATION_TYPE: Record<UsageOperationType, UsageUnit> = {
+  [UsageOperationType.AI_CHAT_TOKEN]: UsageUnit.TOKEN,
+  [UsageOperationType.AI_WORKFLOW_TOKEN]: UsageUnit.TOKEN,
+  [UsageOperationType.WORKFLOW_EXECUTION]: UsageUnit.INVOCATION,
+  [UsageOperationType.CODE_EXECUTION]: UsageUnit.INVOCATION,
+  [UsageOperationType.WEB_SEARCH]: UsageUnit.INVOCATION,
+};
+
+// `workspaceId` + `applicationId` come from the application-access token,
+// never from the body — an app can't charge a different workspace or
+// masquerade as a different app.
+@Injectable()
+export class AppBillingService {
+  private readonly logger = new Logger(AppBillingService.name);
+
+  constructor(private readonly workspaceEventEmitter: WorkspaceEventEmitter) {}
+
+  emitChargeEvent(params: {
+    workspaceId: string;
+    applicationId: string;
+    userWorkspaceId?: string | null;
+    charge: ChargeDto;
+  }): void {
+    const { workspaceId, applicationId, userWorkspaceId, charge } = params;
+    const unit = USAGE_UNIT_BY_OPERATION_TYPE[charge.operationType];
+
+    this.logger.log(
+      `App charge from applicationId=${applicationId} workspaceId=${workspaceId}: ` +
+        `${charge.creditsUsedMicro} micro-credits (${charge.quantity} ${unit}, ${charge.operationType})`,
+    );
+
+    this.workspaceEventEmitter.emitCustomBatchEvent<UsageEvent>(
+      USAGE_RECORDED,
+      [
+        {
+          resourceType: UsageResourceType.APP,
+          operationType: charge.operationType,
+          creditsUsedMicro: charge.creditsUsedMicro,
+          quantity: charge.quantity,
+          unit,
+          resourceId: applicationId,
+          resourceContext: charge.resourceContext ?? null,
+          userWorkspaceId: userWorkspaceId ?? null,
+        },
+      ],
+      workspaceId,
+    );
+  }
+}
@@ -0,0 +1,29 @@
+/* @license Enterprise */
+
+import { IsEnum, IsInt, IsOptional, IsString, Max, Min } from 'class-validator';
+
+import { UsageOperationType } from 'src/engine/core-modules/usage/enums/usage-operation-type.enum';
+
+// $1000 in micro-credits (1 USD = 1_000_000 micro-credits). Bounds a single
+// charge so a compromised or buggy app can't drain credits in one request.
+const MAX_CREDITS_USED_MICRO_PER_CHARGE = 1_000_000_000;
+const MAX_QUANTITY_PER_CHARGE = 10_000;
+
+export class ChargeDto {
+  @IsInt()
+  @Min(0)
+  @Max(MAX_CREDITS_USED_MICRO_PER_CHARGE)
+  creditsUsedMicro!: number;
+
+  @IsInt()
+  @Min(1)
+  @Max(MAX_QUANTITY_PER_CHARGE)
+  quantity!: number;
+
+  @IsEnum(UsageOperationType)
+  operationType!: UsageOperationType;
+
+  @IsOptional()
+  @IsString()
+  resourceContext?: string;
+}