feat: generic web search driver abstraction with Exa support and billing (#19341)
## Summary - Introduces a pluggable `WebSearchDriver` abstraction (interface, factory, service, module) so web search is no longer tied to native provider tools (Anthropic/OpenAI) - **Exa** is the first driver implementation with support for category-filtered search (company, people, news, research paper, etc.) — particularly useful for CRM workflows - Per-query billing for both Exa ($0.007/query) and native provider surcharges ($0.01/query for Anthropic/OpenAI) via the existing `USAGE_RECORDED` pipeline - New config variables: `WEB_SEARCH_DRIVER` (EXA/DISABLED), `EXA_API_KEY`, `WEB_SEARCH_PREFER_NATIVE` (default false — prefers Exa over native when both available) - `WEB_SEARCH` operation type added for usage tracking and Stripe metering ### Architecture ``` WebSearchDriver (interface) ├── ExaDriver — Exa neural search with category support └── DisabledDriver — throws when search is disabled WebSearchDriverFactory (extends DriverFactoryBase) └── creates driver based on WEB_SEARCH_DRIVER config WebSearchService (facade) ├── search(query, options?, billingContext?) ├── isEnabled() └── emits USAGE_RECORDED events per query WebSearchTool (Tool implementation) └── registered in ActionToolProvider, available via tool catalog ``` ### Native search billing gap fixed Anthropic and OpenAI both charge $0.01/search on top of token costs. The token costs were already billed, but the per-call surcharge was not. Added `countNativeWebSearchCallsFromSteps` utility + `billNativeWebSearchUsage` to `AiBillingService`, wired into both chat and workflow agent paths. ## Test plan - [ ] Set `WEB_SEARCH_DRIVER=EXA` + `EXA_API_KEY=...` and verify AI chat can search the web - [ ] Verify category parameter works (ask about a specific company/person) - [ ] Set `WEB_SEARCH_DRIVER=DISABLED` and verify search tool is not exposed - [ ] Set `WEB_SEARCH_PREFER_NATIVE=true` with Anthropic model and verify native search is used - [ ] Verify usage events are emitted in ClickHouse for both Exa and native search paths - [ ] Verify existing billing tests pass (`npx jest ai-billing.service.spec.ts`) Made with [Cursor](https://cursor.com) --------- Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com> Co-authored-by: claude[bot] <41898282+claude[bot]@users.noreply.github.com>
This commit is contained in:
+11
-1
@@ -10,6 +10,7 @@ import { AgentAsyncExecutorService } from 'src/engine/metadata-modules/ai/ai-age
|
||||
import { AgentEntity } from 'src/engine/metadata-modules/ai/ai-agent/entities/agent.entity';
|
||||
import { AiBillingService } from 'src/engine/metadata-modules/ai/ai-billing/services/ai-billing.service';
|
||||
import { UsageOperationType } from 'src/engine/core-modules/usage/enums/usage-operation-type.enum';
|
||||
import { WebSearchService } from 'src/engine/core-modules/web-search/web-search.service';
|
||||
import { AUTO_SELECT_SMART_MODEL_ID } from 'twenty-shared/constants';
|
||||
import {
|
||||
WorkflowStepExecutorException,
|
||||
@@ -27,6 +28,7 @@ export class AiAgentWorkflowAction implements WorkflowAction {
|
||||
constructor(
|
||||
private readonly aiAgentExecutionService: AgentAsyncExecutorService,
|
||||
private readonly aiBillingService: AiBillingService,
|
||||
private readonly webSearchService: WebSearchService,
|
||||
private readonly workflowExecutionContextService: WorkflowExecutionContextService,
|
||||
@InjectRepository(AgentEntity)
|
||||
private readonly agentRepository: Repository<AgentEntity>,
|
||||
@@ -79,7 +81,7 @@ export class AiAgentWorkflowAction implements WorkflowAction {
|
||||
? executionContext.authContext.userWorkspaceId
|
||||
: null;
|
||||
|
||||
const { result, usage, cacheCreationTokens } =
|
||||
const { result, usage, cacheCreationTokens, nativeWebSearchCallCount } =
|
||||
await this.aiAgentExecutionService.executeAgent({
|
||||
agent,
|
||||
userPrompt: resolveInput(prompt, context) as string,
|
||||
@@ -99,6 +101,14 @@ export class AiAgentWorkflowAction implements WorkflowAction {
|
||||
userWorkspaceId,
|
||||
);
|
||||
|
||||
if (this.webSearchService.shouldUseNativeSearch()) {
|
||||
this.aiBillingService.billNativeWebSearchUsage(
|
||||
nativeWebSearchCallCount,
|
||||
workspaceId,
|
||||
userWorkspaceId,
|
||||
);
|
||||
}
|
||||
|
||||
return {
|
||||
result,
|
||||
};
|
||||
|
||||
Reference in New Issue
Block a user