feat: generic web search driver abstraction with Exa support and billing (#19341)

## Summary

- Introduces a pluggable `WebSearchDriver` abstraction (interface,
factory, service, module) so web search is no longer tied to native
provider tools (Anthropic/OpenAI)
- **Exa** is the first driver implementation with support for
category-filtered search (company, people, news, research paper, etc.) —
particularly useful for CRM workflows
- Per-query billing for both Exa ($0.007/query) and native provider
surcharges ($0.01/query for Anthropic/OpenAI) via the existing
`USAGE_RECORDED` pipeline
- New config variables: `WEB_SEARCH_DRIVER` (EXA/DISABLED),
`EXA_API_KEY`, `WEB_SEARCH_PREFER_NATIVE` (default false — prefers Exa
over native when both available)
- `WEB_SEARCH` operation type added for usage tracking and Stripe
metering

### Architecture

```
WebSearchDriver (interface)
├── ExaDriver          — Exa neural search with category support
└── DisabledDriver     — throws when search is disabled

WebSearchDriverFactory (extends DriverFactoryBase)
└── creates driver based on WEB_SEARCH_DRIVER config

WebSearchService (facade)
├── search(query, options?, billingContext?)
├── isEnabled()
└── emits USAGE_RECORDED events per query

WebSearchTool (Tool implementation)
└── registered in ActionToolProvider, available via tool catalog
```

### Native search billing gap fixed

Anthropic and OpenAI both charge $0.01/search on top of token costs. The
token costs were already billed, but the per-call surcharge was not.
Added `countNativeWebSearchCallsFromSteps` utility +
`billNativeWebSearchUsage` to `AiBillingService`, wired into both chat
and workflow agent paths.

## Test plan

- [ ] Set `WEB_SEARCH_DRIVER=EXA` + `EXA_API_KEY=...` and verify AI chat
can search the web
- [ ] Verify category parameter works (ask about a specific
company/person)
- [ ] Set `WEB_SEARCH_DRIVER=DISABLED` and verify search tool is not
exposed
- [ ] Set `WEB_SEARCH_PREFER_NATIVE=true` with Anthropic model and
verify native search is used
- [ ] Verify usage events are emitted in ClickHouse for both Exa and
native search paths
- [ ] Verify existing billing tests pass (`npx jest
ai-billing.service.spec.ts`)


Made with [Cursor](https://cursor.com)

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Co-authored-by: claude[bot] <41898282+claude[bot]@users.noreply.github.com>
This commit is contained in:
Félix Malfait
2026-04-06 12:02:46 +02:00
committed by GitHub
parent 8acfacc69c
commit ea572975d8
34 changed files with 575 additions and 22 deletions
@@ -10,6 +10,7 @@ import { AgentAsyncExecutorService } from 'src/engine/metadata-modules/ai/ai-age
import { AgentEntity } from 'src/engine/metadata-modules/ai/ai-agent/entities/agent.entity';
import { AiBillingService } from 'src/engine/metadata-modules/ai/ai-billing/services/ai-billing.service';
import { UsageOperationType } from 'src/engine/core-modules/usage/enums/usage-operation-type.enum';
import { WebSearchService } from 'src/engine/core-modules/web-search/web-search.service';
import { AUTO_SELECT_SMART_MODEL_ID } from 'twenty-shared/constants';
import {
WorkflowStepExecutorException,
@@ -27,6 +28,7 @@ export class AiAgentWorkflowAction implements WorkflowAction {
constructor(
private readonly aiAgentExecutionService: AgentAsyncExecutorService,
private readonly aiBillingService: AiBillingService,
private readonly webSearchService: WebSearchService,
private readonly workflowExecutionContextService: WorkflowExecutionContextService,
@InjectRepository(AgentEntity)
private readonly agentRepository: Repository<AgentEntity>,
@@ -79,7 +81,7 @@ export class AiAgentWorkflowAction implements WorkflowAction {
? executionContext.authContext.userWorkspaceId
: null;
const { result, usage, cacheCreationTokens } =
const { result, usage, cacheCreationTokens, nativeWebSearchCallCount } =
await this.aiAgentExecutionService.executeAgent({
agent,
userPrompt: resolveInput(prompt, context) as string,
@@ -99,6 +101,14 @@ export class AiAgentWorkflowAction implements WorkflowAction {
userWorkspaceId,
);
if (this.webSearchService.shouldUseNativeSearch()) {
this.aiBillingService.billNativeWebSearchUsage(
nativeWebSearchCallCount,
workspaceId,
userWorkspaceId,
);
}
return {
result,
};