refactor(agents): split tool resolution into native and action rails (#20331)
## Summary
Splits AI agent tool resolution into two independent rails:
- **Native tools** — capabilities baked into the model SDK
(Anthropic/OpenAI `web_search`, xAI `web`/`x` provider options). Bound
by `NativeToolBinderService`, controlled by per-agent
`modelConfiguration` toggles. Opaque to Twenty — executed on the model
provider's servers.
- **Action tools** — registry-scoped tools from `ToolRegistryService`
(code interpreter, send email, record CRUD, etc.). Permission-gated via
the agent's role. Executed on Twenty's server.
Both rails merge into a single `ToolSet` at call time. When both
surfaces expose a search tool the model picks at runtime — coexistence
is intentional (relevant once Exa returns as an action, see below).
## Notable changes worth calling out
**Contract change: `AgentAsyncExecutorService.executeAgent` no longer
accepts `rolePermissionConfig`.** Workflow agents now scope exclusively
by the agent's own permission-tab role (`unionOf: [agentRoleId]`). The
previous role-merging path (caller role intersected with agent role) is
removed. No agent role → no registry tools (fail-closed by design).
**`NativeToolBinderService` relocated** from
`core-modules/tool-provider/native/` →
`metadata-modules/ai/ai-models/services/`. The binder needs SDK-package
knowledge, which lives in `ai-models`. Old location created a backwards
module dependency.
**`NATIVE_MODEL_TOOLS_BY_SDK_PACKAGE` is exhaustive over
`AiSdkPackage`** (`Record<>`, not `Partial<Record<>>`). Adding a new SDK
without thinking about native tools now fails the build. SDKs without
native tools (Bedrock, Google, Mistral, Azure, OpenAI-compatible) get
explicit `{}` entries.
**Discriminated union `kind: 'sdk-tool' | 'provider-option'`** lets one
registry describe both function tools (Anthropic/OpenAI) and runtime
sources (xAI). Follows the local `tool-provider` convention from #19321.
## Deferred to follow-ups
- **Exa web search is dropped from this PR** (along with its
`WEB_SEARCH_TOOL` permission flag and the Exa-specific gating). Exa
comes back as an **action/app tool** once apps can define permission
flags through the SDK — ongoing work in #20481.
- **xAI native search currently errors.** xAI deprecated its Live Search
API (the `web`/`x` provider-option sources this rail maps to), so xAI
returns `410` when native search is actually exercised. The code path
itself is clear — it's only hit if you test xAI native tools. Fixed
separately alongside the broader xAI model fixes.
## Conscious non-decisions
- **No "twenty-native" category.** `native` is reserved for
model/provider SDK features; everything Twenty-owned is just a
tool/action.
- **Coexistence over precedence.** No rule forcing an action search tool
to override native search (or vice-versa) — when both exist, it's the
user's choice in workflow agents and the model's choice in chat.
---------
Co-authored-by: Félix Malfait <felix.malfait@gmail.com>
Co-authored-by: Félix Malfait <felix@twenty.com>
This commit is contained in:
+147
@@ -0,0 +1,147 @@
|
||||
import { Test, type TestingModule } from '@nestjs/testing';
|
||||
|
||||
import { getRepositoryToken } from '@nestjs/typeorm';
|
||||
|
||||
import { BillingUsageService } from 'src/engine/core-modules/billing/services/billing-usage.service';
|
||||
import { ToolRegistryService } from 'src/engine/core-modules/tool-provider/services/tool-registry.service';
|
||||
import { WorkspaceEntity } from 'src/engine/core-modules/workspace/workspace.entity';
|
||||
import { AgentAsyncExecutorService } from 'src/engine/metadata-modules/ai/ai-agent-execution/services/agent-async-executor.service';
|
||||
import { type AgentEntity } from 'src/engine/metadata-modules/ai/ai-agent/entities/agent.entity';
|
||||
import { AiBillingService } from 'src/engine/metadata-modules/ai/ai-billing/services/ai-billing.service';
|
||||
import { AiModelConfigService } from 'src/engine/metadata-modules/ai/ai-models/services/ai-model-config.service';
|
||||
import { AiModelRegistryService } from 'src/engine/metadata-modules/ai/ai-models/services/ai-model-registry.service';
|
||||
import { NativeToolBinderService } from 'src/engine/metadata-modules/ai/ai-models/services/native-tool-binder.service';
|
||||
import { RoleTargetEntity } from 'src/engine/metadata-modules/role-target/role-target.entity';
|
||||
import { getWorkspaceScopedRepositoryToken } from 'src/engine/twenty-orm/workspace-scoped-repository/get-workspace-scoped-repository-token.util';
|
||||
|
||||
jest.mock('ai', () => ({
|
||||
...jest.requireActual('ai'),
|
||||
generateText: jest.fn().mockResolvedValue({
|
||||
text: '',
|
||||
steps: [],
|
||||
usage: {
|
||||
inputTokens: 0,
|
||||
outputTokens: 0,
|
||||
totalTokens: 0,
|
||||
inputTokenDetails: {
|
||||
noCacheTokens: 0,
|
||||
cacheReadTokens: 0,
|
||||
cacheWriteTokens: 0,
|
||||
},
|
||||
outputTokenDetails: { textTokens: 0, reasoningTokens: 0 },
|
||||
},
|
||||
}),
|
||||
}));
|
||||
|
||||
describe('AgentAsyncExecutorService — workflow agent role-scoped tool resolution', () => {
|
||||
let service: AgentAsyncExecutorService;
|
||||
let toolRegistry: { getToolsByCategories: jest.Mock };
|
||||
let roleTargetRepository: { findOne: jest.Mock };
|
||||
|
||||
const agentId = 'agent-1';
|
||||
const workspaceId = 'workspace-1';
|
||||
const agentRoleId = 'role-1';
|
||||
|
||||
const buildAgent = (): AgentEntity =>
|
||||
({
|
||||
id: agentId,
|
||||
workspaceId,
|
||||
modelId: 'openai/gpt-4.1',
|
||||
prompt: 'test prompt',
|
||||
modelConfiguration: {},
|
||||
}) as AgentEntity;
|
||||
|
||||
beforeEach(async () => {
|
||||
toolRegistry = { getToolsByCategories: jest.fn().mockResolvedValue({}) };
|
||||
roleTargetRepository = { findOne: jest.fn() };
|
||||
|
||||
const module: TestingModule = await Test.createTestingModule({
|
||||
providers: [
|
||||
AgentAsyncExecutorService,
|
||||
{
|
||||
provide: AiModelRegistryService,
|
||||
useValue: {
|
||||
validateModelAvailability: jest.fn(),
|
||||
resolveModelForAgent: jest.fn().mockResolvedValue({
|
||||
modelId: 'openai/gpt-4.1',
|
||||
sdkPackage: '@ai-sdk/openai',
|
||||
model: {},
|
||||
}),
|
||||
},
|
||||
},
|
||||
{
|
||||
provide: AiModelConfigService,
|
||||
useValue: {
|
||||
getReasoningProviderOptions: jest.fn().mockReturnValue({}),
|
||||
},
|
||||
},
|
||||
{ provide: ToolRegistryService, useValue: toolRegistry },
|
||||
{
|
||||
provide: NativeToolBinderService,
|
||||
useValue: {
|
||||
bind: jest.fn().mockReturnValue({ tools: {}, providerOptions: {} }),
|
||||
},
|
||||
},
|
||||
{
|
||||
provide: AiBillingService,
|
||||
useValue: {
|
||||
decrementAndCheckAvailableCredits: jest
|
||||
.fn()
|
||||
.mockResolvedValue({ hasNoMoreAvailableCredits: false }),
|
||||
calculateCost: jest.fn().mockReturnValue(0),
|
||||
emitAiTokenUsageEvent: jest.fn(),
|
||||
billNativeWebSearchUsage: jest.fn(),
|
||||
},
|
||||
},
|
||||
{
|
||||
provide: BillingUsageService,
|
||||
useValue: {
|
||||
hasAvailableCreditsOrThrow: jest.fn().mockResolvedValue(undefined),
|
||||
},
|
||||
},
|
||||
{
|
||||
provide: getWorkspaceScopedRepositoryToken(RoleTargetEntity),
|
||||
useValue: roleTargetRepository,
|
||||
},
|
||||
{
|
||||
provide: getRepositoryToken(WorkspaceEntity),
|
||||
useValue: { findOneBy: jest.fn().mockResolvedValue(null) },
|
||||
},
|
||||
],
|
||||
}).compile();
|
||||
|
||||
service = module.get<AgentAsyncExecutorService>(AgentAsyncExecutorService);
|
||||
});
|
||||
|
||||
it('passes unionOf: [agentRoleId] when the agent has a role assigned', async () => {
|
||||
roleTargetRepository.findOne.mockResolvedValueOnce({ roleId: agentRoleId });
|
||||
|
||||
await service.executeAgent({
|
||||
agent: buildAgent(),
|
||||
userPrompt: 'test',
|
||||
workspaceId,
|
||||
});
|
||||
|
||||
expect(toolRegistry.getToolsByCategories).toHaveBeenCalledTimes(1);
|
||||
expect(toolRegistry.getToolsByCategories).toHaveBeenCalledWith(
|
||||
expect.objectContaining({
|
||||
roleId: agentRoleId,
|
||||
rolePermissionConfig: { unionOf: [agentRoleId] },
|
||||
workspaceId,
|
||||
}),
|
||||
expect.objectContaining({ wrapWithErrorContext: false }),
|
||||
);
|
||||
});
|
||||
|
||||
it('does not resolve registry tools when the agent has no role (fail-closed)', async () => {
|
||||
roleTargetRepository.findOne.mockResolvedValueOnce(null);
|
||||
|
||||
await service.executeAgent({
|
||||
agent: buildAgent(),
|
||||
userPrompt: 'test',
|
||||
workspaceId,
|
||||
});
|
||||
|
||||
expect(toolRegistry.getToolsByCategories).not.toHaveBeenCalled();
|
||||
});
|
||||
});
|
||||
+54
-73
@@ -18,7 +18,7 @@ import { isUserAuthContext } from 'src/engine/core-modules/auth/guards/is-user-a
|
||||
import { type WorkspaceAuthContext } from 'src/engine/core-modules/auth/types/workspace-auth-context.type';
|
||||
import { BillingUsageService } from 'src/engine/core-modules/billing/services/billing-usage.service';
|
||||
import { type ToolProviderContext } from 'src/engine/core-modules/tool-provider/interfaces/tool-provider-context.type';
|
||||
import { NativeToolBinderService } from 'src/engine/core-modules/tool-provider/native/native-tool-binder.service';
|
||||
import { NativeToolBinderService } from 'src/engine/metadata-modules/ai/ai-models/services/native-tool-binder.service';
|
||||
import { ToolRegistryService } from 'src/engine/core-modules/tool-provider/services/tool-registry.service';
|
||||
import { UsageOperationType } from 'src/engine/core-modules/usage/enums/usage-operation-type.enum';
|
||||
import { WorkspaceEntity } from 'src/engine/core-modules/workspace/workspace.entity';
|
||||
@@ -39,6 +39,7 @@ import { mergeLanguageModelUsage } from 'src/engine/metadata-modules/ai/ai-billi
|
||||
import { AI_TELEMETRY_CONFIG } from 'src/engine/metadata-modules/ai/ai-models/constants/ai-telemetry.const';
|
||||
import { AiModelConfigService } from 'src/engine/metadata-modules/ai/ai-models/services/ai-model-config.service';
|
||||
import { AiModelRegistryService } from 'src/engine/metadata-modules/ai/ai-models/services/ai-model-registry.service';
|
||||
import { type NativeModelToolOptions } from 'src/engine/metadata-modules/ai/ai-models/types/native-model-tool-options.type';
|
||||
import {
|
||||
AiException,
|
||||
AiExceptionCode,
|
||||
@@ -83,29 +84,11 @@ export class AgentAsyncExecutorService {
|
||||
private readonly workspaceRepository: Repository<WorkspaceEntity>,
|
||||
) {}
|
||||
|
||||
private extractRoleIds(
|
||||
rolePermissionConfig?: RolePermissionConfig,
|
||||
): string[] {
|
||||
if (!rolePermissionConfig) {
|
||||
return [];
|
||||
}
|
||||
|
||||
if ('intersectionOf' in rolePermissionConfig) {
|
||||
return rolePermissionConfig.intersectionOf;
|
||||
}
|
||||
|
||||
if ('unionOf' in rolePermissionConfig) {
|
||||
return rolePermissionConfig.unionOf;
|
||||
}
|
||||
|
||||
return [];
|
||||
}
|
||||
|
||||
private async getEffectiveRolePermissionConfig(
|
||||
private async getAgentRoleId(
|
||||
agentId: string,
|
||||
workspaceId: string,
|
||||
rolePermissionConfig?: RolePermissionConfig,
|
||||
): Promise<RolePermissionConfig | undefined> {
|
||||
): Promise<string | undefined> {
|
||||
const roleTarget = await this.roleTargetRepository.findOne(workspaceId, {
|
||||
where: {
|
||||
agentId,
|
||||
@@ -113,25 +96,13 @@ export class AgentAsyncExecutorService {
|
||||
select: ['roleId'],
|
||||
});
|
||||
|
||||
const agentRoleId = roleTarget?.roleId;
|
||||
const configRoleIds = this.extractRoleIds(rolePermissionConfig);
|
||||
|
||||
const allRoleIds = agentRoleId
|
||||
? [...new Set([...configRoleIds, agentRoleId])]
|
||||
: configRoleIds;
|
||||
|
||||
if (allRoleIds.length === 0) {
|
||||
return undefined;
|
||||
}
|
||||
|
||||
return { intersectionOf: allRoleIds };
|
||||
return roleTarget?.roleId;
|
||||
}
|
||||
|
||||
async executeAgent({
|
||||
agent,
|
||||
userPrompt,
|
||||
actorContext,
|
||||
rolePermissionConfig,
|
||||
authContext,
|
||||
workspaceId,
|
||||
userWorkspaceId,
|
||||
@@ -140,7 +111,6 @@ export class AgentAsyncExecutorService {
|
||||
agent: AgentEntity | null;
|
||||
userPrompt: string;
|
||||
actorContext?: ActorMetadata;
|
||||
rolePermissionConfig?: RolePermissionConfig;
|
||||
authContext?: WorkspaceAuthContext;
|
||||
workspaceId: string;
|
||||
userWorkspaceId?: string | null;
|
||||
@@ -173,56 +143,67 @@ export class AgentAsyncExecutorService {
|
||||
let providerOptions = {};
|
||||
|
||||
if (agent) {
|
||||
const effectiveRoleConfig = await this.getEffectiveRolePermissionConfig(
|
||||
const agentRoleId = await this.getAgentRoleId(
|
||||
agent.id,
|
||||
agent.workspaceId,
|
||||
rolePermissionConfig,
|
||||
);
|
||||
|
||||
// Workflow context: registry tools come from DATABASE_CRUD and ACTION.
|
||||
// Native model tools are bound separately below.
|
||||
const roleId = this.extractRoleIds(effectiveRoleConfig)[0] ?? '';
|
||||
|
||||
const toolProviderContext: ToolProviderContext = {
|
||||
workspaceId: agent.workspaceId,
|
||||
roleId,
|
||||
rolePermissionConfig: effectiveRoleConfig ?? { unionOf: [] },
|
||||
authContext,
|
||||
actorContext,
|
||||
userId:
|
||||
isDefined(authContext) && isUserAuthContext(authContext)
|
||||
? authContext.user.id
|
||||
: undefined,
|
||||
userWorkspaceId:
|
||||
isDefined(authContext) && isUserAuthContext(authContext)
|
||||
? authContext.userWorkspaceId
|
||||
: undefined,
|
||||
const nativeModelToolOptions: NativeModelToolOptions = {
|
||||
webSearch: agent.modelConfiguration?.webSearch?.enabled === true,
|
||||
twitterSearch:
|
||||
agent.modelConfiguration?.twitterSearch?.enabled === true,
|
||||
};
|
||||
|
||||
const registryTools = await this.toolRegistry.getToolsByCategories(
|
||||
toolProviderContext,
|
||||
{
|
||||
categories: WORKFLOW_AGENT_REGISTRY_TOOL_CATEGORIES,
|
||||
wrapWithErrorContext: false,
|
||||
},
|
||||
);
|
||||
let registryTools: ToolSet = {};
|
||||
|
||||
const nativeTools = this.nativeToolBinder.bind(registeredModel, {
|
||||
webSearchEnabled:
|
||||
agent.modelConfiguration?.webSearch?.enabled === true,
|
||||
});
|
||||
// Workflow agent registry tools are scoped exclusively by the agent
|
||||
// permission-tab role. No role means no registry tools.
|
||||
if (isDefined(agentRoleId)) {
|
||||
const agentRolePermissionConfig: RolePermissionConfig = {
|
||||
unionOf: [agentRoleId],
|
||||
};
|
||||
|
||||
const toolProviderContext: ToolProviderContext = {
|
||||
workspaceId: agent.workspaceId,
|
||||
roleId: agentRoleId,
|
||||
rolePermissionConfig: agentRolePermissionConfig,
|
||||
authContext,
|
||||
actorContext,
|
||||
userId:
|
||||
isDefined(authContext) && isUserAuthContext(authContext)
|
||||
? authContext.user.id
|
||||
: undefined,
|
||||
userWorkspaceId:
|
||||
isDefined(authContext) && isUserAuthContext(authContext)
|
||||
? authContext.userWorkspaceId
|
||||
: undefined,
|
||||
};
|
||||
|
||||
registryTools = await this.toolRegistry.getToolsByCategories(
|
||||
toolProviderContext,
|
||||
{
|
||||
categories: WORKFLOW_AGENT_REGISTRY_TOOL_CATEGORIES,
|
||||
wrapWithErrorContext: false,
|
||||
},
|
||||
);
|
||||
}
|
||||
|
||||
const nativeBinding = this.nativeToolBinder.bind(
|
||||
registeredModel,
|
||||
nativeModelToolOptions,
|
||||
);
|
||||
|
||||
tools = {
|
||||
...registryTools,
|
||||
...nativeTools,
|
||||
...nativeBinding.tools,
|
||||
};
|
||||
|
||||
providerOptions = this.aiModelConfigService.getProviderOptions(
|
||||
registeredModel,
|
||||
agent as unknown as Parameters<
|
||||
typeof this.aiModelConfigService.getProviderOptions
|
||||
>[1],
|
||||
);
|
||||
providerOptions = {
|
||||
...nativeBinding.providerOptions,
|
||||
...this.aiModelConfigService.getReasoningProviderOptions(
|
||||
registeredModel,
|
||||
),
|
||||
};
|
||||
}
|
||||
|
||||
this.logger.log(`Generated ${Object.keys(tools).length} tools for agent`);
|
||||
|
||||
+15
-8
@@ -25,7 +25,7 @@ import { type CodeExecutionStreamEmitter } from 'src/engine/core-modules/tool-pr
|
||||
import { CodeInterpreterService } from 'src/engine/core-modules/code-interpreter/code-interpreter.service';
|
||||
import { WorkspaceDomainsService } from 'src/engine/core-modules/domain/workspace-domains/services/workspace-domains.service';
|
||||
import { ExceptionHandlerService } from 'src/engine/core-modules/exception-handler/exception-handler.service';
|
||||
import { NativeToolBinderService } from 'src/engine/core-modules/tool-provider/native/native-tool-binder.service';
|
||||
import { NativeToolBinderService } from 'src/engine/metadata-modules/ai/ai-models/services/native-tool-binder.service';
|
||||
import { ToolRegistryService } from 'src/engine/core-modules/tool-provider/services/tool-registry.service';
|
||||
import {
|
||||
createExecuteToolTool,
|
||||
@@ -62,6 +62,7 @@ import {
|
||||
import { AI_TELEMETRY_CONFIG } from 'src/engine/metadata-modules/ai/ai-models/constants/ai-telemetry.const';
|
||||
import { AiModelRegistryService } from 'src/engine/metadata-modules/ai/ai-models/services/ai-model-registry.service';
|
||||
import { type AiModelConfig } from 'src/engine/metadata-modules/ai/ai-models/types/ai-model-config.type';
|
||||
import { getNativeModelCapabilities } from 'src/engine/metadata-modules/ai/ai-models/utils/get-native-model-capabilities.util';
|
||||
import { SkillService } from 'src/engine/metadata-modules/skill/skill.service';
|
||||
|
||||
export type ChatExecutionOptions = {
|
||||
@@ -163,8 +164,13 @@ export class ChatExecutionService {
|
||||
registeredModel.modelId,
|
||||
);
|
||||
|
||||
const nativeModelTools = this.nativeToolBinder.bind(registeredModel, {
|
||||
webSearchEnabled: true,
|
||||
// Native and action search may both be bound here; the model picks at runtime.
|
||||
const nativeCapabilities = getNativeModelCapabilities(
|
||||
registeredModel.sdkPackage,
|
||||
);
|
||||
const nativeBinding = this.nativeToolBinder.bind(registeredModel, {
|
||||
webSearch: nativeCapabilities?.webSearch === true,
|
||||
twitterSearch: nativeCapabilities?.twitterSearch === true,
|
||||
});
|
||||
|
||||
// Tools the model can call directly: preloaded registry tools (already
|
||||
@@ -172,12 +178,12 @@ export class ChatExecutionService {
|
||||
// serialized). execute_tool routes discovered tools through the registry.
|
||||
const directTools: ToolSet = {
|
||||
...preloadedTools,
|
||||
...nativeModelTools,
|
||||
...nativeBinding.tools,
|
||||
};
|
||||
|
||||
const preloadedToolNames = [
|
||||
...Object.keys(preloadedTools),
|
||||
...Object.keys(nativeModelTools),
|
||||
...Object.keys(nativeBinding.tools),
|
||||
];
|
||||
|
||||
// ToolSet is constant for the entire conversation — no mutation.
|
||||
@@ -393,9 +399,10 @@ export class ChatExecutionService {
|
||||
stopWhen: (step) =>
|
||||
stepCountIs(AGENT_CONFIG.MAX_STEPS)(step) || hasNoMoreAvailableCredits,
|
||||
experimental_telemetry: AI_TELEMETRY_CONFIG,
|
||||
providerOptions: getCallLevelCacheProviderOptions(
|
||||
registeredModel.sdkPackage,
|
||||
),
|
||||
providerOptions: {
|
||||
...nativeBinding.providerOptions,
|
||||
...getCallLevelCacheProviderOptions(registeredModel.sdkPackage),
|
||||
},
|
||||
prepareStep: ({ messages }) => {
|
||||
stepStartedAt = performance.now();
|
||||
|
||||
|
||||
@@ -5,6 +5,7 @@ import { AiModelPreferencesService } from 'src/engine/metadata-modules/ai/ai-mod
|
||||
import { AiModelRegistryService } from 'src/engine/metadata-modules/ai/ai-models/services/ai-model-registry.service';
|
||||
import { DefaultAiCatalogService } from 'src/engine/metadata-modules/ai/ai-models/services/default-ai-catalog.service';
|
||||
import { ModelsDevCatalogService } from 'src/engine/metadata-modules/ai/ai-models/services/models-dev-catalog.service';
|
||||
import { NativeToolBinderService } from 'src/engine/metadata-modules/ai/ai-models/services/native-tool-binder.service';
|
||||
import { ProviderConfigService } from 'src/engine/metadata-modules/ai/ai-models/services/provider-config.service';
|
||||
import { SdkProviderFactoryService } from 'src/engine/metadata-modules/ai/ai-models/services/sdk-provider-factory.service';
|
||||
|
||||
@@ -18,6 +19,7 @@ import { SdkProviderFactoryService } from 'src/engine/metadata-modules/ai/ai-mod
|
||||
AiModelPreferencesService,
|
||||
AiModelRegistryService,
|
||||
AiModelConfigService,
|
||||
NativeToolBinderService,
|
||||
],
|
||||
exports: [
|
||||
DefaultAiCatalogService,
|
||||
@@ -26,6 +28,7 @@ import { SdkProviderFactoryService } from 'src/engine/metadata-modules/ai/ai-mod
|
||||
AiModelConfigService,
|
||||
SdkProviderFactoryService,
|
||||
ModelsDevCatalogService,
|
||||
NativeToolBinderService,
|
||||
],
|
||||
})
|
||||
export class AiModelsModule {}
|
||||
|
||||
+43
@@ -0,0 +1,43 @@
|
||||
import { type AiSdkPackage } from 'twenty-shared/ai';
|
||||
|
||||
import {
|
||||
AI_SDK_ANTHROPIC,
|
||||
AI_SDK_AZURE,
|
||||
AI_SDK_BEDROCK,
|
||||
AI_SDK_GOOGLE,
|
||||
AI_SDK_MISTRAL,
|
||||
AI_SDK_OPENAI,
|
||||
AI_SDK_OPENAI_COMPATIBLE,
|
||||
AI_SDK_XAI,
|
||||
} from 'src/engine/metadata-modules/ai/ai-models/constants/ai-sdk-package.const';
|
||||
import { type NativeModelTools } from 'src/engine/metadata-modules/ai/ai-models/types/native-model-tools.type';
|
||||
|
||||
export const NATIVE_MODEL_TOOLS_BY_SDK_PACKAGE = {
|
||||
[AI_SDK_ANTHROPIC]: {
|
||||
webSearch: {
|
||||
kind: 'sdk-tool',
|
||||
directToolName: 'web_search',
|
||||
},
|
||||
},
|
||||
[AI_SDK_OPENAI]: {
|
||||
webSearch: {
|
||||
kind: 'sdk-tool',
|
||||
directToolName: 'web_search',
|
||||
},
|
||||
},
|
||||
[AI_SDK_XAI]: {
|
||||
webSearch: {
|
||||
kind: 'provider-option',
|
||||
providerOptionKey: 'web',
|
||||
},
|
||||
twitterSearch: {
|
||||
kind: 'provider-option',
|
||||
providerOptionKey: 'x',
|
||||
},
|
||||
},
|
||||
[AI_SDK_GOOGLE]: {},
|
||||
[AI_SDK_MISTRAL]: {},
|
||||
[AI_SDK_BEDROCK]: {},
|
||||
[AI_SDK_OPENAI_COMPATIBLE]: {},
|
||||
[AI_SDK_AZURE]: {},
|
||||
} as const satisfies Record<AiSdkPackage, NativeModelTools>;
|
||||
+134
@@ -0,0 +1,134 @@
|
||||
import { Test, type TestingModule } from '@nestjs/testing';
|
||||
|
||||
import {
|
||||
AI_SDK_ANTHROPIC,
|
||||
AI_SDK_XAI,
|
||||
} from 'src/engine/metadata-modules/ai/ai-models/constants/ai-sdk-package.const';
|
||||
import { AiModelConfigService } from 'src/engine/metadata-modules/ai/ai-models/services/ai-model-config.service';
|
||||
import {
|
||||
AiModelRegistryService,
|
||||
type RegisteredAiModel,
|
||||
} from 'src/engine/metadata-modules/ai/ai-models/services/ai-model-registry.service';
|
||||
import { SdkProviderFactoryService } from 'src/engine/metadata-modules/ai/ai-models/services/sdk-provider-factory.service';
|
||||
|
||||
describe('AiModelConfigService.getNativeModelBinding — xAI search parameters', () => {
|
||||
let service: AiModelConfigService;
|
||||
|
||||
const xaiModel: RegisteredAiModel = {
|
||||
modelId: 'xai/grok-4',
|
||||
sdkPackage: AI_SDK_XAI,
|
||||
model: {} as RegisteredAiModel['model'],
|
||||
};
|
||||
|
||||
beforeEach(async () => {
|
||||
const module: TestingModule = await Test.createTestingModule({
|
||||
providers: [
|
||||
AiModelConfigService,
|
||||
{
|
||||
provide: AiModelRegistryService,
|
||||
useValue: {},
|
||||
},
|
||||
{
|
||||
provide: SdkProviderFactoryService,
|
||||
useValue: {},
|
||||
},
|
||||
],
|
||||
}).compile();
|
||||
|
||||
service = module.get<AiModelConfigService>(AiModelConfigService);
|
||||
});
|
||||
|
||||
it('returns empty options when neither webSearch nor twitterSearch is enabled', () => {
|
||||
expect(
|
||||
service.getNativeModelBinding(xaiModel, {
|
||||
webSearch: false,
|
||||
twitterSearch: false,
|
||||
}).providerOptions,
|
||||
).toEqual({});
|
||||
});
|
||||
|
||||
it('omits sources entirely when neither flag is enabled (no implicit "auto" search)', () => {
|
||||
const result = service.getNativeModelBinding(xaiModel, {}).providerOptions;
|
||||
|
||||
expect(result).toEqual({});
|
||||
});
|
||||
|
||||
it('emits only the web source when only webSearch is enabled', () => {
|
||||
expect(
|
||||
service.getNativeModelBinding(xaiModel, {
|
||||
webSearch: true,
|
||||
twitterSearch: false,
|
||||
}).providerOptions,
|
||||
).toEqual({
|
||||
xai: {
|
||||
searchParameters: {
|
||||
mode: 'auto',
|
||||
sources: [{ type: 'web' }],
|
||||
},
|
||||
},
|
||||
});
|
||||
});
|
||||
|
||||
it('emits only the x source when only twitterSearch is enabled', () => {
|
||||
expect(
|
||||
service.getNativeModelBinding(xaiModel, {
|
||||
webSearch: false,
|
||||
twitterSearch: true,
|
||||
}).providerOptions,
|
||||
).toEqual({
|
||||
xai: {
|
||||
searchParameters: {
|
||||
mode: 'auto',
|
||||
sources: [{ type: 'x' }],
|
||||
},
|
||||
},
|
||||
});
|
||||
});
|
||||
|
||||
it('emits both sources when webSearch and twitterSearch are enabled', () => {
|
||||
expect(
|
||||
service.getNativeModelBinding(xaiModel, {
|
||||
webSearch: true,
|
||||
twitterSearch: true,
|
||||
}).providerOptions,
|
||||
).toEqual({
|
||||
xai: {
|
||||
searchParameters: {
|
||||
mode: 'auto',
|
||||
sources: [{ type: 'web' }, { type: 'x' }],
|
||||
},
|
||||
},
|
||||
});
|
||||
});
|
||||
|
||||
it('preserves source order — web before x — for deterministic provider payloads', () => {
|
||||
const result = service.getNativeModelBinding(xaiModel, {
|
||||
webSearch: true,
|
||||
twitterSearch: true,
|
||||
}).providerOptions;
|
||||
|
||||
expect(result).toMatchObject({
|
||||
xai: {
|
||||
searchParameters: {
|
||||
sources: [{ type: 'web' }, { type: 'x' }],
|
||||
},
|
||||
},
|
||||
});
|
||||
});
|
||||
|
||||
it('returns empty options for non-xAI models even when search flags are on', () => {
|
||||
const anthropicModel: RegisteredAiModel = {
|
||||
modelId: 'anthropic/claude-sonnet-4-6',
|
||||
sdkPackage: AI_SDK_ANTHROPIC,
|
||||
model: {} as RegisteredAiModel['model'],
|
||||
supportsReasoning: false,
|
||||
};
|
||||
|
||||
expect(
|
||||
service.getNativeModelBinding(anthropicModel, {
|
||||
webSearch: true,
|
||||
twitterSearch: true,
|
||||
}).providerOptions,
|
||||
).toEqual({});
|
||||
});
|
||||
});
|
||||
+79
-23
@@ -1,7 +1,8 @@
|
||||
import { Injectable } from '@nestjs/common';
|
||||
import { Injectable, Logger } from '@nestjs/common';
|
||||
|
||||
import { ProviderOptions } from '@ai-sdk/provider-utils';
|
||||
import { ToolSet } from 'ai';
|
||||
import { type ProviderOptions } from '@ai-sdk/provider-utils';
|
||||
import { type ToolSet } from 'ai';
|
||||
import { isDefined } from 'twenty-shared/utils';
|
||||
|
||||
import { AGENT_CONFIG } from 'src/engine/metadata-modules/ai/ai-agent/constants/agent-config.const';
|
||||
import {
|
||||
@@ -10,28 +11,36 @@ import {
|
||||
AI_SDK_OPENAI,
|
||||
AI_SDK_XAI,
|
||||
} from 'src/engine/metadata-modules/ai/ai-models/constants/ai-sdk-package.const';
|
||||
import { getNativeModelToolsForSdkPackage } from 'src/engine/metadata-modules/ai/ai-models/utils/get-native-model-tools-for-sdk-package.util';
|
||||
import {
|
||||
AiModelRegistryService,
|
||||
RegisteredAiModel,
|
||||
} from 'src/engine/metadata-modules/ai/ai-models/services/ai-model-registry.service';
|
||||
import { type NativeModelBinding } from 'src/engine/metadata-modules/ai/ai-models/types/native-model-binding.type';
|
||||
import { type NativeModelToolOptions } from 'src/engine/metadata-modules/ai/ai-models/types/native-model-tool-options.type';
|
||||
import { SdkProviderFactoryService } from 'src/engine/metadata-modules/ai/ai-models/services/sdk-provider-factory.service';
|
||||
import { FlatAgentWithRoleId } from 'src/engine/metadata-modules/flat-agent/types/flat-agent.type';
|
||||
|
||||
@Injectable()
|
||||
export class AiModelConfigService {
|
||||
private readonly logger = new Logger(AiModelConfigService.name);
|
||||
|
||||
constructor(
|
||||
private readonly aiModelRegistryService: AiModelRegistryService,
|
||||
private readonly sdkProviderFactory: SdkProviderFactoryService,
|
||||
) {}
|
||||
|
||||
getProviderOptions(
|
||||
getNativeModelBinding(
|
||||
model: RegisteredAiModel,
|
||||
agent: FlatAgentWithRoleId,
|
||||
): ProviderOptions {
|
||||
options: NativeModelToolOptions = {},
|
||||
): NativeModelBinding {
|
||||
return {
|
||||
tools: this.getNativeModelTools(model, options),
|
||||
providerOptions: this.getNativeSearchProviderOptions(model, options),
|
||||
};
|
||||
}
|
||||
|
||||
getReasoningProviderOptions(model: RegisteredAiModel): ProviderOptions {
|
||||
switch (model.sdkPackage) {
|
||||
case AI_SDK_XAI:
|
||||
return this.getXaiProviderOptions(agent);
|
||||
case AI_SDK_ANTHROPIC:
|
||||
return this.getAnthropicProviderOptions(model);
|
||||
case AI_SDK_BEDROCK:
|
||||
@@ -41,13 +50,30 @@ export class AiModelConfigService {
|
||||
}
|
||||
}
|
||||
|
||||
getNativeModelTools(
|
||||
private getNativeModelTools(
|
||||
model: RegisteredAiModel,
|
||||
options: NativeModelToolOptions,
|
||||
): ToolSet {
|
||||
const tools: Record<string, unknown> = {};
|
||||
|
||||
if (!options.webSearchEnabled) {
|
||||
if (options.webSearch !== true) {
|
||||
return tools as ToolSet;
|
||||
}
|
||||
|
||||
const webSearchTool = getNativeModelToolsForSdkPackage(
|
||||
model.sdkPackage,
|
||||
)?.webSearch;
|
||||
|
||||
if (!isDefined(webSearchTool)) {
|
||||
this.logger.warn(
|
||||
`webSearch requested for sdkPackage="${model.sdkPackage}" but no native binding is registered. Skipping.`,
|
||||
);
|
||||
|
||||
return tools as ToolSet;
|
||||
}
|
||||
|
||||
// provider-option bindings (e.g. xAI) are handled in getNativeSearchProviderOptions
|
||||
if (webSearchTool.kind !== 'sdk-tool') {
|
||||
return tools as ToolSet;
|
||||
}
|
||||
|
||||
@@ -58,7 +84,8 @@ export class AiModelConfigService {
|
||||
: undefined;
|
||||
|
||||
if (anthropicProvider) {
|
||||
tools.web_search = anthropicProvider.tools.webSearch_20250305();
|
||||
tools[webSearchTool.directToolName] =
|
||||
anthropicProvider.tools.webSearch_20250305();
|
||||
}
|
||||
|
||||
break;
|
||||
@@ -69,7 +96,8 @@ export class AiModelConfigService {
|
||||
: undefined;
|
||||
|
||||
if (openaiProvider) {
|
||||
tools.web_search = openaiProvider.tools.webSearch();
|
||||
tools[webSearchTool.directToolName] =
|
||||
openaiProvider.tools.webSearch();
|
||||
}
|
||||
|
||||
break;
|
||||
@@ -79,23 +107,51 @@ export class AiModelConfigService {
|
||||
return tools as ToolSet;
|
||||
}
|
||||
|
||||
private getXaiProviderOptions(agent: FlatAgentWithRoleId): ProviderOptions {
|
||||
if (
|
||||
!agent.modelConfiguration ||
|
||||
(!agent.modelConfiguration.webSearch?.enabled &&
|
||||
!agent.modelConfiguration.twitterSearch?.enabled)
|
||||
) {
|
||||
private getNativeSearchProviderOptions(
|
||||
model: RegisteredAiModel,
|
||||
options: NativeModelToolOptions,
|
||||
): ProviderOptions {
|
||||
switch (model.sdkPackage) {
|
||||
case AI_SDK_XAI:
|
||||
return this.getXaiSearchProviderOptions(options);
|
||||
default:
|
||||
return {};
|
||||
}
|
||||
}
|
||||
|
||||
private getXaiSearchProviderOptions(
|
||||
options: NativeModelToolOptions,
|
||||
): ProviderOptions {
|
||||
const webSearchEnabled = options.webSearch === true;
|
||||
const twitterSearchEnabled = options.twitterSearch === true;
|
||||
|
||||
if (!webSearchEnabled && !twitterSearchEnabled) {
|
||||
return {};
|
||||
}
|
||||
|
||||
const sources: Array<{ type: string }> = [];
|
||||
const xaiTools = getNativeModelToolsForSdkPackage(AI_SDK_XAI);
|
||||
const webSearchTool = xaiTools?.webSearch;
|
||||
const twitterSearchTool = xaiTools?.twitterSearch;
|
||||
|
||||
if (agent.modelConfiguration.webSearch?.enabled) {
|
||||
sources.push({ type: 'web' });
|
||||
if (webSearchEnabled) {
|
||||
if (webSearchTool?.kind === 'provider-option') {
|
||||
sources.push({ type: webSearchTool.providerOptionKey });
|
||||
} else {
|
||||
this.logger.warn(
|
||||
`webSearch requested for xAI but no provider-option binding is registered. Skipping.`,
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
if (agent.modelConfiguration.twitterSearch?.enabled) {
|
||||
sources.push({ type: 'x' });
|
||||
if (twitterSearchEnabled) {
|
||||
if (twitterSearchTool?.kind === 'provider-option') {
|
||||
sources.push({ type: twitterSearchTool.providerOptionKey });
|
||||
} else {
|
||||
this.logger.warn(
|
||||
`twitterSearch requested for xAI but no provider-option binding is registered. Skipping.`,
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
return {
|
||||
|
||||
+12
@@ -0,0 +1,12 @@
|
||||
import { type RegisteredAiModel } from 'src/engine/metadata-modules/ai/ai-models/services/ai-model-registry.service';
|
||||
import { type NativeModelBinding } from 'src/engine/metadata-modules/ai/ai-models/types/native-model-binding.type';
|
||||
import { type NativeModelToolOptions } from 'src/engine/metadata-modules/ai/ai-models/types/native-model-tool-options.type';
|
||||
|
||||
// Parallel to ToolProvider — resolves the complete native-model call payload
|
||||
// (SDK-native tools + provider options), not registry descriptors.
|
||||
export interface NativeToolBinder {
|
||||
bind(
|
||||
model: RegisteredAiModel,
|
||||
options: NativeModelToolOptions,
|
||||
): NativeModelBinding;
|
||||
}
|
||||
+19
@@ -0,0 +1,19 @@
|
||||
import { Injectable } from '@nestjs/common';
|
||||
|
||||
import { AiModelConfigService } from 'src/engine/metadata-modules/ai/ai-models/services/ai-model-config.service';
|
||||
import { type RegisteredAiModel } from 'src/engine/metadata-modules/ai/ai-models/services/ai-model-registry.service';
|
||||
import { type NativeToolBinder } from 'src/engine/metadata-modules/ai/ai-models/services/native-tool-binder.interface';
|
||||
import { type NativeModelBinding } from 'src/engine/metadata-modules/ai/ai-models/types/native-model-binding.type';
|
||||
import { type NativeModelToolOptions } from 'src/engine/metadata-modules/ai/ai-models/types/native-model-tool-options.type';
|
||||
|
||||
@Injectable()
|
||||
export class NativeToolBinderService implements NativeToolBinder {
|
||||
constructor(private readonly aiModelConfigService: AiModelConfigService) {}
|
||||
|
||||
bind(
|
||||
model: RegisteredAiModel,
|
||||
options: NativeModelToolOptions = {},
|
||||
): NativeModelBinding {
|
||||
return this.aiModelConfigService.getNativeModelBinding(model, options);
|
||||
}
|
||||
}
|
||||
+7
@@ -0,0 +1,7 @@
|
||||
import { type ProviderOptions } from '@ai-sdk/provider-utils';
|
||||
import { type ToolSet } from 'ai';
|
||||
|
||||
export type NativeModelBinding = {
|
||||
tools: ToolSet;
|
||||
providerOptions: ProviderOptions;
|
||||
};
|
||||
+1
@@ -0,0 +1 @@
|
||||
export type NativeModelToolKey = 'webSearch' | 'twitterSearch';
|
||||
+5
-3
@@ -1,3 +1,5 @@
|
||||
export type NativeModelToolOptions = {
|
||||
webSearchEnabled?: boolean;
|
||||
};
|
||||
import { type NativeModelToolKey } from 'src/engine/metadata-modules/ai/ai-models/types/native-model-tool-key.type';
|
||||
|
||||
export type NativeModelToolOptions = Partial<
|
||||
Record<NativeModelToolKey, boolean>
|
||||
>;
|
||||
|
||||
+15
@@ -0,0 +1,15 @@
|
||||
import { type NativeModelToolKey } from 'src/engine/metadata-modules/ai/ai-models/types/native-model-tool-key.type';
|
||||
|
||||
export type NativeModelTools = Partial<
|
||||
Record<
|
||||
NativeModelToolKey,
|
||||
| {
|
||||
kind: 'sdk-tool';
|
||||
directToolName: string;
|
||||
}
|
||||
| {
|
||||
kind: 'provider-option';
|
||||
providerOptionKey: string;
|
||||
}
|
||||
>
|
||||
>;
|
||||
+17
@@ -0,0 +1,17 @@
|
||||
import { type AiSdkPackage } from 'twenty-shared/ai';
|
||||
|
||||
import { type NativeModelToolKey } from 'src/engine/metadata-modules/ai/ai-models/types/native-model-tool-key.type';
|
||||
import { getNativeModelToolsForSdkPackage } from 'src/engine/metadata-modules/ai/ai-models/utils/get-native-model-tools-for-sdk-package.util';
|
||||
|
||||
export const getNativeModelCapabilities = (
|
||||
sdkPackage?: AiSdkPackage | null,
|
||||
): Partial<Record<NativeModelToolKey, boolean>> | undefined => {
|
||||
const tools = getNativeModelToolsForSdkPackage(sdkPackage);
|
||||
const toolKeys = tools ? Object.keys(tools) : [];
|
||||
|
||||
if (toolKeys.length === 0) {
|
||||
return undefined;
|
||||
}
|
||||
|
||||
return Object.fromEntries(toolKeys.map((toolKey) => [toolKey, true]));
|
||||
};
|
||||
+9
@@ -0,0 +1,9 @@
|
||||
import { type AiSdkPackage } from 'twenty-shared/ai';
|
||||
|
||||
import { NATIVE_MODEL_TOOLS_BY_SDK_PACKAGE } from 'src/engine/metadata-modules/ai/ai-models/constants/native-model-tools-by-sdk-package.const';
|
||||
import { type NativeModelTools } from 'src/engine/metadata-modules/ai/ai-models/types/native-model-tools.type';
|
||||
|
||||
export const getNativeModelToolsForSdkPackage = (
|
||||
sdkPackage?: AiSdkPackage | null,
|
||||
): NativeModelTools | undefined =>
|
||||
sdkPackage ? NATIVE_MODEL_TOOLS_BY_SDK_PACKAGE[sdkPackage] : undefined;
|
||||
Reference in New Issue
Block a user